How To Upload Files to S3 Using MSBuild

On a recent project, we needed to copy a number of text files containing JSON files and several DLL’s from our solution onto our staging environment. For most companies, I’d recommend looking at Octopus Deploy to do this work. This client didn’t want to use Octopus and they also didn’t’ want to check non-web files into their BitBucket repo, so the available options.

The final solution involved using AmazonS3 to sync the files to the server. Amazon S3 (Amazon Simple Storage Service) is cloud storage solution similar to dropbox. Instead of storing files on your own web server, S3 was designed to make file sharing quicker, more scalable and easier for developers to share files.

In order to use S3 as a deployment tool, we had to create a build script that copied certain files from a designated folder into a zip and then upload that ZIP file onto S3. In our build server, we then needed to follow reverse the process and create a script to copy the zip file form S3 and extract it onto our testing server.

The below snippet should copy all files in a folder called ‘files’ and upload them into S3 for you, enjoy 🙂

<?xml version="1.0" encoding="utf-8" ?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Key>KEY_GOES_HERE</Key>
<Secret>SECRET_GOES_HERE</Secret>
<Bucket>BUCKET_GOES_HERE</Bucket>
<S3FileName>FILENAME_GOES_HERE</S3FileName>
<RootDir>$(MSBuildProjectDirectory)\..\</RootDir>
<OutputDir>$(RootDir)\bin\</OutputDir>
<S3Files>$(RootDir)\files\</S3Files>
<Contents></Contents>
</PropertyGroup>
<PropertyGroup>
<Target Name="Clean">
<RemoveDir Directories="$(S3Files)" />
<MakeDir Directories="$(OutputDir)"/>
</Target>
<Target Name="UploadFiles">
<CreateItem Include="$(RootDir)\Import\**\*.*">
<Output TaskParameter="Include" ItemName="filesToImport" />
</CreateItem>
<Copy SourceFiles="@(filesToImport)"
DestinationFiles="@(filesToImport->'$(S3Files)\Import\%(RecursiveDir)%(Filename)%(Extension)')" />
</Target>
<Target Name="ZipFolder">
<ItemGroup>
<SourceFiles Include="$(S3Files)\**\*.*"/>
</ItemGroup>
<Zip
Files="@(SourceFiles)"
ZipFileName="$(S3Files)..\$(S3FileName)"
ZipLevel="9"
/>
</Target>
<Target Name="UploadToS3" DependsOnTargets="Clean; UploadFiles; ZipFolder;">
<PublishFiles
Key= "$(key)"
Secret= "$(Secret)"
SourceFiles = "$(S3Files)..\$(S3FileName)"
Bucket="$(Bucket)"
DestinationFolder="S3Files/"
/>
</Target>
</Project>

Jon D Jones

Software Architect, Programmer and Technologist Jon Jones is founder and CEO of London-based tech firm Digital Prompt. He has been working in the field for nearly a decade, specializing in new technologies and technical solution research in the web business. A passionate blogger by heart , speaker & consultant from England.. always on the hunt for the next challenge

More Posts

2 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *