“...I've been working since 2008 with Ruby / Ruby on Rails, love a bit of Elixir / Phoenix and learning Rust. I also poke through other people's code and make PRs for OpenSource Ruby projects that sometimes make it. Currently working for InPay...”

Rob Lacey
Senior Software Engineer, UK

Uploading files direct to S3 with AWS gem

Sometimes you just have to upload huge files to S3, today it was a 10Gb mp4. We established a long time ago that uploading via a Rails app via Paperclip was never going to work for files over a certain size. Indeed it is a waste of resources to upload a temporary file to our application server to act as a proxy to S3. So we’ve been uploading to S3 directly for some time with some changes to Paperclip and PLUpload. Today, PLUpload fell over on a 10Gb file and that’s fine but we need a solution when this fails. Personally I find the AWS gem documentation quite difficult to navigate and never having quite as many examples as I would like. So….

s3 = Aws::S3::Resource.new(
  region: 'us-east-1',
  access_key_id: 'ACCESS_KEY_ID',
  secret_access_key: 'SECRET_ACCESS_KEY'
)
# => #<Aws::S3::Resource:0x007fe7a6a81d18 @client=#<Aws::S3::Client> 

obj = s3.bucket('assets.blah.com').object('video/001/011/221/original-files.mp4')

# => #<Aws::S3::Object:0x007fe7a5a2afc8 @bucket_name="assets.blah.com", @key="video/001/011/221/original-files.mp4", @data=nil, @client=#<Aws::S3::Client> 

obj.upload_file("/Users/roblacey/Desktop/original-files.mp4")
GPK of the Day Charred CHAD