Uploading files direct to S3 with AWS gem

20 Jul 2018 09:58

Sometimes you just have to upload huge files to S3, today it was a 10Gb mp4. We established a long time ago that uploading via a Rails app via Paperclip was never going to work for files over a certain size. Indeed it is a waste of resources to upload a temporary file to our application server to act as a proxy to S3. So we’ve been uploading to S3 directly for some time with some changes to Paperclip and PLUpload. Today, PLUpload fell over on a 10Gb file and that’s fine but we need a solution when this fails. Personally I find the AWS gem documentation quite difficult to navigate and never having quite as many examples as I would like. So….

s3 = Aws::S3::Resource.new(
  region: 'us-east-1',
  access_key_id: 'ACCESS_KEY_ID',
  secret_access_key: 'SECRET_ACCESS_KEY'
)
# => #<Aws::S3::Resource:0x007fe7a6a81d18 @client=#<Aws::S3::Client> 

obj = s3.bucket('assets.blah.com').object('video/001/011/221/original-files.mp4')

# => #<Aws::S3::Object:0x007fe7a5a2afc8 @bucket_name="assets.blah.com", @key="video/001/011/221/original-files.mp4", @data=nil, @client=#<Aws::S3::Client> 

obj.upload_file("/Users/roblacey/Desktop/original-files.mp4")