I'm not angry but...

I spent much of yesterday trying to Paperclip working with some S3 extensions that I’ve used in countless projects and it just didn’t work. If you’re interested, well, here’s a slice of it.

module Paperclip
  module Storage
    S3.prepend(Module.new do
      def url(style_name = default_style, options = {})
        with_environment { super }
      end

      def expiring_url(time = 3600, style_name = default_style)
        with_environment { super }
      end
    end)
  end

  module Storage
    module S3
      def download_url(time = 3600, style_name = default_style)
        with_content_disposition("attachment; filename=#{instance_read(:file_name).without_extended_characters}") do
          expiring_url(time, style_name)
        end
      end

      def inline_url(time = 3600, style_name = default_style)
        with_content_disposition('inline') do
          expiring_url(time, style_name)
        end
      end

      private def with_content_disposition(disposition)
        @options[:s3_url_options] ||= {}
        @options[:s3_url_options][:response_content_disposition] = "attachment; filename=#{instance_read(:file_name).without_extended_characters}"
        yield
      ensure
        @options[:s3_url_options].delete(:response_content_disposition)
        @options.delete(:s3_url_options) if @options[:s3_url_options].empty?
      end

      # overrides s3 credentials to serve production assets when in
      # staging or development
      private def with_environment
        # only override if dev or staging
        return yield unless Rails.env.development? || Rails.env.staging?
        # only override if updated in the last day
        return yield if (instance_read(:updated_at) || 2.days.ago) > 1.day.ago

        begin
          @s3_interface = nil
          @bucket = nil
          @s3_bucket = nil
          @s3_credentials = parse_credentials(Settings.get(:s3, 'production-readonly')[:s3_credentials])

          yield
        ensure
          @s3_interface = nil
          @bucket = nil
          @s3_bucket = nil
          @s3_credentials = parse_credentials(Settings.get(:s3)[:s3_credentials])
        end
      end
    end
  end
end

With Settings to extract settings and cope with overriding different environments. Say, to display production images in development using the above. I’ve found it very useful.

module Settings
  def self.get(name, environment = Rails.env.to_s)
    fetch("#{name}.yml")[environment]
  end

  def self.fetch(config)
    yaml = paths(config).inject([]) do |yamls, path|
      yamls << IO.read(path)
    rescue Errno::ENOENT
      yamls
    end.reject(&:blank?).join("\n")

    if Rails.env.production? && yaml.include?('&defaults') && !yaml.include?("production:\n")
      yaml += "\nproduction:\n  <<: *defaults"
    end

    YAML.load(ERB.new(yaml).result)
  end

  def self.paths(config)
    [
      Rails.root.join('config', 'settings', config), # repo
      File.join(ENV['HOME'], application, 'shared', 'settings', config), # server
      File.join(ENV['HOME'], '.settings', application, config) # local
    ]
  end

  def self.application
    Rails.application.class.name.split('::')[0].underscore
  end
end

Except for yesterday when I forgot or didn’t realise that the storage option in Paperclip needs to be a symbol. If it’s a string it gets ignored.

def self.get(name, environment = Rails.env.to_s)
    fetch("#{name}.yml")[environment].deep_symbolize_keys # <---- hours wasted
  end

And finally the methods that extend Attachment::Storage::S3 don’t appear because it defaults to :filesystem unless you supply :s3 as a symbol. Ok, finally I am there… No.

AWS::S3::Errors::AccessDenied Access Denied

But I’ve added the Bucket Policy to use my IAM user, and set the correct permissions. I know this because I compared it to another policy I set up a week prior.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::1234567890:user/username"
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketPolicy",
                "s3:PutBucketPolicy"
            ],
            "Resource": "arn:aws:s3:::bucketname"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::1234567890:user/username"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::bucketname/*"
        }
    ]
}

Files didn’t upload at all. S3 credentials are correct. But still the same error. As then as it turns out, S3 buckets can be on this occasion was set to Bucket and Objects not public. So eventually I realise that the Paperclip upload default is :public and if I am trying to do that then I get AWS::S3::Errors::AccessDenied Access Denied sigh

defaults: &defaults
  storage: :s3
  bucket: bucket
  s3_protocol: :https
  s3_permissions: :private
  s3_credentials:
    access_key_id: accesskey
    secret_access_key: secretaccesskey
    s3_region: us-east-1

development:
  <<: *defaults

production:
  <<: *defaults
  bucket: bucket

So finally…at 2am. This is where we are. I enjoy the chase, I also enjoy sleep.