Rob Lacey

Brighton, UK - contact@robl.me

Software Engineer specialising in Ruby on Rails. I do occasional freelance work, I love working for SARD JV Ltd, they are awesome and give me my pocket money every month. By all means dangle a carrot.


Learn Elixir - Attempt 5 Billion

Started looking at Elixir again. Thinking about building something with MongoDB, and I know that support for MongoDB Ecto but there’s chatter about it and is not quite there but this is an opportunity to start hacking away. Anyway, first point of call. What the hell is Plug anyway, it’s kinda like Rack in Ruby.

https://github.com/braindeaf/unplug

So that’s the Plug docs examples, now to expand on it.

Clutter and getting rid of XCode

I hate clutter. On Kevin ‘s recommendation I bought Disk Daisy to try and clear guff from my Disk, in the same breath as saying XCode isn’t really needed. Yep I just went with it when I got 5Gb updates just so I could install anything. So first call of business….delete XCode. Just nuked that sucker from my Applications folder. Everything still seems to work. Cool.

However, I might have broken something.

Robs-MacBook-Pro:repos rl$ rbenv install 3.1.0-dev
Downloading openssl-1.1.1l.tar.gz...
-> https://dqw8nmjcqpjn7.cloudfront.net/0b7a3e5e59c34827fe0c3a74b7ec8baef302b98fa80088d7f9153aa16fa76bd1
Installing openssl-1.1.1l...

BUILD FAILED (macOS 11.6 using ruby-build 20210928)

Inspect or clean up the working tree at /var/folders/xr/tb40m1q965n0x4z21sjd9mhh0000gn/T/ruby-build.20211110082145.18668.XUsYq9
Results logged to /var/folders/xr/tb40m1q965n0x4z21sjd9mhh0000gn/T/ruby-build.20211110082145.18668.log

Last 10 log lines:
***                                                                ***
***       perl configdata.pm --dump                                ***
***                                                                ***
***   (If you are new to OpenSSL, you might want to consult the    ***
***   'Troubleshooting' section in the INSTALL file first)         ***
***                                                                ***
**********************************************************************
xcrun: error: active developer path ("/Applications/Xcode.app/Contents/Developer") does not exist
Use `sudo xcode-select --switch path/to/Xcode.app` to specify the Xcode that you wish to use for command line developer tools, or use `xcode-select --install` to install the standalone command line developer tools.
See `man xcode-select` for more details.

Ok, so we still need command line tools. I knew that but I kinda assume they were installed alongside XCode not as part of it. It does appear that you can install them manually on their own.

https://developer.apple.com/download/all/?q=command%20line%20tools

Same issue. Do I need to force the Terms and Conditions confirmation again.

Robs-MacBook-Pro:repos rl$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

Not can’t even do that.

Robs-MacBook-Pro:repos rl$ xcode-select -h
Usage: xcode-select [options]

Print or change the path to the active developer directory. This directory
controls which tools are used for the Xcode command line tools (for example, 
xcodebuild) as well as the BSD development commands (such as cc and make).

Options:
  -h, --help                  print this help message and exit
  -p, --print-path            print the path of the active developer directory
  -s <path>, --switch <path>  set the path for the active developer directory
  --install                   open a dialog for installation of the command line developer tools
  -v, --version               print the xcode-select version
  -r, --reset                 reset to the default command line tools path

Only option I really have is to reset the Command Line Tools path.

sudo xcode-select --reset

And boom….!

rbenv install 3.1.0-dev
Downloading openssl-1.1.1l.tar.gz...
-> https://dqw8nmjcqpjn7.cloudfront.net/0b7a3e5e59c34827fe0c3a74b7ec8baef302b98fa80088d7f9153aa16fa76bd1
Installing openssl-1.1.1l...
Installed openssl-1.1.1l to /Users/rl/.rbenv/versions/3.1.0-dev

Cloning https://github.com/ruby/ruby.git...
Installing ruby-master...
ruby-build: using readline from homebrew
Installed ruby-master to /Users/rl/.rbenv/versions/3.1.0-dev

Ok, easier than I thought.

WARNING: Nokogiri was built against LibXML version 2.9.4, but has dynamically loaded 2.9.12

Stopping spewing moany messages at me. Let’s sort this one.

Robs-MacBook-Pro:project rl$ rails c
WARNING: Nokogiri was built against LibXML version 2.9.4, but has dynamically loaded 2.9.12
>
Robs-MacBook-Pro:project rl$ cat Gemfile | grep nokogiri
gem 'nokogiri', '1.9.1'
Robs-MacBook-Pro:project rl$ gem install nokogiri -v 1.9.1 -- --with-xml2-dir=/usr --with-xslt-dir=/opt/local --with-iconv-dir=/opt/local
Building native extensions with: '--with-xml2-dir=/usr --with-xslt-dir=/opt/local --with-iconv-dir=/opt/local'
This could take a while...
Successfully installed nokogiri-1.9.1
1 gem installed

Now it’s compiled against these liibrary locations…

Robs-MacBook-Pro:project rl$ rails c
Loading development environment (Rails 4.2.11.3)
>

And done….

HAR format

The HTTP Archive format, or HAR, is a JSON-formatted archive file format for logging of a web browser’s interaction with a site. The common extension for these files is .har

https://en.wikipedia.org/wiki/HAR_

Awesome. Google Chrome allows you to export a session of network traffic as an HAR, as do a number of others. Nice. If you’re scraping a website and want to capture the requests that help construct that page e.g. a single page JS application that makes 10 calls to a JSON API. Automate this and you’ve got quite an arsenal, you can part scrape a page’s content, grab urls of all images loaded, pinch cookies, etc all from the HAR.

Tell me more, how do I construct my own????

The specification for this format is produced by the Web Performance Working Group1 of the World Wide Web Consortium (W3C). This document was never published by the W3C Web Performance Working Group and has been abandoned

Handy. Thanks.

This might help :S

http://www.softwareishard.com/blog/har-12-spec/
https://www.medianova.com/en-blog/a-step-by-step-guide-to-generating-a-har-file/

What is refactoring?

I can think of a better way to describe refactoring. Nice to see you again Simpkin.

Devise and nuking sessions - Magical Devise Journey II

Not a few days ago I was patting myself on the back for my Magical Devise Journey forcing only particular Devise strategies in some controllers. So after all that. I realised that is no good if we are logged in and that authentication is stored I am still logged in if I try to access API endpoints from the browser. What gives?

Well I was all obsessed over the strategies passed in as args I didn’t spot something.

def _perform_authentication(*args)
  scope, opts = _retrieve_scope_and_opts(args)
  user = nil

  # Look for an existing user in the session for this scope.
  # If there was no user in the session, see if we can get one from the request.
  return user, opts if user = user(opts.merge(:scope => scope))
  _run_strategies_for(scope, args)

  if winning_strategy && winning_strategy.successful?
    opts[:store] = opts.fetch(:store, winning_strategy.store?)
    set_user(winning_strategy.user, opts.merge!(:event => :authentication))
  end

  [@users[scope], opts]
end

Hang on, if you find the user for that scope return them or try our auth strategies..

return user, opts if user = user(opts.merge(:scope => scope))
_run_strategies_for(scope, args)

Dang!

def user(argument = {})
  opts  = argument.is_a?(Hash) ? argument : { :scope => argument }
  scope = (opts[:scope] ||= @config.default_scope)

  if @users.has_key?(scope)
    @users[scope]
  else
    unless user = session_serializer.fetch(scope)
      run_callbacks = opts.fetch(:run_callbacks, true)
      manager._run_callbacks(:after_failed_fetch, user, self, :scope => scope) if run_callbacks
    end

    @users[scope] = user ? set_user(user, opts.merge(:event => :fetch)) : nil
  end
end

So where am I finding it if it’s not by strategy….oh right. I am logged in via a session cookie.

def fetch(scope)
  key = session[key_for(scope)]
  return nil unless key

  method_name = "#{scope}_deserialize"
  user = respond_to?(method_name) ? send(method_name, key) : deserialize(key)
  delete(scope) unless user
  user
end

So I need to nuke that session in my API controllers.

I seem to remember the preferred way to nuke sessions in controllers was something like

class ApiController < ApplicationController
  protect_from_forgery with: :null_session
end

That’s the skinny. It nullifies the session and cookies in some magic Rails-y way.

module ActionController #:nodoc:
  module RequestForgeryProtection
    module ProtectionMethods
      class NullSession
        def initialize(controller)
          @controller = controller
        end

        # This is the method that defines the application behavior when a request is found to be unverified.
        def handle_unverified_request
          request = @controller.request
          request.session = NullSessionHash.new(request.env)
          request.env['action_dispatch.request.flash_hash'] = nil
          request.env['rack.session.options'] = { skip: true }
          request.env['action_dispatch.cookies'] = NullCookieJar.build(request)
        end
      end
    end
  end
end

But…..

module ActionController #:nodoc:
  module RequestForgeryProtection
    def verify_authenticity_token
      mark_for_same_origin_verification!

      if !verified_request?
        if logger && log_warning_on_csrf_failure
          logger.warn "Can't verify CSRF token authenticity"
        end
        handle_unverified_request
      end
    end

    def verified_request?
      !protect_against_forgery? || request.get? || request.head? ||
        valid_authenticity_token?(session, form_authenticity_param) ||
        valid_authenticity_token?(session, request.headers['X-CSRF-Token'])
    end
  end
end

Dang! Dang! Looks like it doesn’t even bother to run the forgery strategy if it’s a GET. So it will nuke sessions but not unless the request should be verified. No worries we’ll just re-use the method of nuking the session and cookies.

class ApiResourceController < ApplicationController
  prepend_before_filter do
    ActionController::RequestForgeryProtection::ProtectionMethod::NullSession.new(self).handle_unverified_request
  end
end

Magic.

Because I can never remember how to concatenate SSL cetificates

Our certificate + The Bundle equals glorious working cert.

$ cat downloads.bigblindmedia.com.crt downloads.bigblindmedia.com.ca-bundle > downloads.crt

Go to Big Blind Media and buy some magic they are lovely.

I do stuff on the internet for CASH Monies

https://smartmockups.com/mockups/multiple-devices

JBuilder Arrays and non-collections

JBuilder may not be the most efficient way to present an API but for simple cases it works pretty well. Our JSON-API standardized API could return the current authenticated user like so.

show.json.jbuilder

json.data do
  json.id current_user.id
  json.type 'users'
  json.attributes do
    json.(current_user, :uuid, :email, :full_name)
  end
end

/api/v1/me

{
  "data": {
    "id": "1a2b3c4d000001",
    "type": "users",
    "attributes": {
      "uuid": "aaaaaaaabbbbbbbbccccccccc",
      "email": "rob.lacey@buttonmoon.co.uk",
      "full_name": "Mr Rob Lacey"
    }
 }

Ideally I want to now add an included block so that can included related objects such as Tenant. Something like

{
  "data": {
    "id": "1a2b3c4d000001",
    "type": "users",
    "attributes": {
      "uuid": "aaaaaaaabbbbbbbbccccccccc",
      "email": "rob.lacey@buttonmoon.co.uk",
      "full_name": "Mr Rob Lacey"
    },
    "included": [
      {
        "id": "1",
        "type": "tenants",
        "attributes": {
          "name": "superadmin"
        }
      }
    ]
  }
}
All of the JBuilder examples talk in terms of Arrays of objects being built from a collection. Most of the time they probably are.
json.array! @comments do |comment|
  next if comment.marked_as_spam_by?(current_user)

  json.body comment.body
  json.author do
    json.first_name comment.author.first_name
    json.last_name comment.author.last_name
  end
end
Or
json.array! @people, :id, :name
 
But not always. If we want to add an array that is made up or arbitrary builder blocks, you find yourself thinking in terms of doing.
json.included do
  json.merge! do
    json.id current_user.tenant_id
    json.type 'tenants'
    json.attributes do
      json.(current_user.tenant, :name)
    end
  end
end
json.included [
  json.data do
    json.id current_user.tenant_id
    json.type 'tenants'
    json.attributes do
      json.(current_user.tenant, :name)
    end
  end
end
]  

Neither of which work, turns out the way to do it is using the undocumented child! method.

json.data do
  json.id current_user.id
  json.type 'users'
  json.attributes do
    json.(current_user, :uuid, :email, :full_name)
  end

  json.included do
    json.child! do
      json.id current_user.tenant_id
      json.type 'tenants'
      json.attributes do
        json.(current_user.tenant, :name)
      end
    end
  end
end
Think I’ll do a pull request for the README as this took me a while to work out. https://github.com/rails/jbuilder/pull/507

The Magical Devise Journey - Limiting Warden Strategies on specific controllers.

Today I was solving how to skip particular Devise strategies on specific controllers. So here’s the thing, we use Devise and recently added a Doorkeeper setup for JWT API Authentication. This is all very nice, but the more strategies you add the more overhead there is when authenticating. 6 strategies when you know you’re only going to need one is too much. But perhaps the biggest problem is if you’re logged in with a session cookie, API responses work out of the box for the current authenticated user and we want/need our public facing API to be stateless.

This begs the question how does all this magic work anyway. Well, Devise is built on Warden, a Rack middleware. You define strategies in your User model and this in turn indirectly configures a Warden::Proxy object which is stored in the Request#env of your incoming request. I assume this is just flexible and can be accessed inside/outside of controllers in middleware, etc. When your controller needs the current user it queries the Warden::Proxy and all of the prefined strategies are consulted in turn to find it.

You can see this here in https://github.com/heartcombo/devise/blob/master/lib/devise/controllers/helpers.rb

module Devise
  module Controllers
    module Helpers
      module ClassMethods
        def self.define_helpers(mapping) #:nodoc:
          mapping = mapping.name

          class_eval <<-METHODS, __FILE__, __LINE__ + 1
            def authenticate_#{mapping}!(opts = {})
              opts[:scope] = :#{mapping}
              warden.authenticate!(opts) if !devise_controller? || opts.delete(:force)
            end

            def #{mapping}_signed_in?
              !!current_#{mapping}
            end

            def current_#{mapping}
              @current_#{mapping} ||= warden.authenticate(scope: :#{mapping})
            end

            def #{mapping}_session
              current_#{mapping} && warden.session(:#{mapping})
            end
          METHODS

          ActiveSupport.on_load(:action_controller) do
            if respond_to?(:helper_method)
              helper_method "current_#{mapping}", "#{mapping}_signed_in?", "#{mapping}_session"
            end
          end
        end

        # The main accessor for the warden proxy instance
        def warden
          request.env['warden'] or raise MissingWarden
        end
      end
    end
  end
end

You give Warden a scope and it will find your User or not.

def current_user
  warden.authenticate(scope: :user)
end

However, this is consulting every strategy and in our API base controller we only care about the JWT strategy. Warden allows you to query all or specific strategies (https://github.com/wardencommunity/warden/blob/master/lib/warden/proxy.rb)

module Warden
  class Proxy
    # Example:
    #   env['warden'].authenticate(:password, :basic, :scope => :sudo)
    #
    # :api: public
    def authenticate(*args)
      user, _opts = _perform_authentication(*args)
      user
    end
  end
end

We can overwrite the authenticate_user!, current_user methods in our API base controller.

def authenticate_user!(opts = {})
  opts[:scope] = :user
  warden.authenticate!(:jwt, opts) if !devise_controller? || opts.delete(:force)
end

def current_user
  @current_user ||= warden.authenticate(:jwt, scope: :user)
end

Or to be more flexible

class ApplicationController < ActionController::Base
  class_attribute :warden_strategies

  def authenticate_user!(opts = {})
    opts[:scope] = :user
    warden.authenticate!(*self.class.warden_strategies, opts) if !devise_controller? || opts.delete(:force)
  end

  def current_user
    @current_user ||= warden.authenticate(*self.class.warden_strategies, scope: :user)
  end
end

class ApiController < ApplicationController
  self.warden_strategies = :jwt
end

Yay, stateless API requests. Phew.

I gone did a Pull Request in case it is valuable to anyone else https://github.com/heartcombo/devise/pull/5392/files