“...I've been working since 2008 with Ruby / Ruby on Rails, love a bit of Elixir / Phoenix and learning Rust. I also poke through other people's code and make PRs for OpenSource Ruby projects that sometimes make it. Currently working for InPay...”

Rob Lacey
Senior Software Engineer, UK

JS Mystery #667

This doesn’t appear anywhere in our codebase. Does iPhone Safari inject this under some circumstances?

undefined is not an object (evaluating 'document.getElementsByTagName('video')[0].webkitExitFullScreen')

When in doubt write that shit down

I do know how to write, but today the only thing I can make out is Nagios and rubocop -A

Somewhere in there there is something about JSONAPI-Resources attributes, maybe.

Rails 6.0.4 mangled SQL

Looks like Rails 6.0.4 upgrade has mangled the SQL JSONAPI-Resources is constructing.

Rswag::Specs::UnexpectedResponse:
        Expected response code '500' to match '200'
        Response body: {
          "errors": [
            {
              "title": "Internal Server Error",
              "detail": "Internal Server Error",
              "code": "INTERNAL_SERVER_ERROR",
              "status": "500",
              "meta": {
                "exception": "Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.\"id\" AS \"users_id\", users.last_name, users.first_name FROM `users` INNER JOIN `' at line 1",

The backtrace has lead me here.

From: /Users/rl/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/activerecord-6.0.4/lib/active_record/relation/calculations.rb:186 ActiveRecord::Calculations#pluck:

    181: def pluck(*column_names)
    182:   if loaded? && (column_names.map(&:to_s) - @klass.attribute_names - @klass.attribute_aliases.keys).empty?
    183:     return records.pluck(*column_names)
    184:   end
    185: 
 => 186:   binding.pry
    187: 
    188:   if has_include?(column_names.first)
    189:     relation = apply_join_dependency
    190:     relation.pluck(*column_names)
    191:   else
    192:     klass.disallow_raw_sql!(column_names)
    193:     relation = spawn
    194:     relation.select_values = column_names
    195:     result = skip_query_cache_if_necessary { klass.connection.select_all(relation.arel, nil) }
    196:     result.cast_values(klass.attribute_types)
    197:   end
    198: end

[1] pry(#<User::ActiveRecord_Relation>)> column_names
=> ["\"users\".\"id\" AS \"users_id\"", "users.last_name", "users.first_name"]
What’s up with that? Looking.

The scariest day of my life. Tainted master branch

You wake in a dark room and someone has only gone and tainted your master branch by merging in multiple commits spanning weeks of work. We can’t possibly unpick 100+ commits, what the hell even were they? But we need master back as it was.

We can however, reset our master to a particular commit in the past that we trust.

git checkout master
git reset --hard <somehash>
git push --force origin <somehash>:master

Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:somerepo/someproject.git
 + <someotherhash>...<somehash> <somehash> -> master (forced update)

There were tears of joy and shaky hands, but it was fine.

ImageMagic not installing on MacOSX

Building RMagick with native extensions is playing up again… I use homebrew so I was looking for the right solution.

Using rexml 3.2.4
Fetching rmagick 2.13.4
Installing rmagick 2.13.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /Users/rl/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/rmagick-2.13.4/ext/RMagick
/Users/rl/.rbenv/versions/2.6.6/bin/ruby -I /Users/rl/.rbenv/versions/2.6.6/lib/ruby/2.6.0 -r ./siteconf20210707-24851-uczhdh.rb extconf.rb
checking for Ruby version >= 1.8.5... yes
checking for clang... yes
checking for Magick-config... no
checking for pkg-config... yes
checking for ImageMagick version >= 6.4.9... yes
checking for stdint.h... yes
checking for sys/types.h... yes
checking for wand/MagickWand.h... no

Can't install RMagick 2.13.4. Can't find MagickWand.h.
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/Users/rl/.rbenv/versions/2.6.6/bin/$(RUBY_BASE_NAME)

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/rl/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/extensions/x86_64-darwin-20/2.6.0/rmagick-2.13.4/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /Users/rl/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/gems/rmagick-2.13.4 for inspection.
Results logged to /Users/rl/.rbenv/versions/2.6.6/lib/ruby/gems/2.6.0/extensions/x86_64-darwin-20/2.6.0/rmagick-2.13.4/gem_make.out

An error occurred while installing rmagick (2.13.4), and Bundler cannot continue.
Make sure that `gem install rmagick -v '2.13.4' --source 'https://rubygems.org/'` succeeds before bundling.

Seems need to reset pkg-config.

brew remove imagemagick
brew uninstall pkg-config
brew install pkg-config
brew unlink pkg-config && brew link pkg-config
brew install imagemagick
gem install rmagick

Thanks to https://github.com/ttscoff/Slogger/issues/344

Mongoid OR queries are seriously dangerous

Upgrading to Mongoid 7.x and found a pretty frightening change in behaviour for OR queries. Any unclear OR declarations could have devastating side-effects.

5.4.1

irb(main):001:0> Train.where(id: 1).or(:expiry.gte => Date.today)
=> #<Mongoid::Criteria
  selector: {"_id"=>1, "$or"=>[{"expiry"=>{"$gte"=>2021-07-07 00:00:00 UTC}}]}
  options:  {:sort=>{"number"=>1}}
  class:    Train
  embedded: false>

irb(main):002:0> Train.or(:expiry.gte => Date.today).where(id: 1)
=> #<Mongoid::Criteria
  selector: {"$or"=>[{"expiry"=>{"$gte"=>2021-07-07 00:00:00 UTC}}], "_id"=>1}
  options:  {:sort=>{"number"=>1}}
  class:    Tenant
  embedded: false>

Previously, or queries would append the selector so where(id: 1) and some OR statement

7.3.0

irb(main):006:0> Train.where(id: 1).or(:expiry.gte => Date.today)
=> #<Mongoid::Criteria
  selector: {"$or"=>[{"_id"=>1}, {"expiry"=>{"$gte"=>2021-07-07 00:00:00 UTC}}]}
  options:  {:sort=>{"number"=>1}}
  class:    Train
  embedded: false>

irb(main):007:0> Train.or(:expiry.gte => Date.today).where(id: 1)
=> #<Mongoid::Criteria
  selector: {"$or"=>[{"expiry"=>{"$gte"=>2021-07-07 00:00:00 UTC}}], "_id"=>1}
  options:  {:sort=>{"number"=>1}}
  class:    Train
  embedded: false>

Now it combines them depending on the order they are declared. I am not entirely sure I like either way of declaring this. Having come from ActiveRecord land I would nearly always assume that you’re AND-ing everything you chain onto a scope.

So in this case we could look for our Tenant by ID, but actually find the first unexpired one instead. That’s one hell of a security hole.

I think the lesson here is to be extra cautious when using ‘OR’ in either ActiveRecord or Mongoid since the outcome is not always as obvious as you might think.

Finally got CSS comments with source location back

An upgrade or two back our Rails 6 app lost line comments. I can’t remember if it was trying to improve speed or the upgrade to sassc. Well it’s fixed now.

# Debug mode disables concatenation and preprocessing of assets.
  # This option may cause significant delays in view rendering with a large
  # number of complex assets.
  # config.assets.debug = true
  # config.sass.line_comments = false
  config.sass.inline_source_maps = true

Thanks https://github.com/sass/sassc-rails

So... I'm writing a book

No really I am. Well two. Over the past year I’ve spent too much time trying to solve the Staff Scheduling Problem and learned a bit about using OR-Tools as a constraint solver. I thought this would be perfect to put my findings into wordsings and publish it. It might just help someone else, how selfless am I? Right?

I could jump in on that but then realistically I’m still learning, but I have spent the last 13 years working with Ruby / Rails and having to fix bugs, and make horrid hacks, and get all of the stuffs working. I realised I could cut my teeth on something I have more experience of and all the sorts of things I post on here, if I remember, and in more detail.

So this year we will have Ruby Expletives (things I learned whilst swearing at Ruby).

Fuck you RVM :(

I really want to play with Ruby 3.0.0, but oooooohhhhhhh noooooooooo. Seems I can’t un tar the rvm archive. I’m just not finding anything :( rbenv it is.

Robs-MacBook-Pro:~ rl$ \curl -sSL https://get.rvm.io | bash -s stable
Downloading https://github.com/rvm/rvm/archive/1.29.10.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc
gpg: Signature made Wed 25 Mar 21:58:42 2020 GMT
gpg:                using RSA key 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
gpg: Good signature from "Piotr Kuczynski <piotr.kuczynski@gmail.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7D2B AF1C F37B 13E2 069D  6956 105B D0E7 3949 9BDB
GPG verified '/usr/local/rvm/archives/rvm-1.29.10.tgz'
.github/: Can't restore time
bin/: Can't restore time
binscripts/: Can't restore time
config/: Can't restore time
config/solaris/: Can't restore time
contrib/: Can't restore time
contrib/hudson/: Can't restore time
docs/: Can't restore time
examples/: Can't restore time
gem-cache/: Can't restore time
gemsets/: Can't restore time
gemsets/jruby/: Can't restore time
gemsets/ruby/: Can't restore time
gemsets/ruby/1.8.5/: Can't restore time
gemsets/ruby/1.8.6/: Can't restore time
gemsets/ruby/1.8.7/: Can't restore time
help/: Can't restore time
help/gemset/: Can't restore time
help/rvmrc/: Can't restore time
hooks/: Can't restore time
lib/: Can't restore time
lib/rvm/: Can't restore time
man/: Can't restore time
man/man1/: Can't restore time
patches/: Can't restore time
patches/goruby/: Can't restore time
patches/jruby/: Can't restore time
patches/jruby/1.7.9/: Can't restore time
patches/libxslt-1.1.26/: Can't restore time
patches/rbx/: Can't restore time
patches/rbx/2.5.2/: Can't restore time
patches/readline-5.2/: Can't restore time
patches/readline-6.2/: Can't restore time
patches/ree/: Can't restore time
patches/ree/1.8.6/: Can't restore time
patches/ree/1.8.7/: Can't restore time
patches/ree/1.8.7/2010.02/: Can't restore time
patches/ruby/: Can't restore time
patches/ruby/1.8.7/: Can't restore time
---8<---
patches/ruby/2.4.1/: Can't restore time
patches/ruby/2.5.0/: Can't restore time
patches/ruby/2.5.1/: Can't restore time
patches/tcs/: Can't restore time
patchsets/: Can't restore time
patchsets/jruby/: Can't restore time
patchsets/jruby/1.7.9/: Can't restore time
patchsets/rbx/: Can't restore time
patchsets/rbx/2.5.2/: Can't restore time
patchsets/ree/: Can't restore time
patchsets/ree/1.8.6/: Can't restore time
patchsets/ree/1.8.7/: Can't restore time
patchsets/ree/1.8.7/2012.02/: Can't restore time
patchsets/ruby/: Can't restore time
patchsets/ruby/1.8.7/: Can't restore time
---8<---
patchsets/ruby/2.4.1/: Can't restore time
patchsets/ruby/2.5.0/: Can't restore time
patchsets/ruby/2.5.1/: Can't restore time
rvm-test-rvm1/: Can't restore time
rvm-test/: Can't restore time
scripts/: Can't restore time
scripts/extras/: Can't restore time
scripts/extras/bash_zsh_support/: Can't restore time
scripts/extras/bash_zsh_support/chpwd/: Can't restore time
scripts/extras/completion.zsh/: Can't restore time
scripts/extras/java_read_properties/: Can't restore time
scripts/functions/: Can't restore time
scripts/functions/detect/: Can't restore time
scripts/functions/detect/system_name/: Can't restore time
scripts/functions/detect/system_version/: Can't restore time
scripts/functions/hooks/: Can't restore time
scripts/functions/manage/: Can't restore time
scripts/functions/manage/install/: Can't restore time
scripts/functions/notes/: Can't restore time
scripts/functions/notes/warnings/: Can't restore time
scripts/functions/requirements/: Can't restore time
scripts/zsh/: Can't restore time
scripts/zsh/Completion/: Can't restore time
tar: Error exit delayed from previous errors.
Could not extract RVM sources.
Downloading https://bitbucket.org/mpapis/rvm/get/1.29.10.tar.gz
Downloading https://bitbucket.org/mpapis/rvm/downloads/1.29.10.tar.gz.asc
gpg: Signature made Wed 25 Mar 21:58:45 2020 GMT
gpg:                using RSA key 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
gpg: Good signature from "Piotr Kuczynski <piotr.kuczynski@gmail.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7D2B AF1C F37B 13E2 069D  6956 105B D0E7 3949 9BDB
GPG verified '/usr/local/rvm/archives/rvm-1.29.10.tgz'
.github/: Can't restore time
bin/: Can't restore time
binscripts/: Can't restore time
config/: Can't restore time
config/solaris/: Can't restore time
contrib/: Can't restore time
contrib/hudson/: Can't restore time
docs/: Can't restore time
examples/: Can't restore time
gem-cache/: Can't restore time
gemsets/: Can't restore time
gemsets/jruby/: Can't restore time
gemsets/ruby/: Can't restore time
gemsets/ruby/1.8.7/: Can't restore time
help/: Can't restore time
help/gemset/: Can't restore time
help/rvmrc/: Can't restore time
hooks/: Can't restore time
lib/: Can't restore time
lib/rvm/: Can't restore time
man/: Can't restore time
man/man1/: Can't restore time
patches/: Can't restore time
patches/goruby/: Can't restore time
patches/jruby/: Can't restore time
patches/jruby/1.7.9/: Can't restore time
patches/libxslt-1.1.26/: Can't restore time
patches/rbx/: Can't restore time
patches/rbx/2.5.2/: Can't restore time
patches/readline-5.2/: Can't restore time
patches/readline-6.2/: Can't restore time
patches/ree/: Can't restore time
patches/ree/1.8.6/: Can't restore time
patches/ree/1.8.7/: Can't restore time
patches/ree/1.8.7/2010.02/: Can't restore time
patches/ruby/: Can't restore time
patches/ruby/1.8.7/: Can't restore time
---8<---
patches/ruby/2.4.1/: Can't restore time
patches/ruby/2.5.0/: Can't restore time
patches/ruby/2.5.1/: Can't restore time
patches/tcs/: Can't restore time
patchsets/: Can't restore time
patchsets/jruby/: Can't restore time
patchsets/jruby/1.7.9/: Can't restore time
patchsets/rbx/: Can't restore time
patchsets/rbx/2.5.2/: Can't restore time
patchsets/ree/: Can't restore time
patchsets/ree/1.8.6/: Can't restore time
patchsets/ree/1.8.7/: Can't restore time
patchsets/ree/1.8.7/2012.02/: Can't restore time
patchsets/ruby/: Can't restore time
patchsets/ruby/1.8.7/: Can't restore time
---8<---
patchsets/ruby/2.4.0/: Can't restore time
patchsets/ruby/2.4.1/: Can't restore time
patchsets/ruby/2.5.0/: Can't restore time
patchsets/ruby/2.5.1/: Can't restore time
patchsets/ruby/head/: Can't restore time
rvm-test-rvm1/: Can't restore time
rvm-test/: Can't restore time
scripts/: Can't restore time
scripts/extras/: Can't restore time
scripts/extras/bash_zsh_support/: Can't restore time
scripts/extras/bash_zsh_support/chpwd/: Can't restore time
scripts/extras/completion.zsh/: Can't restore time
scripts/extras/java_read_properties/: Can't restore time
scripts/functions/: Can't restore time
scripts/functions/detect/: Can't restore time
scripts/functions/detect/system_name/: Can't restore time
scripts/functions/detect/system_version/: Can't restore time
scripts/functions/hooks/: Can't restore time
scripts/functions/manage/: Can't restore time
scripts/functions/manage/install/: Can't restore time
scripts/functions/notes/: Can't restore time
scripts/functions/notes/warnings/: Can't restore time
scripts/functions/requirements/: Can't restore time
scripts/zsh/: Can't restore time
scripts/zsh/Completion/: Can't restore time
tar: Error exit delayed from previous errors.
Could not extract RVM sources.

Docker yet again

I still haven’t got to grips with Docker, I need to sort out our CI environment as it’s a bit weird and stuff is missing so….let’s give Docker a whirl. I’ve already got it installed so.

Robs-MacBook-Pro:rostering rl$ docker version
Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:41:33 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:49:27 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Run a Ruby command without that version of Ruby installed.

Robs-MacBook-Pro:rostering rl$ docker run ruby:2.7 ruby -e "puts :hello"
Unable to find image 'ruby:2.7' locally
2.7: Pulling from library/ruby
d6ff36c9ec48: Pull complete 
c958d65b3090: Pull complete 
edaf0a6b092f: Pull complete 
80931cf68816: Pull complete 
bc1b8aca3825: Pull complete 
e64edafe3f35: Pull complete 
5f7e31a225bc: Pull complete 
a7a2582e3a28: Pull complete 
Digest: sha256:4cdd89e947cc52c78c273953e42a6e9d94ca4cd182875fd50745550580b772f4
Status: Downloaded newer image for ruby:2.7
hello

And now after

Robs-MacBook-Pro:rostering rl$ docker run ruby:2.7 ruby -e "puts :hello"
hello

Ok that’s pretty cool :)

Now if we want to mount an existing directory inside our Docker instance, and boot it with interactive mode

Robs-MacBook-Pro:rostering rl$ docker run -it -v ${PWD}:/apps/rostering  ruby:2.6.6 bash
Unable to find image 'ruby:2.6.6' locally
2.6.6: Pulling from library/ruby
d6ff36c9ec48: Already exists 
root@520c19cb27a6:/# cd apps/rostering/
root@520c19cb27a6:/apps/rostering# bundle install
Traceback (most recent call last):
	2: from /usr/local/bin/bundle:23:in `<main>'
	1: from /usr/local/lib/ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'
/usr/local/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': Could not find 'bundler' (2.1.4) required by your /apps/rostering/Gemfile.lock. (Gem::GemNotFoundException)
To update to the latest version installed on your system, run `bundle update --bundler`.
To install the missing version, run `gem install bundler:2.1.4`
root@520c19cb27a6:/apps/rostering# gem install bundler:2.1.4
Fetching bundler-2.1.4.gem
Successfully installed bundler-2.1.4
1 gem installed

It’s effectively a new version of Ruby with no dependencies, so we need to install bundler, etc.

root@520c19cb27a6:/apps/rostering# gem install bundler:2.1.4
Fetching bundler-2.1.4.gem
Successfully installed bundler-2.1.4
1 gem installed
root@520c19cb27a6:/apps/rostering# bundle install
Fetching gem metadata from https://rubygems.org/.........
Fetching https://github.com/shorelabs/js-exception-notifier.git
Fetching rake 13.0.1
Installing rake 13.0.1
Fetching concurrent-ruby 1.1.6
Installing concurrent-ruby 1.1.6
Fetching aasm 5.1.0
Installing aasm 5.1.0
Fetching i18n 1.8.5
Installing i18n 1.8.5
Fetching minitest 5.14.1
.....