“...I've been working since 2008 with Ruby / Ruby on Rails, love a bit of Elixir / Phoenix and learning Rust. I also poke through other people's code and make PRs for OpenSource Ruby projects that sometimes make it. Currently working for InPay who are based in Denmark...”

Rob Lacey
Senior Software Engineer, Copenhagen, Denmark

Redirecting STDERR

So you don’t like STDERR vomming all over your console?

This script almost works but doesn’t redirect anything that is already hold the original file descriptor, e.g. C extensions.

#!/usr/bin/env ruby

STDERR.puts "Something"
$stderr = File.open(File::NULL, 'w')
STDERR.puts "...and something else"

This however reopens the same file descriptor and covers all bases.

#!/usr/bin/env ruby

STDERR.puts "Something"
$stderr.reopen('/dev/null', 'w')
$stderr.sync = true
STDERR.puts "...and nothing"

And we get…

./test.rb 
Something

yes blocks and metaprogramming are very clever...

…but if you can’t read the code without bashing your head against a wall, its pretty useless.

Old blog posts I never published, except I have now #2

Now I admit that this solution has some merit, it filters incoming params in a particular controller action and DRYs up some of the saving and redirecting that is common place in every controller. But when I took this project on it just complicated things so much that it took 4 times as long to make any changes to the app as it needed to.

do_object_edit("recruiter/new_subscription", :pay_subscription, :agreed_terms, :subscription_type_id) do |o|
  o.transaction_detail = "INCOMPLETE"
  o.amount_paid = 0
end
def do_object_edit(template, action, *fields, &block)
  logger.debug "in do_object_edit"
  logger.debug "allowing edit of #{fields.join(', ')}" unless fields.empty?
  if submitted_using_button?("Cancel")
    if action.is_a?(Symbol)
      redirect_to :action => action
    else
      redirect_to action
    end
    return
  elsif request.post?
    fields.map!{|f| f.to_sym}
    params[:object].delete_if{|k,v| !fields.include?(k.to_sym)} unless fields.empty?
    @object.attributes = params[:object]
    begin
      ActiveRecord::Base.transaction do
        if block_given?
          yield @object
        end
        @object.save!
        flash_message "Details Saved"
        if action.is_a?(Symbol)
          redirect_to :action => action
        else
          redirect_to action
        end
        return
      end
    rescue ActiveRecord::RecordInvalid
      logger.warn $!
      flash_message "An Error Occurred"
    end
  end
  render :template => template
end

Yes is DRY because this is repeated everywhere in some form or other, but its not as readable as this.

s = Subscription.new(params[:subscription]) do |s|
  s.transaction_detail = 'INCOMPLETE'
  s.amount_paid = 0
end

unless s.save
  redirect_to failure
else
  redirect_to success
end

So as another developer taking on a dead project, this just made things harder. So my comment really is, coding an application isn’t just for you its for the client and if you can’t read it easily a month after you’ve written it someone else won’t be able to either.

It stinks of coding arrogance over creating a maintainable project. And the client loses…because it takes 5 times longer to get someone to fix it or make changes in the future.

Ruby &&

Old blog posts I never published, except I have now #1

I really like that the && operator in Ruby returns the final argument, or the result of the last evaluation

rl@bloodandguts:~/repos$ irb
>> "take" && "give"
=> "give"
>> "take" && "give" || "take" && "blum"
=> "give"
>> "take" && false || "take" && "blum"
=> "blum"
> "take" && Numeric.new || "take" && "blum"
=> #<Numeric:0x7f2841bffa70>

Shrinking PDFs

Shrinking PDFs so that they aren’t 50Mb for 10 pages is always a good thing to remember. This command line tool form Adobe is particularly helpful

ps2pdf -dPDFSETTINGS=/ebook ~/Desktop/AnimalHealthCertificate.pdf ~/Desktop/AnimalHealthCertificate-compressed.pdf

Now this Animal Health Certificate will actually get through and not bounce.

rl@loathsome ~ % ls -lah ~/Desktop/AnimalHealthCertificate*
-rw-r--r--@ 1 rl  staff   3.1M 21 Sep 11:01 /Users/rl/Desktop/AnimalHealthCertificate-compressed.pdf
-rw-r--r--@ 1 rl  staff    39M 21 Sep 10:59 /Users/rl/Desktop/AnimalHealthCertificate.pdf

What process is running on port 3000?

Can’t start your Rails app because it’s already running but you can’t find it?

rola@INPAY-RNQP709MTC backend_v2 % sudo lsof -i :3000
COMMAND     PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
Microsoft 14734 rola   28u  IPv6 0xb9dec1e002660162      0t0  TCP localhost:63852->localhost:hbci (ESTABLISHED)
ruby      42857 rola   14u  IPv4 0x2e7917eea3a29d7b      0t0  TCP localhost:hbci (LISTEN)
ruby      42857 rola   16u  IPv6 0xfe39e6fb86a27f74      0t0  TCP localhost:hbci (LISTEN)
ruby      42857 rola   22u  IPv6 0x5c9b52f85ccf9a54      0t0  TCP localhost:hbci->localhost:63589 (CLOSED)
ruby      42857 rola   23u  IPv6 0xb5673178bc310d9b      0t0  TCP localhost:hbci->localhost:63590 (CLOSED)

Assignment still proceeds even with undefined local variable or method

This week I had a really horrible bug. One spec in our suite in our CI knocked out every subsequent spec after skipping. The spec works in isolation. just not on CI and not in a full suite run.

RSpec.shared_contexts 'with some feature' do
  skip('This is not available on CI') if ENV['CI']
  original_driver = Capybara.javascript_driver

  Capybara.javascript_driver = :something_else
ensure
  Capybara.javascript_driver  = original_driver
end

RSpec.describe 'something' do
  include_context 'with some feature'
  it do
    visit '/'
  end
end

Turns out our Capybara.javascript_driver ended up being nil, and our test suite fell over because CI doesn’t support non-headless Chrome. And the reason…

I would appear that variable assignment in Ruby still assigns nil if we attempt to assign a variable from another variable or method that is undefined.

irb(main):001> a = some_method_or_var_that_is_not_defined
(irb):1:in `<main>': undefined local variable or method `some_method_or_var_that_is_not_defined' for main (NameError)

a = some_method_or_var_that_is_not_defined
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	from <internal:kernel>:187:in `loop'
	from /Users/rl/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/irb-1.14.0/exe/irb:9:in `<top (required)>'
	from /Users/rl/.asdf/installs/ruby/3.3.0/bin/irb:25:in `load'
	from /Users/rl/.asdf/installs/ruby/3.3.0/bin/irb:25:in `<main>'
irb(main):002> a
=> nil

Who knew? I didn’t.

Ruby 3.3.0 and Resque

Just deployed a new Ruby 3.3.0 branch to staging and it would appear that our Resque jobs just stopped working. Usual procedure check the logs, look for an error and backtrace. Not logs. Ok there is no log/resque.log. There is nothing in the staging.log. Ok. So the process that kicks off our service is monit. Monit log.

/var/log/monit.log

[UTC Feb 21 09:12:46] error    : 'resque-scheduler' process is not running
[UTC Feb 21 09:12:46] info     : 'resque-scheduler' trying to restart
[UTC Feb 21 09:12:46] info     : 'resque-scheduler' start: '/etc/init.d/resque-scheduler start'
[UTC Feb 21 09:13:16] error    : 'resque-scheduler' failed to start (exit status 0) -- no output
[UTC Feb 21 09:15:16] error    : 'resque' process is not running
[UTC Feb 21 09:15:16] info     : 'resque' trying to restart
[UTC Feb 21 09:15:16] info     : 'resque' start: '/etc/init.d/resque start'
[UTC Feb 21 09:15:46] error    : 'resque' failed to start (exit status 0) -- '/etc/init.d/resque start': /etc/init.d/resque: 12: kill: No such process

Brilliant, it states it has exit status 0 and nothing else. OK. Script tells us that it should boot the process, nothing out of the ordinary here. Nothing to indicate that it is locked to an older Ruby version or anything.

/etc/init.d/resque

PID=/home/deploy/app/shared/pids/resque.pid
CMD="cd /home/deploy/app/current; PIDFILE=$PID BACKGROUND=yes bundle exec rake environment resque:work QUEUE=* RAILS_ENV=production"
echo $CMD

It is backgrounded though, if there was an error it wouldn’t end up in STDOUT. Ok, how about we boot without the background option.

$ bundle exec rake environment resque:work QUEUE=* RAILS_ENV=staging
rake aborted!
NoMethodError: undefined method `[]' for nil
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/resque-2.0.0/lib/resque/logging.rb:8:in `log'
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/resque-2.0.0/lib/resque/logging.rb:13:in `info'
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/resque-2.0.0/lib/resque/worker.rb:854:in `log'
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/resque-2.0.0/lib/resque/tasks.rb:19:in `block (2 levels) in <main>'
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/bundler-2.5.5/lib/bundler/cli/exec.rb:58:in `load'
/home/deploy/app/shared/bundle/ruby/3.3.0/gems/bundler-2.5.5/lib/bundler/cli/exec.rb:58:in `kernel_load'

Ah, there is is. So the Resque::Logging.log proxies onto the Resque.logger if it is defined.

module Resque
  # Include this module in classes you wish to have logging facilities
  module Logging
    module_function

    # Thunk to the logger's own log method (if configured)
    def self.log(severity, message)
      Resque.logger.__send__(severity, message) if Resque.logger
    end

    # Log level aliases
    def debug(message); Logging.log :debug, message; end
    def info(message);  Logging.log :info,  message; end
    def warn(message);  Logging.log :warn,  message; end
    def error(message); Logging.log :error, message; end
    def fatal(message); Logging.log :fatal, message; end
  end
end

Let’s try it.

% rails c
Loading development environment (Rails 6.1.7)
rirb: warn: can't alias measure from irb_measure.
irb(main):001> Resque.logger.debug('blah')
/Users/rl/.asdf/installs/ruby/3.3.0/lib/ruby/3.3.0/logger.rb:384:in `level': undefined method `[]' for nil (NoMethodError)

    @level_override[Fiber.current] || @level

Ah, that’s a problem. The Resque.logger we’re using seem incompatable with Logger.

And we’re not the only ones to see this. https://talk.jekyllrb.com/t/error-when-executing-bundle-install/8822/6

It would appear that Logger has changed, there is a fix in Jekyll 4.3.3 for their issue. So what’s our issue.

https://github.com/jekyll/jekyll/commit/595cc230678952fab244a62dc1811ea39ec34041

Resque by default defines the Resque.logger as an instance of MonoLogger. Never heard of it.

lib/resque.rb

# Log to STDOUT by default
Resque.logger           = MonoLogger.new(STDOUT)
Resque.logger.formatter = Resque::QuietFormatter.new

Our Gemfile.lock reckons we’re using 1.1.1 of mono_logger

Gemfile.lock

mongoid_paranoia (0.5.0)
       mongoid (~> 7.3)
     mono_logger (1.1.1)
     msgpack (1.7.2)
     multi_json (1.15.0)
     multi_xml (0.6.0)

There is a new patch version of mono_logger available. And there it is in the commit history.

https://github.com/steveklabnik/mono_logger/commit/14bfb8302609123d702ace527c9114a8ff7c438a

They have also fixed it. Logger initializer has changed, so if you’re going to use it sub-class Logger use their initializer. Upgrading to bundle update mono_logger bumps the version to 1.1.2 and fixed our issue.

Bundler issue on Ruby 3.3.0 Upgrade

Honestly, Ruby 3.2.2 to Ruby 3.3.0 upgrades have been really painless. I’ve hit every server and installed Ruby 3.3.0 with YJIT enabled and we’re ready to go as soon as the new Ruby_3.3.0 branches specs pass.

However, this one is rather annoying.

% bundle install
Fetching gem metadata from https://rubygems.org/........
Resolving dependencies....................
--- ERROR REPORT TEMPLATE -------------------------------------------------------

NoMethodError: undefined method `version' for an instance of Bundler::IncompleteSpecification

This current Gemfile.lock indicates we last bundled this with Bunder 2.3.20. I know the latest version is 2.5.5

RUBY VERSION
   ruby 3.2.2p53

BUNDLED WITH
   2.3.20

Ok in the Gemfile we have locked to a specific version of bundler, I’m not sure why. I’d think that bundler is pretty safe to allow the host system to have whatever version is installed manage this.

gem 'bundler', '~> 2.3.8'

If I remove this line from the Gemfile, nothing changes. I still get Unfortunately, an unexpected error occurred, and Bundler cannot continue. It’s expecting that version of bundler to manage it still.

Bundler       2.3.20
  Platforms   ruby, arm64-darwin-23
Ruby          3.3.0p0 (2023-12-25 revision 5124f9ac7513eb590c37717337c430cb93caa151) [arm64-darwin-23]
  Full Path   /Users/rl/.asdf/installs/ruby/3.3.0/bin/ruby
  Config Dir  /Users/rl/.asdf/installs/ruby/3.3.0/etc
RubyGems      3.5.3
  Gem Home    /Users/rl/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0
  Gem Path    /Users/rl/.gem/ruby/3.3.0:/Users/rl/.asdf/installs/ruby/3.3.0/lib/ruby/gems/3.3.0
  User Home   /Users/rl
  User Path   /Users/rl/.gem/ruby/3.3.0
  Bin Dir     /Users/rl/.asdf/installs/ruby/3.3.0/bin
OpenSSL       
  Compiled    OpenSSL 3.2.0 23 Nov 2023
  Loaded      OpenSSL 3.2.0 23 Nov 2023
  Cert File   /opt/homebrew/etc/openssl@3/cert.pem
  Cert Dir    /opt/homebrew/etc/openssl@3/certs
Tools         
  Git         2.39.3 (Apple Git-145)
  RVM         not installed
  rbenv       not installed
  chruby      not installed

Even if 2.5.5 is the default.

% gem list bundler

*** LOCAL GEMS ***

bundler (2.5.5, default: 2.5.3, 2.4.21, 2.4.0, 2.3.20)
bundler-audit (0.9.1)
capistrano-bundler (2.1.0, 2.0.1)

So I have to force bundler to use the correct version for this bundle install to work. And success.

bundle _2.5.5_ install
---8<---
Bundle complete! 115 Gemfile dependencies, 321 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

Petty Annoyances #12309123

How hard is it to keep my Rails blog up to date? Honestly, I shouldn’t be worrying about such things because I created this site so many years ago and it was running Rails 4.x up until about a year ago. I them upgraded it to 7.x and everything broke. So now I’m taking a little time to spring clean and today…

rl@loathsome robl.me % bundle outdated
Fetching https://github.com/braindeaf/capistrano-puma
Fetching git@github.com:braindeaf/m3ta.git
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...

Gem          Current  Latest  Requested  Groups
http-accept  1.7.0    2.2.1
rubocop      1.59.0   1.60.2

So what do we have:

  • https://github.com/braindeaf/capistrano-puma - Capistrano Puma is for deploying my application but the latest version I need to support Puma 6.x removes some nice monit support and it deprecates nginx configs. All of which I use. This is just a fork to allow Puma 6.x to be installed rather than holding the dependency back. Perhaps I could added the nginx and monit support back into my own gem. Also I will need some way of maintaining anycable systemd configs too so yes one day.

  • git@github.com:braindeaf/m3ta.git - I built a pretty functional meta tag management tool for my purposes. I haven’t published the latest update. I should. I should remember how the hell I did it last time. I will do that.

Then some other dependencies…

  • http-accept (1.7.0) is held back by rest-client (2.1.0) which is held back by ogpr (1.1.0) - This is just for grabbing Open Graph data from a url, I could do a pull request for OGPR to bring rest-client upto date. I’ll do that. https://github.com/hirakiuc/ogpr/pull/15

  • rubocop (~> 1.59.0) is being held back by standard (1.33.0) simples. I use Rubocop in some client projects and Juniper used standardrb which has it’s own standardised defaults which is fine. I can handle this. I’ll wait and upgrade if and when.

GIT
  remote: git@github.com:braindeaf/m3ta.git
  revision: 8f6b6f98941af3aee8fce4077f75a3896f3aef6a
  branch: rl/refactor_into_view_component
  specs:
    m3ta (0.1.1)
      hashie (= 5.0.0)
      rails (~> 7.0, >= 7.0.4.1)

GIT
  remote: https://github.com/braindeaf/capistrano-puma
  revision: b4cc103868fb680841d523011598a270448f8a1b
  branch: rl/support-puma6-in-5.2.0
  specs:
    capistrano3-puma (5.2.0)
      capistrano (~> 3.7)
      capistrano-bundler
      puma (>= 4.0, < 7.0)

GEM
  remote: https://rubygems.org/
  specs:

---8<---

    http-accept (1.7.0)
    http-cookie (1.0.5)
      domain_name (~> 0.5)

---8<---

    ogpr (1.1.0)
      nokogiri (~> 1.8)
      rest-client (~> 2.1.0)

---8<---

    rest-client (2.1.0)
      http-accept (>= 1.7.0, < 2.0)
      http-cookie (>= 1.0.2, < 2.0)
      mime-types (>= 1.16, < 4.0)
      netrc (~> 0.8)

---8<---

    rubocop (1.59.0)
      json (~> 2.3)
      language_server-protocol (>= 3.17.0)
      parallel (~> 1.10)
      parser (>= 3.2.2.4)
      rainbow (>= 2.2.2, < 4.0)
      regexp_parser (>= 1.8, < 3.0)
      rexml (>= 3.2.5, < 4.0)
      rubocop-ast (>= 1.30.0, < 2.0)
      ruby-progressbar (~> 1.7)
      unicode-display_width (>= 2.4.0, < 3.0)

---8<---

    standard (1.33.0)
      language_server-protocol (~> 3.17.0.2)
      lint_roller (~> 1.0)
      rubocop (~> 1.59.0)
      standard-custom (~> 1.0.0)
      standard-performance (~> 1.3)
    standard-custom (1.0.2)
      lint_roller (~> 1.0)
      rubocop (~> 1.50)

---8<---

DEPENDENCIES
  
---8<---

  ogpr

---8<---

standardrb

---8<---

RUBY VERSION
   ruby 3.3.0p0

BUNDLED WITH
   2.5.5

Being out of date isn’t the end of the world, but it’s nice to think you’re on the cutting edge.

I don't even dev-ops: Part 7- MongoDB Not Starting

Since December I’ve some dev-ops which is not my usual gig. Today, just running round upgrading and rebooting staging servers. Alas MongoDB didn’t come back up on reboot. My immediate thought was that it wasn’t configured to start on start up so let’s get that sorted and bring it up.

sudo systemctl enable mongod.service
sudo service mongod start

Gah…

$ mongo
MongoDB shell version v5.0.24
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect@src/mongo/shell/mongo.js:372:17
@(connect):2:6
exception: connect failed
exiting with code 1

Ok, is the database corrupt or does Ubuntu think it’s already running and it isn’t or something?

$ sudo systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Tue 2024-02-06 08:36:19 UTC; 43s ago
       Docs: https://docs.mongodb.org/manual
    Process: 2483 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=2)
   Main PID: 2483 (code=exited, status=2)

Feb 06 08:36:19 178-79-140-122 systemd[1]: Started MongoDB Database Server.
Feb 06 08:36:19 178-79-140-122 mongod[2483]: Error opening config file: Permission denied
Feb 06 08:36:19 178-79-140-122 mongod[2483]: try '/usr/bin/mongod --help' for more information
Feb 06 08:36:19 178-79-140-122 systemd[1]: mongod.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Feb 06 08:36:19 178-79-140-122 systemd[1]: mongod.service: Failed with result 'exit-code'.

Ok, the /etc/mongod.conf is not readable, it’s seemingly owned by root but something in a recent upgrade means that presumably the config is read by the mongodb user rather than root. I assume.

sudo chown mongodb:mongodb /etc/mongod.conf
ls -la /etc/mongod.conf 
-rw-r-x--- 1 mongodb mongodb 615 Dec 19  2013 /etc/mongod.conf

Cool. Let’s restart the service.

sudo service mongod start

And check the status

sudo systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2024-02-06 08:38:03 UTC; 4s ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 2792 (mongod)
     Memory: 182.8M
     CGroup: /system.slice/mongod.service
             └─2792 /usr/bin/mongod --config /etc/mongod.conf

Feb 06 08:38:03 178-79-140-122 systemd[1]: Started MongoDB Database Server.
Feb 06 08:38:03 178-79-140-122 mongod[2792]: {"t":{"$date":"2024-02-06T08:38:03.246Z"},"s":"I",  "c":"CONTROL",  "id":7484500, "ctx":"-","msg":"Environment variable MONG>

Cool. Beans and Sausages.