“...I've been working since 2008 with Ruby / Ruby on Rails, love a bit of Elixir / Phoenix and learning Rust. I also poke through other people's code and make PRs for OpenSource Ruby projects that sometimes make it. Currently working for InPay...”

Rob Lacey (contact@robl.me)
Senior Software Engineer, Brighton, UK

When Postgresql doesnt start on OSX

New-MacBook-Pro:mini-epic cex$ ./bin/server 
=> Booting Puma
=> Rails 4.2.1 application starting in development on http://0.0.0.0:8001
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
Exiting
/Users/cex/.rvm/gems/ruby-2.2.2@epic-invite/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `initialize': could not connect to server: No such file or directory (PG::ConnectionBad)
	Is the server running locally and accepting
	connections on Unix domain socket "/tmp/.s.PGSQL.5432"?

I tried restarting with brew

New-MacBook-Pro:mini-epic cex$ brew services restart postgresql
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)

Same deal. Thanks to https://stackoverflow.com/questions/13410686/postgres-could-not-connect-to-server Remove the PID and you’re sorted.

-rw-------  1 cex  admin  88  2 Mar 15:02 /usr/local/var/postgres/postmaster.pid
New-MacBook-Pro:mini-epic cex$ rm /usr/local/var/postgres/postmaster.pid
New-MacBook-Pro:mini-epic cex$ brew services start postgresql
Service `postgresql` already started, use `brew services restart postgresql` to restart.
New-MacBook-Pro:mini-epic cex$ brew services restart postgresql
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)

Sometimes....

Sometimes it gets bloody…

AASM kick in the balls bug / typo / misunderstanding

Pretty average AASM block right? Wrong….

I was just dealing with a bug where object.mark_sent! wasn’t saving the record.

included do
  include AASM
  aasm(:state) do
    state :in_checkout, initial: true
    state :processing
    state :available
    state :completed
    state :inactive
    state :sent

    event :start_processing! do
      transitions from: [:in_checkout, :processing, :available], to: :processing
    end

    event :processed! do
      transitions from: [:in_checkout, :processing], to: :available
    end

    event :mark_sent! do
      transitions from: [:available, :completed], to: :sent
    end

    after_all_transitions :set_aasm_timestamp
  end
end

There’s a mark_sent! event, so surely mark_sent! should change the state and save the record right? No.

event :mark_sent! do
  transitions from: [:available, :completed], to: :sent
end

This actually creates two method mark_sent! which makes the state change and it’s counterpart that also saves the record mark_sent!!. You can’t call a method with a double exclamation directly.

[26] pry(main)> e.mark_sent!!
[26] pry(main)* 
[26] pry(main)* e.send(:'mark_sent!!')

When in Vegas, don’t define mark your events with an exclamation. DOH.

How to vendor a gem

I’ve got a really annoying DEPRECATION WARNING. It appears the wck gem is spitting out errors and it’s no longer being updated. All I want to do is calm down the errors. The styles are all still helpful.

DEPRECATION WARNING: Extra .css in SCSS file is unnecessary. Rename /Users/cex/.rvm/gems/ruby-2.2.2@epic-invite/gems/wck-0.0.5/app/assets/stylesheets/wck.css.scss to /Users/cex/.rvm/gems/ruby-2.2.2@epic-invite/gems/wck-0.0.5/app/assets/stylesheets/wck.scss. (called from _app_views_home_index_html_haml___3330030305515264653_70130897119780 at /Users/cex/repos/epic-invite/app/views/home/index.html.haml:9)

Best course of action is to vendor the gem. So All I need to do is unpack it into my application and fix the issue.

New-MacBook-Pro:epic-invite cex$ gem unpack wck --target=vendor/gems/
Unpacked gem: '/Users/cex/repos/epic-invite/vendor/gems/wck-0.0.5'

Gemfile

...
gem 'high_voltage', '~> 3.0.0'
gem 'wck', path: 'vendor/gems/wck-0.0.5'
gem 'cancancan'
...

And bundle install to install from that path.

New-MacBook-Pro:epic-invite cex$ bundle install
...
Using valid_email 0.0.11
Using wck 0.0.5 from source at `vendor/gems/wck-0.0.5`
Using whenever 0.9.4
....

The solution is to just rename the file, and satisfying to avoid seeing so many errors.

mv wck.css.scss wck.scss

Death's Head Vs.... Han Solo

I genuinely expected Solo to be a load of old bollocks, it wasn’t as good as Rogue One but it was still better than the prequel trilogy that we don’t talk about.

Sorry that Death’s Head has a miserable head on that day.

Create MySQL users has changed

After all this time, the way of creating users and granting privileges on user in one statement just doesn’t work anymore. I guess I just missed the memo

New-MacBook-Pro:robl cex$ mysql -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.0.15 Homebrew

Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> GRANT ALL ON robl.* to robl@localhost IDENTIFIED BY 'robl';
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL

Ok, so we need to create the user and then do the grant

mysql> CREATE USER robl IDENTIFIED BY 'robl';
Query OK, 0 rows affected (0.01 sec)

mysql> GRANT ALL ON robl.* TO robl@localhost;
ERROR 1410 (42000): You are not allowed to create a user with GRANT
mysql> CREATE USER robl@localhost;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE USER robl@localhost iDENTIFIED BY 'robl';
ERROR 1396 (HY000): Operation CREATE USER failed for 'robl'@'localhost'
mysql> DROP USER robl;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE USER robl@localhost IDENTIFIED BY 'robl';
ERROR 1396 (HY000): Operation CREATE USER failed for 'robl'@'localhost'
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE USER robl@localhost IDENTIFIED BY 'robl';
ERROR 1396 (HY000): Operation CREATE USER failed for 'robl'@'localhost'
mysql> DROP USER robl;
ERROR 1396 (HY000): Operation DROP USER failed for 'robl'@'%'

Ok, let’s just start again.

mysql> DROP USER robl@localhost;
Query OK, 0 rows affected (0.01 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE USER robl@localhost IDENTIFIED BY 'robl';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL ON robl.* TO robl@localhost;
Query OK, 0 rows affected (0.00 sec)

mysql>

Ok, now I can get this started.

Ruby fuzzy matching

I am running through a data set which is lots of answers and trying to make the data consistent as human input is always going to be wonky from typos, spaces instead of hiphens to complete misspelling. Found this…

https://github.com/seamusabshere/fuzzy_match

2.3.3 :002 > require 'fuzzy_match'
=> true
2.3.3 :003 > fm = FuzzyMatch.new(['Uwe Rosenberg', 'X-Com'])
 =>#<FuzzyMatch:0x007ff02b05f370 @read=nil, @groupings=[], @identities=[], @stop_words=[], @default_options={:must_match_grouping=>false, :must_match_at_least_one_word=>false, :gather_last_result=>false, :find_all=>false, :find_all_with_score=>false, :threshold=>nil, :find_best=>false, :find_with_score=>false}, @haystack=[w("Uwe Rosenberg"), w("X-Com")]>
2.3.3 :004 > fm.find('Use Roenberb')
=> "Uwe Rosenberg"

Perfect. Time to de-dupe some dodgy data.

Postgres isn't starting again

It’s just silently falling over and restarting when trying to start with brew services start postgresql

New-MacBook-Pro:local cex$ sudo -s tail -f /var/log/system.log
Password:
Feb 19 09:52:41 New-MacBook-Pro com.apple.xpc.launchd[1] (homebrew.mxcl.postgresql[27587]): Service exited with abnormal code: 1
Feb 19 09:52:41 New-MacBook-Pro com.apple.xpc.launchd[1] (homebrew.mxcl.postgresql): Service only ran for 0 seconds. Pushing respawn out by 10 seconds.
Feb 19 09:52:51 New-MacBook-Pro com.apple.xpc.launchd[1] (homebrew.mxcl.postgresql[27616]): Service exited with abnormal code: 1

Starting manually and checking the server.log

New-MacBook-Pro:local cex$ pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
New-MacBook-Pro:local cex$ cat /usr/local/var/
cache/    db/       homebrew/ lib/      log/      mongodb/  mysql/    postgres/ run/      
New-MacBook-Pro:local cex$ cat /usr/local/var/postgres/server.log 
2019-02-19 10:00:29.473 GMT [30912] FATAL:  database files are incompatible with server
2019-02-19 10:00:29.473 GMT [30912] DETAIL:  The data directory was initialized by PostgreSQL version 10, which is not compatible with this version 11.1.

Ok, so data dir is incompatible. Damn. Hopefully I can just upgrade the data files with brew postgresql-upgrade-database

New-MacBook-Pro:local cex$ brew postgresql-upgrade-database
==> brew install postgresql@10
==> Downloading https://homebrew.bintray.com/bottles/postgresql@10-10.6_1.mojave.bottle.1.tar.gz
######################################################################## 100.0%
==> Pouring postgresql@10-10.6_1.mojave.bottle.1.tar.gz
==> /usr/local/Cellar/postgresql@10/10.6_1/bin/initdb /usr/local/var/postgresql@10
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
  brew postgresql-upgrade-database

postgresql@10 is keg-only, which means it was not symlinked into /usr/local,
because this is an alternate version of another formula.

If you need to have postgresql@10 first in your PATH run:
  echo 'export PATH="/usr/local/opt/postgresql@10/bin:$PATH"' >> ~/.bash_profile

For compilers to find postgresql@10 you may need to set:
  export LDFLAGS="-L/usr/local/opt/postgresql@10/lib"
  export CPPFLAGS="-I/usr/local/opt/postgresql@10/include"

For pkg-config to find postgresql@10 you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/postgresql@10/lib/pkgconfig"


To have launchd start postgresql@10 now and restart at login:
  brew services start postgresql@10
Or, if you don't want/need a background service you can just run:
  pg_ctl -D /usr/local/var/postgresql@10 start
==> Summary
   /usr/local/Cellar/postgresql@10/10.6_1: 1,706 files, 20.8MB
==> Upgrading postgresql data from 10 to 11...
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
waiting for server to start....2019-02-19 10:05:34.199 GMT [32604] LOG:  listening on IPv6 address "::1", port 5432
2019-02-19 10:05:34.199 GMT [32604] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2019-02-19 10:05:34.200 GMT [32604] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2019-02-19 10:05:34.219 GMT [32605] LOG:  database system was shut down at 2019-02-07 21:30:24 GMT
2019-02-19 10:05:34.223 GMT [32604] LOG:  database system is ready to accept connections
 done
server started
waiting for server to shut down....2019-02-19 10:05:34.362 GMT [32604] LOG:  received fast shutdown request
2019-02-19 10:05:34.362 GMT [32604] LOG:  aborting any active transactions
2019-02-19 10:05:34.363 GMT [32604] LOG:  worker process: logical replication launcher (PID 32611) exited with exit code 1
2019-02-19 10:05:34.363 GMT [32606] LOG:  shutting down
2019-02-19 10:05:34.369 GMT [32604] LOG:  database system is shut down
 done
server stopped
==> Moving postgresql data from /usr/local/var/postgres to /usr/local/var/postgres.old...
The files belonging to this database system will be owned by user "cex".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /usr/local/opt/postgresql/bin/pg_ctl -D /usr/local/var/postgres -l logfile start

Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok

If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.

Performing Upgrade
------------------
Analyzing all rows in the new cluster                       ok
Freezing all rows in the new cluster                        ok
Deleting files from new pg_xact                             ok
Copying old pg_xact to new server                           ok
Setting next transaction ID and epoch for new cluster       ok
Deleting files from new pg_multixact/offsets                ok
Copying old pg_multixact/offsets to new server              ok
Deleting files from new pg_multixact/members                ok
Copying old pg_multixact/members to new server              ok
Setting next multixact ID and offset for new cluster        ok
Resetting WAL archives                                      ok
Setting frozenxid and minmxid counters in new cluster       ok
Restoring global objects in the new cluster                 ok
Restoring database schemas in the new cluster
                                                            ok
Copying user relation files
                                                            ok
Setting next OID for new cluster                            ok
Sync data directory to disk                                 ok
Creating script to analyze new cluster                      ok
Creating script to delete old cluster                       ok

Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade so,
once you start the new server, consider running:
    ./analyze_new_cluster.sh

Running this script will delete the old cluster's data files:
    ./delete_old_cluster.sh
==> Upgraded postgresql data from 10 to 11!
==> Your postgresql 10 data remains at /usr/local/var/postgres.old
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)

Fortunately that seems to have worked.

Adding helpers for RSpec memoization

I thought initially it would be incredibly easy to add a wrapper method alongside existing let, let!, subject memoized helper methods in RSpec.

def current_user(&block)
    u = yield
    allow(User).to receive(:current_user).and_return(u)
  end

  describe 'stuff' do
    let(:artist) { Fabricate(:artist) }
    current_user { artist.user }
  end

But I kept getting. `artist` is not available on an example group when the current user block was yielded. I had to dig into RSpec::Core to find how let is defined and came up with and ensure that artist and current_user were in the same scope when calling one another.

module CurrentUserHelpers
  extend ActiveSupport::Concern

  module ClassMethods
    def current_user(&block)
      raise "#current_usercalled without a block" if block.nil?
      RSpec::Core::MemoizedHelpers.module_for(self).__send__(:define_method, :current_user, &block)

      # Apply the memoization. The method has been defined in an ancestor
      # module so we can use `super` here to get the value.
      define_method(:current_user) do
        u = __memoized.fetch_or_store(:current_user) do
          block.arity == 1 ? super(RSpec.current_example, &nil) : super(&nil)
        end
        allow(User).to receive(:current_user).and_return(u)
        u
      end
    end

    def current_user!(&block)
      current_user(&block)
      before { current_user }
    end
  end
end

Bugs that happen in Continuous Integration but not local

Your CI environment probably doesn’t mirror your development setup 100% of the time. So when stuff spectactularly fails testing elsewhere and not locally you might be tempted to throw your laptop in the bin.

81) BLAH
      Got 0 failures and 2 other errors:

      81.1) Failure/Error: BLAH
            
            IOError:
              closed stream
            # ./vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-3.46.0/lib/seahorse/client/net_http/patches.rb:29:in `block in new_transport_request'
            # ./vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-3.46.0/lib/seahorse/client/net_http/patches.rb:28:in `catch'
            # ./vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-3.46.0/lib/seahorse/client/net_http/patches.rb:28:in `new_transport_request'
            # ./vendor/bundle/ruby/2.3.0/gems/fakeweb-1.3.0/lib/fake_web/ext/net_http.rb:50:in `request_with_fakeweb'
            # ./vendor/bundle/ruby/2.3.0/gems/webmock-3.0.1/lib/webmock/http_lib_adapters/net_http.rb:97:in `block in request'
            # ./vendor/bundle/ruby/2.3.0/gems/webmock-3.0.1/lib/webmock/http_lib_adapters/net_http.rb:110:in `block in request'
            # ./vendor/bundle/ruby/2.3.0/gems/webmock-3.0.1/lib/webmock/http_lib_adapters/net_http.rb:109:in `request'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/default.rb:121:in `response_for'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/default.rb:76:in `request'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/http/common.rb:62:in `call'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/bridge.rb:166:in `execute'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/oss/bridge.rb:584:in `execute'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/remote/oss/bridge.rb:50:in `get'
            # ./vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-3.141.0/lib/selenium/webdriver/common/navigation.rb:30:in `to'
            # ./vendor/bundle/ruby/2.3.0/gems/capybara-3.10.1/lib/capybara/selenium/driver.rb:46:in `visit'
            # ./vendor/bundle/ruby/2.3.0/gems/capybara-3.10.1/lib/capybara/session.rb:265:in `visit'
            # ./vendor/bundle/ruby/2.3.0/gems/capybara-3.10.1/lib/capybara/dsl.rb:51:in `block (2 levels) in <module:DSL>'
            # ./vendor/bundle/ruby/2.3.0/gems/rspec-rails-3.8.0/lib/rspec/rails/example/feature_example_group.rb:29:in `visit'