Much of the work I’ve been doing over the past 10 years has been taking older systems and keeping them current. My first Rails gig was to move hundreds of thousands of users and their services from two older systems effectively white labelled brands written in Perl (CGI) into one new stonking system under just one of those brands. You have to keep your eye on the ball, technology changes and you have to keep up. Anyone still doing web development with Perl are probably thin on the ground now products will have died and the people that wrote them moved onto better things.

Hell, even when you choose a language like Ruby things change all the time and code needs nurturing. This is often burying my head in code quality improvements both in terms of legibility and performances. You make code more legible and when you do you spot things that can turn a controller action from making stupid amounts of database queries to 2 queries for example. As a side-note I sometimes get so buried in these things that when it comes to doing the simple stuff I have to refresh things I haven’t touched in a year. Like how the hell do I set up a collection_select. That aside, this work to some might seem monotonous but I love it.

But as with many refactoring and upgrading tasks, its hard to prove it’s worth to your average outsider. Yeah, I worked really hard on this thing guys and if you look hard enough you’ll see there is less code and less network traffic too and from our DB server. Blank faces.

Ok, that’s not strictly true we have a great team at SARD who understand that we’re not just sitting pretty drink cups of tea and kicking a can down the street. It’s important to know when you’ve made successful leaps in productivity and that has had a purpose. In my case my last piece of work was a means to an end to unlock a blocked upgrade.

It is however intensely satisfying to see a measure of success and today that was to show that on this application the average request from is down from 1500ms to 800ms. Boom.

Always end with a graph with Caramel brown in it.

Loading console and half of your models aren’t defined? They won’t necessarily be just to help boost boot up time.

Rails.application.eager_load!

This will load everything.

As much as Elixir feels quite alien at times. The error reporting is really very thorough.

iex(4)> c = IslandsEngine.Coordinate.new(1,1)
{:ok, %IslandsEngine.Coordinate{col: 1, row: 1}}
iex(5)> Island.new(:square, c)               
** (FunctionClauseError) no function clause matching in IslandsEngine.Island.new/2    
    
    The following arguments were given to IslandsEngine.Island.new/2:
    
        # 1
        :square
    
        # 2
        {:ok, %IslandsEngine.Coordinate{col: 1, row: 1}}
    
    Attempted function clauses (showing 1 out of 1):
    
        def new(type, %IslandsEngine.Coordinate{} = upper_left)
    
    (islands_engine 0.1.0) lib/islands_engine/island.ex:7: IslandsEngine.Island.new/2
iex(5)> {:ok, c } = IslandsEngine.Coordinate.new(1,1)
{:ok, %IslandsEngine.Coordinate{col: 1, row: 1}}
iex(6)> c
%IslandsEngine.Coordinate{col: 1, row: 1}
iex(7)> Island.new(:square, c)                       
{:ok,
 %IslandsEngine.Island{
   coordinates: #MapSet<[
     %IslandsEngine.Coordinate{col: 1, row: 1},
     %IslandsEngine.Coordinate{col: 1, row: 2},
     %IslandsEngine.Coordinate{col: 2, row: 1},
     %IslandsEngine.Coordinate{col: 2, row: 2}
   ]>,
   hit_coordinates: #MapSet<[]>
 }}

I initially thought that an aliased %Coordinate{} could not be pattern matched against an %IslandEngine.Coordinate{}. Wrong. it can’t be pattern matched against a tuple. Wally.

I am not an imposter.

A small amount of history. My first Ruby / Rails / Dev gig was for Advantage Interactive working on lcn.com They are still my domain registration provider and if you look really really hard probably none of my work is left on that site anymore. But times change. The company re-located in 2008 and I found myself needing to look for something else. We gotta eat and we’ve got kittens to feed too.

I got an interview for a company that sold e-commerce solutions to a number of clients and having worked with OSCommerce for a years with PHP and a grounding on a domain registration and web hosting service which had a cart and payment processing and the like I thought it was a good fit. As soon as I got there I was paired up with another developer and set to work on an address book application as a coding challenge. I did my homework and I specifically used the new kid on the block ‘nested routes’ yup know the ’ole /products/1/attributes/17903 malarky.

resources :products do
  resources :attributes
end

Ok, I probably thought I was being pretty smart. What I didn’t expect was that my pair was pretty patronising, he didn’t like this solution for the address book which had a parent object and a multiple child objects so ideal for this scenario. He started to prompt me to find a different solution and he said very slowly so I could hear him clearly “…maybe use a session…?”. He meant instead of using /products/1/attributes/17903. That I should store the product_id in a session when accessing the child object. Effectively.

# set product_id in session
session[:product_id] = 1
# visit
GET /attributes/17903
# and then grab record 
Attribute.where(product_id: session[:product_id], id: params[:id])

To me this is crazy, why in the hell would you use state to manage this. That’s not what a session is for, a session is for having a pretty good try at faking state with HTTP which is basically stateless. So we can maintain logging into a system across multiple requests for example. I just didn’t get his reasoning and we butted heads. “Why would we do that?”. Which on reflection he read as “Why would we do that because I am desperately out of my depth, dude”. He was insistent.

We moved on from the code interview which I had clearly failed in his eyes because I clearly didn’t understand what a session was. I know what a session is. We entered a big room with an oval table, and I was met with another developer. They slouched across the table and told me a bit about the company and that they were migrating a ton of clients over to Rails. I asked “Why Ruby?”, because I wanted to know their reasoning for migrating at this time. If you are going to up and move everything at some significant cost then there has to be a motivation right, something they can’t resolve and can only be resolved longer-term by moving everything over? They misunderstood my question and started telling me how great Ruby was. Dudes I have been working with Ruby full-time for 3 years. I’m not an idiot. But they treated me like one. And eventually said “You’re clearly not a Developer”. Honestly, I was at this point angry because you can’t argue with people who have made up their mind without really taking the time to understand who I was and what experience I had. They didn’t even ask me what I’d been working on.

This is the first time I’d been made to feel bad for not knowing stuff that I do actually know. It’s hard to express yourself when you’re up against confident (or indeed arrogant) people, outward confidence in a meeting room from a lesser experienced person can trump years of experience and that’s crazy. I had clearly not made it to the next stage and I wasn’t likely to work with them anyway given the vibe I got from them.

I continued to look around for work. I had multiple recruiters put me forward for things. I did a code test for Reevoo and got a callback from the recruiter saying that they were impressed with my test and they’d like to move forward, except. “We heard you had a bad interview with insert company>”. What? There are times when you are paranoid that people might not like you or might be talking behind your back, and often you’re imagining it. But no…people had actually been talking behind my back. A recruiter completely unconnected to the company I had interviewed for had information on me and a black mark had been put against my name. How does that work? Who the actual, WTF??? I explained my experience to the recruiter and we went back and forth many times on it. I had to move on sadly and look elsewhere. Not because I was a bad developer but someone had made their mind up and told others they should question my ability even if you had a pleasing code test. Is this what they call Gatekeeping, did they think they were better or that I was just a pretender to the throne?

It’s difficult not to be bitter about such things, I remember all of this pretty clearly and it was a difficult time. Fortunately going the Recruiter route is not the only way. I had to find work pretty quickly and I wasn’t going to continue working in Stevenage with a 3 hour each way journey. Fortunately I met Zach, via a forum and joined PledgeMusic about a month later for 9 years and 11 months. ( we didn’t make it to 10 :( ) and from that first day I never felt like I wasn’t good enough, like everyone else I’ve written bad code, broken things, managed full-time work and at least 6 long-term freelance clients and had some huge successes in that time.

So no I am not an imposter. And if someone thinks you are? well we shouldn’t waste too much time worrying what other people think, unless they are your actual boss.

I’ve been using simple_form for as many years as I can remember. I love it’s simplicity. We’ve been adopting DRY since we all first adopted Rails back when we were less grey. So every time you see a form input like this it pains me.

<div>
  <%= form_for(User.new) do |f| %>
    <div class="field-group">
      <%= f.label :name, class: 'form-control' %>
      <%= f.text_field :name, class: 'form-input', placeholder: 'Name' %>
    </div>
    <div class="field-group">
      <%= f.label :role_ids, class: 'form-control' %>
      <%= f.select :role_ids, options_for_select([][1, 'One'],[2, 'Two']]), class: 'form-select' %>
    </div>
  <% end %>
</div>

This example is a simple one, it doesn’t even cater for errors, placeholders, hints and even left / right alignment of inputs based on different situations.

I know we’re going to repeat this about 400 times in our application and if we want to upgrade from Bootstrap 4 to Bootstrap 5 or another CSS framework, or something custom we’re going to have to rewrite every form on the site which could take months and a developer out of circulation to do this. If you ever need an excuse not to do something. (Let’s not upgrade to Bootstrap 5 it will take a year and cost $1Billion). This is a big one.

SimpleForm offers the ability to reduce that code into one line per form group. It makes reasonable guesses about what kind of form element you want based on the data type of the Boolean is check_box, String is text, etc. However it’s customizable so you can determine on a per input basis what it should be.

<div>
  <%= form_for(User.new) do |f| %>
    <%= f.input :name, placeholder: 'Name' %>
    <%= f.select :role_ids, collection: [[1, 'One'],[2, 'Two']] %>
    <%= f.select :bio, as: :text %>
  <% end %>
</div>

SimpleForm is ridiculously customisable and it caters for so many eventualities. Your form element could have 100 variations and SimpleForm’s not so simple config file will aid you to nail your form layouts. Trouble is your simple_form config can end up looking like this. Show that to your UX/UX peeps and they will stare blankly back at you and say things like WTF, or “you’re having a laugh” or worse “NO”.

config.wrappers :select, tag: 'div', class: 'form-group row', error_class: 'form-group-invalid', valid_class: 'form-group-valid' do |b|
    b.use :label, class: 'col-sm-7 col-md-6 col-lg-5 form-label'
    b.wrapper tag: 'div', class: 'col-sm-17 col-md-18 col-lg-19' do |ba|
      ba.use :input, class: 'form-control', error_class: 'is-invalid', data: { select: true }
      ba.use :full_error, wrap_with: { tag: 'div', class: 'invalid-feedback d-block' }
    end
  end

This kind of thing works if you’re a seasoned developer, and you can remember exactly what you did last week. But if you’re anything like me I jump from thing to thing and often don’t get a chance to take stock of everything before jumping into something else. But it occurred to me this morning that it’s not SimpleForm that I’m in love with it’s the simplicity I’m in love with and the implementation I can accept is pretty cool and I like to see elaborate things but I’m not particularly in love with the way it doesn’t allow everyone in the team to understand it without a week of poking it with a stick and in the end it’s a hinderance to development.

I can offer a different solution however, Rails has offered us a way to use custom FormBuilder and has since forever. It’s very nice to have a singing dancing solution, but a simple one will suffice. A simple solution needs to have some flexibility but it only needs to cater for the needs of our team not for every Ruby outfit on the planet.

We could easily just build our own solution but for the sake of our sanity make it compatible with SimpleForm arguments so we can just swap this in by changing simple_form_for with another_form_for.

class AnotherFormBuilder < ActionView::Helpers::FormBuilder
  def input(method, options = {})
    # delegate to a single method for each kind of form input based on type or options[:as]
  end
  
  def string
  end  

  def time_picker
  end

  def toggle_switch
  end
end

def another_form_for(name, &block)
  form_for(name, builder: AnotherFormBuilder, &block)
end

Now that I’ve written this I’ve found this rather nice article after coming to this conclusion, it appears they are thinking along the same lines.
https://brandnewbox.com/notes/2021/03/form-builders-in-ruby/

Feeling slightly bad that the same week that I do a PR for SimpleForm then I’m off the mindset to move on .

I did a bad and ploughed into creating an update function for my form and woah begads I broke something.

UndefinedFunctionError at GET /rooms
function DungeonWeb.RoomView.render/2 is undefined (module DungeonWeb.RoomView is not available)

Now DungeonWeb.RoomView.render/2 is not there. In the backtrace it even suggests there’s nofile. There totally is…

defmodule DungeonWeb.RoomView do
  use DungeonWeb, :view
end

Not entirely sure what could cause this or cause the room_view.ex not to be loaded :S

UPDATE: 29/11/2021

It would appear that saving and changing just whitespace on my rooms/index.html.heex fixed everything. Huh? So there’s some kind of template cache that can get messed up?

There is nothing worse than working through a programming textbook and everything working first time. Sometimes the best way to learn is to see backtraces, and broken stuff all over the place. I’ve been working full time as a programmer for over 10 years and while I’m experienced, if there is a possibility for something to break I seem to find it without fail.

I also systematically go through eBay and pick up older Pragmatic Programmers books because I just love the presentation, they’ve nailed it and every read is just a comfortable. However, my £4 Programming Phoenix book means that I’m about 5 years out of date and so everything doesn’t work first time. But that’s ok, I’m learning and finding out stuff by breaking stuff.

Today it’s working out Ecto changeset. Firstly the cast function was missing, so I needed to add import Ecto.Changeset

defmodule Dungeon.Room do
  use Ecto.Schema
  import Ecto.Changeset

  @primary_key {:id, :binary_id, autogenerate: true}
  schema "rooms" do
    field :name, :string
    field :x, :integer
    field :y, :integer
  end

  def changeset(model, params \\ :empty) do
    model
    |> cast(params, ~w[name x y], [])
    |> validate_length(:name, min: 1, max: 30)
  end
end

And now….

Ecto.CastError at GET /rooms/new
expected params to be a :map, got: `:empty`

It appears that cast takes a :map and we’ve defaulted to :empty and cast is now kicking off :S

I can’t replace it with…

def changeset(model, params \\ %{}) do   model
  |> cast(params, ~w[name x y], [])
  |> validate_length(:name, min: 1, max: 30)
end

…since cast is expecting name, x, or y. Oh right. Ok the fourth argument to cast/4 is optional keys. This will be pretty lame since every empty Room struct will have no name, x, or y. So there’s got to be something else missing here :S

def changeset(model, params \\ %{}) do
  model
  |> cast(params, [], ~w[name x y])
  |> validate_length(:name, min: 1, max: 30)
end

Hmmzzz….

It would appear that my name, x, and y are all empty.

def changeset(model, params \\ %{}) do
  model
  |> cast(params, ~w[name description x y]a, [])
  |> validate_length(:name, min: 1, max: 30)
end

Seems we need a list of atoms rather than strings so.

~w[name x y]a

Ok, queen….I’ll check the Ecto documentation. I can read that too right?

defmodule User do
  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :name
    field :email
    field :age, :integer
  end

  def changeset(user, params \\ %{}) do
    user
    |> cast(params, [:name, :email, :age])
    |> validate_required([:name, :email])
    |> validate_format(:email, ~r/@/)
    |> validate_inclusion(:age, 18..100)
    |> unique_constraint(:email)
  end
end

OK, looks like this \\ :empty isn’t the right thing to do.

Looks like Ecto 3 compatabilty for mongo-ecto was committed 4 days ago. So trying to flip over my default PostgreSQL setup to MongoDB.

diff --git a/config/dev.exs b/config/dev.exs
index 72122a0..07fb393 100644
--- a/config/dev.exs
+++ b/config/dev.exs
@@ -1,13 +1,18 @@
 import Config
 
 # Configure your database
+# config :dungeon, Dungeon.Repo,
+#   username: "dungeon",
+#   password: "",
+#   database: "dungeon_dev",
+#   hostname: "localhost",
+#   show_sensitive_data_on_connection_error: true,
+#   pool_size: 10
+
 config :dungeon, Dungeon.Repo,
-  username: "dungeon",
-  password: "",
+  adapter: Mongo.Ecto,
   database: "dungeon_dev",
-  hostname: "localhost",
-  show_sensitive_data_on_connection_error: true,
-  pool_size: 10
+  hostname: "localhost"
 
 # For development, we disable any cache and enable
 # debugging and code reloading.
diff --git a/lib/dungeon/repo.ex b/lib/dungeon/repo.ex
index 09502cf..df8c03b 100644
--- a/lib/dungeon/repo.ex
+++ b/lib/dungeon/repo.ex
@@ -1,5 +1,8 @@
 defmodule Dungeon.Repo do
+  # use Ecto.Repo,
+  #   otp_app: :dungeon,
+  #   adapter: Ecto.Adapters.Postgres
   use Ecto.Repo,
     otp_app: :dungeon,
-    adapter: Ecto.Adapters.Postgres
+    adapter: Mongo.Ecto
 end
 
diff --git a/mix.exs b/mix.exs
index bcc220c..ce79a70 100644
--- a/mix.exs
+++ b/mix.exs
@@ -20,7 +20,7 @@ defmodule Dungeon.MixProject do
   def application do
     [
       mod: {Dungeon.Application, []},
-      extra_applications: [:logger, :runtime_tools]
+      extra_applications: [:logger, :mongodb_ecto, :ecto, :runtime_tools]
     ]
   end
 
@@ -35,8 +35,9 @@ defmodule Dungeon.MixProject do
     [
       {:phoenix, "~> 1.6.2"},
       {:phoenix_ecto, "~> 4.4"},
-      {:ecto_sql, "~> 3.6"},
-      {:postgrex, ">= 0.0.0"},
+      # {:ecto_sql, "~> 3.6"},
+      {:mongodb_ecto, github: "elixir-mongo/mongodb_ecto"},
+      # {:postgrex, ">= 0.0.0"},
       {:phoenix_html, "~> 3.0"},
       {:phoenix_live_reload, "~> 1.2", only: :dev},
       {:phoenix_live_view, "~> 0.16.0"},

iex(1)> Dungeon.Repo.all(Dungeon.Room)
[debug] QUERY OK db=1.8ms decode=1.3ms idle=1930.8ms
FIND coll="rooms" query=[{"$query", []}, {"$orderby", %{}}] projection=%{_id: true, name: true, x: true, y: true} [[{"$query", []}, {"$orderby", %{}}], %{_id: true, name: true, x: true, y: true}]
[]

Not a Dungeon in sight.

iex(2)> w = %Dungeon.Room{x: 1, y: 1} 
%Dungeon.Room{
  __meta__: #Ecto.Schema.Metadata<:built, "rooms">,
  id: nil,
  name: nil,
  x: 1,
  y: 1
}
iex(3)> Dungeon.Repo.insert!(w)
[debug] QUERY OK db=96.8ms idle=1253.4ms
COMMAND [insert: "rooms", documents: [[_id: #BSON.ObjectId<619c493af7fdf5aad1189f38>, x: 1, y: 1]], writeConcern: %{}] [[insert: "rooms", documents: [[_id: #BSON.ObjectId<619c493af7fdf5aad1189f38>, x: 1, y: 1]], writeConcern: %{}]]
%Dungeon.Room{
  __meta__: #Ecto.Schema.Metadata<:loaded, "rooms">,
  id: "619c493af7fdf5aad1189f38",
  name: nil,
  x: 1,
  y: 1
}
iex(4)> Dungeon.Repo.all(Dungeon.Room)
[debug] QUERY OK db=0.4ms idle=422.0ms
FIND coll="rooms" query=[{"$query", []}, {"$orderby", %{}}] projection=%{_id: true, name: true, x: true, y: true} [[{"$query", []}, {"$orderby", %{}}], %{_id: true, name: true, x: true, y: true}]
[
  %Dungeon.Room{
    __meta__: #Ecto.Schema.Metadata<:loaded, "rooms">,
    id: "619c493af7fdf5aad1189f38",
    name: nil,
    x: 1,
    y: 1
  }
]

Every time I try and upgrade anything through brew I appear to get node linked to v17. Nope, v17 doesn’t work with our current set up. Something to do with node-sass that isn’t resolved yet. Anyway, I still have v14 installed and I need to keep re-linking it like so.

Robs-MacBook-Pro:rostering rl$ node -v
v17.0.1
Robs-MacBook-Pro:rostering rl$ brew link --force --overwrite node@14
Linking /usr/local/Cellar/node@14/14.17.0... 3986 symlinks created.

If you need to have this software first in your PATH instead consider running:
  echo 'export PATH="/usr/local/opt/node@14/bin:$PATH"' >> /Users/rl/.bash_profile
Robs-MacBook-Pro:rostering rl$ node -v
v14.17.0

Started looking at Elixir again. Thinking about building something with MongoDB, and I know that support for MongoDB Ecto but there’s chatter about it and is not quite there but this is an opportunity to start hacking away. Anyway, first point of call. What the hell is Plug anyway, it’s kinda like Rack in Ruby.

https://github.com/braindeaf/unplug

So that’s the Plug docs examples, now to expand on it.