OnlineOrNot Diaries 9

Another Saturday morning here in Toulouse, let's go into how OnlineOrNot went this week.

It was a coding week (that makes it three in a row, for those counting at home), and I took the time to clean-up a ton of tech debt. This article gets a little bit technical.

For context, I launched OnlineOrNot approximately two years and two months ago, and I never really treated the code like it might stick around for a while. A lot of my git commit messages are just me swearing. These are things you can get away with when you know you'll be the only person reading your code.

I built OnlineOrNot as a "mono-repo", but the folder structure was pretty chaotic, and if I wanted to use packages, I had to publish them to npm. I had the main Next.js app in one folder, a folder for my terraform configuration, and every AWS Lambda function in another:

infrastructure/
lambdas/
|   |-- browser-checker/
|   |-- checker-scheduler/
|   |-- cron-tasks/
|   |-- uptime-checker/
|   |-- uptime-checker-runner/
|   |-- weekly-emails/
|   |-- ...more lambdas here
web/

For a long time that worked well. There were a couple of utility functions I copy pasted between lambda functions, and I had comments to remind me to update the code in multiple places (obviously not a great practice when you have a team), and never had any issues.

Adding fly.io complicated things

At some point last year, I duplicated OnlineOrNot's stack and made it run on fly.io. The resulting folder structure looked like this:

infrastructure/
fly/
|   |-- checker-scheduler/
|   |-- uptime-checker-runner/
|   |-- ...more apps here
lambdas/
|   |-- browser-checker/
|   |-- checker-scheduler/
|   |-- cron-tasks/
|   |-- uptime-checker/
|   |-- uptime-checker-runner/
|   |-- weekly-emails/
|   |-- ...more lambdas here
web/

The trouble with this is, a good 95% of the code between the fly.io and AWS versions of OnlineOrNot were the same, adding a significant amount of code duplication. I figured it wasn't such a big deal at the time, since I'd just delete the AWS code one day, but then I realized OnlineOrNot could survive a hosting provider outage by keeping both stacks online.

So the duplicated code had to stay, until I had time to clean it up properly.

Moving to a "real" mono-repo

Since OnlineOrNot is mainly written in Node.js, I actually had a relatively easy way to split up code into packages that can be shared between apps: npm workspaces.

The gist of the change was:

1. Run npm init -y in the root of my mono-repo, and add a workspace config to the package.json:

   "workspaces": [
       "apps/*"
       "packages/*",
   ],
   

2. Move all the apps into the apps/ folder, and cut common code out of apps, and move them into their own packages in the packages/ folder

In the end, it looks like this:

infrastructure/
apps/
|   |-- checker-scheduler-aws/
|   |-- checker-scheduler-fly/
|   |-- uptime-checker-aws/
|   |-- uptime-checker-fly/
|   |-- web/
|   |-- ...more apps here
packages/
|   |-- common-queries/
|   |-- time-utils/
|   |-- ...more packages here

With the old folder structure, things were starting to feel a bit risky: if I updated duplicated code in one place but not others, parts of OnlineOrNot could behave differently as I changed hosting providers.

With the new structure, I don't even need to publish packages to npm to use them in my apps - it Just Works quite nicely.