The root cause of “works on my machine”

I recently read an article about rare illnesses. Right in the opening sentence, it said something that confused me: Rare illnesses are the most common illnesses. How does this make sense?

It makes sense because the “rare” refers to the number of people affected by one individual type of rare illness, while the “most common” refers to the total number of people who suffer from any arbitrary type of rare illness.

It is exactly the same with the problem of “works on my machine”. The number of different things that can go wrong when deploying software is so big that each individual problem might only arise under very specific circumstances and is therefore rare in itself.

This is the root cause of “works on my machine”: the most common types of problems are rare problems. Most problems are unique. When looking at a graph that represents different problems that arose when deploying a software in different environments, this leads to a long tail of rare and unique problems.

This long tail of problems makes it extremely difficult to ensure reliable deployment processes that work in many different environments. When deploying software in a production setting, this is typically solved by standardising the deployment environment. But this is pretty much impossible when it comes to local deployment on individual developer’s laptops. Even if the basic setup is standardised, the sheer variety of hardware and software that can – and will – be present on developer’s laptops very often leads to problems in local deployment of software. And the majority of these problems are unique.

Problems that affect a majority or even just several developers are typically automated away quickly. The problem is that this still leaves a lot of problems that affect only one or a few developers. Addressing these rare problems in deployment automation often doesn’t warrant the time and effort that would go into it, because so few people are affected. So individual developers are left to deal with these problems on their own.

As a consequence, many developers spend a lot of time troubleshooting their local deployments. Depending on the complexity of local deployment, this can amount to a little, or a lot of developer’s time. On average, developers spend around 10% of their time troubleshooting their development environments. For some developers, this number is a lot higher.

How to squash the long tail

There is only one way to get rid of this time sink: By addressing root causes of several individual issues, and providing solutions that prevent them from arising in the first case.

Containerisation is one such solution that prevents a lot of issues from arising, by providing isolated and standardised environments for software components to run in. Package managers that allow to install and use several versions of a package next to each other (isolated packages, e.g. Nix) are another neat approach that chop off a large section of the long tail of unique problems. Standardisation of developer’s laptops is an often-tried approach with mixed outcomes (Read: Problems with the local development environment: Is containerisation the solution?), but at least conceptually it is sensible, because it tries to address an underlying cause instead of thousands of individual problems.

Recently, another approach has gained popularity: remote development environments (RDEs) or cloud development environments (CDEs). These are work environments that are provided remotely where software developers can deploy and run the software they work on, together with any development tools they need.

As complete environments, RDEs go several steps further than containers by providing isolated environments from the operating system up. RDEs are also a lot simpler to standardise because they exist in addition to developers laptops: Developers are free to use whatever hardware they like and install whatever software they want on their laptops, without interfering with their standardised RDEs. As such, RDEs will get rid of the long tail of unique problems almost entirely.

If you want to learn more about RDEs/CDEs, here are some additional resources: