I hadn’t heard of the formal definition of the XY problem, until I came across it, procrastinating watching Primeagan talk about a recent idiosyncratic rant from Linus Torvalds. Primeagan goes on to explain the rant as the XY problem: I’m not convinced if the analogy is correct as I don’t fully understand Linus’s problem. Especially, as the thread goes onto a conciliatory tone from both sides.

Nonetheless, many people have experienced the XY problem both in life and at work. I prefer to think of the problem as a lack of First Principles thinking. In other words what are the foundational truths of the problem at hand: Not the symptoms or 2nd/3rd order affects of the problem. This lack of First Principles thinking is pervasive throughout science, technology and society at large. We especially see it in medicine where drugs manage the symptoms of an underlying problem without addressing it directly. Primagean goes onto to make a convincing case about XY problems being pervasive in JS world and SWE as a whole; The idea that we like to abstract away complexity(because it is easy to do with present SWE tools) and reduce repetition (because of DRY principles).

All engineers at least at one point in their career have succumbed to the practices of over abstracting. Where does one draw the line between abstraction and being explicit? Or repeating oneself rather than abstracting? I don’t think there is a definitive rule, at least not one that comes to mind. Certainly, a quantitive approach could help in choosing.For example:

• How often is this set of functionality used within an organisation?
• Is the abstraction a meaningful cost benefit?
• What are the implicit and explicit costs of maintaining that abstraction?

Most importantly, if you are choosing to abstract, have you considered the First Principles of what it is you are attempting to solve? In other words what is this abstraction solving? Other than shifting the complexity elsewhere. The case made by Primagean for learning new things is relevant — far too often, one would rather abstract away the need to learn something new within an organisation. Moreover, we are taught DRY is the modus operandi of all good engineers.

Docker is a great example of a beneficial abstraction. The world is definitely a better place with containers. However, it doesn’t mean that the problems of administrating a complex system have gone away. Docker solves the problem of having expert skills to deploy complex SW. It doesn’t solve the problems of operating complex SW. At the other end of the spectrum left-pad is a horrible abstraction and should never have been written.

Case closed.