Software is difficult, that’s no surprise. But why? Einar W. Høst offers an excellent critique which is so common-sense as to be nearly tautological. Software is difficult not because the problem it’s trying to solve is difficult, but because “the problem” is poorly defined and understood very differently by different people.

That’s certainly what I’ve seen: technical debt arises commonly because either a) developers differ on their understanding of what constitutes good code vis-a-vis requirements or b) deadline pressure pushes development teams to cut corners and deliver quickly. Both of these scenarios are at heart about determining what the problem actually is! In the first case it’s about different understandings among developers; in the second, it’s about the team’s efforts to understand users better. (Here I follow the Lean Startup concept that shipping code is an experiment that seeks to validate am hypothesis about the user. Even for teams not following this strategy explicitly, I think this concept is a reasonable way to understand deadline pressure.) Høst refers to the difficulty of understanding the problem as irreducible informality, which is a term I really enjoy - it calls to mind so readily the problem of decidability in classic complexity theory. The post is worth reading in full, and so are its many links.

Indeed, this meta-problem, that the problem itself confounds precise definition, is with us in many endeavors. I like to think that software as a discipline has a lot in common with politics and literature, and in this case I think software developers are re-learning ancient lessons discovered long ago by politicians and writers both. Machiavelli, if memory serves, wrote about how citizens rarely know what they want, centuries before the first digital computer. Writers are explorers, Emerson said, decades before the first computer program was written - their job is to find the problem that begs description in literature.

What software brings to the conversation, I think, is a realization that the meta-problem is in fact tremendously exciting - because it generates innovation. Viewed from this angle, solving a problem is relatively unexciting. Rather, it is the fact that a solution helps us find more and better problems, and to discover the language which helps us to understand the world more clearly.