The Tyranny of Functional Specifications

I want to discuss one of the reasons why most large enterprise IT is garbage. I'll try to keep the language and story simple.

It starts with the fact that large IT clients prefer almost to the point of exclusivity to deal with large IT companies. A part of this is because a client prefers to know the vendor isn't going to go under because they couldn't pay the electricity bill (problematic assuming they have moved beyond abacuses) before they hand over the software. Another part of it is that at bigger enterprises, you just don't hire noname vendors. Large IT vendors build brand for one reason: so that the CIO can get a quick "yes" from management because they know the vendor name and feel smart for it; the board knows the name IBM and since they do, they feel reassured and sign off on the contract. Don't mock them; this is why you buy Hellmans Mayo and not Horowitz Mayo.

This would be fine if human competence scaled. Competence, sadly, scales about as well as Microsoft Access.

Enter Sturgeon's Law. At the risk of statistics abuse, we'll postulate that 90% of IT professionals should not have gone into IT. Beyond a certain size, it's Sturgeonly impossible to keep the competence average from dropping. Here, look. I've drawn a graph for you.It's really good. The green line is the number of employees, and the red line is the collective competence of said employees.

Nice graf. For you. Iz made many hours, you happy?

The story of this graph which I went to great trouble to draw in Visio Fisher Price Enterprise Edition is simple. As the company grows, it becomes impossible to maintain the original core level of actual bona fide IT competence. And when the HR and executive positions fall prey to this inevitability, the transformation becomes irreversible. Developer candidates are no longer fizzbuzzinated, and non-developer positions are filled with people in smart threads on the virtue of their ability to draw boxes and coerce otherwise innocent arrows into pointing senselessly from one box to another on whiteboards in the name of enterprise architecture. A skill they have been certified to wield, reassuringly.

It's inevitable. No group of competent people can survive the horror of scaling. When God designed the universe, he started with Sturgeon's Law. The rest is filler.

At this point, the growing enterprise is getting pretty good at hiring project managers, enterprise architects, scrum coaches, agile gurus, and IT governannce consultants and getting worse at hiring people who can actually construct software. And that this level, the client side of enterprise software contracts is also pretty big and has grown beyond the level where they feel the need for or are able to attract and retain sensible technical decision makers. Sensible and technical. Because being technical and at the same time business-aware is usually only mutually exclusive in the minds of non-technical people.

To reinforce this mutual exclusivity of scale and competence, I have drawn using the latest SOAP ESB technologies a UML-safe representation of the situation. It has also been reviewed by the UX team and is pending QA review. In Coldfusion. Here:

This architectural diagram coincidentally also represents a complete technical specification of Microsoft bob.

So when information systems are specified at this level, it is of course the functional specifications that matter. not because they are more important, but because most of the people involved in the process can only handle functional specs. Because they are about what the user and what the user wishes they could do, not about how the cold silicon actually should go about making this happen. Silicon is just silicon, right? The technical people are only consulted to find out how much something would cost, not how it ought be done. Boilerplate non-functional specs drawn up by some long-gone technical consultant that people prefer not to remember are pasted in and mandate things like sub-3 second response times on webpage presentation and compliance with the letter of RFC 1436 wherever possible.

In the name of business-centricity, what the user will magically be able to do negates any need to consider how these new computor-enabled superpowers are actually implemented and this is the tyranny of functional specifications. To celebrate this tyranny, here is a picture of a gopher:

It might not look like a gopher, but it fulfills the functional spec you provided us sir.

Lip service is paid to non-functional snoozepills like WCAG because the government requires it. AJAX is mistaken for a functional requirement, and oft-recurring IT media buzzwords like agile, ISO 27001, SOA, and many more are pasted in in lieu of actually having an opinion - a business-centric opinion - on how the system should in fact be constructed and do what is is supposed to so.

This results in a lot of systems that - in the best case - do something approximating what the users need it to for a few years until it has to be rebuilt because the platform was EOL'ed or because it is such spaghetti that it is almost cheaper to rewrite it to implement some otherwise minor new features. Or it doesn't scale. Or management has got mobile computing religion and wants the BizTalk app ported to Symbian. Or a ton of other possible futures, all containing much pain. That was the best case, and you know it. I've only got 15 Gb on my VPS so there isn't space to go into the worst case.

There are a lot of problems with the tyranny of functional requirements, but I am most concerned with the sustainability aspect. All software is crap, but at least some of it is viscous enough that one can swim. Given this, the most one can hope for is to not have to reinvest over and over again in the same functionality and the only way to accomplish that is to step the disrespect for non-functional design down a notch. Here is a scientific representation of down a notch:

TODO: change blue bars to orange to match new corporate identity.

This is basic business sense. Specify your procurements such that you don't pay more than once for the same thing. I went to business school and I can confirm that this principle is not explicitly taught, but I think that is less an oversight and more an optimistic assessment of general levels of common sense.

There is plenty wrong with larger scale enterprise IT, but focusing more on non-functional specs is really the low-hanging fruit here in terms of righting what's wrong.