Thursday, February 04, 2010

Is IT evolution a bad thing?

One of the tenants of IT is that enabling evolution, i.e. the small incremental change of existing systems, is a good thing at that approaches which enable this are a good thing. You see it all the time when people talk about Agile and code quality and clearly there are positive benefits to these elements.

SOA is often talked about as helping this evolutionary approach as services are easier to change. But is the reality that actually IT is hindered by this myth of evolution? Should we reject evolution and instead take up arms with the Intelligent design mob?

I say yes, and what made me think that was reading from Richard Dawkins in The Greatest Show on Earth: The Evidence for Evolution where he points out that quite simply evolution is rubbish at creating decently engineered solutions
When we look at animals from the outside, we are overwhelmingly impressed by the elgant illusion of design. A browsing giraffe, a soaring albatross, a diving swift, a swooping falcon, a leafy sea dragon invisible amoung the seaweed [....] - the illusion of design makes so much intuitive sense that it becomes a positive critical effort to put critical thinking into gear and overcome the seductions of naive intuition. That's when we look at animals from the outside. When we look inside the impression is opposite. Admittedly, an impresion of elegant design is conveyed by simplified diagrams in textbooks, neatly laid out and colour-coded like and engineer's blueprint. But the reality that hits you when you see an animal opened up on a dissecting table is very different [....] a haphazard mess that we actually see when we open a real chest.

This matches my experience of IT. The interfaces are clean and sensible. The design docs look okay but the code is a complete mess and the more you prod the design the more gaps you find between it and reality.

The point is that actually we shouldn't sell SOA from the perspective of evolution of the INSIDE at all we should sell it as an intelligent design approach based on the outside of the service. Its interfaces and its contracts. By claiming internal elements as benefits we are actually undermining the whole benefits that SOA can actually deliver.

In otherwords the point of SOA is that the internals are always going to be a mess and we are always going to reach a point where going back to the whiteboard is a better option than the rubbish internal wiring that we currently have. This mentallity would make us concentrate much more on the quality of our interfaces and contracts and much less on technical myths for evolution and dynamism which inevitably lead into a pit of broken promises and dreams.

So I'm calling it. Any IT approach that claims it enables evolution of the internals in a dynamic and incremental way is fundamentally hokum and snake oil. All of these approaches will fail to deliver the long term benefits and will create the evolutionary mess we see in the engineering disaster which is the human eye. Only by starting from a perspective of outward clarity and design and relegating internal behaviour to the position of a temporary implementation will be start to create IT estates that genuinely demonstrate some intelligent design in IT.

Technorati Tags: ,

PS. I'd like to claim some sort of award for claiming Richard Dawkins supports Intelligent Design


Martijn Linssen said...

Fine post!

One of my blog posts ( comes pretty close to what you're saying. It is very important that we do not create a mess so it all looks nice & smooth on the outside. However, project steering vs ALS does make that increasingly harder...

mamund said...

I read Dawkins a bit differently. To me, he is saying that the animal (the system) has an outer simplicity that belies the haphazard way in which the animal is assembled. However, the individual parts of the animal (system components) are, in fact, quite efficient and effective (the class eye ball is a case in point). And these parts have experience gradual change over time _independent_ of the whole.

To bring this home to IT, each component needs to be able to evolve into a single unit that works as efficiently as possible. That component should be free to undergo modifications as needed without disruption to the entire system. In practice, this only works well when the system is tolerant of a good amount of ambiguity and abstraction. This ambiguity usually results in a system that has many inefficient couplings, but works very successfully overall.

IOW, it's not acceptable to simply 'hide the mess' inside. Instead, the system must be arranged in a way to allows for local modifications and efficiencies to continue w/o adversely affecting the whole. When this is possible each component can grow to it's most effective level.

This is _much_ more difficult to pull off than creating singular, monolithic systems each time you want to fix one small item.

Steve Jones said...

Mamund but the point that Dawkins makes in the book is that it would be a truly RUBBISH engineer who designed animal internals in this way (the eye being a classicly rubbish piece of engineering) as it requires a massive amount of hacks and compensations to get over the initial engineering efforts.

The eye is completely rubbish in humans, its built back to front and the cabling goes out through the front before going back down which means we have blind spots. You'd fire someone who designed something that badly.

The point I'm making is that Evolution is a bad way to design a system the whole point of evolution is that of selection, selection of an individual against others. In IT we have just one individual (our IT estate) and so selection doesn't apply. Dawkins point (made elsewhere in the book about moving from the propeller to the jet engine) is that a sensible engineer would have gone back to the drawing board on lots of things. From our perspective in IT this makes perfect sense and all I'm saying is that this should be the default plan.

I'm still saying that internal service or system evolution in IT is actually a bad thing and doomed to fail and approaches that promote it are fundamentally flawed.

Anonymous said...

Sorry, but mamund I read that much more in-line with mamund.

Intelligent designer-or-not :) you are not going to get it right.

The only question is how effectively you are able to respond to change. Besides - most of the (r)evolutionary changes you must address will be external and outside of your perspective.

Steve Jones said...

The point of revolutionary change is that it may require a drop back to the beginning and starting again. This isn't possible in an evolutionary model.

Integral ):( Reporting said...

IT doesn't have any money to always go back to the drawing board each time Microsoft, Oracle, IBM or a group of people come up with a "better" way of doing something. It's hard to admit, but I side with Mike, even though he cannot articulate a versioning strategy that would make evolution possible in REST.

Steve, just think how much an IT budget should be if one of the governance policy would be "There shall be no IT component or system that is not supported".

Evolution is the only sustainable mechanism for IT to exist, even if the price to pay is suboptimal functionality. Revolution does not guaranty success in itself.


Steve Jones said...

Lets be clear here, I'm not talking about when a VENDOR claims they have a better way. I'm saying that sometimes it is cheaper to go back to the drawing board behind a well defined contract than it is to continually evolve an existing system. Most legacy systems, and by that I mean any system that is a pain to maintain, originally set out to be "flexible" and "evolve" and this was often done on the back of some fad or other pushed by a vendor or buzz-word compliant fad. The problem is that most of these "flexible" systems are in reality a single application which happens to expose multiple interfaces and the flexibility is a complete myth.

REST doesn't fix this, WS-* doesn't fix this, EAI didn't fix this, OO didn't fix this, "modules" didn't fix this. The fundamental problem is that the plumbing inside these systems becomes a mess and the external interfaces, which are often just an afterthought, just hide the mess beneath. It would be much better if we concentrated more on those interfaces and considered the internals something that could be disposed of then we'd create solutions that better survived rather than systems that we have to drag forwards/

Martijn Linssen said...

I agree, no technique will ever solve this because it's a physical and functional problem - although EAI with a canonical model is the best way to bridge the gap between that very diversity in the IT-landscape, and those very standardised (tbd) services on the outside

After a while, any application becomes a mess, because we focus on delivering projects rather than applications. RUP, Agile, it's all about one-night stands, never about marriage

By the way, when the dinosaurs were in the process of extinction, evolution was going back to the drawingboard so we could have a dialogue on that over a few beers...

IT (and business!) would be much better if we'd started thinking for ourselves. Put a fine interface on top of an application, call it a service, and then start to build federations out of these services, allowing you to slowly become loosely-coupled from the mess underneath. You won't be able to manage your IT unless you have a constantly up-to-date view of all components, simple as that. Could you raise your kids if you saw them one day a month, a year?

Rob Eamon said...

@Steve, "the external interfaces, which are often just an afterthought, just hide the mess beneath."

We're in violent agreement on the "afterthought" aspect. I'd offer that often they don't hide the mess, they add to it.

@Martijn, "RUP, Agile, it's all about one-night stands, never about marriage."

That's awesome! Can I use that?

"Could you raise your kids if you saw them one day a month, a year?"

Excellent point. But that seems to be the predominant aspect of systems. Most don't pay attention to the architecture/design docs after deployment. They know they exist only if someone could point them to it. :-)