The People's Democratic Republic of IT

IT is a communist state in many organisations, one that believes in rigid adherence to inflexible approaches despite clear indications that they inhibit growth and a central approach to planning that Mao and Stalin would have thought is taking things a little too far. This really doesn't make sense in the capitalistic world of business and the counter-revolution is well under way. Its

I don't think the word 'Enterprise' is really worth anything in terms of something being a single standard Enterprise approach.  Whether that is Enterprise Resource Planning, Enterprise Data Warehouse, Enterprise Service Bus or Enterprise Architecture you either end up with multiple solutions or a central solution that isn't used to the level it was envisaged so you get lots of solutions on the side.

Part of this is because in the capitalistic world of business it appears that communist style central planning has been, and remains, the normal approach.  This People's Democratic Republic of IT approach has two key parts to it

1. IT knows best and will give everyone 'each according to their needs' and decide what those needs are.
2. Cultish following of other communist plans, independent of whether the users want them.

The world of integration is a great example of the latter.  Do you know how much the business cares about whether you integrate two systems using REST, SOAP,sockets or flying monkeys?  ZERO.  Hell probably even less than zero in that they have an active disinterest in it.  Yet in IT we don't take this as a guidance of 'its not important, lets commoditise the fuck out of it'.  Nope we continue to 'innovate' where it really doesn't matter and we do so because a whole heap of hype tells us to... business hype?  Of course not, its hype from people who think they've discovered the universal hammer that turns everything into a nail.

On the former its the realm of 'Enterprise Architecture' and EDWs that really underline just how much IT often resembles the politburo.  Here groups of worthy individuals set about on the business equivalent of the Cultural Revolution or Stalin's grand plan for agriculture.  They just know that if everyone would just work in the same way then everything would be so much better.  So off they trot pushing a single solution and historically this was pushed all the way through to production and the business went:

"Well its not what I wanted but its a bit less shit than what I've got"

So IT created grand strategic plans (and I've said before there is no such thing as IT strategy) often in areas that the business really didn't care and off the business went and started using DropBox, salesforce.com and Amazon.

In effect the Shadow IT efforts of the business are analogous to the black market economies that often thrived in communist countries in the 80s.  Getting on doing what they need to do and being a lot more efficient than the state in doing it.  What we are seeing today is that as budget shift more and more towards the business the shadow IT market is getting bigger and bigger and the central planning has suddenly hit an issue.

The business understands technology

 Maybe not in the depth that IT does, but what the business understands is a bit more valuable

They understand how to focus on outcomes that add value, not technology hype.

So now as the Enterprise Architect says "you cannot do that, it is against our policy" the business says "stuff that for a game of soldiers, your policy doesn't work for us.".  The business is having its Berlin Wall moment, and while the IT communist state, the People's Democratic Republic of IT (because communist states love claiming they are democratic) might hold on for a while the reality is that the world is beginning to come crashing down.

Its time for IT to embrace capitalism, embrace value over technology and outcomes over acronyms.

Why your IT strategy failed or why the business hates IT

One of the most depressing things in IT is our inability to learn.  From the 'Oh look our massive waterfall project ran over budget' to the 'I really can't maintain the code we wrote that doesn't have a design or documentation' we do the same things as an industry over and over again.  Most depressing however is the phrase 'The IT Strategy would work if the business would just change'.

To illustrate this I'd like to tell you a tale, the names will not be used to protect the guilty but it sums up nicely the lunacy of many IT strategy efforts.

Many moons ago I was working for the business side of a large and very successful company.  This company was seen as a genuine market leader and I was working on some very complex mathematics around predictive analytics that the business folks wanted to make their supply chain run better.  There were two highlights through the process from the IT department.

The first was when discussing how such a solution would be implemented and integrated into the operational systems.  IT had a strategy you see and by strategy I mean they had picked a couple of vendors, the solution I was working on had some very specific requirements and wasn't available from those vendors.  An Enterprise Architect from the IT department said in a pretty well attended meeting

'It doesn't matter what the business wants, if we say it isn't going in, it isn't going in.'

The project continued on however as the business saw value in it and wanted to understand what could be done.  One of the key pieces was that we'd need some changes in how operational processes worked, not big ones but more we'd change the way people worked within the existing processes by giving them better information.  To this end we had a workshop with the business and certain key IT folks and worked out how we'd have to design the interfaces and processes to work within the current business environment and culture.  It was a good workshop and the business folks were very happy.

Then came IT, IT you see had a big strategic project to replace all of the existing systems with 'best of breed' solutions.  I'd always assumed that given the massive budget for that program the business was fully engaged... then this happened....

One of the IT folks chirped up and said : "We need to have a workshop so we can tell you what your new operational processes are going to be"

Note the 'tell'... to which the most senior business guy there (a board member IIRC) said

"How do you mean tell us?"

IT Guy: "The new systems have new processes and we need to tell you what they are so you can change."

Business Guy:"Have you done an impact analysis against our current processes?"

IT Guy: "No we've just defined the Best Practice To-Be processes, you need to do the impact and change management.  We need the meeting so we can tell you what the To-Be processes are"

Business Guy in a voice so dripping with sarcasm I thought we'd have a flood: "I look forward to our IT department telling the business what Best Practice is for our industry."

IT Guy, completely failing to read the sarcasm: "Great we'll get it organised"

This is one of the most visible examples of my career on why IT strategies fail.  I've said before there is no such thing as IT strategy its the job of IT to help automate and improve the business strategy, that means thinking tactically and taking strategy from the business model.

"Culture eats strategy for breakfast"

This is the reality and an IT approach that seeks to drive over the culture and dictate from a position of technology purity will fail.  You can change the culture, its hard and its not a technology thing, but you always need to be aware of the culture in order to succeed.

IT Strategy, if such a thing exists, is there to make the business better not to make IT better.

The single eye of enterprise architecture

There is a famous phrase

In the kingdom of the blind, the one eyed man is king

IT has always had a problem communicating with the business, and the business communicating with IT.  To fix this IT created something called Enterprise Architecture which aimed to provide a framework around the internal IT estate in a manner that helped that conversation. We can argue how successful that was but what is clear is that with the rising consumerisation of IT and the massive increase in technical literacy amongst business people that this boundary approach to IT with EA as the gatekeeper isn't really going to work.  Historically the EA function was that one-eyed man, it had enough depth in IT to provide some control and enough engagement with the business to act as the leader through the challenges of IT.

The business isn't blind to IT anymore.

So what has been the reaction?  Unfortunately it appears to be in some cases to take a single-eyed view of the problem, a single eye looking through a single lens.  That single eye recognises the increase in sophistication of the business and sees the rise of Value Networks and Business Architecture and thinks 'Oooooh, we have to do the business architecture stuff as well' and makes that a sub-section in the overall encompassing enterprise architecture.

This is missing an opportunity.  A huge opportunity.  EA was created in an era where IT organisations were internal only and where external integration was extremely rare.  In the world of SaaS, Cloud, Social and OpenData however this is no longer the case, outside-in is the norm.

Sure we need groups to looking at integration standards and help with IT operations, but the communication with the business doesn't need an interpreter in the same way anymore. Business Architecture is a rightly growing area but its not a subset of EA, or even an extension of EA its a discipline in its own right, one that owes more to business schools than to IT departments.  So what does IT need to do?  Well it needs to be able to translate that architecture into solutions much more quickly.

IT needs to look at the business with two-eyes again, not try and extend and old concept through its own view of the world.

Architectural Homeopathy

Sometimes when you are in a meeting someone says something brilliant, I had that experience today when someone said 'We have a sick body, can we please stop pretending everyone is a surgeon'.  Her point was simple, historically in the company they have had challenges of people having opinions and critically of decisions being made without data and whose implementation success isn't tracked.

I talked about 'Thinking is dead' previously and this is a clear manifestation of that and in the world of IT I'd like to give it a name... Architectural Homeopathy in other words people making IT decisions which

1. Are based purely on an individual opinion on what will succeed
2. Are not backed by a case as to why it will succeed
3. Whose implementation success is not measured (beyond 'it went live')

This approach also has another aspect which is related to comments and challenges to recommended practices.  A good architectural challenge is to say that 'X won't work because we don't work in a centralised way, we need to do Y because we are federated and Y has already been proven to work here', the Architectural Homeopathy challenge is 'X won't work, we should do Y'  or more likely just 'X won't work'.  No evidence will be proposed to support this 'feedback' and no constructive change can be based around it, but if any issues occur the Architectural Homeopaths will say 'You should have done as I said'.

Architectural Homeopaths often build whole careers out of this, building approaches based on 'it worked for me' and failing to see the broader elements that drove success or criticising a proven approach based on a perceived technical flaw while missing the real problem (I'd argue that REST for enterprise integration is a great example of that).  These are the folks who propose great views of architecture without actually having delivered it themselves into operation and coached others how to use that approach.

The point here is that Architectural decisions need to be defensible based on data, even if it turns out to be wrong that gives you the ability to learn.  Proven approaches with justifiable approaches that are measurable against a set of criteria are what professionals should strive to do.  'Advisors' who promote approaches and cannot demonstrate their success and more importantly how others have adopted their approach and delivered success should be treated the way a Homeopath is treated by the medical profession.

I have no objection to the phrase that coding is an art not a science, that it is a creative endeavour not simply mechanistic.  That is fine, but unlike art its success is quantifiable, it either works as intended or it does not.  What I object to is people promoting architectural approaches (or indeed business approaches) which are based on a series of powerpoints and opinions with all of the real support evidence of Homeopathy.  These Homeopaths throw in comments based on this ignorance and personal 'belief' and disrupt progress and love to claim that 'it would have been better my way' while not explaining in detail what their way really would have entailed.

Architectural Homeopathy... get into the sack...

Getting lost at the EDGE

Speaking with a business & commercial chap today he had a bit of a moan to me about architects worrying about edge scenarios. His complaint was that they kept trying to consider all of the edge conditions and then design around them when in fact he

a) didn't think those things would happen
b) even if they did happen he'd be fine for the system to collapse
c) thought the investment the architects had made in edge conditions was a waste of time

The point here is that sometimes its okay for a system to fail and also its okay to specify what a system won't handle rather than trying to make it handle everything.

Now some architects may leap forward and say "well what about the future, what about extension"

But you know what? If the business doesn't want to pay for these edge conditions then why on earth are you bothering? Sure you should force the point and say "the system won't handle X,Y,Z or pigs learning to type in ancient greek, is that okay" rather than "The system doesn't handle X,Y,Z or greek typing pigs so what we should do is redesign the keyboards to enable pigs to work with the system when they learn to type".

The sort of architectural "perfectionism" that underpins this mentality is, IMO, one of the worst traits of architecture, namely the avoidance of getting into the solution. By putting these edge cases in the way the architecture and design phase takes longer and the dirty solutioning piece when the architecture actually has to be proven can be postponed.

Edge conditions can be non-functional as well "What if your carbon blade procurement system is turned into a Facebook app and gets 10m hits in an hour?" The answer is of course "That won't happen, and even if it did we sell carbon blades to the airline industry so why on earth would we worry about a Facebook app?". Still architects will argue about the "most scalable" way to build the solution that is currently targeted at 5 end-customers with up to 20 concurrent users max.

Sometimes architects claim this is them being "professional" and considering the "future" while taking a "holistic" view. What rubbish, its about architects focusing on architecture and losing track of the big, and simple, picture and merrily pursuing their own mental exercises to demonstrate their architectural prowess.

Edge conditions can be hard to deal with, but that doesn't mean you HAVE to deal with them. Often, in fact most times, the right approach is to declare them out of scope and make it clear what the system won't do as much as what it will.

Technorati Tags: SOA, Service Architecture

We're all going on an architecture hunt

I'm not scared, we can't go over it, we can't go under it we'll have to go through it.

Duane talk about Forensic architecture. Its a great read on the lessons that he has learnt. The only thing I'll add is that I don't think we are CSI, we are more Bones, hell its pretty much Jurassic Park out there. Documenting architecture after the fact, as Duane says, isn't so much the exception, its the career.

Technorati Tags: SOA, Service Architecture

Why people matter more in architecture than technology

I've said before about Ivory towers v long term but I thought it was briefly worth bringing it up on the back of the Convenience v Correctness piece.

Correctness is normally used to describe the technology side, solution X is more "correct" because it obeys more rules or exhibits better technology pieces. The goal in this mindset is to be as correct as possible, rather than being good enough for he job at hand.

If architecture is to be about delivering what the business wants then there are three things that should be at the front of an architects mind

1. Is it good enough
2. Can my team deliver it?
3. Can we support it?

These are the three primary elements to consider, not whether the solution is "the most correct" or whether it represents the "best" technology but whether what is being proposed is good enough and whether the people that are going to have to cope with the proposed solution will be able to do so.

Convenience over Correctness therefore should mostly focus on convenience as that will help deliver something at a better business price and which more developers (read cheaper developers) will be able to deliver and which more people in support (read cheaper support) will be able to maintain.

Architecturally it is the convenience that represents the best choice even though technologically it is not the correct choice. If I have a team of VB 6 developers and have to build a distributed multi-threaded application then the technologically correct choice might be to architect using REST and Erlang, however the business correct decision would be to architect around the limitations of the team and the platform to enable that team to deliver the solution. Yes it could be argued that hiring 20 Erlang guys with REST experience could have got the job done quicker head-to-head, but not if you factor in hiring time, recruitment costs and the increased salary costs and this was also not the challenge that the architect was presented with.

Architecting from the Ivory tower is easy. Assuming everyone is as smart as you are is easy. The real skill is in understanding the complexities and picking the technologies and approach that is most convenient for the people you have and good enough for the problem being solved.

If architects are to change the perception of IT then we must focus on business objectives over technological correctness. Brunel was right that the "correct" gauge should be wider as the previous one was merely a convenience. That was the engineering and technology view, the business view was that standardisation and the majority was the correct view for economic development.

In delivering business solutions it is the business correctness that matters over the technology correctness this means focusing on the people available for delivery and matching their skills to the architectural proposals and focusing on support to ensure that it will operate effectively.

Architecture should be about making it work in reality, not making it work better on paper.

Technorati Tags: SOA, Service Architecture

Is REST architecture or a design pattern?

Mark Baker commented on a presentation by Roy Fielding that claimed that "SOA was a null architectural style". Now part of it was that Roy seems to think that SOA = WS, which is wonderfully naive, but a bigger bit is the idea that REST is an architectural thing.

Is it? After all what is the goal of architecture in an IT context? Surely its to represent what the business wants to achieve. Looking at the Wikipedia definition of Enterprise Architecture for instance it says

Enterprise Architecture is the description of the current and/or future structure and behavior of an organization's processes, information systems, personnel and organizational sub-units, aligned with the organization's core goals and strategic direction.

And System Architecture goes down as

A system architecture or systems architecture is the design or set of relations between the parts of a system.

Now this is what I've always thought of as being architecture in an IT context. Even something very very techy like ADML describes software architecture as

A software architecture describes the structural properties of software, typically the components and their interrelationships, and guidelines about their use.

The key bit here is it describes the OPERATIONAL pieces (components and interactions) rather than the implementation mechanics.

Now I don't think this fits what REST is at all. REST deals with the implementation mechanics and the principles and practice of that implementation, it doesn't provide you with a style for the modelling of an architecture and its interactions, it provides you with a framework for its implementation.

In his presentation Roy uses the phrase "Architecture is trying to find the common design" before going to talk more about architectural style being more abstract that that. Now the issue here is that in architecture he talks about implementation protocols as being the key to the Web "architecture". REST then becomes one level above that talking not about the protocols but about the "style" of the Web. Now this really got me thinking. This is a long way away from what I mean when I talk about business architectures and it really is about implementation practice... then it dawned on me, this isn't an argument about architectural styles, in fact REST isn't an architectural style.

REST is a design pattern

In software engineering ,a design pattern is a general repeatable solution to a commonly occurring problem in software design. A design pattern is not a finished design that can be transformed directly into code. It is a description or template for how to solve a problem that can be used in many different situations.

Now that to me sums up exactly what REST is, and how the Web is the implementation of the REST design pattern, and explains how the REST design pattern can be used to implement your projects and programmes. This doesn't make REST any more, or less, relevant to decisions but for me it explains the different between SOA, as a business driven concept, and REST as an implementation model for that approach. It is fair to say that Web Services has a lack of patterns and practice and that this is an area that should be addressed in terms of Web Service implementation patterns (and anti-patterns), but both REST and Web Services are fundamentally about technology and implementation.

Architecture needs to be about understanding the goals and objectives of the business problem being solved and about the effective modelling of that solution. This then needs to be translated into technology by creating a software architectures which are based, or constrained by, firm design patterns that ensure repeatability of solutions. Confusing the goals of architecture by making it focused on the technology implementation really doesn't help us bridge the gap between IT and the business, indeed it reinforces the gap by emphasising the IT focus as being on technology rather than on the business problems.

REST may well have a place within the implementation of an Enterprise or Solution Architecture, but we shouldn't confuse the goals of architecture (business problem) with patterns (IT implementation), and REST belongs firmly in the later.

Technorati Tags: SOA, Service Architecture

CYA - or the art of doing bugger all through paperwork

Cover your arse (or ass for those in North America, although why you cover donkeys I don't know) or CYA is one of the most crippling inhibitors to change in any organisation. Often dressed up in the weasel words of "considering all the possibilities" it is all about making any job seem so much bigger than it actually is.

IT, in particular Enterprise Architecture and IT operations, can be the major creators of CYA within a project or organisation. Asking people to consider every potential outcome, and embarking on long studies to consider the full long term impacts on the Albanian watch industry of that SAP upgrade or application to support the marketing campaign.

These people delight in the "what if" scenario and fight back against suggestions like "There aren't even 10 billion people on the planet" by accusing the rationalists of being unprofessional and just trying to get things out the door without the proper controls.

You can tell organisations that suffer from this problem because they tend to be burdened down with paper, paper that is very rarely read even once after being written. Projects have to complete lots of different assessments and studies to get anything working, all due to the fear that if something does go wrong that it will bounce back. Because after all if you do all the paperwork and still produce a crappy system at least you can prove you did the bad work according to the process.

During projects this comes down to project managers asking for daily status reports, Enterprise Architects asking for "justification documents" on technical decisions that make you think about "how will this work in 5 years time" while operations ask for a full set of documentation, training courses, security audits, penetration tests and the like before you can even speak to them.

Now some people will howl "but you do need to think about these things" and to that I say "sometimes you do, sometimes you don't" and there in is the problem. IT organisations rarely think about what is appropriate for the job being done. In the same way as IT sets up projects to be rubbish in support so the CYA culture makes sure that everything moves at the same snails pace.

What is the solution? Well in part its about more formalism around specifications so you can get rid of all of these reports because you can say "look the SLA on the service says 99.99 availability and here is the test suite for it" and "look the Security Policy says that all Albanian Watch sellers are banned from using this". This increase in formalism means a decrease in the amount of documentation required because these elements can be used to enforce the boundaries of the service and highlight when those boundaries are in danger of being breached.

The other part is for IT people to be more professional, so not producing crap software and projects, not having documentation so poor that support can't use it and not continually releasing things that don't scale to the market. The CYA crowd also needs to become more proactive in determining what success is for a given element and then applying the right set of principles and audit to that solution rather than a blanket one size fits all.

Formalism and Professionalism, is it too much to ask?

Technorati Tags: SOA, Service Architecture

Enterprise Architecture - a tool not a destination

There is a scary thing going on in IT at the moment, its around Enterprise Architecture, now I've always maintained that There is no such thing as IT strategy but I'm getting worried that Enterprise Architecture is starting to be turned into a goal in its own right. There are some good reasons to have an Enterprise Architecture function, it helps to provide consistency and governance. The challenge however is when Enterprise Architecture starts trying to lay down 5 or 10 year plans, and even worse when it starts trying to make technology based decisions today based on those plans of tomorrow. As Sam Lowe put it to me the other day. The goal of Enterprise Architecture should be purely at the IT level, it shouldn't be at the technology level.

Now I have a slight problem with Sam's idea in that IT does actually mean "Information Technology" so clearly IT isn't abstracted from technology. What I think Sam was trying to say however is that EA isn't about solutions its about the general goals that IT should have within and organisation. I'll go along with that as an approach. There is a rule of thumb I've always used: Plan for a year, have a goal for 2, a target for 3 and vague fag packet stuff beyond then.

The problem is that EA is beginning to be seen as the goal in itself, so Enterprise Architects lay out the "vision" of where the organisation should get to in 5 or even 10 years time and create huge amounts of documentation in which to detail this end state. Simply put that is a massive waste of time and effort and something that will only impact in a negative way on the projects being done today.

Enterprise Architecture should never be the goal, it should aim to guide and steer and help to select the good from the bad. This should not be done against some grand IT or technology vision but should be done against the business value that elements will derive. The reality is that this will change significantly over a 5 year period, let alone more, and so any decisions that are made today on the basis of progression towards the mythical enterprise architecture are going to be an increased burden which is done to achieve something that won't happen.

Note here I'm talking about enterprise architecture. If you have a 5 year programme of work to deliver against a new business area or goal than that is fine, because that isn't an enterprise thing, its a solution thing.

Enterprise Architecture isn't a goal, in the same way as IT strategy isn't a goal, the point is that the business changes and IT needs to adapt in line with the business and someone needs to herd the cats to make things consistent. This is why the key skills in EA are not the creation of big documents that try and define some vague future but the ability to influence people and to help them reach the right choice for their current solution that also makes sure that the next thing down the line isn't going to be screwed.

Technology changes far too fast for anyone to make bold ten year plans. Back in 1997 some companies hadn't caught onto the concept of the internet, I even interviewed with an IT company back then who didn't have email that went outside the company! Anybody who attempted an Enterprise Architecture with a 10 year goal back then would almost certainly have looked like a muppet after only 3 years, and I doubt that the rate of IT development has slowed since then.

Enterprise Architecture is required, its required as an active day to day part of the operation of how IT operates, it should continually evolve and iterate the "to be" state of the IT estate in line with the demands of the business and it should never be allowed to start defining goals that sit way beyond the technology horizon of today. Enterprise Architecture is not R&D, its meant to be a practical solution to ensuring that solutions are consistent. Word documents rarely help make anything better, normally they make things worse, this means that EA has to be about active participation and direction of projects and programmes and about the people and communication skills that this requires. This also means that while EA is about the general cases and direction of IT that it must understand the technologies of today and those that will soon be arriving. Without a proper understanding of technology it is hard to see how EA can ensure that pieces are progressing as required.

Technorati Tags: SOA, Enterprise Architecture