My way or the highway - anti-pattern

Description
This anti-pattern is about delivery processes, given a major goal of SOA is to enable business flexibility its amazing how organisations still insist on having a single delivery process for every type of delivery, or a best two approaches, one for package and one for the rest.

Effect
The effect here is that there is no link between the business drivers and requirements and the type of delivery process employed. No matter whether agility is required or a single compliance driven delivery its always a single process that is applied. This means seeing package projects capturing requirements rather than doing a best-fit analysis, it means seeing agile processes where a vanilla package implementation would be better and it means seeing software development projects doing waterfall because that is what is used on vanilla package implementations.

The end effect is that while the services might map well to the business objectives the delivery process prevents this being realised in a way that matches the way in which the business wants the SOA to be delivered. This gets SOA a reputation as either expensive (e.g. when packages get over customised) or slow (waterfall on a software development package).

Cause
The cause of this tends to be an over reliance on project management in the IT department and a fixed approach. The use of Waterfall across the board is often driven by the finance departments approval processes which mandate that everything must be known and priced before a project can start. Often these fixed processes are part of a broader "quality" initiative where the quality approval people find it easier to have a single process to audit than understanding and coping with different ways of working that actually deliver more quality

Resolution
The resolution is to understand the differing business and technical objectives that business services have and to tailor your process appropriately. This means increasing the ceremony where you have complex contracts and external interactions, it means doing waterfall when you have a vanilla package implementation and are going to change the business to fit and it means having co-located teams when you want true agility and pace of change.

This means convincing the quality people, or convincing the business that the quality people are getting in the way of quality. It also means shooting the sacred cows in the project management organisation and investing in the training and mentoring required. It also involves explaining to finance how different financial approval models make sense in different areas of the business. Its a whole lot of explaining but there is thankfully a whole lot of data out there on how single process approaches tend to destroy IT estates.

Technorati Tags: SOA, Service Architecture

DOA - antipattern

Description
The opposite end to the percolating process anti-pattern this is the one where people assume that data is the centre of the universe.

Effect
With this anti-pattern you see data being driven as the centre of everything, services are all related to data processing and the first set of services being defined are about "key" data objects in the organisation. The actual actions and capabilities aren't captured its all about the data so the Customer service has a series of CRUD methods on it, but nothing that captures how a new customer is really added to the organisation. The impact of this is that it becomes an abstraction above the database but can lead to data silos being created, it can also lead to very inflexible architectures especially where these data services try to include both the "live" and in-process data as well as the post transactional data.

These architectures match nicely to reporting systems but badly to active goal or process oriented areas of the business.

Cause
The cause is pretty simple, its because people have started to do SOA from a Data Oriented perspective and not noticed the difference between the letter S and the letter D.

Resolution
The resolution is pretty simple. Keep the data services as a post transactional base for data, effectively these become the reference data for your systems. Then do a proper mapping of the business services and use these data services as support services and to help communication between different areas of the business, for "live" data keep it local to the service doing the processing and only submit back to the support data services once processing has completed.

Technorati Tags: SOA, Service Architecture

Selling SOA

There is a conversation that I've had close on a hundred times now in the past 8 years about how to sell SOA to the business it goes like this

Business: Why do I want to do SOA? Isn't it just another IT acronym
Me: Do you understand how your current IT estate works?

B: No
M: Would you like it to look like the business?

B: Yes
M: Then that is what SOA is about, its about making the IT estate look like the business, evolve like the business and be costed in line with the business value it creates

Then you show them what the Business Service architecture piece is and how you will create a heatmap for them of where IT adds value and where you should be moving towards Utility (OpEx) spend and now you have a business person bought in and a goal on what the IT estate and its supporting organisation should look like.

Its not rocket science and it has nothing to do with technology.

Technorati Tags: SOA, Service Architecture

Yes Sir, No Sir, three service views full sir! - Anti-pattern

Description
This is where the time isn't spent to get a clear business service architecture and get the business bought into the approach and where IT doesn't have the confidence, or maybe the ability, to effectively negotiate with the business.
Effect
The effect is that the service definitions are changed on a regular basis, sometimes its just about a data field, sometimes a new capability and occasionally a complete re-writing of the service interfaces. Every meeting that happens results in changes at the interface and service level and the leads to a fracturing of the overall architecture around business silos. Eventually the whole SOA programme is canned because its become too complex to maintain and isn't delivering any benefits.
Cause
The Cause here is IT going it alone and defining a strawman and, unlike IT2B, not having the knowledge or confidence to enforce or structurally modify it. In IT2B the issue is the IT department thinking it knows best, here the issue is the IT department thinking it knows nothing and therefore must always say yes. Often this is driven by a lack of good communicators in IT and through a view from the business of IT as purely an "order taker" rather than a partner.
Resolution
Organisations that have this problem need to drive SOA as a cross-boundary business programme with IT providing support. IT need to invest in highly skilled communicators who can work with the business as partners and help bring together the differing perspectives. At the heart of this issue is one of confidence, communication and ownership, the solution is to make the business the owner and invest in the communication and confidence side of IT.

Technorati Tags: SOA, Service Architecture

ADHD SOA - Anti-pattern

Description

ADHD SOA is where, normally, a group of architects continually shift their mind about what "good" looks like from an SOA perspective.

Effect

This is all about continually "refreshing" the technology stack. On the technical side it means that rather than focusing on getting things into operations and industrialising their approach the architects instead concentrate on the upfront elements, especially the first few weeks of requirements and development and look for ways to continually shave fractions of effort or add additional features.

Both of these result in lots of unneeded work and an increase in the complexity of the IT estate under management, they slow down progression while generating activity.

Cause

The cause tends to be a focus on design and development time optimisations. Unlike the Shiny Nickel pattern which is always about the latest buzz the drive here is always to be different to the last project phrases like "we should try X out" ring around. The most common issue here however is a lack of measures, various different pieces are tried and compared based on personal preference and then combined together on the next project "lets try X with Y but not Z this time". A lot of this comes from vendor product upgrades, after all if you've paid for it then you should use the new features. This isn't driven by industry buzz-words but just pulled along by availability of options and a desire to try new things, often you will end up with bizarre cases where people are pushing using very old technologies along with the latest versions because "we know this worked in 2002" or similar justifications.

The basic cause here is a lack of focus on industrialisation of the delivery process, a lack of measures to demonstrate impact or improvement and a complete disregard for the testing, deployment and operations part of SOA.

Resolution
So how to fix it? Well first off make people really care about operations. Give the architects a target for operations. If the cost of operations doesn't go down then they aren't meeting their goals. Next up get some proper measures on how long it takes to get new people up to speed on a project and how much of the project is automated. Set the architects the task of improving these metrics. The ADHD approach tends to result in an increase in the time it takes to onboard people as its always new to everyone.

Technorati Tags: SOA, Service Architecture

SOA Anti-patterns revisted - Part 1

Back in 2006 I wrote an article, with some help, at InfoQ on SOA Anti-patterns and I thought it was about time to revisit and extend them.

The format again remains the same

Each anti-pattern presented in this article will follow the following format:

• Description - What it is?
• Effect - What you see?
• Cause - What behaviours led to that effect?
• Resolution - What can be done to solve the problem?

First off I'll revisit the old patterns to see if they are still valid.

Antipattern: The Shiny Nickel

Hell yes, people are still lobbing in new technologies without worrying about the consequences so they can get it on their CV.

Antipattern: The Technology Altar

Again its a Hell yes. Some of the REST preaching out there is right out of the technology altar book.

Antipattern: Percolating Process

Again its something I see over and over again. People are still saying things like "Process on top of Services" and I still see people mapping out processes and then calling activities "services" and claiming SOA.

Antipattern: Point to Point Web Services

Its still an anti-pattern but I'll admit that its one that is happening less and less. People are doing ESBs to avoid Point to Point and the REST crowd are claiming the dynamic nature of the URIs combined with DNS.

Antipattern: Splitting Hairs

Again its about splitting process and services, some people are even going further and advising you split further into data/service/process. Its still an anti-pattern and its still being done over and over again.

Antipattern: IT2B

Not really an SOA anti-pattern but an IT organisation pattern and its certainly still a problem.

Antipattern: DIY Transport

Its an anti-pattern but I've not seen someone do this recently. The ones who did this before are now leaping on REST as their transport, I'm still not sure that its any better than the custom XML-RPC approach of before as it still gives a completely unstandardised application interface. But the problem now probably isn't at the transport level. So if you are doing this you are seriously behind the times, its beyond an anti-pattern now and into the category of reasons for commital to an asylum.

Antipattern: Nobody Home

Are people building services without thinking about how they will be used? Oh yes they are. Some people have even talked about not being concerned about the interface design and that "you should at least start somewhere". Well you shouldn't start by making up services based on a set of perceived rather than actual needs.

Antipattern: Too many Cooks in the SOA

Yup still valid.

Antipattern: UBER service

I've not seen this one for 12 months or so, but its still valid.

Antipattern: A Million Services all in a row

Still seeing this, the focus at the edge, build up the volume and "they will come". Muppetry

Antipattern: Architectural Stovepipe

Big strategic views from Enterprise Architecture that result in hard to deliver solutions... almost a best practice in some places it appears. Its still an anti-pattern.

Antipattern: Defensive SOA

Getting worse this one. More and more companies are "already doing SOA" and when you look its the same thing they did before but with a software upgrade.


So to my mind they are all still valid but some of them are less of an issue than 2 years ago. Now what are the new SOA anti-patterns?

Technorati Tags: SOA, Service Architecture

DOA your SOA?

I like a lot of the stuff that Dave Linthicum writes but I don't agree with his latest post on why people should start with the data when doing SOA.

First off lets be clear, data is important its one of a few really important things in SOA

1. Services
2. Capabilities
3. Real World Effects
4. Data

Now I've deliberately but them in that order because that is the order I think is important. I've said before about SOA v POA and how I think you know that you are doing SOA. The key point on this is that a stack based view of

But the fact is you need to start with the data first, than work up to services, than the agile layer (process, orchestration, or composite). If you follow those basic steps you'll find that the solution is much easier to drive.

just doesn't work for me. It first of all implies a technology centric view of SOA that I don't agree with and secondly places services as just an intermediary between process and data, with the goal surely being to do more and more in the "agile" layer (although those of us who have done large BPM and composite projects probably aren't sure about the agile bit). This stack based view isn't going to progress IT, it might help with a single project but surely the goal of SOA is more than that?

To me this plays into the technology centric change view of much SOA advice out there. To me data is important but it is only important where it works this means understanding the services first.

Now if Dave is talking about having done the business services how he'd look to realise them with technology and that he'd have a single business governance piece over the service dealing with the data, process and other elements then I'm with him. But starting with data for your architecture.

To me that means you have a Data Oriented Architecture, and I can't see anyone selling to the business that the next project is going to be DOA and the goal is the whole IT estate to be DOA within 5 years.

So start with the business services, understand the business capabilities, understand the real world effects and then understand what data is being exchanged and managed to deliver that. Data is important, very important, but if you are taking a business view of modelling SOA then its just one of the technical artefacts, not the major purpose for existence.

Technorati Tags: SOA, Service Architecture

Vive la France?

Okay so recently I've read a bunch of articles about how France is becoming relaxed about the rise of English. It appears however this is just a front to make us relax so we won't see the real goal.

Unfortunately I've unearthed this plot via Apple, looking at the MacPro and going through the "HOW much could I spend on a PC?" test. I saw the following

See it?

What about now? Seriously its there in black and white

Yes you get a British keyboard, but you have to learn French to understand the instructions. This is a truly nefarious plot.

Now this actually has a good point from a globalised project perspective. You can't assume what language people are going to speak so you need to be very clear about what you are going to support (i.e. keyboard in English, but instructions in French) otherwise you can end up in a hell-hole of trying to support every language in existence.

Fortunately English in general remains the lingua franca ;)

Technorati Tags: SOA, Service Architecture

Fibonacci - word of the week

I haven't rolled this into the project yet but its something that I've done before to raise the sense of community. Basically you pick an unusual word, but not too unusual, that people need to get into either a conference call, document or email. The word must fit within the context of the communication, not just a random shout out, and the non-included people on the call must not notice.

Today I decided on the word Fibonacci to test out the theory again, this was placed into a call with the phrase "we are starting from 1 and with any Fibonacci or exponential growth that means its about getting the next one".

Its the sort of little thing that helps bring people together and works very well in small teams within large scale programmes. Don't make it ridiculous or offensive just make it so its a word that could be used but requires some elegance to get it into context.

Technorati Tags: SOA, Service Architecture

Will the Credit Crunch kill the hype?

At the Netherlands Java Night 08 I moderated a discussion on Java futures and chatted with a bunch of people doing real, and large, Java projects. The overwhelming plea was "please stop the hype and the crap, lets make it work properly first". Now with the credit crunch this got me thinking about what a credit crunch could mean to IT spending.

Now apart from the obvious of more outsourcing there is the impact on technology selection, namely not as much new product being bought and a focus on the TCO more than just the development time. Now this could be a very good thing for SOA programmes out there as it helps them focus on what really matters and away from technology bells and whistles and helps them to upgrade slower and thus have more consistency and thus reduce their governance and overheads.

The other bit could be its impact on things like scripting and the various other language developments going on at the moment. These are great if there is loads of cash to spend on investigating new things but not so good if you are interested in keeping your TCO down (which is aided by consistency more than by technology) and don't want to spend loads of cash training people in something new.

Clearly the credit crunch isn't a good thing, but there could be some advantages in terms of its impact on IT spending and in particular its impact on practicallity over hype. Less of the big software purchase and random building where there is no value and more of a focus on repeatable and measurable reduction in TCO.

I can hope anyway.

Technorati Tags: SOA, Service Architecture

IT Hype and the valley reality distortion field

One of the problems that I have in my job is the problem of IT Hype, most especially the huge distortion field that appears to fall over Silicon Valley. When I'm working with companies who are going through a procurement phase its amazing to see this field spread across the globe via the blogosphere. Its the sort of field that tends to ignore companies like IBM or SAP because they just can't be encompassed within this idea that everything must come from the valley. So how do you get around that problem and make sure you make a smart decision?

Now its certainly true that the valley does have a disproportionate impact on IT, its a pretty small strip of land to have given birth to the likes of HP, Sun, Oracle, BEA, eBay and Google. That is damned impressive. The problem is that when you are looking for really good small companies to work with I'd argue that the bullshit to content ratio is never higher than in the valley.

Recently I've been looking at companies in a few spaces recently, which unfortunately I can't go into, and I've started to come up with a valley weighting system of how to score a company before you even go and see them, this is for the small guys not the established guys

1. Current Revenue
2. Current Customers

These two numbers give you the base line for the company, divide revenue by customers and this gives you an average spend, what you are looking for here is companies that appear reliant on a single customer. i.e. if they are talking to you about $40,000 spend and the average is $100,000 then either everyone loves it, or they've got a big marque client who they care the most about. Very common in finance this problem.

1. Projected Revenue
2. Projected Customers

This gives you where they are expecting to get. Again divide the two to see what the future spend per customer is, also look at current/projected for revenue to see the growth path.

Now we factor in the valley factor

1. In the valley - Bullshit factor = 5 year growth projection * 0.5 (i.e. if its 10x growth in 5 years, bullshit = 10)
2. West Cost - BF = 5 years *0.3
3. Mainland US - BF = 5 years * 0.25
4. Europe - BF = 5 years * 0.2
5. India - BF = 5 years * 0.15

This gives you a basic financial BF which should then be weighed against the hype. Basically for each post on Digg or Slashdot add another 0.1 to the BF. Then look at the length of time the company has been around and not cash positive, add 0.2 BF for each year, finally look at the length of time the company has been around and not grown at similar rates (i.e. they are predicting a hockey stick graph after ten years of solid 5% growth), add 0.3 for each year of growth that just doesn't fit the projection.

The final BF then gives you the amount of convincing you should take in order to believe anything they say. If they have a BF of 1 then they need 1 reference for every point they make. If they have a BF of 5 they need 5 references.

Part of the reason for doing this is that these new companies often look shiny and its important not to get caught up in the hype. As an example of hype v reality its quite funny now to look at transmeta and ARM now. Back then in 2000/2001/2002 they were the darlings of the Slashdot crowd and when I went to the valley the talk was about how they were going to completely dominate then chip market. Meanwhile a tiny company in the UK which had been doing chips for years and growing strongly continually had made a move into licensing IP and moving into the mobile phone market. Now in 2008 their results look a bit different, revenue for transmeta in the 2nd quarter failed to break $400k revenue while expenses nearly hit $2m, ARM meanwhile struggled by with $128.1m revenue and a healthy pot of cash.

What has this to do with SOA? Well really its to do with T-SOA, WOA and all the other things out there. When you are looking at companies as part of a long term strategy you need to know that they are going to be around in 10 years time and still helping you evolve, otherwise you are doing something as a point solution and will accept that support and maintenance is your own problem. You also need to figure out whether that wonder technology really is amazing or is just some empty hype, the more amazing the claim, and the growth predictions, then the more evidence you should be asking for.

The other point is that what you are saying is that if you think their business model doesn't stack up but the technology does that you will need access to the technology when they go bust. This is why lawyers invented escrow if you want the tech but don't trust the model then make sure you get the rights to access the code (or other tech) for your own purposes if (when?) they go bust.

This is part of that shift about moving IT away from technology and into a business frame of mind, it means not following the hype but following the numbers.

Technorati Tags: SOA, Service Architecture

Be clear - document

A number of years ago I had a period of time working at a smaller company where I found myself up against the massed ranks of the "big american consultancy" (BAC) on a number of projects. Now clearly they didn't like me as I was doing some high value delivery work and might have occasionally pointed out some minor flaws in the way that they were working. Now what this did teach me however was the following

1. Don't delete any emails
2. Always send a summary email after a meeting even if it isn't full minutes
3. Document decisions - if someone decides something then send them an email confirming the decision
4. Point out the obvious - if someone changes their mind. Point it out to them. If they are a senior stakeholder then do this one on one. Even if they do change their mind then at least they'll feel that they owe you one. If they are competing against your project then make it visible to everyone.
5. Don't rely on a conference call to get you the status. Build up a social network and trust to the water cooler conversations
6. Go to the raw data. If its about development then do code reviews, if its about data warehousing then go to the data quality and volumes, if its about user interfaces then go to the usability studies.

Now what I'm not advocating is lots of massively verbose documents, in those lie doom as the detail is where the weasel professors can live. The important piece here is making documentation clear and small. One notable experience I had with a BAC was where they refused to do Unicode as it was "too hard" for their mainframes (this is despite pointing out that UTF-8 is basically an extended ASCII set) so we went to the standard ASCII set. Testing began and the data had what looked like typos all over the place. Clearly this was an integration (my) problem. But thanks to the very brief documentation we quickly proved that they were sending an 8 bit character set through when ASCII is 7 bit. Oddly the decision was then to move to UTF-8.

The point on documentation is short, sharp, concise and irrefutable. Its there to make decisions visible and obvious and make non-adherence to those decisions an act of wilful stupidity or disobedience.

As a client said to a BAC on one of my more entertaining projects.

"Are you a complete idiot or an asshole?" (it was an American project)

Documentation isn't there to hide behind (CYA) its there to document facts and Just the Facts.

Technorati Tags: SOA, Service Architecture

Its about WHAT not HOW

A shout out to Todd Biske with a brief post on his concerns on the current REST/WS/Monkey debate. I completely agree.

Lets focus on the WHAT not on the HOW. We've tried focusing on HOW for decades and its one of the things that has got IT into the mess it is.

Technorati Tags: SOA, Service Architecture

You don't need an SUV - keeping the scope down

Okay so Gerald has been going a month now and we are into that dangerous stage where people begin to realise what the project is doing. Suddenly they want more, lots more. Rolling out to 30,000 people isn't enough, why not roll it out to 500,000? Why not have amazingly complex security requirements and some AI?

Pretty much all of these have now been suggested on the project and the mantra I respond with is

do one thing well

What I mean by this is that we don't want to boil the ocean at this stage, we want to prove that the solution works for a, pretty large, user group and then build on that success. If we start lobbing in more requirements and expanding the scope then the project will be delayed or fail and we won't achieve any of the aims. Once we have that first piece done then its open season on what is next, but right now lets keep it concise.

The most important phrase at this stage is "Phase 2" as in "we'll add that to the requirements for Phase 2". The reason is that 90% of Phase 2s never happen and the 10% that do look completely different to what was being asked at the start of Phase 1. Smile the California shop assistant smile, give them the feeling that you care, and then file the thought away in the bin of "future requirements". People will bleat and moan that its "required" but the reality is that its better to have something doing 80% than nothing doing 100% and many of the requirements you hear at this stage are "SUV" arguments.

An example of fake creep in the real world is the SUV. 99.999% of SUV owners have absolutely no reason for owning one, but they come up with "requirements" that end up with them buying the SUV.

• It snows around here sometimes
• I drive up hills
• The roads are a bit rough
• I might go off-road
• I want to feel safe

Now all of these are rubbish reasons to own an SUV. Unless you are a farmer or equivalent, and its amazing how many of those don't buy a Porsche Cayenne then these are all fake requirements. In the French Alps you will see people with a Renault Twingo doing fine in the snow and as for the rough bit and the hills... well I'm here in Bangalore today and the roads could be described as "patchy" at best and everyone has a sub-compact.

The point is that your project should be a sub-compact. It works, it gets from A-B and you can get 5 people in at a squeeze. Building an SUV for your project means adding in gadgets, weight and features that take you time and money and at the end of it you are just shifting those 5 people in slight more comfort.

The smart move is to build the basics and then look at what actually creates value. Say "no" to all of the scope creep and keep it simple. My goal is to have all of my user groups equally annoyed that they are only getting 80% of what they asked for, but for that 80% to be 100% useful.

Technorati Tags: SOA, Service Architecture