Ranking the SOA vendors... Starting with the Big Boys

Update: There is now a 2nd attempt at this that I'm happier with.

Like everyone else out there I've read the various analyst reports on SOA stacks, visions, products and all of those elements. So I thought I'd have my own crack of the whip at rating what is out there, the following assessment is just my view based on using the products, talking to the vendors and should be taken with the same pinch of salt that all analyst reports should be accompanied with.

Rather than rating every product I've decided to split it into four groups

1. The big boys - SAP, Oracle, Microsoft, IBM
2. The pack - BEA, Sonic, Sun, RedHat/JBoss
   
3. Going to be bought or die - the rest

The last group, and some could argue if Sonic should be in there, are those elements which are either at the cutting edge or are for people who (IMO) are missing the boat. I'll start with the big boys.

Group 1 - Rating the big boys - Today

These are the companies where you are probably going to have some of their SOA suite whether you like it or not. You've bought the ERP, welcome to the SOA type of thing.

First Place
IBM - Clear and out there by a mile and a half.

Strengths:
SCA, SDO, ProcessServer, SOMA and a decent set of research creds, IBM are the intellectual leaders in much of the SOA space. Thought-leaders in the standards space. They've got a full stack of technology to match pretty much everyone.

Weaknesses:
The only criticisms I'd level at them is not being totally honest on product life expectancy, I mean two years ago WBI Server with Crossworlds and MQWorkflow was a vibrant product, now it makes CICS looks cool. The "Advanced" ESB that is MQSI (possibly the most re-named product in the history of IT) can't be far behind. The other element is when Rational will catch up with SOA and when IBM will actually have an SOA tool rather than just process oriented tools. Currently there isn't really a joining vision like ESA or Fusion its all a bit disjointed.

Need to:
Be more honest about what they are doing. Get a real vision for the future and drive everything towards that. Need to admit there is a level about WBI Modeller that they just don't have yet.

Runner-Up
Oracle - Hey look, technology that can work with multiple vendors.

Strengths:
Oracle's strategy of buying smart has given them a stack that is heterogeneous out of the box. Good applications server base, good and improving security suite, top notch BPEL product and the start of BAM. Killing interconnect was a good move, and they've even started moving the Application platform over to using the app server, though it will take a while. Good grasp and pushing of standards. Very complete stack, and an amazing change in the last few years away from its "all in the database" towards a middleware view.

Weaknesses:
Lack of a decent ESB, and a stripped down BPEL engine doesn't do it for me. An obsession with their own middleware framework (remember BC4J? Well now we have ADF) that surely can't last. They too don't have a decent service modeller, or any service modeller in fact, and the number of "services" being generated out of apps seems to indicate a real desire to create quantity over quality, and they look suspiciously like stored procedure front-ends.

Need to:
Complete the suite, this means a good ESB and getting a decent business service view of what Apps should look like, not a "we've got more services than SAP" dick-swinging competition.

Third and Gaining
SAP - The last to the party.

Strengths:
Speed to market, sure they've been developing everything themselves, but its what 3 or 4 years since they had nothing now they have an integration suite, Process engine, Portal, J2EE application server and a vision for SOA called ESA. Very impressive and the products are improving. Good grasp and pushing of standards

Weaknesses:
Still seeing the world in SAP coloured glasses, SAP to SAP is great, but lots of the world isn't. Very very process oriented view of the world, many of the "services" they are exposing are singular process (hell like Oracle its database oriented).

Need to:
Complete the suite, for them this means admitting that there is more than SAP in the world. Again like Oracle they need to stop going for volume in their "services" and create a decent business service view of their suites.

Must try harder
Microsoft - Oh dear where is the strategy?

Strengths:
Decent base technology and one of the few Business oriented SOA approaches with Motion.

Weaknesses: No obvious strategy, Window Workflow has a different base to BizTalk 2006, BPEL support remains static and there is nothing in the ESB space or really for enterprise workflow. Failure to adopt or push open standards and a totally new modelling approach that is still unproven. Biggest challenge appears to be that the enterprise software revision is waiting for Longhorn Server, not even Sun are dumb enough to link an OS upgrade with Enterprise software features.

Need to:
Buy BEA, or get a vision for SOA that allows the enterprise to evolve at a different pace to the operating system.

Technical SOA - The IT Departments drug of choice

After reading yet more articles about the wonders of technology without thinking, you know the sort that say "Web Services = SOA" or "its easy then just to reconfigure your global supply chain using the Web Interface changing process and rules without requiring re-testing or re-deployment". Also DanC declared SOA doomed with which I disagreed . Well all of this got me thinking about technologists and how we become obsessed with a given technology or approach and just can't help using it everywhere. Sort of like addiction, so here it is, how to recognise if your organisation is suffering from Technical SOA addiction (thanks to this link for the basics on addiction)

1 - Use - The use of web-services or other SOA technologies within a project. It helps you do some things and you don't suffer from the consequences, in fact it makes you feel better because it speeded things up.

2 - Misuse - In a desire to use SOA technologies you decide to use BPEL for basic page flow in an application or you lob a Web Services on something just because you can. The server crashes or the project over runs and the business shouts at you to clean it up.

3 - Abuse - SOA Addition is kicking in, despite the warning from above and the over-run you decide to try BPEL for page flows again, or you create yet more Web Services and deploy a global UDDI "just because you can". More failures start occurring and governance starts going out the window

4 - Dependency/Addiction - Everything has to be SOA and use SOA technologies, large bulk uploads and ETL work is done using BPEL and Web Services, project rules dictate that Web Services must be created on all elements that could theoretically be re-used elsewhere. SOA technologies are compulsively used regardless of the impacts on projects. The IT department starts trying to tell the business how to operate and prioritizes SOA technologies over delivering anything to the business. The aim of IT is to consume and create as many web services as possible with an almost macho desire to boast about where SOA technologies were used where they had never been used before.

Do you recognise your organisation? If you are currently at stage 1 or 2 there is no reason for you to be moving towards 3 or 4, but you need to start making sure you see Technical SOA as just implementation so it doesn't become an emotional crutch that you need every hour of the day.

If you, or an organisation close to you are misusing technical SOA or slipping into dependency it isn't too late, there is a simple 7 step process to help you and them get over it

1) Admit you have a problem - face up to the fact that you've become dependent on technical SOA, talk to your software vendors about the issue and tell them that you'll need to talk to them differently from now on.

2) Start cleaning up your act - get rid of those Web Services you know were just created to give you another hit, refactor out those badly deployed technologies

3) Ask the business to help them - admit your problem to them and ask for help, they can work with your problem if they understand it

4) Read the Mythical Man Month - understand that your aren't the first, and won't be the last, to think that technology is a silver bullet and do things badly.

5) Focus totally on project delivery for a while, get your head down and don't think about IT strategy

6) Get some help, go and speak to other people who have either had similar problems and got over them, or managed to avoid your destiny and created something more balanced and beneficial.

7) Get a life - if you became that obsessed with technology you need another hobby.

Technical SOA - its okay in the right social occasion and when you've got responsible adults making sure it doesn't get out of control.

Remember - Web Service responsibly.

(NB The same addiction approach should be used for people who try and use PHP, Perl and similar languages, sometimes its okay, but when it becomes an addiction you become a danger to yourself and to those around you)

Why Governance isn't an SOA after thought

One of the bigger challenges with SOA implementation is when to start worrying about governance, you don't want to get all heavy handed and stop anyone doing anything, and after all all these web service things make stuff easy so governance shouldn't be as much of an issue.

Historically governance has been about making people re-use things, mainly because re-use was hard, or at least tricky, so people would either cut and paste the code or just do it themselves. This was paticularly true when you had a great big EAI team that acted as a major preventative to getting anything actually done due to them being the single biggest IT bottleneck.

Now however we are in a world where point-to-point integration has never been easier, and anyone can call a web-service with a couple of button clicks. This means that the emphasis of governance has changed to enforcing policy which has a couple of implications

1. You need a policy to enforce
2. You need a mechanism to enforce it by

This means that in SOA governance is more about direction than it is about championing the approach. What SOA governance needs to be focused on is enabling good behaviours and preventing bad ones. This doesn't mean it turns into a Security Department and says "no" all the time, but it does mean that there needs to be an active communication plan on what "good" looks like for SOA in the enterprise.

This is one reason that organisations get bitten much quicker with Web Service technologies than they have done before, the time to create the "rats nest" that EAI was meant to solve is much less and the ability to by-pass company policy and security can quickly get out of hand (right-click, expose "CompanyPayDetailsService"). So when planning to move towards SOA be aware that the technologies you will be implementing much of this with enable some spectacularly complex (messy) infrastructures to be created in a hurry, and some amazingly dumb things to be done equally quickly.

When planning for SOA don't think of governance as something that you will look at in Phase 2, or it will be the only problem you are looking to solve in phase 2! Create a simple, lightweight, briefly documented policy on how services should interact (having a Business Service model helps here) and put in place some basic measures of "who is calling what". As your SOA grows so will the governance, so its a fine line to balance, you have to increase the formalism as the SOA implementation grows, start light and build up, but always plan to have governance.

And that really is the point, SOA aims to link up business and IT more effectively this means that planning is critical and governance of these stakeholders and groups is what will enable the flexibility you want.

If you create a free for all it will be a dream for a consultancy, but a nightmare for your company.

Oh boy its the 90s again... OO arguments

Not much to do with SOA, but more to do with the wonder of our golgafrinchian approach to IT

A very short while ago I blogged about SOA being doomed and how it was similar to what happened in the 90s around OO, well blow me down if I don't then come across an article from the folks who do Drupal about OO and whether they do it or not...

Its like a timewarp, it come about because people have dared to suggest that in the 21st century that OO is the most effective way to build software systems. And then (very like the C developers in the 90s) trys to show how they are in fact doing OO, mainly by missing the point of what OO is about....

Now I thought that Encapsulation was all about limiting access to inner workings... but

"Like most other object-oriented systems, Drupal does not have a way of strictly limiting access to an object's inner workings"

Now I'm a bit unclear as to how Eiffel, Java, C# and even C++ can't limit access to an objects inner workings, I'd sort of naively thought that "private" helped in that way. So what Drupal does is use namespaces to help.... umm so sort of like Ada could claim with its packages... but less effective.

Under polymorphism it uses the old C stalwart of claiming that passing variable "somethings" to a function is the same as polymorphism. This seems to be confusing polymorphic functions with polymorphic classes.

Oooh but its better than the 90s because now we have GoF patterns to play with too.. they use Singleton (aka Global Variables) and a few others that you could have a crack at in C.

They appear to have a nice project, but rather than try and retrofit OO concepts onto what they have, why not just admit what they say at the end

"Drupal is on the surface a procedural system, because it is built in a procedural language [.....] A great deal of the power of Drupal comes from its underlying relational database, and relational programming techniques that mirror it"

So its procedural code on stored procedure (procedural) type elements. Be honest about it, be proud about it, but don't try and claim that you couldn't build it in an OO language, or that there is still a sensible discussion about whether procedural or OO is best.


Those who cannot learn from history are doomed to repeat it. --George Santayana

Driving IT from the Business on ARCast

At the recent Microsoft Insight conference (download all the presentations, or just mine) I got to do an ARCast with Ron Jacobs, which was talking about getting IT more business focused and some of the interesting things we tend to do in IT around re-inventing wheels. One of the areas of focus was of course using the OASIS SOA Reference Model rather than inventing your own. Other ARCasts from the event include Ivar Jacobson who I had the pleasure of being on a panel with around software development practices.

Meyfroidt's Law of developer skills

A friend of mine, Steve Meyfroidt, made a statement a couple of years ago that I've used several times and its holding more and more true as time goes by. The statement is

"We like to pretend that its India that's rubbish, but actually its pretty much everyone"

As applications become more complex, but with the amount of basic work in them becoming larger as well, its becoming more and more common to have a clear split in development between those people who define and govern, and those who do standard coding. In defiance of the Mythical Man Month more people are added to projects, rather than increasing the quality threshold. People blame poor coding on India or China, but a rudimentary analysis indicates a whole lot of rubbish being created on-shore as well.

The trend seems unstoppable to keep adding more bodies to the industry, and the average skill level continues to move downwards, both onshore and offshore. If you are planning a project, a programme or your entire enterprise IT strategy keep Meyfroidt's Law in mind, you can't plan for everyone to be brilliant, you must plan to enable the brilliant to succeed and enable the rest to deliver without causing damage.

Think about Brunel building the SS Great Britain or the GWR lines, a great visionary, a bunch of fantastic engineers, and then a load of people with hammers. You want to plan your IT in the same way, target the smart people to direct the rest, and only give the rest hammers.

As IT continues to expand its reach and pull more and more people from non-technical into semi-technical roles keeping Meyfroidt's Law in mind is going to be critical to success.

Why Metadata isn't magic

There is a lot of talk with Web Services about "Metadata", which after all is just "data about data". The problem is that people sometimes seem to think that "metadata" means that it isn't actually code and shouldn't be treated as such. So Business Rules, and even BPEL sometimes, are described as metadata which can magically be handled differently. Several companies enable Business Rules to be changed on the running system via a web browser, and talk about end-users doing this on the production system.

Of course when you actually talk to the vendors and use phrases like "please god tell me you are kidding" they admit that while you CAN change rules at runtime its not often a very smart idea. Mostly they then talk about changing parameters to rules (so fraud check is above $2000 becomes above $1000) rather than adding in complex new rules at runtime.

The problem is that these elements are being pushed in the same way as "configuration" was pushed on packages a few years ago. The belief is that just because it isn't old style "development" it is some how easier and requires less rigour. The reverse is actually true, because these elements are so central to the operation of the system and contain core logic they require MORE testing and MORE formalism than the rest of the application.

Java is metadata, it describes in an abstract sense a set of operations on the JVM, the JVM itself has instructures which are metadata, describing as they do some abstract concepts that are mapped down onto the processor. And using the latest Java VMs you could change the class at runtime and hotswap it... you could even build a web browser interface to let anyone do it.... this wouldn't be smart, so why is it smart when its rules or process?

Just because something is described in an XML document, or an Excel sheet, doesn't mean it might not actually be code. And if it is doing business logic, whether than be process, decisions or whatever then it should be treated in the same way.

Calling Business Rules, Process or transformations (like XSLT) "metadata" and pretending they won't go boom is a sure fire way to create one hell of a mess.

The measure I use is simple : If its got an "if" statement in it... its code.

And it reminds me of a comment made to me a few years ago about a package vendors implementation "they said it was just configuration, but 50 people configuring a product for 2 years sure looks like development to me"

Service Description for Discovery or Function?

One thought that occurred to me last week was that when people are looking to locate services we use a different method of describing their discovery than we do to describe their function. Thus while we may ask for a hairdressers, the reply will come back in terms of routes, colours and individuals and its by this association that we will find the actual hairdresser described.

This got me wondering as to whether a similar challenge exists with technical services, paticularly when numbers get large. Is there one description of service that defines its functionality, what WSDL and WS-* try and do (badly) and another that tells us how to find the "right" service? In the real world these elements are done to make discovery easier, thus someone is described not by what they are good at, but by their features as its fairly unlikely that in a room full of people you will spot the one "who knows all about Linux". With services there is a similar challenge, namely one of recommendation. If there are multiple similar services, either internal or external, how do you refer someone to the right one?

With services this description is liable to be part of the many documents that relate to the service ("its the payment service that uses the penguin example") this can even lead to services being used in ways that the original writers hadn't thought about ("There was a service about payment that talked about penguins, but it had a cracking currency converter on it"). Thus the ancillary descriptions help the service be discovered for another, and originally unintended, use.

This is why when you are looking at registries, or service repositories its vital that they should index not just the functional elements, but also the myriad of other information that relates to it. By increasing the information footprint of the service you increase the chance of there being some unique information that people can use to describe it to others. These "soft" information sources are also often the most valuable to people trying to re-use your service as they help provide much more context over the standard WSDL approach. They will include the business context, the technical context and the examples that you've used to demonstrate its use, its as important to find all of this information as it is to find the functional definition.

Why SOA won't fail except for people who miss the point

Fight Fight...

Reading Dan Creswell's weblog and his recent post on "SOA definitely doomed" and a couple of replys (including me) I think SOA is heading towards the "trough of dissolution" which is great news. Why? It means that the shiny happy technology wave is going to crash soon and we can get on with what SOA is really about this means

1) SOA is not about technology, not ESBs, Web Services, EJBs, Spring, .NET, Java etc etc etc
2) SOA is not about RPC, Messaging, Events or anything else
3) SOA is about changing the way you think about applications and enterprises

First off Dan goes after the "we don't know what SOA is" - We now have SOA RM, not everyone is onboard yet but its a start.

This is the shift that SOA represents and is the thing that will continue on. Undoubtedly there will be people who don't do SOA in the future, hell there are people who keep doing waterfall today (and failing as a result), and there will be people who say they do SOA in the same way as people say they do agile today when all they are doing is traditional iterative.

SOA is an important shift that has been used successfully in massive systems for many years, and which is leaking through to the big, middle and small mainly via vendor product marketing, in a similar way that OO really made the break through in the 90s, with various companies pushing "their" take on OO, paticularly OOD.

The rest of Dan's issues are focused around the development of SOA systems, which is part of the challenge for anyone trying to understand SOA. The vendors products are NOT about architecture they are about delivery, but SOD of IT was never going to catch on. Dan's points on the confusion in this technology area are very similar to those in the 90s where people argued C++ v Smalltalk v Eiffel etc etc and lost the fact that the importat shift was the change in thinking, not in technology.

His point about complication is well made, SOA is aiming at a more complicated challenge, its aiming to create a common language between diverse stakeholders. But then at the moment we have nothing with which to communicate between these groups effectively and that cannot be allow to continue. But I remember only 5 years ago being told that business people would never understand Use Cases, and now its become a natural way to discuss requirements.

SOA is about a simple principle, making your IT look like your company and enabling it to change in the same way as your company. This isn't about STOPPING small agile projects, its about enabling them, but enabling them in a way that doesn't make them the maintenance nightmare of tomorrow.

OO has failed in the large, its time to try an approach that works.

For the record Dan could definitely kick my arse.

And for an example of the OO wars of the 1990s, here is a cracker from the inventor of C++ and a troll from 1997 and back in 1994 people were not sure what OO was about