Technologists to shoot...

Thanks to Gervas Douglas and the yahoo list I've got a cracking example of the sort of thing I mean....

According to Forrester, an ESB typically includes communication
infrastructure; request routing and version resolution (mediation);
transformation and mapping; service orchestration, aggregating and
process management; transformation management; security; quality of
service; service registry and metadata management; extensibility for
message enrichment; monitoring and management; and support for the
service life cycle.

Oh god, how complex do you want one product to be?

But by far the biggest change in the ESB market, he said, is "you find
more and more larger vendors adding either a standalone ESB, like IBM
and BEA, or including those features in basic integration suites, like
Tibco and WebMethods and Oracle. Over time, I believe standalone ESBs
will gradually be consumed by the larger vendors and become more basic
infrastructure technology," Vollmer said.

Genius, this doesn't actually make any sense at all. Standalone ESBs (as produced by IBM/BEA) will gradually be consumed by the larger vendors. What the hell does that mean, he has just said that the larger vendors actually are either choosing to a) consider ESB as a lightweight platform to link things or b) are lobbing stuff onto existing products. I have three issues with this

1) Oracle's ESB is miles more like IBM/BEA than it is Tibco/Webmethods... and I'd bet more on them going for that model in future
2) The consumed line makes no sense
3) How on earth can the ESB with its millions of features he described before ever become a "basic" infrastructure technology.

This is exactly the sort of thing I was ranting about in the last post. The whole article is purely about how technologies will "solve" problems and move you towards SOA, and there isn't even a consistent line about what the technology is.

Technologies won't solve those old problems. If SOA is just about building applications in the same old way using Unicode based transport mechanisms then we've regressed. If its about using EAI type tools, then we've regressed.

Not one of these people actually suggested that SOA was about altering the way you think about systems. It read exactly like a bunch of people arguing that C++, Java, Smalltalk, Eifflel, UML, BON, Schloer-Mellor et al were the "way to do OO" rather than OO actually being about a shift in the way people think about designing systems. SOA works a level above OO in that it is about the architecture of multiple as well as single systems, but still the line is pushed that buying this technology will solve the problems created by the one they sold you last year.

Technorati Tags: SOA, Service Architecture

SOA - shoot the technologists

There are lots of statements made about SOA not being something you can buy from a vendor. I'm getting more extreme in my views... I think it isn't even something you can build. Clearly there can be technology delivered that meets an SOA, but can you actually do SOA if you are thinking about it as something that is just built?

People seem to be approaching SOA more and more as something that is resident in IT only, or even worse is where IT "understands" the business and builds things that are more responsive... without actually getting the business to own or define anything.

If SOA is going to actually make a difference to what is a failing industry then it needs to impact the structural problems, rather than just trying to deliver a new set of technology projects. 80%+ of IT spend is on "business as usual" aka "keeping the lights on" the support and bug fixing of current solutions, paying the maintenance licenses, patching things and generally just keeping them on life support. And then with the 70% or so of new projects that actually fail to deliver what was expected this means...

This means that around 6% of IT spend actually delivers new value to the business. This is fundamentally broken. Therefore if SOA is going to succeed it needs to help more projects succeed, and more importantly either reduce the 80% spend or help deliver more value for that 80%. This means that SOA truly has to be about how you govern and deliver IT, including how you continually deliver IT to the business once it has gone live.

This is why I'd argue that SOA isn't even something that gets built, its something that becomes a part of your culture, it changes all aspects of your IT organisation. Otherwise it really just will be yet another techy idea that delivers little or no value down the line.

IT is a broken industry, it takes more than an ESB, some Web Services and changing some design guidelines to fix a problem this large.

Technorati Tags: SOA, Service Architecture

When to kill your SOA project

There has been a decent amount of press recently about SOA projects "failing to deliver value" or indeed just plain failing. This isn't that much of a surprise really in an industry where failure has been the repeated norm for many a decade. What I'd like to know about these failures is whether they have read Mythical Man Month, Peopleware and Death March. In other words are the projects failing because of "SOA" or because the organisation wasn't able to do that sort of project using the tools that it had? Organisations that see SOA as a technology based thing will be particularly in danger to this sort of problem as they try and solve problems with technology without understanding truly what they are delivering. This "lipstick on the pig" approach to SOA tends to deliver failing projects pretty much all the time as it is an attempt to solve an underlying major problem by applying a new technology as sticking plaster.

SOA projects are not "special" and mystically better than non-SOA and you need to know when to kill them off and the three books above will give you some good indicators. But some very obvious ones are

1. Project has slipped 50% on its timescales, and you aren't even developing yet
2. You are using early access software and you don't have a link to the dev team
3. Most projects failed under the old process and it hasn't been changed
4. There was no additional time added for learning new technologies
5. The vendor cackled when selling you the product
6. You've included several brand new products, and its taking much longer to integrate than expected

All of these should help you to can the project early and do a proper post-mortem on the project so you can get it right next time.

In many ways you should be more often looking to kill of new SOA projects as they will have new technologies, new process requirements, new governance requirements and the odds are you'll get it wrong (particularly if you don't get outside help).

If you really want SOA to succeed then you have to look to fix the basics first, this means understanding the business services, the right process and governance model and putting in place the right training to make sure it all works.

SOA isn't magic pixie dust to be added to projects to make them succeed, technology based SOA in particular suffers from the challenges that any technology driven exercise does, namely it rarely delivers what the business wants.

Kill early, kill often. I've seen far to many projects fail at year 2 that had been known as failures since week 1.

Technorati Tags: SOAService Architecture

SOA is more about old technology than new

I've been spending a lot of time recently looking at how a service oriented approach helps around older applications in particular how it helps to move them from being static systems into more agile and responsive ones. During this I've done quite a bit of research into how much people spend just on "keeping the lights on" as opposed to actually moving forwards.

Its about 80% of the IT spend out there. Which means that for SOA to actually matter, namely making the 80/20 rule apply to SOA then its actually going to be not about new technologies, web services and the like, its going to be about how we reduce that 80% and start making existing systems pay for themselves.

The goal of SOA is often touted as being the regeneration of these estates, but most of the time I'm just seeing people proposing new IT projects that aim to put "lipstick on the pig" of legacy applications. This isn't solving the problem its just trying to hide it, and is liable to be as successful at hiding it as EAI was when it promised to do the same thing in the late 90s early 00s.

This is why I argue that SOA has to be about changing the way you think about your IT organisation and the IT estate. It has to impact not just new technology builds, but more critically it must impact the old technology. This means that SOA must change the way that application maintenance is done, the way that help desks are set up and the way that change requests are handled. It even means changing the way IT is organised and in particular the current "handover" of applications to the life-support system that most applications move into once they go live.

SOA's buzz and hype is driven in the desire to sell new projects and sell new technology licenses, this misses the opportunity for most companies which is to move their IT organisation away from a project or technology focused approach towards being truly organised and responsive to how the business operates.

People have often said "you can do SOA with CICS/COBOL" and they are right, I'd go further in fact. If your organisation isn't doing SOA in its legacy and old ERP systems then you are just playing with new toys and aren't actually solving the problem that IT has created.

Technorati Tags: SOA, Service Architecture

AXIS2 debugging between London and Mumbai

Well I had a 9 hour flight today so I thought I'd try and get AXIS2 working... it must be something I've done... right?

I'm using a simple WSDL file with 3 methods, used the WSDL2Java to generate the stub and the skeleton and the test code. This didn't work, however thank god for open source as this meant I could trudge through the code to understand why on earth things didn't work properly. So the service was basically

interface GeoLocation{
 GeoLocation findByIATA(iata String);
 GeoLocation findByICAO(icao String);
 GeoLocation[] findByName(geoName String);
}

And no matter which method I called it ALWAYS saw it as findByIATA on the server, a bit of a challenge as the SOAP message and responses are typed to the specific request, so SOAPFaults abounded as it complained I was passing the wrong information to the method, in fact I was getting the wrong method for the request!

First top tip is that Axis2 doesn't have hot deploy, if you upload the same WebService archive this doesn't take effect until you restart tomcat, which is a bit naff.

Second top tip is download the Java VM documentation, being at 35,000 feet over Iran isn't the best point to forget the arcane syntax for starting tomcat remotely in debug mode.

Third tip is that when Tomcat says "Failed to shutdown" its lying, its shutdown, dead gone, buried and out of there.

These two combined led to a very painful cycle of adding logging messages through the code in a great bug hunt. First off I thought it was a client side problem, this was denied by the client by the cunning rouse of actually having the right SOAP envelope. So this meant that the fault was on the more painful to debug element, the server. A quick cointreau to stiffen my resolve and it was confirmed that all of the right information was passing across the wire.

Next up the great issue of the Axis 2 src build not actually producing something that runs. So its the world's nastiest development approach, taking a working jar file, exploding it and inserting a new .class file... nice. This narrowed it down further to say that the loading of the different end-points was being done correctly but it was the mapping of the request on to the end-point that was going horribly wrong

Landing in Mumbai brought up the solution for the debugger, setting JAVA_OPTS in Tomcat to be -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n, the problem I'd had on the flight was that I'd put the address in as localhost:8000 and it didn't work.

So surely now moving down the home straight for debugging, a simple watch expression helped to determine when the AxisOperation was set to the wrong thing... oh wait it wasn't 100% simple because the naming conventions are all over the place, msgContext, msgctx, messageContext... this is why naming conventions do actually matter. The target then was identified as the SOAPActionBasedDispatcher from axis2.

Then we find it after a couple of step intos, surely this must be the answer "findOperation"... ummm but SOAPAction is a blank string. I hate it when the problem is actually right at the start but you find it at the point where the variable is finally processed. The reason its finding SOMETHING to call is that the mapping table in AXIS2 has a bunch of options including "" which is assigned to (I assume and now I've got to find out) the first operation it gets.

So now its about debugging right from the start again and finding out why that SOAPAction isn't being set correctly.... oh great we are into the servlet API now, but its now looking like I'm back to square 1 and it might actually be an issue on the client side after all (the HTTPHeader doesn't appear to have the SOAPAction set it in to anything). Then its down to MessageContext, oh yes, here we have a cracker.

MessageContext.getSoapAction() returns options.getAction(), this is null. However msgContext.getAxisOperation().getSoapAction() returns the right value. Another cracking example of how not to code, having two different variables with the same name that aren't actually the same variable isn't a bright idea.

Now we move onto another patch on Axis2, and I run into another annoyance, the maven build is now farting over some test element it can't do, something to do with jibx... errr I don't care if jibx tests fail, I want my sodding jar file, so its explode and combine time again....

And then it all works fine. So to the chaps at Axis, the fix is to modify MessageContext.getSoapAction() to just reference the SoapAction of the AxisOperation. Bug raised and reference here, but it really was a bit worrying to see some basic development practice errors, particularly the thing that the bug turned out to be, duplicate places where information could be stored.

The worst bit about this is that I've just wasted 4 hours that I should have been asleep because I just couldn't let it lie.

Technorati Tags: AXIS2, Java, Web Service

Schema envy

I've been messing around with some of the official schemas out there with the goal of breaking tools. The one linked from the title is an impressive 240k or so and includes 6 other schemas which aren't exactly slim either. Its a cracker to throw at DOM based parsers out there, for some reason they have issues.

But what it made my think, there is the ACID test for CSS, but where is the evil test for Schema that really sets out to break tools? One that implements all of XML Schema and uses it in fully compliant, but deeply twisted ways.

Anyone know of the ultimate test schema?

Technorati Tags: SOA, Service Architecture

Axis 2, Maven and the problems with distributed systems

I've had a lesson this evening in the fragility of distributed systems. I decided to try out Axis 2, the 1.0 release, and to further the education I brought down the source version and decided to build it. Now Maven works with a whole list of repositories and Axis has a whole heap of dependencies, which in themselves have dependencies.

And like any good distributed system where you don't have control over the remote servers... these dependencies don't exist anymore. One was maven-itest-plugin, the other was a rather specifically named stax-utils-20060501.jar and there could have been others. The solution for anyone who wants to know is to edit the project.properties file and add in http://ws.zones.apache.org/~dims/maven/ as a repository, it all seems to work fine then.

But here is a good open source project, run by good guys who know their stuff, and they are being bitten by the age old problem of distributed systems. Namely that they've designed something, or are using something, that assumes that everything is always available. This is a common mistake that people make when building distributed systems and SOA isn't going to solve this problem any time soon. What this example does show is that building robust systems is all about being prepared for failures and building systems that are designed to fail gracefully.

Oh and after I got it built and deployed it exceptioned with a NullPointer in the init...

Technorati Tags: SOA, maven, Axis, Service Architecture

Virtual Machine top tip of the day

After wondering why my computer was running like a dog, I checked the CPU usage and saw that much to my surprise that the Linux VMWare image was running at 100% of one of the cores. What was going wrong I wondered?

Then I suddenly realised that the default screen saver for Fedora is one of those pretty animated ones. This reminded me of when people had X terminals rather than workstations and the bane of the sys admins life was people running XEarth and the like, which destroyed both network and CPU.

So if you are using VMs, and aren't running them headless, then remember the screen saver!

Technorati Tags: VMWare