Monday, December 17, 2007


I'm doing some reviewing of JavaOne presentations at the moment (SOA/EAI) and the number of vendor presentations that are basically a booth demo where they'd like a bigger audience is just staggering. Sometimes they are dressed up as an "investigation" but most of the time its literally a standard booth demo (even including "how to install" on occasion).

Death to Boothware.

Technorati Tags:

Tuesday, December 04, 2007

Why corporate IT means something

Over on Joel on Software there is a post about his talk at Yale where he warns people not to go into "in-house" corporate development because its soul suckingly bad. Why?
Number one. You never get to do things the right way. You always have to do things the expedient way. It costs so much money to hire these programmers—typically a company like Accenture or IBM would charge $300 an hour for the services of some recent Yale PoliSci grad who took a 6 week course in dot net programming, and who is earning $47,000 a year and hoping that it’ll provide enough experience to get into business school—anyway, it costs so much to hire these programmers that you’re not going to allowed to build things with Ruby on Rails no matter how cool Ruby is and no matter how spiffy the Ajax is going to be.

"No matter how cool Ruby is", "Spiffy"?!?! this just about sums up why vendors don't understand their customers. Now most companies I've been in are doing just those sorts of things and are doing it when it makes sense and the people doing this tend to the leading IT lights of those companies. Having worked with lots of product companies as well I can safely say there are legions of developers in those companies who certainly don't have the ability to choose a new programming lanaguage and who are never going to lay their hands on Ajax. So maybe the issue isn't corporate development but the sort of development that Joel did when he was in a corporate.
You’re going into Visual Studio, you’re going to click on the wizard, you’re going to drag the little Grid control onto the page, you’re going to hook it up to the database, and presto, you’re done. It’s good enough.

Now apart from the last bit (which after all is just smart engineering over turd polishing) this does sound frighteningly like average development. There is a bit around never having time to do quality and refactor to your hearts content but that really does miss the point as I've never met a good developer who ever thought what they had produced couldn't be improved.
Now, at a product company, for example, if you’re a software developer working on a software product or even an online product like Google or Facebook, the better you make the product, the better it sells.

Ahh I know Joel is talking to students here, but is lying really the option? The best software sells the most? Only if you define best as sells the most. The history of software is littered with examples where better software was ignored for inferior fare. Hell in IT it often seems that we deliberately go for the worst option out of some sort of perversion (C v Ada syntax for instance). The implication here is that product software is the best quality. My experience has always been I've never met a piece of commercial software I couldn't break. Things like having a messaging product, Java VM, Operating system and hardware all from one vendor and it wouldn't work at all not as in a slight issue but as in they could never have tested it because it didn't work in any way whatsoever. Things like vendors shipping software products without decent test tools being available. Things like evil class loader work arounds due to vendor stupidity. Things like Rational XDE which came exactly from people "optimising" (the new stuff is soooo much better and started from a different place IMO).

Corporate in-house software on the other hand is designed to do a specific purpose and when it does that thing it works. There are less edge cases to worry about and you have a defined reason for doing it. Now there is indeed lots of crappy inhouse software in the same way as there is lots of crappy vendor software the question should be really about comparing the best of corporate IT with the best of vendor IT. Now excluding the "found your own IT company" which is always the best if you can pull it off the question is which is better to work for?

Now the problem is that Joel clearly worked at a crappy job in the crappy part of an corporate company. He complains about the pay and conditions and seems to think that software companies always have the best perks. This isn't true for several reasons
  1. Societal balance - by which I mean women. Tech companies are male domains and the male/female ratio is normally completely dreadful. If you work in a corporate then the odds are this will be much more balanced. This makes for a better social life
  2. HQs of corporates are better than tech companies. I've been to lots of the supposed "best" tech company offices and the HQ of a pharma, bank, oil company, airline or even traditional manufacturing company tend to be miles better. Now only the best in IT get to be at HQ, but were you planning on being average?
  3. Flying - The policies at most vendors appear to be "coach unless a VP" whereas in corporates it tends to be "we are in business so we fly in business".
So a clear win for the corporates on that one. The next up is the worry that you can't become the CEO if you work in a corporate in IT. Now this is pretty fair, but you can become the CIO and be responsible for a multi-billion dollar budget which doesn't seem too bad and given the choice between that an CEO of a small product company then I have to say I'm with the CIO of a large corporate. Lets be generous here and say that the vendors win this one.

Next up Joel talks us through his hell hole job at Viacom. It really does seem that he was a completely unempowered programmer and this is a crap job at any company, where he makes the mistake is thinking that these people don't exist in vendors. They absolutely do and they have many of the same issues. Field sales engineers are good examples, often very talented but having to put up with whatever is thrown at them from the mothership. The point here isn't corporate v vendor but empowered v munchkin, and you don't want to be a munchkin.

So the question in terms of what is best is what sort of impact can a good and empowered person have on a corporate or a vendor? In this day and age I'd say that the impacts are pretty similar. The key is finding what matters. With a vendor company you could take them into a new market and you can do the same in a corporate, with a vendor you could create a competitive advantage over the field, something you could do at a corporate. The point is that these days when decent companies look at changing the game they make sure IT is in that decision. Its a good place to be.

The real reason to me that corporate IT is a great place to work, if you are good and have good communication skills, is that you can actually see what you do make a difference. I'm incredibly proud that I wrote software that means air traffic controllers can do their jobs and be alerted quickly to any collision risks. Now sure the folks who wrote the graphics library and the OS, the software vendors, had a part in it but it was me who brought it together and gave it purpose.

Corporate IT is the point of vendor IT. Vendor IT is there to make money and its the corporates that it predominately gets this from. This means that while as a vendor you can say "look at our shiny app server" or "look at my bug tracker" the only point to your software is if some corporate people turn it into reality. Thus its corporate IT where the real achievement is. Software Vendors provide the bricks and mortar, they quarry the stone and provide you with the rough hewn pieces for you to carve and give purpose to.

The key in corporate IT is to be one of two things. Firstly you could be very talented and looking at real edge challenges (for instance forecasting in retail & supply chain) where you'll have to push the boundaries of technology to the edge in order to stay ahead of the game. Secondly you could be the person tasked with sculpting a result out of all the raw materials of people and the software the vendors provide to create a new solution to a business problem. This is where, IMO, the greatest challenges and talents in IT are. People who can understand technology, people and process and bring them together. Making people think in different ways, using technology in ways the vendors hadn't expected and doing this all successfully to a properly formulated business case. That takes talent.

The majority of the hardest technical challenges I see in IT are in the corporate space. There are of course challenges in the vendor space, especially where the sale is direct to the customer (but is Amazon really IT company or a next generation retailer?) but the challenges in the corporate space are just as large and hairy and often just a lucrative. The hardest challenge in IT however is the same as it has always been, how to take multiple technologies and large numbers of people and deliver a system, changing the business as you do so to make the adoption of the system successful. These are the people who give a point to all of IT and who can point to things in the real world and say "I made that happen".

So vendor or corporate? If you are talented and have good communication skills I'd say go for the corporate. Especially when it comes to the Christmas party.

Technorati Tags: ,

Monday, December 03, 2007

Christmas SOA

I've been reading some things lately that talk about business people being "in control" of BPMN diagrams so I thought it was worth while reprising something I wrote back in 2005 and which is in the book around how there is a big difference between the perceived business process and the actual execution process.

Now I have two kids and its coming up to Christmas and their view of what happens at Christmas is of course completely different to what actually happens. The kids are the business at Christmas its all about meeting their business expectations and goals. Heather and I as the parents have only one job to do and that is to deliver Christmas in the manner the business expects to see it. This means that we have to hide from the business the technical delivery elements and just show Lana and Louis the vision they want to see... now of course the question is how do we deliver an SOA Christmas?

At the top level 0 its all pretty simple all the business wants to see is presents coming from as many different channels as possible. As the kids are both still young the metric that will matter is volume of presents and its by this that success will be judged.

At Level 1 they have the concept that Santa Claus is the centre of the present giving universe and it is to Santa that the list of request should be made. Santa is then responsible for distributing the list to everyone else for the things he doesn't want to get.

So at a process level the business view is that the list goes to Santa, who sends it on, there is a view that there are things called shops out there involved in some way, but that isn't important to the business. Santa then delivers presents to the stocking while the parents deliver to the tree. Any person coming through the door over this period is also expected to enable the business to get presents.

This is the perceived business process and its this that the kids want to be able to effect. Can the list be delivered in person? Emailed? Sent via some slightly dubious chat service? Or maybe posted the old fashioned way? There is a, reluctant, acceptance from the business that the 25th December is the delivery date and that this isn't movable and that some suppliers will miss the delivery date and provide presents at the first available point after the main day. Santa however is never late.

Now the goal of the parents (IT) is to deliver Christmas in the way that the kids expect and to ensure a successful and happy Christmas period. Unfortunately the parents also know that Santa needs some manual intervention to get things moving. So first of all they have to provision the infrastructure, namely Santa and the Tree, then they have to co-ordinate the present buying to minimise duplicates. A spree of present buying is then underway, as is an increase in debt, followed by delivering the presents in the right manner to the places the business expect them to be.

This therefore is the execution process. This process might get much more complex if you have to add in balancing acts between the business people so there is the perception of present equivalence or if the parents are actually hosting Christmas this year so the workload goes through the roof. This all demonstrates that within SOA, there might be a commonality in an understanding of the services being provisioned, but the actual manner of provisioning and the additional actions are only of interest to the implementors and should be hidden from the business, and other consumers as they are irrelevant to them as long as their objectives are being met. This is one reason why SOA makes good BPM but BPM makes bad SOA

This is just one of the reasons I tend to say that business process is an IT myth. You need to understand both the business view of what must be achieved (often goal driven) and how it is measured (the implicit process) and then map this down to how IT will deliver that goal and measures (sometimes via an explicit process). In this world of separation its nonsense to argue that the business is "in control" of the execution processes beyond the level of defining the important elements of KPIs, goals and implicit process. Having this explicit process implemented within the services rather than across them (for a blog outline there is chapter on this in the book) also helps to scale as the services are coordinating effort rather than having to have a process per child.

Switching back to more normal IT should the sales director worry that that simple two steps of "check fraud then get payment" might require a complex series of processes and integration to external 3rd parties? Should the LOB manager who wants to get their current key suppliers need to know that this integrates with 20 backend systems and has a complex Master Data Management solution? No that is the reason IT is there to take the business goals and KPIs and best surface them in the way that the business wants. This is why business people shouldn't care about their BPMN diagrams being what are being executed they should just care that they are being delivered.

Remember folks, SOA isn't just for Christmas.

Technorati Tags: ,

Accenture still not getting SOA

A while back I pointed out some comments by Accenture's CTO which (for me) indicated that Accenture don't get SOA well they've now established a venture with BEA and it appears that Accenture still don't get SOA.
He [Accenture CTO Donald Rippert] described the four phases of SOA implementation, which begins at using XML as an interface, then implementing legacy systems as Web services, and then using an ESB to connect Web services and use composite processes. The fourth phase involves using BPEL (Business Process Execution Language for Web Services) in which a business application is revised by making changes to the process model rather than the code.

But SOA, for the most part, is not enabling this yet, said Rippert. "Maybe someday, but not today; I don't see it today," Rippert said.

Wow its still impressively wrong. Hands up everyone who has seen BPEL used in a commercial environment? Does that feel like the fourth and final phase of SOA to you? No me neither. This really is dreadful advice on what SOA is and what SOA maturity is about. I've worked with customers where their first technical SOA project was all about using BPEL (without an ESB) to do something where that made sense.

You can take what Mr Rippert is saying and easily replace WS and XML for some other basic form and an EAI tool, then have that EAI tool be the ESB and put a process engine on the EAI suite. In other words his vision is from a functional and conceptual perspective equivalent to EAI.

If SOA, or any IT change, is just about a series of three and four letter technical words then it will deliver almost no benefits. So has to be about changing the way people think or its pointless.

Technorati Tags: ,

Friday, November 30, 2007

SOA 2.0 comes back... now with added schwaz

Via the Tibco blog I came across this post which referred to a pdf which.... well you get the picture. There is a box which talks about "2nd generation SOA" in there which sounds oddly like SOA 2.0.

The challenge is that these "advances" are still focused on the next set of technologies that people want to sell you and not on making the structural and mindset changes that will actually deliver the benefits.

So not so much 2nd generation as the 2nd set of SKUs.

Technorati Tags: ,

Wednesday, November 28, 2007

Nodding dog alignment - the perils of aligning to people not business

I've seen a several organisations over the years that boast proudly of their business alignment but who still have an unsuccessful IT department. The litany in these IT departments is always the same
We are aligned to the business because we deliver what the business wants

When you delve deeper however what they are actually doing is aligning to business people rather than to the business goals. In effect the IT department just turns into a nodding dog and says yes to all requests made by anyone who can claim to be a business person. This is what leads to hideously configured packages because "the business said so" and to a fragmented IT estate and ever increasing IT spend for ever decreasing business value.

What alignment should mean is actually aligning to the Business Value and then making the tough decisions based on that. If there isn't the business drive to differentiate in an area then don't. Almost no-one will ever admit their area isn't actually strategic so doing what business people say is not the same as IT being aligned to the business. IT is there to govern and manage how IT is implemented and operated that means saying "No" as well as "Yes" and it means understanding the business objectives rather than the personal goals of individuals.

Business alignment isn't about people alignment.

Technorati Tags: ,

Tuesday, November 27, 2007

REST scales? Only if you make it

Last night the "Ramsey's Kitchen Nightmares" programme dealt with The Priory Restaurant and what a mess it was in at the start of the programme..... the website that is. Prime time programme, their URI flashed onto the screen in the first five two minutes and what happened? Oh straight into the not responding territory the website went.

Now this was using a "GET" and we all know that REST scales, so what was the problem? Mr Creswell summed it up when he talked about common sense architecture. So sure REST can scale but it isn't magic. It does various things around state management that can help but it still requires basic fundamentals to be done, its not magic and its not a silver bullet for scalability.

So does the Web scale? Yes. But when Google is allegedly the 4th largest server manufacturer in the world, maybe its not REST but hard work and brains and an awful lot of hardware....

Technorati Tags:

Monday, November 26, 2007

Why Unit Tests don't help with crap programmers

I had an entertaining chat a few weeks ago with a rather fundamental agile proponent. Now we agreed on lots of bits where "agile" could help (with me mainly point out that "iterative" and other practices have been there for a while) and then we found a point of real contention. He state that TDD (test driven development) would improve the quality of code for all developer, while I maintained that muppet developers would produce muppet unit tests.

Fortunately we had some empirical data to test this on as we had access to the code base for the project he was running. He had TDD going through out the system and although there were "minor" coverage issues in terms of branches the unit tests were all green and there were lots of them so everything was fine.

find . -name * -exec grep -A 4 catch {} \;

Ohhh dear the look on his face, very entertaining, the screen began to fill up with statements that looked a lot like this

catch(Exception e){
System.out.println("Test failed");

Yes folks a number (4) of his developers didn't have a scooby-doo how to write a decent unit test or how to write decent code and were just "going for green" by trapping the exceptions. The point is that the poor chap had been so used to having a good quality team that he had naively assumed that things that had helped graduates from top line universities improve would also help muppets improve. The problem was that the muppets pair programmed with the other muppets and no-one else in his remote team would pair with the muppets so the four of them ended up regularly changing partners but within a very small, and in bred, gene pool.

The point of this is that there are very few things that automatically find muppets for you and getting them to write more code to check their own code isn't one of them. You still should be doing things like code reviews and you should always be looking out for System.out if you are doing Java.

Anyway he has caught the issue early enough, extended the build to have some more checks and has instituted a weekly code review hour where two developers a week will have to present their code for critique and improvement.

Technorati Tags:


Reading the Wiktionary definition for automagically left me feeling a bit cold. To me its not a replacement for the term "automatically" its actually one of two things
  1. Its something done by IT that the business doesn't need to care about. As in "don't worry we will do that automagically" this means that actually (as in magic) there is an awful lot of work to be done behind the scenes to make the trick look simple.
  2. Its what vendors REALLY mean when they say "automatically" with respect to upgrades. What they mean is that you have to do a huge amount of prep, it makes massive assumptions on what you have done previously, including often placing impossible constraints and normally it will fail. In other words (like magic) its actually a lie.
These are two very different uses of the term that I've heard and both are explicitly not "automatically" in the true sense of that word. The point of automagically is when its not automatic at all and either you don't care (case one) or its a lie (case two). Now if people agree I'll update the wiki entry but I thought I'd first check what others had seen.

Technorati Tags: ,

Thursday, November 08, 2007

Professionalism v the consumer view of Java

Sometimes I read vendor views on a market and just groan. Over at El Reg I read just such an opinion from a Sun employee on Java's future and he is advocating that Java SE 7 should leap down the same road of bloat and anti-enterprise thought that caused Java SE 6 to be an enterprise class dud and not think about upgrades. Chet Haas seems to think that this was a good road to go down and we should lob more into the core including "I said I would also like to see further improvements in the platform libraries, to offer increased functionality and easier programming paradigms." which sounds idyllic but of course he isn't proposing that we remove anything to enable these new, improved(?), paradigms but that we just add more functionality.

The reason for this, and the reason he demonstrates a lack of understanding of real enterprise computing is his comment "Java is what it is and you cannot now start taking things out of it. If you do so, you will break existing applications.". You see here is the issue, people like myself who are advocating removing stuff and creating a minimal core for Java (see this presentation) isn't that we are saying "delete all the libraries from existence". What we are saying is that in an enterprise computing environment where there are billions (tens of billions) of dollars spent on people, education, software and planning that we really can decide what we need and what we don't.

What I want is a minimal core and the ability to add only those libraries and features that I want because I know what I'm doing. Sure if someone wants the full bloat, or as the consumer platform, it makes sense to have the full kitchen sink edition (Java KS?). In the former case because the person can't be arsed to work it out and in the later it gives an assured platform for execution.

This is the problem. People of this mindset seem to see the consumer platform as being the ultimate Java environment the one that really matters and the ones whose problems should always been addressed to the detriment of all others.

Quite frankly it was ridiculous in Java SE 6 the justifications that were trotted out to explain why having things like JAX-WS 2.0 in Java SE 6 and a lightweight webserver made sense. The systems I work on have teams of professionals on them who are better able to decide for that problem whether they need a basic Java SE environment with a web service implementation or whether they are going to use an application server. Forcing these people to undertake hideous class loader workarounds just sums up how divorced from enterprise reality the Java SE 6 decisions were.

The problem is that like many vendor employees Chet doesn't have to cope with the myriad of issues that "everything in the box" creates for organisations. The "just don't use it" line doesn't work when you fall victim to Meyfroidt's Law or to the issues of wanting to use something different to that which comes in the box and suffering the pain of ripping it out (no vendor worries about that). The reality of enterprise computing is that the minimal set of functionality that gets the job done is what you want and you want to restrict people to that as it makes support (where the cost is) easier.

If Mark Baker wants to use REST then let him, if Paul Fremantle wants to use Axis 2 or Synapse then so be it, if someone wants to use JAX WS 2.0 on Java SE 6 their funeral (if they want to upgrade to JAX WS 2.1 then that should be easier than using the older version), if Dan Creswell wants to use Jini then so be it and if lots and lots of people want to use a J2EE application server then fair play to them. The point is that all these people (baring the Java SE 6 muppet) have made an informed choice about what works for them and that doesn't mean we want to see all the stuff that other people are using, if we did we'd download the stuff and use it. This is how professionals work, they make informed choices about what they want to do and then select the right approach for the long term. This is good architecture and good engineering, picking the core of the solution, making it architecturally consistent and then enforcing that consistency. Hey its Mythical Man Month territory again, yes Java SE is that wrong.

For the consumer market Chet has a point, for the enterprise market he does not. One of these is worth several billion dollars, the other is not. Its about time that Java Enterprise users got the standard platform that they need rather than the one designed for Joe Sixpack.

Technorati Tags:

What is it with vendors view on time?

There is one fascinating thing that I've seen from vendors over the years and that is their different perspectives on time when the talk about their products. I call it the "Temporal Perception" of a company. The basic challenge is that vendors talk about time as if the only important thing is their release dates and they have two different views on whether that date is in future or in their minds represents the present.

As an example I was talking to a vendor recently about their strategy and they kept talking about their next release (due in 2009) in the present tense. Clearly the development and marketing chaps need to be focused on that release but they were talking about it as if they'd already done everything, boxed it and shipped it to customers. Every question around the current issues and challenges was met with a "We've solved that in X" as if that would help me in the next EIGHTEEN MONTHS before the product shipped. Quite a few companies fall into this group and its easily the worst.

Another group of companies talks about their products in the future tense all the time saying "we haven't done that yet but it is on the roadmap" and then you find out they are shipping it next month but their perception is that anything that isn't already out the door is vapourware. They talk about a month as if its a distant possibility and even next week is just too far out to assume that it will happen. They have a long term vision of what they want to achieve and they'll talk about what they want to do but always in the distant future. This is a very small group of vendors.

Another one just refuses to recognise that there is a world beyond the next six months. Products are either shipping this quarter, just going into Beta or simply don't exist. They won't talk about the strategy and direction and won't admit to anything that hasn't been formally announced and passed through about a million lawyers first. They certainly won't admit that the product they are selling you today will be dead in 12 months. This is sadly common as well and doesn't help customers plan for the future.

I could go on and on about the different views on time and product that vendors have, fundamentally it comes down to their culture and perspectives and tends to fit into these three groups.

This really causes issues for companies when they work with software companies because the one thing that these vendors (except sometimes the ones in the middle group) are never focused on is the time frames that make sense for the customer and understanding the actual issues and challenges of the customer's present. So when you speak to a vendor make sure you understand their temporal perspective and recognise what that means to your relationship with them and what it will mean.

Technorati Tags: ,

Wednesday, November 07, 2007

SEATAC or how monopoly makes service delivery a pain

Sometimes when I look at SOA infrastructures and there are a couple of single points of failure or bottle necks. Security is normally one and often its an ESB or a registry. The point is that all requests go through these elements. Now if they are scalable this can be alright, the challenge however comes if the funding and the KPIs for these areas aren't aligned with the overall goals or even compete with the individual service goals.

What do I mean? Well last Monday I had one of those nightmare travel days going into SEATAC from LHR. First off I wasn't in business, secondly we had a delay due to issues in Iceland so it took longer... then we landed.

The US Immigration folks are a great example of the problem that comes when something, whether in government or in business, has a monopoly over a gateway or access to other services. While the service provision goal might be to get out out the airport as quickly as possible and make it as painless as possible this is nothing to do with the objectives of the immigration (and customs people). Two 747 planes landed at the same time from Europe, to handle the several hundred Europeans getting off these flights there were.... THREE people assigned by immigration to handle the volume.

Seriously THREE people for two 747s full of europeans. This was followed up by customs people who turned the simple task of taking a piece of paper off people into something that generated three queues that were over 50 people deep each. The point is that neither immigration or customs (gateway services) are tasked with helping to deliver the end-to-end QoS that either the consumer or the service provider (airport) wants. This creates a contention point that has no resolution because the actual governance is set up to create the problem. Immigration are tasked on letting through only desirables and don't care if you stand in a queue for an hour (or TWO hours in the case of the unluckiest person in our group).

This challenge is replicated in businesses when these gateway elements are put under IT management and are charged with cost reduction and costed as a single element. This means that their focus is on being cheap and therefore being poor.

What is the solution therefore? Well for the US immigration its probably to notice that European flights tend to have lots of Europeans on them. For businesses its to think of these gateway elements as utilities rather than as cost elements. This means that the cost for them is based on the usage and is linked back to the original consumer/provider interaction. If you want good service as its important then you need to be able to pay for a better SLA, if you don't care about the performance then you get a worse SLA.

Infrastructures need to be linked to the SLAs of the consumer/provider not have a generic SLA that attempts to cover everything and be viewed as a purely cost item in the equation, otherwise they become blockers to change and experience and you end up taking 2 hours to get through an airport and a load of pissed off users.

Technorati Tags: ,

Thursday, November 01, 2007

How to create customised Business Services on a package

Having said Change the Business not the Package, something I will stick by, there is the problem of what if that package only implements part of a business service? What if you have legislative elements that need to be done or local policies that need to be enforced? What if you need to add in Decision Support? Now the first thing to do if you think this is


Seriously, right now. Are you really sure that this is special to your business? Because the odds are that what you are really doing is just adding in some customisation for the hell of it. Can you change your business to fit the way the package works?

Now lets assume that you actually do have a good reason to have some differentiation in this area, how do you add in customisation? The answer isn't rocket science its simply
  • Treat the package as an integration point
  • Put customisation in.... a CUSTOM application
This is why SAP has Netweaver and Oracle has Fusion Middleware (I'm still surprised SAP aren't going after BEA) and of course why lots of other integration and middleware stacks exist.

So what you look to do therefore is to use the interfaces that exist on the package as the integration points for these new customised applications. A basic rule of thumb is always use mediation when communicating with the package, or between different services. Another good idea is to think of the business services that are exposed to the enterprise as not always being straight one-to-one mappings to those definitions from the package. This might be due to data schemas but most normally will be because the package has lots of interfaces that are fine grained and you only want to put some coarse grained services out there for the world to use.

The other bit on the diagram is that business service areas may combine information and function from a package with that from other applications either via a composite application or just by creating a facade over different applications to create a single consistent business view for the enterprise.

So the basic rules for package customisation are
  1. Don't
  2. No seriously don't
  3. If you have some differentiation then put it outside the package
  4. Consider the package as an integration point
  5. Don't customise the package itself
  6. Think of the business services to be exposed
  7. Remember that a business service can contain information and function from multiple IT systems.
So customise in the middleware, this is what these platforms are designed to be good at.

Technorati Tags: ,

Wednesday, October 31, 2007

The Ivory Tower of Implementation Optimism v Long term viability

One of the things I've been banging on about during presentations recently is how to bits of IT thinking are the major contributions to the current mess that IT finds itself in.

First off is the view that new technologies and solutions will clean up the mess. This is wrong for the simple reason that in almost all companies this represents less than 20% of the total IT spend (often its less than 10% or even 5%). This means that 80% of the spend isn't being touched or looked at. A simple 80/20 rule would tell you that this approach is wrong. Unfortunately IT continues to believe that yet more lipstick on the pig will make the pig attractive.

The second one is that projects and in particular the focus on Cost to Live (CTL) over TCO are set up to create bad IT estates. By focusing people's mind on the go live date it encourages people to make short cuts and decisions that while helping the project hit its cost and time figures ensures that the TCO is driven further up. In other words the decisions in the 20% actually make the 80% worse.

One of the things I've worked with companies on, and which is in the book, is that different parts of the business require different delivery approaches and technologies. This means looking at the business and understanding where you should be targetting your best people. I've said it before that if I had a bunch of Dan Creswells to work with then I'd be making sure that they were focused on the highest value areas, the ones that require constant change (i.e pretty much never drop into standard support) and just letting them do it the way that works for them, and I'd do this making sure that they were responsible for the TCO not just CTL.

However that is about 2% (at most) of the heads out there in IT (remember Meyfroidt's Law). The skill in architecture in these other areas therefore is much, much harder. Architecting a solution in a team of talented people that will be supported by talented people is easy, almost trivial. To architect a solution that will be built with people without your skills and talent and which can then be supported by people who don't have the talents of those developers is a real and challenging art.

The way I look at it is this. I went to university in York and in York there is York Minster, one of the most impressive buildings I've ever seen or been inside. When it was built the art of building was a real craft that required master craftsmen all over the place to get it built. The end result is a fantastic achievement that is testament not just to the architect who designed it but to the talents of the builders.

Move on to the 19th Century (and even before) and building had become more of a commodity. That cathedral to scientific knowledge that is the Natural History Museum was built with much lower skilled builders who were working to a defined plan. The end result is almost equally impressive as York Minster and yet the skills profile was dramatically different.

The other lesson to be learned from building is that the long term viability of solutions is more important than the initial build cost. The Tacoma Narrows disaster is a great example of focusing on CTL, with disastrous long term consequences.

The goal therefore of architecture when constructing it for the 98% is to enable the solution to be built and sustained successfully by that 98%. You are not architecting the solution for yourself you are architecting it for others. This requires a much higher degree of skill and architectural competence than simply aiming for the "best" technology for the best developers. This is why I disagree with Dan that there is a risk of Architecture RIP when looking at these areas, its more of a risk in the project and developer optimistic area IMO.

This is also why I have to disagree with Steve Vinoski's fatalistic assumption that this means that there is No hope for IT. In fact I'd say that there is no hope for IT if we take Steve's advice and concentrate on CTL
If you’re in a position of technical leadership or project management and you’re asked to come in ahead of schedule and under budget, my advice is that you’re generally more likely to succeed with REST and dynamic languages than with the alternatives because their inherent constraints allow for better focus.

Now I'd like to see some references to back this up (a nice manufacturing plant example for instance) but even so this just repeats the mantra of "this is best for CTL". It doesn't challenge the status quo and just tries to focus on doing "new" things over improving the old. It also assumes that the quality of the developers is at the same level as Steve, which is more than slightly unrealistic.

Steve is right that there are great developers out there in the enterprise space, but he is wrong in assuming that it is viable to just employ the talented. Simply put these people won't work in support of legacy systems (for starters) and are wasted when implementing low value or commodity systems. The reality of a good enterprise IT community is that it is a distended bell curve of quality v quantity

Now some people might find it depressing, but tough its the reality. If you aren't looking at architecture from a business perspective and constraining it by the reality of the world then you aren't doing good architecture.

The hope for IT and the reason that architecture is required is to start focusing on the 80% and on the long term viability of solutions and to stop focusing on the short termism and Ivory Tower of "lets only use brilliant developers". If we can focus as architects on enabling the talented to link up with the rapidly evolving and enable the rest to deliver and support successfully then we will finally start progressing IT forwards.

Talented developers create architectures that they can build and support. Talented architects create solutions that can be built and supported by others.

There is much more actual talent in the later than there ever will be in the former.

Technorati Tags: ,

Tuesday, October 30, 2007

Separation of Policy from implementation or why I've turned on the human check

One of the basic rules of SOA is that you should keep the operational polices of a service separate from the logic. In other words the bits that dictate how, when and whom for a service are managed differently from the "what" of the service.

Today I've had to turn on the human check for comments as I've had a series of SPAM links put onto the blog. This is in effect implemented as a policy, changing the policy doesn't alter the implementation (i.e. the process of adding a comment) it just adds in an additional step that fulfills the policy requirement that there should be a person commenting.

This is why things like WS-Policy, WS-Security and the WS-RX group specifications are important. They keep the policy pieces away from the implementation pieces and enable you to manage them independently.

Technorati Tags: ,

Monday, October 29, 2007

Why republicans don't like SOA

I often look around for non-technical examples to use of why the people piece is more important than the technology piece and today via the wonder of YouTube and an American friend who sent me a couple of links I have a cracking one.

You know in companies the problem you have within IT or with business folks who just won't accept reality? You know the people who say "yes but if things we different then..." or "I don't care what the facts say I think this is better". You know these people, they are the types who make the same screw ups over and over again just because that is how they screwed up before, almost like a bad track record is proven track record.

The challenge often is that these people act as blockers to any change programme because it challenges, or refutes, their basically held beliefs and opinions. These might be "It must all be SAP" or "We can't change the business" or "Waterfall/Agile/Flying Monkeys/REST/Web Services are the only way to do anything" or some other single view point which doesn't like the idea of change and hates the idea of multiple ways of working and change (as an aside can people who don't like change please get out of IT).

Now to the bit that proves the republican party doesn't like SOA and prefers monolithic software delivery, well some of them anyway. Three republican presidential candidates indicated that they didn't believe Evolution was true. One of these Mike Huckabee even questions the temerity of considering this a reasonable question to ask. This is the standard response when you question people's opinions (or cushy jobs) and look to have change in an organisation. Things are declared as "off limits" for political reasons or marked down as inviolate elements that cannot be challenged.

People are the problem with change and technology isn't enough as even facts and reason often don't work.

Technorati Tags: ,

Wednesday, October 24, 2007

UML and SOA a beginners guide

One of the most common questions I get asked is how Business SOA works with UML. How do you capture Use Cases in an SOA world? How do you do activity diagrams, sequence diagrams etc.

The first point is that you should define the Business Service Architecture first, before you start into the IT project elements. This is the bit that gives the SOA context to everything else.

Now lets look at UML and in particular those pesky Use Cases. The simple rule here is that
  • Service => Use Case Package
  • Capability => Use Case.
In other words the requirements for each service need to be gathered within the bounds of that service, not from end to end of the application. The point here is that when doing the Use Case for a service the actor doesn't need to be a person the actor is any external entity which wishes to interact with the service. The interaction with other services can be done by <> statements to the capability definitions from another service package. The good thing about this is that you can use Use Case stubs (i.e. no actual detail) where the service is implemented using another approach (e.g. when using BPMN to implement SOA). This means that your Use Case model is complete for the services that it defines and still allows the flexibility of choosing different approaches for different services.

So that is how I'd say to do UML Use Cases with Business SOA.

Next up there are sequence diagrams and activity diagrams. For these I'd say sticking to the same rules for BPMN is a must, namely have a core thread within one service and only make invocations and requests to other services. This makes it much easier to enforce the separation of concerns and also helps to stop the leaking complexity that often comes into these diagrams.

The business SOA gives the structure, the UML the definition. This is the power as it gives context to the rest and enables different approaches to work together.

Technorati Tags: ,

Friday, October 19, 2007

How to create performant Web Services and XML

I was at an event the other day when a chap said that XML/Web Services etc were fine "unless you wanted to do thousands of transactions a minute in which case you needed something much more streamlined".
Now I was sitting at a table at the time and we all pretty much agreed the statement was bollocks. So I thought I'd do a quick guide to create performant Web Services and XML applications.
  1. Use XML at the boundaries. If you are writing Java/C/Ada/Python/Ruby etc then once you've made the hit to translate into your native language... keep it there. When you cross another service boundary then that probably needs XML, but don't keep marshalling to/from XML every time you move around inside a service.
  2. KISS. If you have 25 XSLTs and indirections in your ESB then it will run like a dog
  3. Don't have a central bus that everything goes through, think federation
Oh and of course the most important one
Seriously, hardware is cheap so my basic rule (and it hasn't failed yet) is as follows:

If you are out by a low multiple (under 3) or a few percentage points then scale your XML processing like a Web site (lots of thin servers at the front). If you are out by high multiples then you've screwed up somewhere.

Now the later one can be that the vendor has screwed up, so make them fix it. But in the last 7 years of doing XML and Web Services I can say that in environments running lots and lots of transactions (thousands a second even) that I've yet to find that XML and Web Services didn't scale.

The key if you have a performance problem is to really look at the pipeline and see where the real time is being spent. Several times I've had people claim "Web Service performance issues" then found that 1ms is being spent in the WS bit and 5 seconds in the application, or people blaming a specific element (XSLT/XML parser/etc) and then finding out that it is a bug in an implementation (one in a certain vendors stack looked almost like a wait loop bug). These elements aren't performance issues with Web Services or XML, they are bugs and issues with the applications and implementations.

XML and Web Services are not the most efficient things in the world. But then I'm currently working on a computer where this browser (inefficient editor) is running in side a VM (inefficient) on a box with 3 other VMs running on it (very inefficient)... and it is still working fine. A home computer now comes for a reasonable price with FOUR CPUs inside it (remember when a front end started at 4 x 1 CPU systems?) and Moores Law continues to run away. The issue therefore isn't whether XML/WS (or event XML/REST) is the most efficient way of running something but whether it is good enough to be used. New approaches such as streaming XML take the performance piece on still further and make it even less of an issue than it was before. This is about changing the libraries however and not the principles. XML/Web Service works from a server perspective.

So server processing isn't the issue. So maybe its network bandwidth... again I'd say no. Lob on gzip and you will burn up some more CPUs but again that is cheap in comparison with spending a bunch of time writing, testing and debugging software. This creates a much lighter over the wire message and again just seems to be fine in implementations where the wire size is important.

The only piece I've found that continues to be an issue is something that is nothing to do with Web Services or XML, but which is more of an issue in REST, and that is chatty behaviour over the network. Being chatty over a network gives latency, no matter what your bandwidth is there will still be a degree of latency and you can't get away from that.

So basically the solution is to communicate effectively and accurately and to do so in reasonably coarse grained ways. This is hardly news at all as it has applied to all communication systems in IT. As networks speed up and latency comes down then maybe even this will become less of a problem, but for today its certain a place to avoid excess. Google for instance on a second search (cached) to "Steve Jones SOA blog" responds in 80ms. A coarse grained approach that has 1 interaction per request and 10 requests will have a network induced lag of at least 800ms, a chatty approach that has 5 interactions per request will have 4000ms or 4 seconds. Chatty = non performant.

So basically I've found that Web Services and XML do scale, hardware is cheap, that Stupidity doesn't scale and that networks still introduce latency.

Its not rocket science but some people seem to think that its still 1999.

Technorati Tags: ,

Thursday, October 18, 2007

Change the business not the package

I've spotted a bit of a worrying trend around SOA and package delivery namely that people seem to be thinking that SOA magically means that you can now customise packages safely. The goal becomes once again "fit the package to the business" this unfortunately is the road to ruin. It doesn't matter what the package is, whether Finance, ERP, niche or even a small off the shelf thing for a point solution the basic rule remains the same. You are going package because it doesn't differentiate you, it isn't an area where you will gain competitive advantage and if everyone in your sector ran the same package, it really wouldn't matter.

This is a critical starting point. If you want to use a package solution you have to accept that this means you have chosen to take a non-differentiated route. If your content is the thing that is your USP then this doesn't mean that you shouldn't use a CMS it just means you should recognise that it is the content that is the USP, not the publishing and managing of the content.

Too often people go through extensive requirements gathering before selecting a package and then try and find the "best fit" to those requirements. The project kicks off and they try and implement the remaining requirements. This is suicide, don't do it.

First off start with a set of objectives and principles that will guide the package selection. One of these principles must be that the goal will be to change the business approach to fit the selected package. Then choose the package. The next state is then to ask "how can I cope with what the package does" this leads to the understanding of what business change can be done.

So how does SOA help in this process? Well first off you can use a Business Service Architecture to understand the drivers and interactions at a high level, this really helps to clarify where the package will be used and to make clear the broad business context in which it fits. So if you are looking at packages across multiple areas you should still be thinking about the different areas based on their different business context and drivers, one size doesn't fit all.

So SOA doesn't mean that you can customise the package anymore today that you could before. There is a rough calculation that I've seen a few times around package implementations (the numbers vary a bit but the thought is important)

So basically the more you customise the more likely you are to fail. Now while it looks okay down the bottom end, don't be fooled.

Now these aren't hard and fast numbers, indeed they could be under playing the impact of customisation.

SOA doesn't help in terms of package customisation. A decent BSA can help you understand how the package will work (if at all) and the boundaries that it will have, but the same rules on customising the box remain.

There is however a new choice when it comes to adding differentiation into a business service which is partly implemented using a package.... but not in this post.

Technorati Tags: ,

Wednesday, October 17, 2007

REST is dead long live SIP

One of the things that always amazes me in technology is how people seem to think that technology revolutions have suddenly stopped because they've found a hammer. Whether it be Web Services, REST, EAI, ERP, Excel or anything else certain classes of IT people seem to become blind to the history of IT and think that because they are happy with what they are using that there will be no other developments.

As an example of this lets look at REST (as implemented by HTTP) and SIP, now REST lays down a bunch of constraints and has its HTTP implementation as the way to do everything. HTTP is of course limited in what it can achieve and certain tasks it just doesn't seem to be the right way to go, VOIP being one example, now SIP provides some similar elements in terms of philosophy of simplification (which is nice) but enables multiple protocol interaction and can (in theory) support application to application communication.

Now I've been looking at it and I'm not sure why you couldn't use SIP as a boot strap before electing to use HTTP, VOIP, Video or something else for communication between applications and from applications to people. After all wouldn't it be nice for the application to send you an SMS when something changes rather than polling a web server? These people talk about using SIP to kick off VoiceXML as a reasonable approach today, and VoiceXML is an HTTP based standard.

This would of course mean that HTTP, and potentially REST, would be relegated to specific application problems with an increased requirement for sophistication around the actual mode of interaction between systems and between systems and users.

The real point here is that no one technology will ever remain for ever as the pinnacle of achievement. It is either replaced (e.g. CORBA to Web Services) or commoditised (e.g. TCP to HTTP) and in both cases it ceases to be the cutting edge of IT delivery.

Will SIP relegate HTTP to "one" of the options? Who knows. But what I do know is that backing HTTP to be the cutting edge of IT delivery for the next 20 years is on a hiding to nothing.

Technorati Tags: ,

Tuesday, October 16, 2007

Why procurement is meant to be hated

There are three hated departments in most companies, they compete to be the most loathed of the bunch but one stands out from the other two as different.

The departments? HR, IT and Procurement.

These are always loathed and cursed and accused of preventing progress or just creating a set of bureaucratic processes that stifle peoples soul and remove the joy from the world. Only one of them has an excuse however.... procurement.

Now the reason that procurement is hated is that in fact this is the job of procurement. Have you ever wondered why procurement systems are almost always a pig to use, and why every "upgrade" seems to work in a completely different and more perverse way?

The reason is simple, the goal of procurement is to stop people buying things. If the CFO wanted to make it easy they would just give everyone credit cards and a cheery wave of "whatever you want is fine". The reality is that the purpose of procurement, a centralised service in the business, is to monitor and control all purchasing decisions, from a new pen through to a new building. The system should be painful to both suppliers and users, ensuring that both sides try and avoid interaction unless completely required. In this way procurement helps to focus spending on what is really required and removes spending on items that are just "wanted" because its not worth the effort.

This is an important thing when looking at building a business service architecture, understand the true drivers of services and don't assume that there are common elements across them all or that the end-users opinion always is the most important. Usability, performance and reliability are all great things when interacting with your customers, but in procurement they are positively to be avoided and a focus on annoying the user and making things slow and painful and prone to failure helps to deliver on the goal of reducing spend.

Don't assume that the goal is always to make things better, or that a system that is seen as "rubbish" isn't actually doing its job. Understand the business value that a service has and the drivers that create that value. This helps you to understand what "good" looks like in different parts of the business and focus your investment and effort accordingly. Don't assume that everything in a business needs to improve and get better, somethings either aren't important enough (general ledger anyone?) or are actually better off being bad (procurement).

So that is why procurement is meant to be hated. The more hated it is, the more successful it is actually becoming.

So that really leaves HR and IT in their daily fight to the bottom of the satisfaction league.

Technorati Tags: ,

Monday, October 15, 2007

Should IT deliberately create chaos?

There is an old IT joke

A farmer, an architect, a gardener and an IT guy are talking about which is the oldest profession. Now these chaps are bible literalists so they think the Garden of Eden was the start of everything.

The farmer says "Mine is the oldest profession as Adam had to tend all the animals and get all the fruit from the trees, if he wasn't a farmer he would have starved."

The gardener jumped in saying "yes but someone had to design the garden of Eden and put all the plants in place, so God must be a gardener"

"No, no" said the architect "First God had to create order and structure out of the chaos and build the universe so he must have been an architect"

"Ahhh" said the IT guy "but where did you think the chaos came from?"

Now the reason for saying this is that I saw one of the best pro-REST presentations at the Enterprise SOA conference in Belgium last week (which actually talked about business SOA). One bit I did have issue around was a concept that disorder and fragmentation was the "norm" of enterprise IT and thus the problems were the same as the web (its slide 37)
Internet vs. Enterprise
One is a gigantic, uncontrollable anarchy of heterogeneous systems with varying quality that evolve independently and constantly get connected in new and unexpected ways.

The other is a worldwide, publicly accessible series of interconnected computer networks that transmit data by packet switching using the standard Internet Protocol (IP).

Now in part Stefan has a good point, namely that IT systems currently suck. But where I'd disagree is that the goal of IT should be to create such a chaotic system with so little governance and control. This is one challenge I have when people talk about applying Web principles to the enterprise, it misses out on a fundamental difference between businesses and the internet. Namely that of compulsion.

It is true that IT unfortunately does create IT estates that are a mess and which have in the main part been driven by people who focused on the technologies first. This isn't however true for the businesses that they support, there is the ability in companies to compel change and to enforce conformity. Companies establish procurement departments, HR departments and even IT departments with these aims. Taking the Web view and applying it to business seems to imply that the chaotic nature of the web is the correct approach to take in creating an IT estate that works for the business. This doesn't make sense.

We need to understand in IT that sometimes compulsion is a better approach, if you are putting in SAP its best to fit the business to the package not the other way around, equally if you are looking at links between businesses is it better to take the standardised approach of procurement and enforce conformity of contract or to enable a "dynamic discovery" approach (whether UDDI or via MIME)?

IT needs to learn from the business an understand the power of enforcement and contracts, rather than thinking that its the business that needs to learn from the chaos that IT creates.

(Oh and Stefan, join the revolution. My presentation on SOA Methodology is web delivered :) )

Technorati Tags: ,

Sunday, October 07, 2007

Great parody on technical IT

Pete Lacey has a great parody on the technical focus of IT folks. Its a wonderful trawl through the way that business people ask for solutions and how IT in return focuses on technology.

In this hilarious romp through how businesses continued to ask the same questions and the IT department continued to offer them yet another technology that would this time solve the problem. There is a great bit in there where he demonstrates his comedic brilliance by pretending that he thinks that VOIP is a business requirement (rather than the requirement of course being to reduce the phone bill), this wonderfully highlights how technologists can't see the difference between a business requirement and a technology solution.

There are great bits in there to highlight, almost too many to mention, but for me the real bit that made me realise how great a parody was this bit

"And… Well, you know the rest. They picked the wrong technology again, despite the fact that the right choice was staring them in the face. Like CORBA before it, SOAP..."

Brilliant, a superb highlighting of the extreme arrogance of technologists in thinking that the next technology really will be the silver bullet. He then drives the point home by brilliantly misunderstanding SOA and equating it to network oriented computing (although I think just saying Distributed Computing would have underlined the parody a little better).

Thanks to Pete for the shout-out at the end to my BSA post. He hilariously pretends to misunderstand the difference between architecture and requirements gathering, a common problem for technical IT.

I don't think I've seen a better post that sums up the problems of technologists thinking that technology is always the solution and that anything more contextual is a bad idea.

Truly well done Pete, a great post on the issues of technologists focusing only on the technology and assuming that this will meet the business needs.

Technorati Tags: ,

Wednesday, October 03, 2007

Business Service Architecture - is it time to admit defeat on SOA?

Over on the Yahoo email list there are a bunch of emails around SOA/REST etc etc and as ever Mr Tilkov summed it up well when he talked about Business level SOA being implementable using REST, but technical level SOA being different to REST.

So what I'm thinking is it time to bring clarity to all of this and admit that the techies and vendors have "won" the battle for the SOA name? Time to give up on the SOA name as something that has been co-opted purely to the technology and which is sometimes more focused on selling products than on delivering solutions?

My proposal is that we should start being clear when we mean business SOA by calling it a Business Service Architecture. The goal is to clearly differentiate business SOA from technical SOA and help move the debate on from the current technology centric debate towards one that is about changing how IT works to make it focused more on business solutions than technical implementations.

Business Service Architecture - Business problems solved, no matter what the technology.

Technorati Tags: ,

Monday, October 01, 2007

Exception Management and SOA - a Webinar

Last month I recorded a Webinar with Vitria (one off registration required) on how exception management fits into SOA. The key point I was trying to get across in the presentation is that exception management is a business issue and that SOA should be about the business and therefore thinking about business exceptions is a core requirement for any successful SOA.

Exception management is about when things go wrong, two core points here
  1. Sometimes its okay for things to fail and ignore them
  2. Exception management is a cost overhead, getting people back onto the primary flow is value creation
SOA as technology wouldn't think about exception management, SOA as business must.

Technorati Tags: ,

Thursday, September 27, 2007

Is REST architecture or a design pattern?

Mark Baker commented on a presentation by Roy Fielding that claimed that "SOA was a null architectural style". Now part of it was that Roy seems to think that SOA = WS, which is wonderfully naive, but a bigger bit is the idea that REST is an architectural thing.

Is it? After all what is the goal of architecture in an IT context? Surely its to represent what the business wants to achieve. Looking at the Wikipedia definition of Enterprise Architecture for instance it says
Enterprise Architecture is the description of the current and/or future structure and behavior of an organization's processes, information systems, personnel and organizational sub-units, aligned with the organization's core goals and strategic direction.

And System Architecture goes down as
A system architecture or systems architecture is the design or set of relations between the parts of a system.

Now this is what I've always thought of as being architecture in an IT context. Even something very very techy like ADML describes software architecture as
A software architecture describes the structural properties of software, typically the components and their interrelationships, and guidelines about their use.

The key bit here is it describes the OPERATIONAL pieces (components and interactions) rather than the implementation mechanics.

Now I don't think this fits what REST is at all. REST deals with the implementation mechanics and the principles and practice of that implementation, it doesn't provide you with a style for the modelling of an architecture and its interactions, it provides you with a framework for its implementation.

In his presentation Roy uses the phrase "Architecture is trying to find the common design" before going to talk more about architectural style being more abstract that that. Now the issue here is that in architecture he talks about implementation protocols as being the key to the Web "architecture". REST then becomes one level above that talking not about the protocols but about the "style" of the Web. Now this really got me thinking. This is a long way away from what I mean when I talk about business architectures and it really is about implementation practice... then it dawned on me, this isn't an argument about architectural styles, in fact REST isn't an architectural style.

REST is a design pattern
In software engineering ,a design pattern is a general repeatable solution to a commonly occurring problem in software design. A design pattern is not a finished design that can be transformed directly into code. It is a description or template for how to solve a problem that can be used in many different situations.

Now that to me sums up exactly what REST is, and how the Web is the implementation of the REST design pattern, and explains how the REST design pattern can be used to implement your projects and programmes. This doesn't make REST any more, or less, relevant to decisions but for me it explains the different between SOA, as a business driven concept, and REST as an implementation model for that approach. It is fair to say that Web Services has a lack of patterns and practice and that this is an area that should be addressed in terms of Web Service implementation patterns (and anti-patterns), but both REST and Web Services are fundamentally about technology and implementation.

Architecture needs to be about understanding the goals and objectives of the business problem being solved and about the effective modelling of that solution. This then needs to be translated into technology by creating a software architectures which are based, or constrained by, firm design patterns that ensure repeatability of solutions. Confusing the goals of architecture by making it focused on the technology implementation really doesn't help us bridge the gap between IT and the business, indeed it reinforces the gap by emphasising the IT focus as being on technology rather than on the business problems.

REST may well have a place within the implementation of an Enterprise or Solution Architecture, but we shouldn't confuse the goals of architecture (business problem) with patterns (IT implementation), and REST belongs firmly in the later.

Technorati Tags: ,

Wednesday, September 26, 2007

Video from QCon conference

Over at InfoQ the interview that Stefan Tilkov did with me back at QCon in the UK (next one is in SF in November) is now available online, I'm talking about Business SOA and its impact on IT, with only a minor dig at the technology driven people.

Technorati Tags: ,

Thursday, September 20, 2007

Professional IT and why you don't hear it

Speaking to a very interesting and smart chap from IBM at the conference today called Max from IBM Research who is looking at the Web 2.0 technologies and their growth he asked a very good question....

First the setting. There was a panel on SOA Runtime and the WS-* v REST question came up. I repeated the quote "want to be cool do REST, want a career do WS" just to be provocative really (and interestingly got a round of applause from the audience...). The proper answer was the one that Gregor said which is that there are two worlds, the enterprise, transactional and integration one (the big one) and the interactional one (the small one) and that they require different models. Something I agree with.

The discussion with Max afterwards was around what he sees as the "explosion" of REST APIs (900 at the last count apparently) and I pointed out that Oracle and SAP alone probably have double that, even before you go to internal and commercial implementations. His question was simple

"Why don't you hear about that?"

The answer?

The people who work in these area tend to be employed and doing it as part of their job. Their main interest isn't therefore in the technology as a thing but in the objective of getting it live. Tony put this well on the panel saying that "the focus isn't on technology its on operation, if its not operational it doesn't count". People who work on large scale enterprise IT therefore on the most part see technology as a tool to get the job done, rather than as the end in itself. Its also something that they do for a living and therefore "normal". This is the big reason that you don't hear from people in all of these enterprise companies, they often don't realise that what they are doing is special, because hell it works why is that impressive? The point for them is doing business better so the story is "new automated fish selling solution for the abstract art industry" not "we used Web Services to connect some stuff".

This later one is also the "where is the news" point. "Company uses Web Services successfully" isn't news, its not shiny so why would anyone write about it? News is about what is next and what is.... well "new". Doing something successfully with the existing technologies isn't a big deal, its what people expect. They get written up as Case studies where the word "Web Service" is just a remark (at best) as the concentration is on the operational solution.

Things like Java SE 6 are indicative of the problem that we face in IT these days, it was aimed squarely at what I like to think of as the ADHD part of IT, or as Andy Hedges puts it the "ooooh string" crowd. The voices from professional IT do it for a living and don't have much of an interest in raising their heads above the parapet and pointing out that they are already using Web Services and have been for many years and that they are now using BPEL successfully in their organisation. There is an assumption made that because they don't blog about it that it isn't happening.

There is a way that these people make their voices heard however, its via standards bodies, professional bodies and most importantly in the procurement of billions of dollars of IT systems, technologies and solutions every year. The likes of SAP, Oracle and IBM are the bell weathers of this market and you can see what their clients are demanding by seeing where the main focus of their work is (they often also have the "string" as well as it helps them sell better). The problem is that because these people have a day job that is focused on objectives they don't have the time to blog, join technology committees, etc So their involvement is very low per capita in comparison with vendors and the blogosphere, its also not "interesting" as its their day job so why on earth would they want to talk about it more?

As Gregor said there are two models, one around interaction and one around transaction and they require different approaches and thought. But one thing that is for certain is that it is the IT professionals in enterprises and the large software vendors who will eventually decide how both worlds will work, and as before the successful ones will continue to see technology as part of the solution to achieve their objectives and not as the objective in itself.

Convincing more of these organisations and people to get actively involved with standards and vendors is a big challenge, because they already have a day job. But without exception when they do get engaged it makes for better standards and better products.

Noise on the Web != where the dollars are going. William Shakespeare had a good line that might work here

"Life is a tale told by an idiot - full of sound and fury signifying nothing"

Or to rephrase a famous saying

"If a Web Service is deployed in the enterprise and nobody blogs about it, does it still get used?"

Technorati Tags: ,

Wednesday, September 19, 2007

Dave Chappell - another reason why people matter

I've just sat through a presentation from Dave Chappell on what he is looking to do at Oracle now he is there. Its a message around data-grids, high availability and scaling. Looks very interesting in terms of how it further takes Oracle away from the database (but not data) being important.

When I meet companies I place a big weight on the people there and the clarity of the vision. Simply put companies that don't have forward looking people will just produce average copies of what everyone else does. The SOA Grid presentation by Dave was a great example of this as it really talked about a problem area in the current generation of technologies (box dependency, operational scaling and database overloading) and a vision of what a SOA Grid would actually look like and what the important parts of that message would be.

One of his points was that the reason people do stateless services (although of course they aren't really stateless) is because that is the way that you are forced to scale applications today. This doesn't make it the only way to scale, just the way that you scale based on the current generation of infrastructure. Dave's proposal is that Oracle will develop in-memory data grid infrastructure that scales horizontally and only offloads to the database as a transactional store and does this asynchronously, rather than as an active datastore the database becomes a record keeper of what has already happened. This means.... LESS DATABASES and also that databases become less important... no news on whether Dave is based in Boston for personal security reasons.

My learning from this is that the vision comes from the individual, and getting it successfully delivered by Oracle will require that clarity through out the delivery process. Oracle have been very successful at hiring the smart people at the right time to help them move from being a database company that did Java badly into a middleware and applications company that does a good database. More on this in a SOA World Article that Dave wrote.

So learn from Oracle and get the people right first, they'll then get the technology right for you.

(Oh and the organizing genius behind the keynotes and the panel's I'm in was Robert D. Johnson who has the word SOA on his business card twice)

Technorati Tags: ,

SOA Governance isn't about technology either...

Today I was on a panel at the International Conference on Service Oriented Computing which was about SOA governance. The panel members were
And of course myself. Now Chris had done a keynote on Governance from the OpenGroup perspective and talked about the "Three C's" of Governance, Culture, Communication and (IIRC) Control. We had a good set of debates and I'd like to summarize a few of the points made.
  1. There must be an impact to violating governance, too often people ignore governance knowing that they will get away with it
  2. Governance is not one-size fits all, find appropriate governance models in different areas
  3. Governance is about people and process much more than it is about technology
  4. The Business needs to be accountable for SOA governance and the business Services
  5. IT needs to be responsible for the delivery of SOA and the technical parts of the services
  6. Too many companies are buying registries as if that is all they need to do for SOA governance (a top point by Dave)
  7. Governance needs to not create big document jockey processes, it needs to be effective
  8. Bad governance kills innovation (a good point from the floor)
  9. Governance isn't a new IT problem with SOA, projects have failed for years due to bad governance. The point is that SOA provides a better structure for governance than previous project based approaches.
  10. SOA governance isn't an IT thing, it impacts the business accountability, the funding model and the full lifecycle.

Myself and Tony disagreed about the most effective way to enforce governance, I preferred using a stick with nails in it while Tony felt shooting someone and leaving the body around was more effective. Personally I prefer the stick with nails as its personal, but I see where Tony is coming from :)

The basic piece though was that if you don't switch you governance model to actually be based around the services and the business then you are being wildly optimistic if you think that applying a new technology (like a registry) will solve your problem.

One of the more enjoyable panels I've been on. They were video'ing it but I don't know what happens with the videos, if I find out I'll link it.

Tomorrow its SOA Runtime headed up by Dave Chappell.

Technorati Tags: ,

Tuesday, September 18, 2007

Emacs 2.0 ... one instance for everyone

There have long been arguments of Emacs v vi and which is better. I was one of those who saw Emacs as the stronger enterprise tool, and vi as the quick hack it editor when you didn't have time to boot both the computer and Emacs. With all this talk of "rich desktops" and "rich applications" it made me think....

Well with Web 2.0 and Google Docs showing us the way its time for Emacs to finally dominate. One of the features I loved in XEmacs was being able to have multiple people working on one code base in one editor, just fire up a window on another machine and you are away. Both editing the same code, pair programming in the truest sense of the word.

Now we can go a stage further, Emacs 2.0, the Web hosted single instance of Emacs served up to every browser on the planet. Everyone editing inside the same Emacs session, everyone able to share the same information, everyone able to use a Mayan Calendar to plan their projects.

This is what the vi people never realized. Emacs has just been waiting for the Web to become sufficiently mature and the PCs to become big enough to really enable to true power of Emacs. Web 2.0 is that infrastructure, it will become the plumbing that will put Emacs in every browser and finally finish the argument once and for all.

Never needs to reboot, never shut it down, always available. Need to browser the web? Do it inside Emacs 2.0. Need to run a global procurement system? Do it inside Emacs 2.0.

Emacs 2.0 will run as a virtual application hosted on all of the browsers on which it runs, it will become the base Operating system install on all new PCs and servers and it will provide a single shared environment for everyone on planet earth.

One editor, one instance, 6 billion users.

Technorati Tags: ,

Saturday, September 15, 2007

Why contracts help migration

Now there are some people who think that validation is a bad idea, but recently I've seen a few cases where contracts have really come to the fore again. One of these is around migration. The scenario is pretty simple, you want to start moving to a new solution area but you don't want to do it in one go, you want to clearly split the area into two pieces, the consumer bit and the producer bit. Most likely its the producer that is going to be replaced but you don't want to like the two parts tightly together so they have to upgrade at the same time and the same pace.

So what do you do? Well first off you define a formal contract between the two areas, one that can be fulfilled by the current implementation and which can also be supported by the new service when it arrives. Effectively you've now split up the two worlds and placed an agreed boundary between them which can be enforced, tested against and measured against. You are now in the situation where the new implementation has to meet the contract and where the consumer can now rely on that contract independently of the implementation.

When looking at splitting up areas, or migrating applications and software the enforcement of clear contracts is a very simple way to start introducing business centric contracts and to provide a controlled way in which upgrades can be handled for both producers and consumers. Not using contracts means there isn't something to test and validate against and thus the new implementation is aiming more at an abstract goal than a concrete reality and the consumers have nothing to rely on as they plan their own future improvements.

Contracts therefore increase the flexibility of complex systems explicitly because they enforce the boundaries.

Technorati Tags: ,