The ultimate Mashup - something to do before you die

What is a mashup? Well there are lots of definitions out there but I think the one that references Music is best, where a Mashup is where you create something new by combining two or more existing things.

Yesterday I had what has to be the ultimate Mashup experience. How else to describe the food at the Fat Duck? The combination of Snail and Porridge can only be described as a fantabulous culinary mashup. The "Sound of the Sea" that combines an iPod with a litany of wonderful sea food and all around combinations that make you raise an eyebrow before hand and then be brought to tears by the sheer brilliance of the food.

I've eaten at some great restaurants before and had some amazing food. Without a doubt however the Fat Duck was in a complete league of its own as a culinary experience.

And isn't that what successful enterprise mashups should be about? Combining elements that haven't been brought together before to produce the sort of result that delivers customer satisfaction in a measurable and repeatable way.

Heston Blummental, mashup genius.

Technorati Tags: Mashup

In defence of design

There are some things that I thought had been accepted in Software Engineering as a good thing, one of these was that design time clarity around interfaces is a good thing and helps multiple teams work together more effectively. This however isn't apparently the case as Mark Baker has made clear in a recent InfoQ article

So what's the caveat? It's the issue of when the WADL is consumed. Some Web services proponents who have taken an interest in Web based solution have been using WADL as they would use WSDL, as a design-time artifact. Using WADL this way though, is akin to developing a Web browser with built-in knowledge of, say, the Google homepage HTML form at the time the browser was compiled: if Google changes the form in a backwards-incompatible way (i.e. not just adding a new optional parameter) after the browser is deployed, then the browser will not be able to use that resource/service. Using the hypermedia constraint with WADL means that the client should consume the WADL at runtime. So be careful when choosing your WADL tooling, as some tools that try to help you, don't.

Now I have to admit to being happy at seeing the growth of WADL in the REST community (next up they can deal with policy, reliability, security and other challenging topics) but the implicit suggestion here is that having reliable interfaces at design time is a bad thing and that its much more effective to have dynamic run-time only interfaces that can change, and which you can automagically (in a vendor sense) adapt to.
The problem here is that its one of those beguiling silver bullets, suddenly lots of things that have previously been a challenge will melt away by using REST and letting everything adjust at Runtime. The problem with this approach is that it is spectacularly difficult to engineer in complex real-world applications.

Put it this way, if you are implementing VMI between twenty vendors and a manufacturer do you really want each of those twenty vendors to have their own unique spin on stock updates and ordering, do you really want the manufacturer to change the schema for its stock report but only find out at runtime? Before we get the "late validation" argument again would you really bet a million dollars on an XPath getting the right value?

Its a wonderful theory around all of this dynamism and automatic change, but writing applications that have to dynamically adapt to schema changes and corporate policies. The last bit is a kicker...

if you're developing client-side code which makes assumptions which aren't true of all resources (or server-side "APIs" which requires this of clients), then you're not using the uniform interface.

Now I've argued before on the fact that things like Invoices cannot not be deleted (they can be cancelled, but that is not a deletion) while other elements (e.g. stock identifiers) can clearly be deleted. If you make an assumption that delete is valid for an invoice then you are a muppet... but you might be using a uniform interface.

Design time interfaces and contracts are a good thing, there are thousands of successful projects and programmes that testify to the fact. Wishing reality away doesn't change this.

Design is a good thing, engineering is a good thing, early agreement reduces back-end problems and the earlier you find the problems the cheaper it is.

Technorati Tags: REST

G is for Google

In trying to book my snowboarding holiday I spoke to a very nice french lady at the resort, we went through various things and she needed my email address. She spelt it back to me using a sort of phonetic alphabet (E is for Elephant etc).

V for Victory as well, J for Janvier, O for Orange. Normal sort of stuff.

G however stood out. G was for "Google".

Now THAT is brand recognition.

Technorati Tags: Google

Think People not roles when driving change

December and January are traditionally months of change in organisations. New things are kicked off, people get promoted, people get moved around and generally things shift around a bit.

This means that you get occasions when there is the discussion about the "organisations chart" or the "roles" that will be required. Now if you are planning and don't know the people then its fine to talk in terms of roles, but remember that fundamentally people drive what they will do and a role specification could help or hinder them. If you know who the people are going to be then its much more effective to think directly in terms of people.

The problem is that quite often people don't like talking about what they want to do, they prefer to talk in terms of abstract roles as "sensible" while of course meaning "I would like to do that". These discussions really do tend to go nowhere as they either end up with people creating very vague roles so everyone thinks they are doing what they want and they all end up conflicting as they try and do it.

Also if you are working within your team or area and you know the skills that you have available then don't say "What the role needs is someone with A grade technical skills who can communicate with the business as well as managing the team perfectly and keep the budget under 5 quid". Well yes this would help, but you don't have access to that person so why define an organisation chart or role definition that can't possibly succeed?

This is critical in SOA when you are looking at assigning people into any form of new approach to delivery or business change. Success will be driven by specific people, not by specific roles. Too often there is the perception of equivalence where none exists just because a title or role is the same.

So plan to succeed with the people you have, and if they don't cut it this means you need different people. Don't hide behind role definitions and complain that people didn't meet them.

Technorati Tags: SOA, Service Architecture

Invocation v Intention

Just a quick note here on Invocation v Intention. In the SOA Methodology there is the "Why" bit that talks about why services (and actors) communicate.

I thought it was worth briefly explaining the difference between invocation and intention in these communications.

The invocation is the actual act itself, it is one service initiating the communication and asking for a specific capability to be delivered. Simplistically the invocation could be seen as an event reception by a service, a message being sent from one service to another or a standard procedure call. The point is that the invocation is about the capability on the destination service.

The intention is why the consumer made the call in the first place and why the producer wanted them to. What was their reason for doing it? Now the uber simplistic view is "to cause the capability to be delivered" but that is just "to get to the other side" to chickens. It doesn't really explain why an invocation was made.

So lets take why sales talk to finance. Nominally the invocation is "Report Sale" or some such. This can have some nice pre-conditions and post-conditions around what that invocation will do but it doesn't explain why Sales every actually calls the capability.

The reason Sales makes the invocation can vary. If they are bonused then the intention of the invocation is "to get paid my bonus". If they are measured on the reporting time then its "to comply with policy". Understanding these differences helps you understand what IT needs to do. Where it needs to smooth the way (policy) and where crappy IT will be overcome by the end users (bonus). This isn't to say you can deliberately have crappy IT if the sales guys are bonused but it does mean that while the users will gripe about the issue they will be personally driven to overcome it.

The other bit is on Finance. What is the intention of finance when they receive that report, why do they want them? Partly it is to measure the sales team and partly to make the finance departments life easier by having accurate numbers. The point here is this.

If your sales numbers suck because of a poor IT solution then you can look at the intentions and see if they can be changed to get around the IT, e.g. if the sales team aren't bonused can you bonus them in a way that is cheaper than replacing the IT system but means the reporting accuracy goes up. If you can't then its quite clear that the IT project to improve the sales reporting to finance isn't something that Sales will be very interested in (after all its a Finance policy they are complying with), this means that the people who will pay for the project will probably be Finance. If the current Sales system does a good job for the sales people but gives crappy reports for Finance then there isn't a motivation for Sales to focus on that area.

This is why its important to understand both the Invocation/Capability and the Intention when modelling services and their interactions.

Technorati Tags: SOA, Service Architecture

Web 2.0 a definition for managers

About 2 years ago now Andy Hedges summed up Web 2.0 to me as

Big Fonts, rounded edges

Although he claims he didn't invent this I'm going to credit it to him as I've not managed to track down a similar quote to get a better reference point for a presentation I'm writing.

Now of course there is the drive towards intimacy and of course there is the concept of the internet and the web as a platform. But seriously, how many sites are just the same as before but with "Big fonts, rounded edges"? Its almost as if the developers convinced management that this was all that was needed.

Same pig, different lipstick.

Technorati Tags: SOA, Service Architecture

Nerds who communicate - or why Europe leads in SOA

Joe McKendrick asks a question in his recent blog post "Europe leads with SOA: if so why?" and it got me to thinking. Reading The Economist in the bath tonight drinking a nice glass of Hine it suddenly stood out from the page

American kids grow up knowing that "nerds are bad, jocks are good". (His focus is exclusively American: in many other countries academically high-achieving children are revered by their peers.)

Now this got me thinking. When I work with European companies the gap between IT and the business tends to be less than that in the US. A main reason for this is respect which comes back to that jock v nerd thing that just doesn't exist and Europe and which is summed up by a conversation I had in Paris one night.

Watching a football game I got chatting to some guys, one of whom turned out to support the same team as me (the one time best team in the world Wolverhampton Wanderers). Later on that night we got chatting at the bar to two French ladies and one of the girls asked "what did you study". Somewhat embarrassed I admitted to having a Bachelor of Engineering degree in Computing (lets face it I assumed they all studied philosophy) while the chap I was with proudly said he studies English at Cambridge. The response back floored the both of us "couldn't you get into an engineering polytechnic?" (Polytechnic meaning something completely different in France).

The point is that I find much bigger cultural differences between IT and the business in the US than I do in Europe and much of it goes back to school and university. In enterprises in Europe I find many more people who can "hold their own" when talking with the business (not about technology) than I do when I'm in the US. The technical skills of the two groups are pretty much similar, potentially with the US being ahead, but the communication skills in Europe are much, much higher.

So to answer Joe's question as to why. IMO its because in Europe being smart, particularly in Engineering, Maths and Science is seen as a positive thing while in the US it appears that to remain popular people have to subjugate this interest or face exclusion and ridicule as a "nerd". Before my first JavaOne (my first big US conference) I remember thinking of myself as nerd, because after all I was CompSci BEng, I did the late hours and I knew the technology. I came back realising that a European nerd can't hold a candle to the social exclusion of their US equivalent.

So why does SOA do better in Europe? Well its because in Europe the business, respects what IT does and IT is able to speak properly to the business. The focus of IT is much less on technology than it is on the business solution and (to be blunt) the girls in marketing will date the architects in IT.

Technorati Tags: SOA, Service Architecture

The rise of the JVM as a platform?

There has been a lot of talk about the issues of Java bloat recently and where Java should go. Back last year (1 day after Google presentation was released) I did a presentation on Java futures at Java Night.

But via Stefan to Ted there appears to be more talk about using the JVM as a platform rather than extending Java to do everything. I once said on a public forum

Remember people Java is LANGUAGE while JVM is just another target PLATFORM albeit a portable one.

Now this makes a lot of sense really, especially when you consider virtualisation, so the JVM becomes the basic execution environment and you have a (limited) set of languages with defined interoperation via the JVM. I also like the solution of having a cut down Java that just has the core in it (something I've proposed for JavaSE 7)

Oh and now for the smug bit. I said the above quote in January 1997, glad to see people catching up :)

Technorati Tags: Java

The problem with clever people - SOA and talent management

I've had the pleasure in my career of working with some really very clever master technologists. The sort of people who really do deliver 10x more than the person they sit next to. I've observed over the years however that these people create a series of problems in IT but one in particular that is lethal.

• False equivalence - This is a really bad one. You get people who are nominally the same level or grade as the master and assume they must therefore be as good

I've seen this quite a few times. The egos of the others at the same level are bruised due to the success of the person with actual talent and so they try and drive a similar agenda or result often with disastrous consequences for the company. The talented person will often be forced into collaboration with their "peer" and have to submit to some form of group decision on the way forward.

The challenge here for companies is talent management, and oddly this is something where SOA can really help when viewed from a business perspective. The point is that the talented person has the greatest ability to make a major change in how the company approaches the market and will probably get bored when doing cost reduction and spreadsheet based tracking... the sort of stuff the average folks seem to prefer as its simple. What you need therefore is some sort of way of understanding where you should put these people

(Seriously I'm loving Google Presentation right now)
The purpose of this heatmap is to show where the company should invest (red) and where they should either cost reduce or move to a utility model (blue). This means aligning your smart folks directly with the business and giving them the challenge of delivering the value and the change and aligning your average folks with the IT challenges of cost reduction and shift to utility. Putting the talent in areas that drive no value is dumb, and is probably a quick way to lose those people.

This is all about changing the way you organise IT and making sure that you don't create false equivalence. Different KPIs means different agendas which means different mentality and approach which in turn means ensuring that the talent can have the impact without the rest trying to compete.

Technorati Tags: SOA, Service Architecture

It doesn't have buy-in? Then cancel it.

I thought I'd share a little story with you today. Speaking with a friend the other day he told me a great experience he had with a project... he cancelled it.

Why was this so great? Well he looked at the project and saw that the business wasn't really committing the people to it that it needed to properly succeed. This would mean that the project would drag on longer than it needed to and would most likely fail. His response was to pull the technical resources off the project until the business could focus on it. It wasn't there the budget didn't exist it was that the budget looked like being wasted.

The end result was a bunch of saved money and a project that has slipped by over a year because actually there was more important stuff to do.

All to often I hear the complaint in IT that a project failed because the business didn't engage. The question is why did they try and deliver if the business wasn't interested?

Technorati Tags: SOA, Service Architecture

Reuse v Use and why projects deliver failure

The normally rational Stu Charlton made a claim recently that

SOA reuse is a pipe dream for most organizations because they are not willing to change their investment evaluation windows or mindset about the economics of software. Most are just looking to improve their agility -- which is about the way we design interfaces & interactions, not about reused logic.

Now there are quite a few problems with this because what it implies is that an organisation that doesn't change their mindset can get agility through technology but can't do reuse.

This really doesn't make sense. I've argued for a long time that the important element is changing mindset around SOA and that if you don't do that you just get the same problems delivered via newer technology. This means you won't get agility for exactly the same reasons that reuse is difficult when the mindset doesn't change. Agility also means caring about the lifecycle costs and it means making investments in those interfaces and interactions that are broader in scope than simply the development being done today.

There is a lot of other sense in there (except the slightly odd bit where it seems to imply that Google and Amazon could change the definition of Pi, but I think that might be a different Pi to the Pi that Indiana attempted to change).

The point to be made here is that all of these things like flexibility, agility and reuse require a mindset change for most organisations. Primarily this comes down to a simple case of measures and KPIs. Most IT is delivered by projects, the goal of projects is to hit the budget and hit the delivery date, as I've said before this is about CTL v TCO. If you are targeted on hitting a date and budget then any corner that is cut results which means you hit those measures means you are a superstar, any extra investment you make that means you overspend or slip means you are a failure.

The other piece is that Reuse is really an IT concept, the business has a different one, namely use and this is where SOA comes in and where it becomes actually possible to deliver standardised functions that are used by all. Services are about Use not reuse. This behaviour is much easier to create in organisations than reuse as demonstrated by the ERP consolidation programmes, single customer view solutions and other elements that are regularly commissioned by the Business and IT to deliver standardisation.

If IT thinks the same way about delivery then there is no way that either Reuse or agility will be delivered. But its nonsense to say that Reuse is more of a pipedream than Agility when the operational reality of IT is that large parts of the IT estate are standardised in packaged solutions which are being exposed as services into the rest of the enterprise.

I agree with Stu that the problem is the mindset, but I'd extend it to say that the mindset challenge is primarily with IT and its continued religious belief in technology as the primary answer.

Technorati Tags: SOA, Service Architecture

SOA Methodology

Last year the nice folks at BeJUG invited me to their Enterprise SOA event to talk on the topic of "SOA Methodology". They've now converted the video, linked it up to the presentation and its all online at their parley site. The presentation deck is embedded here for those who want to play "guess what the pictures mean" before listening to the presentation.

Technorati Tags: SOA, Service Architecture

Business Service Bus (BSB) - a specification

Last year I put up a post about how I viewed the ESB market. This is basically the model I recommend to large organisations when looking at the various ESB and EAI products, namely don't think that a kitchen sink approach will work and quite often you want a less powerful product that will help you enforce policy.

With help from the folks at Progress Software we wrote a definition of what the BSB would be. This was pretty much ready early last year but I've been a bit busy with work so it had to stay on the back burner so its just been used by some companies internally. Now however its time to put this BSB specification out there.

Comments welcomed. The point of the BSB specification is to be the complete opposite of most product roadmaps these days, it isn't about having more widgets or buzzwords its all about having a very simple set of capabilities that can be used in very powerful ways. I've thought about it as a sort of MQSeries for SOA, not the MQSI/MQWorkflow/CrossWorlds extensions but the core superb product that is MQSeries (or WebSphere MQ to give it the new branded name). The bit that made MQSeries powerful wasn't that it did loads of fancy things but that it did (and does) spectacularly well a small set of important features.

The point of the BSB specific is to declare a minimum set of functionality that is required to manage interaction between disparate business domains. This minimum set of functionality is also (pretty much) the maximum allowed set of functionality. The Non-requirements are in many ways more important that the requirements as they exclude things that make compliance and management more difficult. These are the core requirements and non-requirements from the BSB Specification (which explains them in more detail).

Requirements for a BSB
The following elements must all be true for a BSB to be considered complete:

1. Homogeneous in both technology standards and document formats, typically this means WS-* and XML.
2. Specifically the BSB must support WS-Reliable Messaging, WS-Security, WS-Policy and WS-Addressing as a minimum.
3. Interact across different business concerns using asynchronous and synchronous Web Service calls, (although all uses of async and sync must remain symmetric—i.e. no async to sync conversion within the BSB).
4. Be targeted at communication and interaction and not application development
5. A BSB product must allow the automated and scripted migration of it’s configuration between development, testing and production to reduce any technical overhead in it’s deployment
6. A full BSB environment must provide a mechanism for the testing and validation of services and service interactions through the Bus
7. A BSB must be deployed in a federated model and contain no single point of failure
   

Non-Requirements

The following are the things that the BSB must not allow

1. Although the BSB is to support both synchronous and asynchronous requests we explicitly exclude async to sync conversion or sync to async. Requests should be made in the right format.
2. Transport to Transport conversion (e.g. SMTP to HTTP or JMS to HTTP) inside the BSB.
   
3. Any process implementation beyond the simple pipe-lining described below
4. Routing that would involve the use of external application-level meta-data or introspection into message content will expressly be outside the role of the BSB. Such logic should generally exist with in the DSB layer but may be permissible within an exit in certain circumstances. Use within an exit however, should be confined to only inter-domain routing policies and should not reflect application/business level considerations unique or local to one or more connected domains.
5. System driven routing (i.e. those routings that may require system availability status as meta-data to drive the routing behaviour) should as well be confined to those routings that are expressly relevant to the inter-domain policies being enforced, such as SLA. BSB level routing behaviour should never exert a policy assertion that would be specific to business logic associated with a domain.
6. Likewise, transformations should be thought of the same way—as transformations necessary for inter-domain operation and interoperability.
   

The reason for writing this was that I was a bit fed up with the ever growing complexity of ESBs and the focus more on new functionality rather than a small, tight core that would work effectively and allow organisations to govern their SOA environment successfully. Less really is more when it comes to governance. I don't want all the bells and whistles, I want to mandate a set of standards and approaches that can be simply enforced and thus use simplicity to drive re-use and adoption.

Technorati Tags: SOA, Service Architecture

Take it away till it works

Back in the 90s when I was doing CORBA and GUI work there was a little mantra that I'd learnt that I can't for the life of me remember where it came from, its something however that applies equally to IT systems as it does to GUI work. Its a bit like JAGNI but its used to actively help reduce requirements. The rules are simple

1. What do you want to do - state it clearly and concisely
2. Define your requirements
3. Take away requirements until you don't meet the objective

Basically what you should do in a project is look at all the requirements and remove them one by one, only replacing them if you now don't deliver 80% of the required business case. The key is to keep saying "NO" over and over again and saying "its in stage 2" to all of the other requirements. What we used to find in GUIs is that once people used a clean interface that didn't have all the bells and whistles people didn't ask for them to be added as they liked using the clean interface. I've found the same thing with other solutions and in particular the deliver of Services. Its amazing how much more powerful a simple service that does a given job well can be and how people come to cope with a simple service and don't demand more complexity.

So in 2008 don't ask what you can do, ask what you can't.

Technorati Tags: SOA, Service Architecture

Chairman Mao and the problems of IT

Reading the economist over Christmas (oh and seriously get a subscription) there was a great article about how Chairman Mao is an ideal "role model" for the poor quality manager. Reading it however it became clear that 90% of IT also appears to follow the way of Mao when approaching their jobs.

So if you aren't actually any good at IT then here is how to bluff your way using the Mao guide to IT. This applies to Architects, Managers, Vendors and even developers who just don't feel they have the talent to get anywhere

A powerful, mendacious slogan

For Mao it was "Serve the People", given that he "... lived like an emperor, carried on litters by peasants, surrounded by concubines and placated by everyone. this really was an impressive slogan. The point here is that the slogan is an outright lie, but its an outright lie that is talked about honestly and with conviction but doesn't bear any relationship to reality.

In IT things like "Business Focused", "Customer Service" and "Delivering value" can be used to mask what is essentially a continual drive to use new technologies and to avoid talking to the business or customers if at all possible. In fact its a case of business as usual with everything being now justified as "required" by the over arching vision.

Ummm looking at all those vendor slogans makes me think that Chairman Mao's PR department is alive and well in IT. Huge massive friendly slogans backed by software that makes you cry and costs a fortune.

Ruthless media manipulation
Have posters printed, set up a web page, have an award in fact do anything that makes it look like you are doing this while in fact doing nothing of the sort. This tends to be a real management favourite, a new initiative, a new name and the same old behaviours and thinking. A big internal email campaign, possibly even some external press and most of all controlling all of the communication that goes between IT and the business.

Ever had a manager tell you "you can't talk to the business"? Or how about "tell me what you want and I'll find out" and then become convinced that they haven't talked to the business at all? What about architects who claim to "understand the business" and therefore don't let anyone else go direct?

Other bits here are in the creation of fake statistics and successes is another area here. The classic "well it was fine when I left it" and claiming improvements in areas where there aren't any metrics being formally gathered. This is where people claim "we've improved quality/productivity/time to market by X%" when all the evidence seems to show that in fact its got worse. The key is that the person manipulating and broadcasting the stats is the person responsible for them.

In IT departments the media manipulation tends to be internal rather than with the press but its the continual planting of stories and broadcasting of success (and others failures). A few other good ones here are where a project fails but its "all the fault of the vendor" or where a technology turns out to be rubbish and that is "down to the implementation team" the key is spinning and there are lots of people in IT who seem to think that spinning is as important as delivering.

Vendors are truly the best at this in terms of their external marketing. The product might be a bug-riddled piece of crap that won't even install from the CDs but thanks to some smart marketing, demoware and presentations its lauded as being a market leader.

Sacrifice of friends and colleagues
Now here is where IT often moves away from Mao in that I tend to see Cabals and cliques being the dominant factor with competing groups battling for supremacy. But then again isn't this exactly what Mao was about? Basically using different groups within the organisation as the scapegoats? Project fails because in reality the architecture was completely rubbish? Blame the project manager, the development team or India but never, ever allow people to point out that the architecture was unimplementable.
This is something I see over and over again as IT projects fail. The finger pointing starts and the "winner" is the one who gets the least blame and places the most blame elsewhere. Again this isn't actually about reality its about using other people to take the fall for you so you don't have to admit to failure yourself.

With Vendors I guess this would be how they stab each other in the back, but I'm not sure its as good an analogy.

Activity substituting for achievement
Now this is the area where IT, and in particular Architects, come into their own. Those constant internal meetings, those document reviews and those endless process improvement efforts that never improve anything. Its hard to imagine a group of people who create more noise for little achievement that those found in IT. In 2007 I saw people proudly demonstrate how they could use WSDLs in their tools... 2007? That was done in 2001. There are all the various frameworks out there, something IT developer love to do. Don't develop the solution, develop a framework and never get around to the solution.

With vendors this is all about adding new bells and whistles to the products, and to be fair to the vendors this is what the Chairman Mao impersonators in corporate IT and the blog world are asking them to do. Its not about making it better its more like "Pimp my product" with lots of new bling being added on to the same old crap in the mistaken belief that this actually makes it better.

Part of the problem is that the talent gap in IT is ridiculously huge. There really are a huge number of people who are, when all is said and done, just bluffing at their jobs and who follow (unknowingly) Mao's plans for success on a daily basis.

IT and Chairman Mao, maybe the required reading shouldn't be The Mythical Man Month but the mass murders' Little Red Book.

Technorati Tags: SOA, Service Architecture