Define the standards FIRST

One of the bits that often surprises, no infact not suprises it stuns me, is the amazing way that people don't define the standards they are going to use for their project, programme or SOA effort right at the start. This means the business, requirements and technical standards.

Starting with the business architecture that means picking your approach to defining the business services. Now you could use my approach or something else but what ever you do it needs to be consistent across the project and across the enterprise if you are doing a broader transformation programme.

On requirements its about structuring those requirements against the business architecture and having a consistent way of matching the requirements against the services and capabilities so you don't get duplication.

These elements are about people processes and documentation and they really aren't hard to set up and its very important that you do this so your documentation is in a consistent format that flows through to delivery and operations.

The final area are the technical standards and this is the area where there really is the least excuse. Saying "but its REST" and claiming that everything will be dynamic is a cop-out and it really just means you are lazy. So in the REST world what you need to do is

1. Agree how you are going to publish the specifications to the resources, how will you say what a "GET" does and what a "POST" does
2. Create some exemplar "services"/resources with the level of documentation required for people to use them
3. Agree a process around Mocking/Proxying to enable people to test and verifying their solutions without waiting for the final solution
4. Agree the test process against the resources and how you will verify that they meet the fixed requirements of the system at that point in time

This last one is important. Some muppet tried to tell me last year that as it was REST that the resource was correct as it was in itself it was the specification of what it should do and the test harnesses should dynamically discover only what the REST implementation already did. This was muppetry of the highest order and after forcing the individual to ingest a copy of the business requirements document we agreed that the current solution didn't match the business requirements no matter how dynamically it failed to do so.

So with REST there are things that you have to do as a project and programme and they take time and experience and you might get them wrong and need them updating. If you've chosen to go Web Services however and you haven't documented your standards then to be frank you really shouldn't be working in IT.

So in Web Service world it really is easy. First off do you want to play safe and solid or do you need lots of call-backs in your Web Services. If you are willing to cope without callbacks then you start off with the easy ones

1. WS-I Basic Profile 1.1
2. WSDL 1.1
3. SOAP 1.1

Now if you want call-backs its into WSDL 2.0 and there are technical advantages to that but you can get hit by some really gnarly XML marshalling and header clashes that exist when going between non-WS-I compliant platforms. You could choose to define your own local version of WS-I compliance based around WSDL 2.0 but most of the time you are better off investing in some decent design and simple approaches like having standard matched schemas for certain process elements and passing the calling service name which can then be resolved via a registry to determine the right call-back service.

Next up you need to decide if you are going WS-* and if so what do you want

1. WS-Security - which version, which spec
2. WS-RM - which version, which spec
3. WS-TX - your kidding right?

For each of these elements it is really important to say which specification you are going to use as some products claim they support a specification but either support an old version or, more impressively, support a version of the standard from before it was even submitted to a standards organisation.

The other pieces is to agree on your standard transport mechanism being HTTP. Seriously its 2010 and its about time that people stopped muttering "performance" and proposing an alternative solution of messaging. If you have real performance issues then go tailored and go binary but 99.999% of the time this would be pointless and you are better off using HTTP/S.

You can define all of these standards before you start a programme and on the technical side there really is little excuse in the REST world and zero excuse in the WS-* world not to do this.
SO

Technorati Tags: SOA, Service Architecture

REST Blueprints and Reference Architectures

Okay so the REST-* stuff appears to have rapidly descended into pointless diatribe which is a shame. One of the questions is what it should be instead (starting with REST-TX and REST-Messaging wasn't a great idea) and around a few internal and external discussions its come down to a few points

1. What is "best practice"
2. What is the standard way to document the interactions available & required
3. How do we add new MIME types

Quite a bit of the technical basics have been done but before we start worrying about a "standard" way of defining reliability in a REST world (yes GET is idempotent.... what about POST?) we should at least agree on what good looks like.

Back in the day Miko created the "SOA Blueprint" work around Web Services, an attempt to define a concrete definition of "good", unfortunately it died in OASIS (mainly due to lack of vendor engagement) but I think the principles would be well applied here.

The other piece that is good (IMO) is the SOA Reference Model, Roy Fielding's paper pretty much defines that reference model but what it doesn't have is a reference architecture. Saying "The internet is the reference architecture" doesn't really help much as that is like saying that a mountain is a reference architecture for a pyramid.

Now one of the elements here is that there appears to be some parts of the REST community that feel that Enterprise computing must all "jump" to REST and the internet or are in fact therefore irrelevant to REST. This isn't very constructive as the vast majority of people employed in IT are employed in just those areas. B2B and M2M communications with a decent dose of integration are the standard problems for most people, not how to integrate with Yahoo & Amazon or build an internet facing website.

For the enterprise we have to sacrifice a few cows that hopesfully aren't sacred but I've heard bandied around

1. You can't just say "its discoverable" - if I'm relying on you to ship $500m of products for me then I don't want you messing around with the interface without telling me
2. You can't just say "late validation" - I don't want you making a "best guess" at what I meant and me doing the same, I want to know that you are shipping the right thing to the right place
3. You can't just say "its in the documentation" - I need something to test that you are keeping to your side of the bargin, I don't want just English words telling me stuff I want formal definitions... contracts.
4. You can't just say "look at this URI" - we are embarking on a 5 month project to build something new, you haven't done your stuff yet, you don't have a URI yet and I need to Mock up your side and you need to Mock mine while we develop towards the release date. Iterative is good but we still need to have a formal clue as to what we are doing
5. You can't say 'that isn't REST' if you don't have something objective to measure it against

So what I'd suggest is that rather than having the REST-* piece looking at the technical standards we should really be focusing on the basics mentioned above. We should use Roy's paper as the Reference Model from which an enterprise Reference Architecture can be created and agree on a standard documentation approach for the technical part of that reference architecture.

In other words

1. REST Reference Model - Roy's paper - Done
2. REST Reference Architecture - TBD and not HTTP centric
3. REST Blueprints - Building the RA into a concrete example with agreed documentation approaches (including project specific MIME types)
   

Right now burn me at the stake as a heretic

Technorati Tags: REST, Blueprints

REST-* can you please grow up

Well didn't Mark Little just thrown in a grenade today around REST-* by daring to suggest that maybe just maybe there needs to be a bit more clarity on how to use REST effectively.

As he said this "The REST-* effort might end up documenting what already exists which indicates that part of the challenge is that lots of people don't really know what REST is and certainly struggle as they look to build higher class systems and interoperate between organisations.

Part of this is of course about up-front contracts and the early v late validation questions. But part of it also appears to be pure snobbery and a desire to retain arcane knowledge which goes back to that "Art v Engineering" debate.

A few choice quotes from twitter

"Dear REST-*. Get a fucking clue. Atom and AtomPub already do messaging. No new specification needed, that's just bullshit busy work." - Jim Webber


"REST might lack clear guidelines, but something called REST-* with a bunch of vendors is hardly going to help!" - Jim again

"and if they think REST lacks guidelines for messaging/security/reliability/etc.., they're not looking hard enough" - Mark Baker

Now part of Mark Little's point appears to be that we need more clarity around what good should be in the REST world and this needs to be easier to access than it currently is. I've seen some things described as REST that were truly horrific and I've seen other bits in REST that were a superb use of that approach. The problem that all of them had was in learning about Atom, AtomPub how to use them, how to use MIME types and of course the balance between up front contracts and late evaluation.

Would it really be such a bad thing to have an effort that got people together and had them agree on the best practices and then have the vendors support developers in delivering against that practice?

The answer of course is only yes if you want to remain "1337" with your arcane skills where you can abuse people for their lack of knowledge of AtomPub and decry their use of POST where quite clearly a DELETE should have been used.

If REST really is some sword of Damocles that can cut through the integration challenges of enterprises and the world then what is the problem with documenting how it does this and having a few standards that make it much clearer how people should be developing. Most importantly so people (SAP and Oracle for instance) can create REST interfaces in a standardised way that can be simply consumed by other vendors solutions. It can decide whether WADL is required and whether Atom and AtomPub really cover all of the enterprise scenarios or at least all of the ones that count (i.e. lets not have a REST-TX to match the abomination of WS-TX).

This shouldn't be an effort like WS-*, its first stage should be to do what Mark Little suggested and just document what is already there is a consistent and agreed manner which vendors, developers and enterprises can agree on as the starting point and that this starting point would be clearly documented under some form of "standards" process.

Would that be a bad thing?

Update: Just found out that one of the two things that they want to do is REST-TX... its like two blind men fighting.

Technorati Tags: SOA, Service Architecture

Start on the road to SaaS with SPUD

Lots of companies are leaping onto SaaS in the manner of a drowning man grabbing onto a tiger shark in the vain hope that all will be fine. The problem is that they haven't really thought about what they want and what it takes to properly adopt SaaS so what they do is take the same old approach to package adoption

1. Work out what you want to do
2. Buy a package/SaaS solution to do some of it
3. Customise the package/SaaS solution to a point where its specific to your business
4. Act surprised when it goes over budget, over time and turns out to be a nightmare to maintain

SaaS can actually be worse in the 4th point that traditional packages as at least if you do the truly dumb of changing the database schema under Oracle or SAP then you have the choice of not upgrading something that just isn't possible with SaaS where all of your customisations and specific developments have to be upgraded when the SaaS vendor says you do.

The right way to adopt SaaS is really the right way to adopt packages. Its not rocket science and comes down to a very a simple point

You've chosen to use a package as you don't see this area of your business as differentiating

If you do think its differentiating then why on earth are you picking something that all of your competitors can buy? If you think its "10%" differentiating then you are probably kidding yourself and in reality its a case of its 100% non-differentiating but there is something you should be building on-top of the package that could add something. In otherwords you've got something extra that differentiates, not something different.

Anyway so once you've faced the reality what is the obvious conclusion?

Package implementation is about changing the business to fit the package not the other way around

So you need to focus on fitting the business to the best-practice that you've purchased from the package vendor and delivering a Standardised Package (SP) as quickly as possible. Not unsurprisingly companies that do this tend to find their package implementations hit time and budget much more often and they tend to deliver greater operational efficiencies as it challenges locally held truths that turn out to just be historical relics. The point here is that in reality you are undertaking a business programme that is just enabled by IT and the package vendor is providing the business processes.

Now the next question is of course how do you pay for this sort of solution? Well its now not about differentiation so its really about cost to serve which means that what you want to do is pay based on the business utility that is being provided. Implementation aside this means that the running of the solution should scale up and down in-line with that utility. The utility could be number of staff, number of transactions, value of transactions or anything else that represents what the business is actually paying for. This means that the package needs to be utility delivered (UD).

Hence the best way to adopt SaaS is really to look at it as just one delivery mechanism the core is really around two key decisions, the first is the business decision

• Standardised Package (SP)

The second is the economic decision

• Utility Delivered (UD)

Hence the best way to adopt SaaS is to ignore SaaS and concentrate on what you are trying to achieve as a business, which is SPUD. SaaS is the UD part of SPUD but to make it successful you need to do the SP bit as well.

Technorati Tags: SOA, Service Architecture

Contracts are for producers and consumers

Jetset Dan Creswell is attending a rather interesting sounding conference in Seattle and put up a post around scalability which talks about a presentation he has been to.

A key part of Amazon’s approach to defining service contracts is SLA’s. Conventional wisdom for SLA’s is that they are a one-way contract but in fact they should be considered as two-way contracts (what the service promises and how it is to be used).

Now saying that "Amazon are spot on" might sound like a fan-boy comment, but this really is one of the major issues with contracts and SLAs as they are currently provided by standards and technology vendors. I wrote a paper back in 2005 for IEEE Software on "Toward an acceptable definition of service" which was pretty much all about how contracts and SLAs are important (and I've since talked more about the need for WS-Contract and WS-SLA).

Again this stuff isn't new. Bertrand Meyer's "Design by Contract" approach went into some detail around the concept of contracts, grouping them into three bits

• Pre-conditions - things that must be true before you call me
• Post-conditions - things I promise will be true after I complete
• Invariants - things I promise won't change

It is very sad indeed that the level of formalism available in Eiffel is not present in today's "modern" technologies. Contracts apply to the interaction between consumer and producer and hence should apply to both parties, and they cover the data, functional and non-functional aspects of the interaction. Most importantly for the future of IT and the sharing of services these contracts need to be formally defined, managed and enforced. Having this information on paper helps only in arbitration after there has been an issue, which helps only lawyers and accountants. By having technical standards for contracts and SLAs we can simplify the process of sharing services and better understand the impacts that using a service will have.

Now can anyone think of a good reason why vendors aren't helping to create contracts?

Technorati Tags: SOA, Service Architecture

Predictions for 2007

Okay its time to don my cap of predictions for 2007, the following should be thought about as reliable as the average horoscope, i.e. I'm making them up off the top of my head.

Products and vendors

1. First off 2007 will the year of the SOA technical "platform" not exactly news as quite a few people are claiming this today, but 2007 will be the year that really sees this become useful. With Oracle, BEA and IBM all looking like having pretty major releases next years its going to be an entertaining marketplace.
2. Smaller vendors will struggle even more, the justifications for picking niche products will increase.
   
3. The "rich and fat" ESB model will die on its arse. Products will start to clearly split the communications infrastructure from the application infrastructure.
4. Microsoft will slip further behind in the enterprise space while they wait for Longhorn server
5. IBM will finally admit that it does have a cohesive strategy based around J2EE and that the non-J2EE bits are going to EOL.
6. Rumours of BEA being bought out by Chelsea FC will abound, then die away once Roman Abramovich realises they don't posses a top quality international striker.
7. SCA will become the "accepted" enterprise way of doing things
8. REST will be delivered into the stacks so they can remain buzzword compliant, REST advocates will denounce them both as heretics and as proof that REST is the "answer".
   
9. Business level modelling will continue to be a pipe dream, filled either with overly complex tools or insanely technical ones.
10. Oracle will buy some more companies, probably including some sort of registry
    
11. IBM will buy some more companies, probably focused around provisioning and management
    
12. Windows Workflow exploits will show up in the wild
13. Some product vendors will finally get the difference between product and application interfaces and stop confusing the two.
14. Questions will be asked about why you have to pay so much money for an invoicing process in an ERP.
15. Java being Open Sourced will not be the big "wow" that Slashdot predicted
16. ERP vendors will start to get their heads around SaaS licensing models
17. Hardware virtualisation will become the "norm" for new deployments
    

WS-*

1. WS-Contract and WS-SLA will remain in the future while WS-* concentrates on more technical challenges.
2. WS-* will continue to be plagued by insanely simple bugs in various implementations, but vendors will hopefully each have just one WS stack (rather than all having multiples like they do now).
3. BPEL 2.0 will go up a hype curve like almost no technology in history... people will then complain about their visual COBOL applications being unmaintainable.
4. WS-* will split into competing factions, those that think everything must be done in "pure" WS-*, and those that think that sometimes its okay to not use the standard way if its actually simpler.

REST

1. REST will start re-creating the sorts of things that WS-* has, so await "RESTful security" and "RESTful reliability" as well as "RESTful resource descriptions" being bandied about.
2. REST will aim for a MIME type explosion, this won't get very far and lead to lots of "local" standards that are nothing of the sort.
3. REST will split into competing factions, those that hold to the "literal truth" of the REST paper, and a more progressive sect who treat it as a series of recommendations that are to be applied with thought.

IT/Business

1. IT will continue to not care about the TCO and will focus on the cost of development
2. Some major IT horror stories will emerge based on SOA and REST "failures" the reality will be that the project was screwed from the start but it was a damned fine scapegoat
3. More engineering and measurable solutions will be required by the business
4. Business will demand more visible value from IT
5. Offshoring will continue, and South America will rise further as an offshore location
6. The business will want to see IT clearly split the utility from the value
7. IT will continue to focus on technical details and miss the business big picture
   

Well that is it for starters like all good horoscopes there are some specific elements that won't come true and a bunch of generalities that I can claim did. But my big prediction for 2007?

1. Sun will finally get their act together and pull all those brilliant minds into a cohesive enterprise strategy and ditch all the fan-boy bells and whistles that have dogged their recent past.

Well I can dream can't I?

Technorati Tags: SOA, 2007,Service Architecture

Rating WS-*

I blogged back in January on the lack of a WS-Contract, so I thought it was time to review the landscape and see what WS-* standards there are these days, and how much wasted effort there has been. I'm excluding all the vertical standards from this (like Translation Web Services et al) as they are addressing specific business problems and so at least its an attempt to work from a business perspective, the goal here is to look at the technical standards.

To this end I'm going to split the WS-* standards into five groups

1. Good Idea - The standards that make sense and I can see being used
2. Good Or Dumb - The standards that have potential to go either way
   
3. DOA - The standards that have either died or shouldn't have been looked at
4. Dangerous Idea - The standards that are down right dangerous and will cause more harm than good
5. MIA - The standards that just don't exist, but which should

Good Idea

• WSDL - Its the interface, its a good idea, and WSDL 2.0 has some decent extensions that will cause issues (callbacks) but are there for a decent reason
• WS-Addressing - It was needed and it will help more complex deployments
• WS-Policy - Great idea, shame its not being universally adopted elsewhere
• WS-BPEL 2.0 - Just need that human workflow in there chaps
• WSDM - Good idea, and it seems like the implementation might be on the way too.]
• WS-RM - Reliability is a good idea
• WS-Trust, WS-Security etc - Damn good idea to do this in a standardised way
• WS-Federation - Federated things are good, centralised things are bad
  

Good Or Dumb

• Semantic Web Services - Could very easily turn into intellectual masturbation and very little real world practicality. Could also be fronted by decent tools and provide a great way to add more formalism to a contract.
• WS-Context - Looks like a good spec on paper, the devil in this one will be in the implementation, bit worried about the WS-CTX service which looks like a big central attempt to manage context, and the lack of reference to process standards such as WS-BPEL, could DOA in a year.
  
• UDDI - Never realised its grand vision, sure its still going inside some decent products but it clearly isn't the standard it was cracked up to be
• WSN - Web Service Notification, again its down to the implementations here, I haven't seen much out in the wild that indicates its going strong, even though its a 1.3 standard.
• WSRF - Resource management, it sounds beguiling but I'd bet on this moving into Dangerous as product implementations start coming out and making resource state into some complex beast, they could however make it trivial and please the "stateless" invocation crews.

DOA

• WS-Choreography - sounded so good, but just doesn't seem to have the weight behind it
• FWSI - Hadn't even heard of it till I started this post
• WSRP - RSS does a better job
• Web Service Quality Model - Sounded good... but has it gone anywhere?
• WS-Reliability et al - killed off by the better WS-RM standards
  
• WS-Discovery - Jini broadcast discovery for Web Services... oh god.

Dangerous Idea

• WS-TX - Two Phase Commit via Web Services, I'm sorry this is in my dumb ideas group. People will expose this via the internet and then "wonder" why they are having massive performance problems. If something is so closely bound in that it needs 2PC, then build it tightly bound, not using Web Services.
• WS-BusinessActivity - Shifts logic around into the ether, not a great idea

MIA

• WS-Contract - Defining business contracts on services, include pre, post and invariants
• WS-SLA - defining the SLA for both the service, and the invocation.

So there are quite a few good ideas, and a whole heap of not very good ideas. But it is good to see that the basics of security, reliability and identity are being covered. To be honest its better than I expected to see, and I've deliberately excluded all of the random WS-* proposals that have never made it into a multi-consortium group or a standards body.

Any I've missed or any ratings that are wrong?

Technorati Tags: SOA,WS, Web Services,Service Architecture