Why contracts give flexibility....

As per the last post there has been the standard "flashback" from a bunch of people who crusade first and think later.

So lets explain in simple terms how strict contracts give flexibility, and to make it really easy I'm going to use four examples, firstly REST (which the "don't validate" crowd follow) and secondly LEGO(r) which the "I don't understand SOA but I'm going to use this as an analogy" crowd try to go for. The final examples are... well I won't spoil the surprise... So I should cover lots of bases here

So first off lets look at HTTP, it has a very strict contract with a limited number of possible actions, GET/POST/PUT/etc and an extremely limited way of getting to those actions. Now interestingly the entire concept of REST is that this limited and tightly specified set of actions when rigorously enforced provides an infinitely flexible architecture on which to built things. So HTTP is in fact an area which has a pretty tight set of up front validation (try sending random packets to Apache or Firefox to see where it gets you) and an extremely limited and well specified set of actions with which to interact with the server.

So simply put the REST people who are claiming that you shouldn't validate are exactly the same people who are claiming that you should rigidly adhere to a tight specification and reject anything that (for instance) stops GET being idempotent. So here is a great example of a tight specification being used as the implementation approach for complete flexibility.

Next up lets take Lego, you have a "brick" it has a bottom which has set dimensions and a top which has set dimensions, amazingly despite limiting the specification of the brick to having exactly the same form and function top and bottom the potential for Lego is practically infinite, you can do Star Wars, Air Force One and really odd stuff. This is all managed via a very strict contract as to how the different bricks connect.

My third example is the wonderful world of IKEA. IKEA uses pretty much the same screws in every thing they do, they use the same sort of connectors in everything they do, and lots of times they use the same size of wood, just combined in different ways. Having tight specifications like this that mean they only need certain sizes of doweling or screw gives them an immense (and profitable) flexibility in what they can create.

Tight specification can often be used as the basis of flexibility rather than rigidity, but having these tight specification means that you, as an engineer, need to understand the boundaries within which you operate.

The final example comes from the wonderful world of Pimp my Ride and more particularly the world of tyres and "rims". Having a set size for wheels and a tight specification doesn't decrease the flexibility it positively increases it. There are a massive amount of tyre and rim combinations available so one car could have three tyres from one manufacture alone and a set of 4 rims from the first hit on Google. For the mathematically minded that is 12 different combinations on a single crappy Ford Focus.

This is why specifications are engineering, if you could say "I wish that I could just do this because its cool" then you'd be a liberal arts major. Having specifications can increase flexibility when you know how to use them and use them in a smart way. This is the reason that screws, nails, hinges, doors, trains and lots of other things obey a specification. Specifications are the best way to get things to inter-operate. The REST crowd should know this better than most with their mantra of "standard HTTP", but unfortunately too often people are beguiled by an idea which makes them contradict the benefits of what they are saying is right.

Specifications give flexibility because they tell people what can and cannot be done, people are then very adept at modifying their behaviour and their world to cope with these limitations.

It would be completely wrong to say that this was Darwin in action, but its completely right to say its how adaptation works.

Technorati Tags: SOA, REST, Service Architecture

Can we please be engineers not optimists?

Via Stefan Tilkov I came across this Mark Baker article and well I'm going to set myself up on the opposite side of the fence here, in fact I'm going to go over that fence, across the field and sit in the pub watching him sink slowly into a quagmire.

One of the worst things I see in IT over and over again is the rage against formalism and verification, its just bizarre how often it happens. Now Mark might have a great new idea for time dependent evolvable validation... but for me he is wrong in principle.

First of all lets start with a the basic tenant that everyone should hold dear

KISS - Keep it Simple Stupid

And another is one of the few new Agile terms that I liked which formalised an age old saying

YAGNI - You aren't going to need it.

And before I get into it also have a look at The second System effect from the best book in IT.

When dealing with 3rd parties a basic rule of thumb is trust no-one and CYA (Cover your ass) this means verifying both what you send out as well as verifying what you receive. There are very good engineering reasons for this. Lets say that you have said that there is a limit of two items per order, as that is the business decision. If someone sends 2344 you want to reject that straight away, and schema verification is a great way to do that.

Engineering means designing a system that works for the task it is required to do, and if one of the things it needs to do is allow change then it needs to allow that change in a controlled and measured way.

The concept that Mark puts forwards of validating only at the last possible moment is somewhat beguiling as it has the idea that everything will turn out right, but if the poor bugger on the order system is expecting massive numbers then the odds are he won't have coded it expect them. This isn't their fault, if you say "its 1 or 2" then its pointless him writing loads of code to handle a million items.

Validating at the edges however does require you accept that when change happens it may break things. But the reality is that most of the time this is what you want. if you are dealing with someone who suddenly sends in a new document with a field that says "by responding to this message you are agreeing to give us all your money and your first born child" then you'd like to know.

Validate as early as possible, make sure that things are correct and hold people to account as early as possible. I've seen companies lose literally millions of pounds (billions of dollars :) because they didn't enforce interface contracts effectively, this led to problems down the line and systems that were very hard to untangle.

There are an awful lot of stupid people out there and it is your job to protect your systems and your business from their stupidity, this means not trusting what they sent you and checking it yourself for validity. Optimism is not a good engineering trait, pessismism is. That is why you validate what you send to make sure it doesn't come back on you, and validate what you receive to make sure the other person isn't an idiot.

To use a physical engineering analogy, you check that the wing is put on properly as soon as it is attached, not by looking out of the window at 30,000ft to see if it is still there.

Technorati Tags: SOA, Service Architecture

REST URI naming convention

Over on the Yahoo SOA list there has been a REST discussion where I've continued to not see the benefits of REST for prime time business development....

But there was also a discussion about URIs and whether they should be opaque (meaningless) or sensibly named. I'd like to make a REST proposal

All URIs in REST must be meaningful, they should clearly articulate the type of resource and its place in the hierarchy, individual resource identification should be done via a GUID. The GUID is used to standardise the use of IDs and to provide a clear identifier of the meaningful and opaque parts of theURI.

Thus

http://www.foo.com/customer/invoice

would be a "good" URI starting point and

http://www.foo.com/customer/invoice/8900183b-fb31-4ee7-a5be-c78ada4b2312

would be an individual resource identifier.

http://www.foo.com/customer

would be the customer starting point and

http://www.foo.com/customer/1cb792d5-a521-4123-b081-5bd07328bca5

would be a specific customer.

To me the lack of agreement on naming conventions to REST is indicative of its prototyping feel. Agreeing that URIs MUST have meaningful names would make it simpler to debug, simpler for developers to understand and will clearly separate roots from resources.

Comments?

Technorati Tags: REST

Predictions for 2007

Okay its time to don my cap of predictions for 2007, the following should be thought about as reliable as the average horoscope, i.e. I'm making them up off the top of my head.

Products and vendors

1. First off 2007 will the year of the SOA technical "platform" not exactly news as quite a few people are claiming this today, but 2007 will be the year that really sees this become useful. With Oracle, BEA and IBM all looking like having pretty major releases next years its going to be an entertaining marketplace.
2. Smaller vendors will struggle even more, the justifications for picking niche products will increase.
   
3. The "rich and fat" ESB model will die on its arse. Products will start to clearly split the communications infrastructure from the application infrastructure.
4. Microsoft will slip further behind in the enterprise space while they wait for Longhorn server
5. IBM will finally admit that it does have a cohesive strategy based around J2EE and that the non-J2EE bits are going to EOL.
6. Rumours of BEA being bought out by Chelsea FC will abound, then die away once Roman Abramovich realises they don't posses a top quality international striker.
7. SCA will become the "accepted" enterprise way of doing things
8. REST will be delivered into the stacks so they can remain buzzword compliant, REST advocates will denounce them both as heretics and as proof that REST is the "answer".
   
9. Business level modelling will continue to be a pipe dream, filled either with overly complex tools or insanely technical ones.
10. Oracle will buy some more companies, probably including some sort of registry
    
11. IBM will buy some more companies, probably focused around provisioning and management
    
12. Windows Workflow exploits will show up in the wild
13. Some product vendors will finally get the difference between product and application interfaces and stop confusing the two.
14. Questions will be asked about why you have to pay so much money for an invoicing process in an ERP.
15. Java being Open Sourced will not be the big "wow" that Slashdot predicted
16. ERP vendors will start to get their heads around SaaS licensing models
17. Hardware virtualisation will become the "norm" for new deployments
    

WS-*

1. WS-Contract and WS-SLA will remain in the future while WS-* concentrates on more technical challenges.
2. WS-* will continue to be plagued by insanely simple bugs in various implementations, but vendors will hopefully each have just one WS stack (rather than all having multiples like they do now).
3. BPEL 2.0 will go up a hype curve like almost no technology in history... people will then complain about their visual COBOL applications being unmaintainable.
4. WS-* will split into competing factions, those that think everything must be done in "pure" WS-*, and those that think that sometimes its okay to not use the standard way if its actually simpler.

REST

1. REST will start re-creating the sorts of things that WS-* has, so await "RESTful security" and "RESTful reliability" as well as "RESTful resource descriptions" being bandied about.
2. REST will aim for a MIME type explosion, this won't get very far and lead to lots of "local" standards that are nothing of the sort.
3. REST will split into competing factions, those that hold to the "literal truth" of the REST paper, and a more progressive sect who treat it as a series of recommendations that are to be applied with thought.

IT/Business

1. IT will continue to not care about the TCO and will focus on the cost of development
2. Some major IT horror stories will emerge based on SOA and REST "failures" the reality will be that the project was screwed from the start but it was a damned fine scapegoat
3. More engineering and measurable solutions will be required by the business
4. Business will demand more visible value from IT
5. Offshoring will continue, and South America will rise further as an offshore location
6. The business will want to see IT clearly split the utility from the value
7. IT will continue to focus on technical details and miss the business big picture
   

Well that is it for starters like all good horoscopes there are some specific elements that won't come true and a bunch of generalities that I can claim did. But my big prediction for 2007?

1. Sun will finally get their act together and pull all those brilliant minds into a cohesive enterprise strategy and ditch all the fan-boy bells and whistles that have dogged their recent past.

Well I can dream can't I?

Technorati Tags: SOA, 2007,Service Architecture

SOA is people

With all this REST v WS-* type of arguments flying about I thought I'd remind everyone of the most important thing in delivering successful Service based projects, systems and enterprises.

People

Yes folks the thing that will actually make or break your systems in the majority of cases will be the people. This means the talent of the people defining and delivering and the cultural changes in the wider organisation to accept and take advantage of the services effectively.

All too often in IT's obsession with technology we argue about the pointless and ignore the critical. If your SOA strategy and direction isn't centred around the people and practice changes, then it won't deliver the benefits you expect.

Technorati Tags: SOA, Service Architecture

Rating WS-*

I blogged back in January on the lack of a WS-Contract, so I thought it was time to review the landscape and see what WS-* standards there are these days, and how much wasted effort there has been. I'm excluding all the vertical standards from this (like Translation Web Services et al) as they are addressing specific business problems and so at least its an attempt to work from a business perspective, the goal here is to look at the technical standards.

To this end I'm going to split the WS-* standards into five groups

1. Good Idea - The standards that make sense and I can see being used
2. Good Or Dumb - The standards that have potential to go either way
   
3. DOA - The standards that have either died or shouldn't have been looked at
4. Dangerous Idea - The standards that are down right dangerous and will cause more harm than good
5. MIA - The standards that just don't exist, but which should

Good Idea

• WSDL - Its the interface, its a good idea, and WSDL 2.0 has some decent extensions that will cause issues (callbacks) but are there for a decent reason
• WS-Addressing - It was needed and it will help more complex deployments
• WS-Policy - Great idea, shame its not being universally adopted elsewhere
• WS-BPEL 2.0 - Just need that human workflow in there chaps
• WSDM - Good idea, and it seems like the implementation might be on the way too.]
• WS-RM - Reliability is a good idea
• WS-Trust, WS-Security etc - Damn good idea to do this in a standardised way
• WS-Federation - Federated things are good, centralised things are bad
  

Good Or Dumb

• Semantic Web Services - Could very easily turn into intellectual masturbation and very little real world practicality. Could also be fronted by decent tools and provide a great way to add more formalism to a contract.
• WS-Context - Looks like a good spec on paper, the devil in this one will be in the implementation, bit worried about the WS-CTX service which looks like a big central attempt to manage context, and the lack of reference to process standards such as WS-BPEL, could DOA in a year.
  
• UDDI - Never realised its grand vision, sure its still going inside some decent products but it clearly isn't the standard it was cracked up to be
• WSN - Web Service Notification, again its down to the implementations here, I haven't seen much out in the wild that indicates its going strong, even though its a 1.3 standard.
• WSRF - Resource management, it sounds beguiling but I'd bet on this moving into Dangerous as product implementations start coming out and making resource state into some complex beast, they could however make it trivial and please the "stateless" invocation crews.

DOA

• WS-Choreography - sounded so good, but just doesn't seem to have the weight behind it
• FWSI - Hadn't even heard of it till I started this post
• WSRP - RSS does a better job
• Web Service Quality Model - Sounded good... but has it gone anywhere?
• WS-Reliability et al - killed off by the better WS-RM standards
  
• WS-Discovery - Jini broadcast discovery for Web Services... oh god.

Dangerous Idea

• WS-TX - Two Phase Commit via Web Services, I'm sorry this is in my dumb ideas group. People will expose this via the internet and then "wonder" why they are having massive performance problems. If something is so closely bound in that it needs 2PC, then build it tightly bound, not using Web Services.
• WS-BusinessActivity - Shifts logic around into the ether, not a great idea

MIA

• WS-Contract - Defining business contracts on services, include pre, post and invariants
• WS-SLA - defining the SLA for both the service, and the invocation.

So there are quite a few good ideas, and a whole heap of not very good ideas. But it is good to see that the basics of security, reliability and identity are being covered. To be honest its better than I expected to see, and I've deliberately excluded all of the random WS-* proposals that have never made it into a multi-consortium group or a standards body.

Any I've missed or any ratings that are wrong?

Technorati Tags: SOA,WS, Web Services,Service Architecture