Why thought is more important than action

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation.
- Fred Brooks - No Silver Bullet

Now that essay is in what I think of as the entrance exam to a job in software, if you haven't read it you really shouldn't be working in computing.

I thought it was worth bringing up again as there have been some bits recently around SOA v BPM which have been misunderstood as me bashing BPM. This isn't the case at all, what I'm saying is that you need to think first before implementing BPM and that SOA represents the best way to create the conceptual construct for BPM, and that BPM is about the labour of representing that construct.

The key here is that the important piece is the thought and planning of that initial piece, not the implementation of the ideas. It is much more valuable and powerful to have the correct conceptual framework before you start on the implementation.

This is why I say that BPM screws up SOA because BPM does not create a conceptual framework, it creates an implementation approach which is a set of steps linked together. There are people who practice true Business Process engineering who create the conceptual framework for process, and to me that looks much like the Business Service approach I try and do in that it considers the parts of the business and their boundaries before worrying about any simple issue of steps.

Hell don't stop at Fred though, Adam Smith apparently has some good ideas as well.

"A journey of a thousand miles begins with a single step" is another famous saying, of course its not actually true, the first thing to work out is exactly where you are going and how to get there otherwise you could spend 40 years just wandering across a pretty small desert.

SOA is about thought, and thought makes better action.

Addition: It appears to be a bigger meme today than normal, someone else is bigging up the book

Technorati Tags: SOA, Service Architecture

CYA - or the art of doing bugger all through paperwork

Cover your arse (or ass for those in North America, although why you cover donkeys I don't know) or CYA is one of the most crippling inhibitors to change in any organisation. Often dressed up in the weasel words of "considering all the possibilities" it is all about making any job seem so much bigger than it actually is.

IT, in particular Enterprise Architecture and IT operations, can be the major creators of CYA within a project or organisation. Asking people to consider every potential outcome, and embarking on long studies to consider the full long term impacts on the Albanian watch industry of that SAP upgrade or application to support the marketing campaign.

These people delight in the "what if" scenario and fight back against suggestions like "There aren't even 10 billion people on the planet" by accusing the rationalists of being unprofessional and just trying to get things out the door without the proper controls.

You can tell organisations that suffer from this problem because they tend to be burdened down with paper, paper that is very rarely read even once after being written. Projects have to complete lots of different assessments and studies to get anything working, all due to the fear that if something does go wrong that it will bounce back. Because after all if you do all the paperwork and still produce a crappy system at least you can prove you did the bad work according to the process.

During projects this comes down to project managers asking for daily status reports, Enterprise Architects asking for "justification documents" on technical decisions that make you think about "how will this work in 5 years time" while operations ask for a full set of documentation, training courses, security audits, penetration tests and the like before you can even speak to them.

Now some people will howl "but you do need to think about these things" and to that I say "sometimes you do, sometimes you don't" and there in is the problem. IT organisations rarely think about what is appropriate for the job being done. In the same way as IT sets up projects to be rubbish in support so the CYA culture makes sure that everything moves at the same snails pace.

What is the solution? Well in part its about more formalism around specifications so you can get rid of all of these reports because you can say "look the SLA on the service says 99.99 availability and here is the test suite for it" and "look the Security Policy says that all Albanian Watch sellers are banned from using this". This increase in formalism means a decrease in the amount of documentation required because these elements can be used to enforce the boundaries of the service and highlight when those boundaries are in danger of being breached.

The other part is for IT people to be more professional, so not producing crap software and projects, not having documentation so poor that support can't use it and not continually releasing things that don't scale to the market. The CYA crowd also needs to become more proactive in determining what success is for a given element and then applying the right set of principles and audit to that solution rather than a blanket one size fits all.

Formalism and Professionalism, is it too much to ask?

Technorati Tags: SOA, Service Architecture

Google Roulette

Recently I've been travelling quite a lot and I've been playing a new and dangerous game, one that unfortunately is soon to be outlawed so get your kicks while you can.

What you do is this. You decide to not use Word to create a new document and decide to give Google Docs instead. Everything goes fine in the office and then it comes time to go home. Now I'm a big fan of the train system over cars for commuting so I then have the choice... do I shutdown and not work on the train, do I create a scratch pad then cut or paste or do I play Google Roulette and recently I've been choosing the later every time.

All you do is not shutdown the browser and carry on editing even when you are offline. Google starts freaking out with warning messages after a minute or so which in a Windows Vista sense have to be okayed everytime (I want a "I'm playing Google Roulette leave me alone"). On the short to commute to my home from work the risk is minimal, I'm on a MacBook Pro so the reliability is pretty solid, I can add to the excitement running it inside the Parallels image and getting the random reboot requests that corporate IT likes to throw in.

But the real excitement comes on a long train journey or a flight, especially a flight. On the train its just a matter of will the battery fail, is there enough power to keep sleep mode until I connect back to the power, should I have the battery at 10mins to go, or is 3 enough? Then when I get to the hotel and there is no internet connection... will Google upgrade Docs while I am disconnect thus meaning that the sync won't work?

On planes the excitement gets more extreme as it becomes a challenge between myself, security and the airline... will they see that flashing light and insist that I fully turn the machine off... will the security people ask me to take the battery out for no good reason... will I make the mistake of thinking "oh bugger 5 mins power left, I'd better switch batteries" and suddenly realise that you can only do the switch when the thing is plugged in.

So far in about a month of travel I've had 2 machine failures (one via security and one via a battery change) and nothing else which isn't bad really... its almost exactly the same as I would have had if I'd been editing the document on my computer normally.

Google Roulette... its as dangerous as fishing for goldfish.

Technorati Tags: SOA, Service Architecture

PATCH for REST? Let the religious wars begin

From Stefan through to a change request memo for HTTP it looks like some people think that REST needs more than just PUT, POST, GET and DELETE. The new addition is "PATCH", which is basically a sub-section PUT on a resource.

So how will REST cope with this new interloper, a new verb being added to the mix? But surely GET, PUT, POST, DELETE are all you need to describe everything in the multi-verse? This can only go one way..

Religious war time!

I can see it now, one camp loudly proclaiming the sanctity of the 4 true verbs and holding onto the original fielding paper as proof of their validity. These people decry the "PATCHites" as heretics and bemoan their lack of understanding of the one true way. "You are doing it wrong" they will say "if you have to use patch its because your resources are to complex" and advocating that large documents should be split into many resources, thus meaning that PATCH is redundant because it can be replaced with PUT on these sub-resources. They will storm into conferences and meetings where PATCH is proposed shouting "Down with PATCH, hold true to your belief in Fielding", they will create badly drawn T-Shirts, most probably in Black with "NO to PATCH" emblazoned across them. A few will promise not to shower or shave until PATCH is defeated, but many will suspect that this is just laziness and actually what they do anyway.

Updated after comments from Stefan and Mark:
Some RESTafarians elect to retreat to distant mountain tops and observe the battle from on high, looking down on the destruction below they tut "REST is an abstract concept untainted by mere practicalities. We are unconcerned by practicality and as in theory there is no difference between theory and practice
end update to take account of REST != HTTP :)

Meanwhile on the other side the PATCHites decry the fundamentalist Fieldingites and their lack of acceptance that the world has moved on and that architectural purity is not always possible "Look" they shout "we have use cases and real world examples of why this is needed". The PATCHites will decry the lack of vision of the Fieldingites and accuse them of living in ivory towers. The PATCHites will adopt a campaign of extensive blogging and commenting and will try and fix ballots by creating multiple accounts. PATCHites will convince random business people to pretend they care and have quotes from analysts obtained after heavy drinking sessions and photoshopped pictures. PATCHites will create White T-Shirts with a picture of Darwin on it and a slogan that says "HTTP must evolve, support Patch" in letters so small as to be unreadable from over 2 feet away.

PATCHites will point to the draft specifications that were authored by Fielding and which contain a reference to PATCH saying "The original vision was always for PATCH", the Fieldingites will respond "That was put in there by evil forces trying to tempt RESTafarians away from the one true path, the only document we believe is the final published specification that has been anointed" they will claim that using unanointed specifications is a grand heresy is it implies that REST was not perfect in its conception. Using early specifications means nothing the Fieldingites will claim as they show only the tempation and challenge rather than being a definitive truth.

And there will be much gnashing of teeth and smiting on websites, newsgroups, blogs and email lists, eventually the PATCHites will be faced with a break-away faction who propose not only PATCH but that "CREATE" should be considered as the correct way to create new resources rather than a simple "PUT", these Creationists will be decryed by both the PATCHites and Fieldingites as heretics and cast into the outer darkness. PATCHites will resource to kidnapping of Fieldingites by promising them free lunches at the Googleplex and then make them work in a real company for six months as a PERL code maintainer. The war will escalate to the stage at which no debate involving any term HTTP, REST or Web will be safe and will descend into a bloody flame war.

Finally a peace will fall over the land as the PATCHites and Fieldingites realise that no-one has been listening and that people have either being getting around the lack of "PATCH" by having parametrised PUTs to automatically create sub-resources, or just didn't need it in the first place anyway because they were using only small size resources. So they will agree to put away those differences and gang up on the true heresy that is WS-*.

Technorati Tags: REST

YAGNI, Requirements and why scaling isn't always important

I've had a few discussions recently where people have gone on about using approach X (o r R) because it would give "full security", "massive scalability" or some other non-functional nirvana. This is normally said without actually asking what the requirements are and a response from me of "but it doesn't need that sort of security" or "its only got one user" is met with "you must prepare for the future, what if it goes on the internet tomorrow?".

This is a classic technical argument, and one that is hugely divorced from business reality. Having the perfect solution for a business problem does not mean that the solution has the "best" technical architecture, it means that it is good enough for the job.

Scalability is the easiest place to over engineer solutions. Back in 2001 I architected a solution that used stateful web services, I did this as the web service provided a security session into the back end application (basically if you didn't authenticate then the service wasn't connected to the backend). It worked a treat, scaled fine because there was only one consumer of the service they were a call centre and they had a single point of interaction with the service and all their requests went through that one session. It worked, it went live. Would it scale to 10,000 users? Nope. But then that was NEVER in the business case and was NEVER going to happen.

By separating the backend from the information exchange it then becomes possible to have different interfaces on the same logic that provide different scaling approaches. All to often however people want to architect the whole system based around that information exchange.

Split information exchange from the business services, and worry about the scaling that is appropriate for your information exchange. Don't worry about technical purity and some "wonder" architectural approach. Don't over engineer because if you do X (or R) then it will scale to 100,000 users, but your requirements say "6".

Business requirements should drive your decisions on scalability, not a technical discussion on what is possible. Scale to what is needed, not to what is dreamt.

Obsession with technical purity is a major challenge in IT, and its unlikely the business will take IT people seriously while they continue to obsess about things that don't have business requirements.

Technorati Tags: SOA, Service Architecture

Bold and old predictions from Microsoft

Remember all those Bill G predictions that turned out to be not so visionary?... well It appears that this ability isn't limited to the top of the company, BillG has managed to instill this through out the organisation.

While flying to Amsterdam the other day (a very nice walking city) on business I was reading through the British Airways in flight magazine "High-life". On pages 125 and 126 was a double page advert from Microsoft entitled

"Everything My Computer Science Teacher Taught Me Was Wrong"

Its an article by Andrew Herbert who is one of their R&D leads. Its basically trying to say how technology is rapidly changing and what that means and it makes 5 predictions for major change in IT, which refers back to the CS teacher being wrong.

The first one is The Single Threaded Program in which he says this is a "20th century idea that has become obsolete", now I have a bit of an issue here because I'm sitting in my home office right now and looking behind me I can see one of my University text books "Communicating Sequential Processes" which was published in 1985. Now I don't know what they were teaching at Cambridge in the 20th Century, but certainly up at York it was assumed that multi-threaded was the way to go. So this is certainly an old prediction as its predicting that the past will happen. Its a good point to stress however that single threaded applications are very limiting but I hope that Mr Herbert isn't right when he says "it's going to change the way we teach programming" as I'd hope that all good universities had been teaching multi-threading for decades.

Next up is an even less bold statement when Andrew predicts the end of "Low-level programming languages", which contains the cracking line (this is 2007 remember) "Once considered an extravagant use of memory, compilers are now essential tools", this is even less of a prediction than the multi-threading one. The debate on assembler v compiler was pretty much answered in the 1970s and by the 1980s it was a question of how high level the programming language was rather than assembler v C/Ada/Smalltalk/LISP etc. This area finishes up with an amazingly bold prediction however "We are moving towards designing yet higher level languages with greater levels of automation and self-checking that eliminate programming mistakes"... hang on did I get that right

moving towards [...] higher level languages [...] that eliminate programming mistakes

Yes I did read it right.... Oh boy, moving from the old to the practically and mathematically impossible, apart from issues like the Halting Problem and Busy Beaver there is the basic challenge that nothing can eliminate wilful stupidity. Reduce yes... eliminate no.

The next one is the first real prediction as Andrew predicts then end to "Screens on Desks" basically saying that future displays will just be "there" as part of the wall or surface by either projection or direct integration. Now here I'm with him. The ability to not have this monitor and use all of the wall space in front of me would be cracking when I'm working. Its not overly bold as some of this technology exists today, but its a pretty good prediction that can be measured in a reasonable (10 year or less) time frame and you can see the business case for it.

"Virtual Memory" or Disk swapping of memory to disk is the next thing that is going to die as memory is ceasing to become an issue. Fair enough really again as its a solid prediction that can be measurable, for instance by the next version of Windows after Vista not having Virtual Memory support. What are the odds on that though?

In the last paragraph for Virtual Memory is another sub-prediction (like the programming languages one) that boldly predicts two things "Within a few years we will probably be able to carry a terabyte of personal storage, enough to hold all the audio and video you'd want to use in a lifetime". So its a "few years" (lets say 5) to get 1TB. This seems fair enough as you can get 1TB USB drives these days and the iPod is already at 80Gb, with other players significantly higher. So 1TB in 5 years seem a rock solid prediction that can be measured by having 1TB personal players available in that time. The 2nd part though is whether 1TB is enough, and here I'd have to say that I've already passed the 1TB level and I'm hopefully years away from being dead. I've taken 30+ hours of video, and lets assume that in future everyone will do this in HD (or more) which means a (post compression) bit rate of around 750KBs, which means I've already created over 1TB in video alone. So the question is whether I've watched over 30 hours of video/film/TV in my lifetime... and the answer clearly has to be yes. So its a prediction but its definitely not a valid one. 1TB is a lot, but its not all you will ever need.

The final prediction is the death of "Hierarchical File Systems" By which he means the OS storing stuff in that sort of format and users accessing it like that. His prediction, again measurable by the next version of Windows (and Linux), is that this will be replaced by "Modern Database technology" which is where it puts his predictions at odds with Google who seem fine with just leaving stuff around and letting the search find it. And isn't this what BeOS sort of did?

Its always brave when people predict the future, so good on Andrew Herbert for doing that. But to describe these things as "top five obsolete software ideas" says much more about the mindset of an organisation that thought they were valid approachs in the late 20th century than it does about a radical shift happening today. Out of the 5 predictions, two are already mainstream and have been for over 20 years, and 2 are predictions for the future but about hardware (screens & Virtual Memory) and the other one is about operating system specifics that has already been released in a product in the 20th Century, and which was originally meant to be in Windows Vista.

But remember folks "1TB of storage is all you will ever need" can now be added to the list of foolish predictions.

Technorati Tags: Microsoft

Why BPM screws up SOA

I've often argued that Services are containers for implementation and that processes are not the ultimate thing in business, and a key part of this is the idea that Service != operation. This last piece appears to be one of the most beguiling myths about BPM and SOA and its surfaced yet again:

"SOA is an architectural style the recognizes services (functionality representing process steps) as components."
Jack Van Hoof

Emphasis mine. But I'm willing to give Jack the benefit of the doubt as there are two ways of reading that. The less charitable reading is the one that was jumped on someone talking about CRM (quell surprise)

"Let me start by defining that a service is a specific business process step that can be composed and reused in different business processes."
Kjell-Sverre Jerijærvi

Now this really does leave no room for maneuver, and it is completely and utterly wrong. This is one of the big challenges of BPM and SOA in that if you start with BPM, which is about co-ordinating steps, then suddenly every service looks like a step. I've seen this problem on several occasions now, and heard it repeated by many others so it looks to be pretty endemic in BPM driven solutions.

To be 100% crystal clear. If you are doing BPM and then just saying "step = service" then you are doing VISUAL Cobol and replacing function calls with service. The fact that you are using WS-* or XML does not make these elements services. As the SOA-RM says "A service is a mechanism to enable access to one or more capabilities", so it is possible for it to be a single capability, but that is certainly not the only, or indeed the most likely, number of capabilities in a service.

BPM driven SOA tends to make bad SOA as it is driven from a procedural and process view, has poor separation of concerns and is mostly all about driving things from the technology perspective.

SOA makes great BPM, BPM makes crappy SOA.

Technorati Tags: SOA, BPM, Service Architecture

Why business people say SAP

Over on the Yahoo email list there was a comment made that business people say "SAP" and "Oracle". This is true, but only because they've been made to say it. Get three sales execs in a room talking about their sales approach and strategy and you'd be amazed how often they don't say "Siebel". Get a supply chain guy chatting to a warehouse guy and its amazing how often SAP, Oracle, GLog or whatever won't pass their lips.

Business people know IT terms because we've made them learn because most of IT is too lazy to understand the business terms and its too easy just to fob people off with a brand or a code word. What the business really wants to ask for is optimisation and efficiency in their language, and SAP comes closer to doing that than the IT department that implements it.

Technorati Tags: SOA, Service Architecture

How to use Javascript to circumvent security

One of the things that I've found playing around with Google gadgets and Javascript recently is that these XML libraries very helpfully enable you to pull down XML content from pretty much any site you like. I found this out by accident while playing with the Yahoo libraries and some external software to see if I could pull stuff from my internal machine and put it out onto the external ones. I could.

Now what this means is that with a bit of basic coding, maybe something as simple as doing a loop on http://10.0.0.1/feed and just upping the ip address you might be able to get hold of internal feeds and then submit that content externally. Because the javascript is living in the browser there is no security around the retrieval or submission, because the users SSO will get them internally and they are already sorted on their company proxy.http://www.blogger.com/img/gl.link.gif

As more platforms support "standard" RSS & Atom feeds for information which have standard URI names then the ability of such hunt and find techniques will be much more effective, and because you can use a remote script load approach you can keep your cracking script up to date based on what is working and what is not.

IBM have a good article that goes into the challenges of javascript (and the script tag) and security and what this means for mashups.

Technorati Tags: Javascript, Security

People and consistency - or Republicans and Science

One of the challenges of dealing with people is that given almost exactly the same information they can leap to two completely different conclusions. I've just sat opened jawed in front of the Republican Candidate debate on two successive questions

First was "Do you accept Evolution"
Second was "Do you accept Global Warming"

On the first one a candidate (don't know the name I'm afraid) said the great phrase "some people might want to believe that they come from primates, however far that gets them", this was then followed up on the next question by "we have to accept what the scientists tell us".... sheer genius.

Now what has this to do with SOA? Well simply put its all about communication and expectation. If you have two services that look similar it isn't enough to just assume that they are because the reference points for the developers might have been completely different, one might be paranoid about security and require a million hoops, while the other doesn't care about reliability so runs it off their mobile phone. Equally when dealing with consumers you shouldn't assume that they are doing the right thing, or maybe even know what they are doing.

The Republican candidates give a great example as to why its good to validate and check, and cross check. Its just a shame the guy asking the questions didn't spot the glaring inconsistency and allowed what can only be called an error condition to pass un-noticed with who knows what unintended consequences.

Check and verify, the people might be dumber than you think.

Technorati Tags: SOA, Service Architecture

Globalisation and the Jungfrau - its a good thing folks

One other thing that I noticed on the Swiss trip was just how good a thing Globalisation is. Now I know there are a bunch of people in IT who are massively in favour of protectionism, this is normally because they fall on the wrong side of Meyfroidt's Law and are part of the IT problem rather than its solution.
Anyway back to Switzerland, normally on a jaunt such as this the demographics of foreign tourists goes

1. Japanese
2. Americans
3. Brits
4. Germans

Sometimes the Americans move up, sometimes the Germans beat the Brits, but pretty much always this is the "top four" in the tourist stakes.

Well on the way up the Jungfrau the list was completely different

1. Indians
2. Japanese
3. Americans
4. Brits
5. Australians
6. Germans

The Aussies slightly cheated by all being under the age of 30 and on a "round Europe" bus and train tour (with no hotels). But the Indians were there at the top on merit, easily the largest set of foreign nationals around.

This is exactly what globalisation is about and exactly why it is a good thing. When people bleat about "jobs going abroad", especially in IT, what it means is that someone elsewhere can do your job for cheaper, which means you need to improve your skills and offer something more... or (and I know its cruel) pick another career. In return those people who now have a better job, earn more money and want to get ahead will come and spend that money in the countries that are actually paying them... because Switzerland really is a cracking place to visit.

The end result is everyone gets a bit richer, and overall everyone is happier. Now I know it sucks if you find your skills out of date and replaced by either machines or people in cheaper economies, but that is exactly the system that works in the world, and exactly the system that has been created by the countries that are now complaining about it.

Globalisation of IT means more people from India spending money in the US and Europe, not just money going to India, and as bunch of tourists they are pretty considerate which is nice for a change.

Technorati Tags: Globalisation

Will Eclipse Solve the BPEL/WSDL problem?

I raised a bug/feature request last year on Eclipse to enable properWSDL/BPEL mapping after I'd talked about the problem of current tools having only BPEL to WSDL (as in 1 BPEL = 1 WSDL) mapping, quick as a flash the eclipse folks responded . But now it looks like it might not make it after all... the latest comment says ominously that

------- Comment #2 From Michal Chmielewski 2007-05-31 18:30:48 [reply] -------

We'll see if we have time for this ...

It will be a real shame if the eclipse project just churns out another 1-to-1 mapper as that really doesn't help BPEL become a first class citizen in a decent SOA.

Technorati Tags: SOA,BPEL,eclipse, Service Architecture