What to do when insanity reigns. CYA for a reason

Recently I've had one of those experiences where you just have to stand back and think "Either I'm insane or they are". Reviewing the facts its pretty clear that the insanity is on the other side. Slightly worse than that the decision that is about to be made is actually tacitly admitting that its an insane decision.

This is when you need to implement a real CYA policy. CYA is Cover Your Arse this isn't the bad old EA CYA approach, its not even about clear documentation. This is about making sure when the shit inevitably fan that people don't turn round and say "why didn't you say" but also making sure that they don't say "its your fault because you didn't support it".

So the line here on your programme is simple, you know its going to fail, you have to support it through the failure, but when it goes tits up you need to make sure that your well reasoned arguments into the clinically insane decision are well versed.

Stage 1: Store ALL the documents that went into the decision

Stage 2: Write an email saying that you respect the decision and will of course support the approach, but you feel it doesn't address some of the issues that you previously raised

When this email is ignored that is fine, the people making the decision clearly know its insane but have decided that right now insanity is a valid defense. If they reply with "We think it does address those problems" then again don't reply.

Stage 3: Write down a list of the things that are going to go horribly wrong. Get them flagged in the risk register. You don't have a risk register.... holy crap you need more help than I can ever offer, get one in a hurry

Stage 4: It now becomes the PMs job to track against the risks, if you are the PM then make sure you got Amber early, not Red as you will be "obstructive" but go Amber with your concerns. The worst thing in the world is the project that flicks from Green to Red with you saying "told you so".

Stage 5: Its going down hill, your risk report looks like the dance floor in Saturday Night Fever and people have forgotten what colour green actually is. Now is the time to start pushing "resolutions" you really want to be the person who pulls it from the fire, this will involve however Napalming lots of the people who screwed up, you do not want a problem child responsible for clean up as that just means that it will go more wrong.

Stage 6: You are responsible for clean up. The key phrase here is "Drawing a line under what happened before" this means you are going to ignore any previous decisions and base your judgement on what goes forward is what you want. Its the same line as when a failure is put in charge, the difference here is you will be firing people.

Stage 7: Remember that people screwed up against advice. Those people need to find other place to screw up, unless they are in a desperate career saving type place, then they are your perfect allies. Think of reformed smokers or born again Christians... but with a pay cheque.

Stage 8: Do an Obama. The smartest thing Obama has done so far is declare everything screwed. Don't be nice, be ruthless. If there are six months to go live but it will never happen blame it on the prior administration. Then work out what is achievable.

Stage 9: Plan to an end. Don't get caught in fire-fighting, break the current project and start a new plan for a real end game.

Stage 10: Deliver to live. There is nothing that will help you more than meeting the expectations you set in Stages 8 and 9. This makes you a rock star and this gives you the future right to say "this is wrong".

Insanity and stupidity are horrible things, but don't try and ignore them. The worst things I've ever seen are where bright people have dug stupid people out of holes without making that visible. I've seen some really dumb people in roles they couldn't handle as a result and some smart people burnt out because of it.

The Risk Log is your friend. Do your job, do your best. But this isn't the navy. Over throwing the stupid captain is fine, as long as the number back you up.

Technorati Tags: SOA, Service Architecture

User Adoption matters

One of the most annoying things in the WWW and most especially the Web 2.0 world is the Field of Dreams mentality of "build it and they will come". With package projects and business services this is the resort of two groups of people.

1. People who think the technology is the only thing
2. People who are scared of users

Sometimes people fall into both of these groups but the underlying principle is always the same. The technology is enough, its enough to "let people know" and you will then "build a community" which will make it all successful. The problem is that while there are successful internet businesses that were created in part by this approach they also had a couple of other things

1. Marketing
2. A user population the size of the internet

If you are aiming at the mass consumer market on the web then it might be enough to launch it and do some marketing, if however you are doing this internally to your organisation then quite simply it isn't enough.

So how do you drive user adoption? Well the first thing is to find out why users might not use what ever you are proposing. Be negative, get the worst things out there and then one by one mitigate those risks. To do this you've of course got to identify your users and be realistic about them. If you are doing a bug tracking system or a service for fraud analysis its highly unlikely that everyone in the company is going to use it, so set your objectives realistically and see why your user community might not switch.

Next up think about how you are going to market it to the users, yes that's right market it. Again its not enough just to lob an article on your intranet, think about how you are going to communicate what is coming before it is there. Create a comms plan and work out what you are going to tell them when. Maybe even create an internal buzz campaign to make people interested.

Next up look at how you transition users to your system gradually (if you can). If it is a green field system then this is easier as you don't have the data migration challenge. The point of a gradual migration is to start building a reputation for success that can then be used to go after the more challenging groups. If you have to go big bang then make sure it works on day 1. If this means delaying the launch a couple of weeks then try and do that because if you bugger up the launch day they'll remember for a long time, no matter how good it is a few weeks later (look at Heathrow T5 for an example of that).

Finally, and most importantly, don't stop on go-live day. Track usage and adoption and look at who is, and isn't, using the service/package/solution/etc go out and find out what has worked and then have a follow up campaign to get people more engaged. Keep doing this as a core part of the run for the system to make sure that the system is successful in 24 months time, not just 24 minutes after being turned on.

Technorati Tags: SOA, Service Architecture

If you get made redundant

As someone who has been made redundant twice (startups that didn't) and fired once (saying that a company would go bust in 12 months if it didn't change its approach, I was wrong.... it took 18 months) and who graduated during the last downturn I thought I'd give some advice to anyone who finds themselves in IT without a job in the next year, and a few things to do while you are in work to make sure you either keep your job or are better prepared for what ever comes next.

First off lets be clear, being made redundant sucks, its a crappy feeling and it will hit your self-esteem, so the first point is

It sucks, it sucks for everyone, its not just you

You are entitled to mope around for a bit, hell the first time I was made redundant was six months into my first job, my uni girlfriend dumped me and Wolves went out the FA Cup with a dreadful performance... trust me it sucked. The key point however is that you need to drag yourself and realise a few key points

1. No-one owes you a job - you have to go and get it
   
2. You will be competing for your next job - its not an end of year promotion round
3. Your skills need to be marketed

The first two are pretty obvious but its the later where I've seen people struggle. Your internal profile at your company that lists out your skills is completely different to the CV that your competitors for the job are putting in. Its not a question of lying its a question of putting it the right perspective for the job you are going for. If you are after a Java developer job then your CV needs to reek of Java, don't lob in the fact that you did VB for six months on a nightmare project as most Java guys will turn their nose up at that. If you are after an architect job and your last project was on time and budget... don't expect people to know that.

Its really worth spending time crafting your CV and making it something that markets yourself. When you review CVs how often do you look at page 2? How often do you look behind the first half of the first page (especially when reviewing electronically)? Well that is how most other people review as well.

The other piece that some people have to face up to as well is that maybe IT isn't he best area for them to be in. Lots of people have fallen into IT because there were lots of jobs and will struggle when it gets more competitive. Sometimes its worth taking advantage of redundancy to recognise that your career lies elsewhere.

Key things to do while you are at work is get your Google score for work up if you can. Nothing says "different" like being able to find relevant stuff to your job on Google and someone thinking "hey, this guy knows his stuff". I've had people say they are "world class" in a given subject but I've given the job to the person who the world actually knew about.

Another key thing to do at work is start profiling your work for your career. This doesn't mean turning down work it means thinking about how it best reflects what you want to do. If you are managing a team of offshore developers, but you want to be an architect, then make sure you are also doing the governance and oversight stuff.

If the worst does happen and you are made redundant make sure you don't dress like it at interviews, a new suit and shiny shoes (like when you went for your first graduate job) are a must at many companies.

Being made redundant can also open up new doors, I was lucky. My first taste of redundancy got me into Air Traffic Control which ended up with me meeting my wife in Paris. My second led to be working in Denmark which led to my third (getting fired) which led to a solution architect role that got me to where I am now. If I'd been in the same job since university then I wouldn't have done half the things I've done to date.

So start planning now because you aren't just planning for redundancy you are planning for your career, and its that focus that will make sure you aren't out of a job for long.

Being made redundant sucks but not having a plan for your career sucks even worse and is especially critical at a time like this.

Technorati Tags: SOA, Service Architecture

REST of SOA the questions

After the REST of SOA a couple of people came up and asked me questions that could basically be summarised in the following

We've got a server development team that loves REST but as a Flash/Web/Ajax developer its really hard for me to work with

The two main reasons citied were

1. The lack of PUT/DELETE from a browser, but server teams who still wanted to use it
2. The size limit on GET
   

Now on the GET limit and the PUT/DELETE I suggested that the dev team should think about using a Proxy and having that map the URIs from those that work to the web to those that they want to use internally. But what it really came down to was that you had this amazing disconnect between the people doing the server side stuff and the poor buggers who had to consume it.

Technological fanatisicm doesn't work no matter what the technology.

Technorati Tags: SOA, Service Architecture

REST of SOA

So last week at AdobeMAX I did my first public presentation on doing REST and SOA together. Thanks to Duane for that and to the person who dropped out leaving me with the baby :)

Now I know they recorded the audio and video so when I find that I'll link to it.

What I said was that the REST model works in the interactional space of applications, especially in those which are focused around data navigation. I admitted that I found it a bit fan-boyish when it first came out but that there are areas where it does deliver value.

Thanks to Ben Scowen I had a whole set of detail around REST as he has done a massive REST Web programme, so kudos to Ben on that. I also wanted to make sure that people who attended would have some real detail around REST rather than just the picture presentations I normally use.

My major bit on the presentation around REST was the concept of state management in REST and the fact that (for me) this is the bit that really differentiates REST and which is the hardest to get your brain around.

The other major bit was the concept of thinking about the services and then using the URIs and methods as the way to separate the implementation of the services, I used an internal example as a way to do that.

So until Adobe release the audio et al, here is just the powerpoint


What I said throughout was that it was about picking the right tool for the job and understanding what works right in your environment. Some people followed up with questions afterwards that indicated that REST isn't quite the happy place for everyone.

Technorati Tags: SOA, Service Architecture

Have Adobe done what Sun wouldn't?

Sitting here at AdobeMAX I can't help but think that Adobe have created exactly what a bunch of people, including myself, have been campaigning for in the Java space for a while. Namely a lightweight profile that concentrates on the desktop area.

I've said before about profiles being an important direction for Java and to my mind that exactly what Adobe are doing with Flex and AIR. They've created a restricted profile based on a limited (but therefore powerful) set of functionality. What they don't have is a consistent model that works on the server (they use Java for that) but they have clearly defined a profile that works.

The point here is that the "kitchen sink" mentality pushed by the core Java SE team has clearly retarded the ability of companies to innovate and deliver different approaches for different environments. We are seeing this in part in the mobile space where Apple, Google and others (including Adobe) are moving away from J2ME towards more specific approaches which are often around taking desktop technologies and shifting them onto the powerful smartphone platform. With Java SE 6 being so large this just wasn't an option in the Java space (SavaJE have done some work, but who wants all that stuff on a mobile phone? MIDI support?

So while companies like Adobe are demonstrating, as are Microsoft with Silverlight, that new models work we are seeing Java turn into an ever bigger beast. This is really sad as many of the things here at Max are stuff I've seen demo'ed in Java many, many, years ago. The problem therefore is not that Java can't do this stuff but that the trajectory that Java has been on has prevented it.

So can Java get back into the lead? Potentially, but it requires a fundamental reassessment of how Sun view Java, both its core and the profiles and models that are acceptable. With the JCP appearing to be abandoned for Java SE 7 the omens are not great, but you never know. Until then it means we have to all start working in a mixed technology environment which is a pain in the arse, but it just isn't sensible to insist people download a web-server to run your client application.

Technorati Tags: SOA, Service Architecture

Driving IT from the business

A short post here just to once again go over the basic principles of Business driven SOA. The goal here is that IT should be driven from the business this means

1. Understanding what the business is about
2. Understanding where and how IT can help the business
3. Letting the business interact and drive IT on its terms
4. Not thinking that the latest buzzword is the important thing
   

Now the goal is of course the same as before, you want to deliver an IT estate that

1. Looks like the business
2. Is costed based on the value it delivers to the business
3. And evolves like the business

This therefore is about understanding where to apply new technologies and where to just cut costs and cope with what you have, or even move towards outsourcing it because it just isn't important to your business at all.

Business driven SOA is about using the right tool for the right job in the right place in the right way. Its fundamentally got to be about the delivery of technology inline with the overall business. This means it must be constrained by what can be delivered and must be inspired by what could be delivered.

As we hit the down-turn this becomes more important. Taking a technology approach is just burning company money and applying new technologies where they don't deliver the benefit is just intellectual masturbation, but worse as you are wasting the business' money doing it.

So look at the business model, look at the business services and think

"If I had half the budget what could I do and what would really have an impact"

Then look at the saving areas, look at SaaS, look at outsourcing and mainly look at how you measure and reward people to cut costs. After that think about the top line

"Now that I've saved that 50%, where can I invest 25% to drive the company upwards"

That is where you look at impact and look more at modern technologies and approaches.

To do this you have to shift your mind out of IT for a bit and look from the business side. Use your experience and knowledge to frame the approaches and answers but don't forget that the goal here is for the business to be driving.

Business SOA is hard, it requires a real breadth of skills, this isn't delivery to PowerPoint its delivery to live. The business doesn't want to drive a video game, its a real business and it needs a real solution.

Technorati Tags: SOA, Service Architecture

Making the pilot a success

Okay so Gerald is off and running now, platform is deployed, some cracking problems that I'll blog about once we are 100% rolled out but the key right now is that we've got over 20,000 people to roll this out to and we need a champion. This means we need to bite off a bit that is a decent reference case but is also manageable.

All too often people go at these programmes and just because they can roll the technology out to all the people it means that its successful. It isn't successful until everyone is using the solution. This means good references and good credentials and a strong roll-out plan that includes briefings and training.

Technology deployment is easy, but doing it too fast can really hit your ability to deliver successfully. Provisioning everyone of those 20,000 people could be done tomorrow but that would create confusion and push-back which would ultimately lead to the failure of the programme.

So pick your champion, pick your pilot and make them look like a rock star.

Technorati Tags: SOA, Service Architecture

Can the US Government now hack my computer?

Okay so I've decided (for once) to get ahead of the game and register with the new big brother system that will allow me to travel into the US, something that you have to do in advance now

The warning popup is quite a sight

This Department of Homeland Security (DHS) computer system and any related equipment is subject to monitoring for administrative oversight, law enforcement, criminal investigative purposes, inquiries into alleged wrongdoing or misuse, and to ensure proper performance of applicable security features and procedures. As part of this monitoring, DHS may acquire, access, retain, intercept, capture, retrieve, record, read, inspect, analyze, audit, copy and disclose any information processed, transmitted, received, communicated, and stored within the computer system. If monitoring reveals possible misuse or criminal activity, notice of such may be provided to appropriate supervisory personnel and law enforcement officials. DHS may conduct these activities in any manner without further notice. By clicking OK below or by using this system, you consent to the terms set forth in this notice.

The highlight is mine. Does this mean that in order to go to the US I must allow the US government to hack my computer?

The other good bit is that the dialogue only has an OK button, you can't even cancel and get out.

I just thought "you can't do that" then suddenly remembered what the response will be "yes we can".

Technorati Tags: SOA, Service Architecture

MSN v Google.... does Microsoft own MSN?

Just updating a home PC to XP SP3 to install the wonderful Adobe CS4 so I did a search on IE on MSN.com for "XP SP3" and got....

Google of course redirect straight to the XP SP3 download.

Seriously if you can't get your own terms right how credible will people think you are?

The general point here is about credibility. Don't make up numbers that can be easily disproven and make sure that operational tests actually work.

Simply put undersell and over deliver.

Technorati Tags: SOA, Service Architecture

iPhone as a corporate device

Okay, since the 3G came out I've been using it as a corporate device which means email/calendar/contacts. Previously I was on a windows mobile device with a slide out keyboard, which is touted sometimes as being the "best" for corporate comms.

As a corporate device the iPhone was simply superb. There have been some people citing some minor security pieces but seriously the odds of my losing the iPhone as opposed to the last device? My last device was stolen at Seattle Airport by a pick-pocket. My iPhone however was in a more secure pocket... because I care about it.

So lets just look at usability.

Exchange.... well the push sync was as good as expected and it burnt the battery as expected. But the big question is whether the touch keyboard was as "good" as the old slide out. Once yes, Twice yes, a hundred times yes. Simply put its become a joy to use. The only slight niggle I have is that cut-and-paste would help.

Calendar was great, contacts worked great, wish there was voice dialing as standard but its still superb.

And the best bit?

ITS A BRILLIANT PHONE

Which is where most of the Windows Mobile devices I've had fall down. They've been okay as a mobile office client but rubbish as phones. The iPhone is a good phone. It switches to phone mode really well (and from anywhere) the call quality is good and its never dropped a call.

So my conclusion is that not unsurprisingly the iPhone is a great corporate device because its a usable device. Now what would really help for corporate adopt is a "local" version of the iTunes store as a way to manage enterprise client application deployment, deploying corporate apps has long been a pain onto mobile devices, either "click on this link and pray" or "Insert this SD card" with the challenges around updates that this brings. A Corporate iTunes store and iTunes extension would solve all of that. No idea if Apple would do it, but that would turn them from being the cool device in the corporation to being the managed cool device in the corporation.

Technorati Tags: SOA, Service Architecture

A note to vendors on the Church Turing Thesis

At work I regularly get vendors throwing about words like "unique", "revolutionary", "game changing", while of course promising to reduce costs, increase agility and pretty much everything else bar world hunger.

I normally ask standard questions in reply at this point, mainly about what it build upon. What really annoys me is when people then say "its completely new".

No it isn't, its an evolution of something, it might be a clever idea but its not going to be a completely and utterly new solution that no-one in the whole world has ever done anything like it before. Let me introduce you to the Church Turing thesis

Every effectively calculable function (effectively decidable predicate) is general recursive

In other words just TELL me what you are building on and I will have a lot more respect and a lot more interest than just telling me that it is 100% new and original. Snake-oil salesman try that approach a decent vendor should know the roots of their product and be able to explain the previous things that they have built upon. If Isaac Newton was admit standing on the shoulders of giants then a software vendor had better be standing on shoulders of something if they want to be credible.

Your product might be great, but if you can't say from where it comes then I'll just cry snake-oil and ignore you.

Technorati Tags: SOA, Service Architecture

Now we're caching on GAs

GAs = Google AppEngineS. Rubbish I know but what the hell. So I've just added in Memcache support for the map so now its much much quicker to get the map. Now the time taken to do these requests isn't very long at all (0.01 to do the mandelbrot, 0.001 to turn it into a PNG) but the server side caching will speed up the request and lead to it hitting the CPU less often...

Which means that we do still have the option of making the map squares bigger now as it won't hit the CPU as often, which means we won't violate the quota as often.

So in other words performance tuning for the cloud is often about combining different strategies rather than one strategy that works. Making the squares bigger blew out the CPU quota, but if we combine it with the cache then this could reduce the number of times that it blows the quota and thus enables it to continue. This still isn't effecting the page view quota however and that pesky ^R forces a refresh and the 302 redirect also makes sure that its still hitting the server, which is the root of the problem.

Technorati Tags: SOA, Service Architecture

Resize not helping

Well the resize to 100x100 isn't working because its now crapping out because I've got 200+ requests that all smash the CPU limit which blows another quota limit.

So basically you can bend the CPU limit, but if you send 200+ requests in a couple of seconds which all break the limit then that one kicks in as well.

Back to 54x54 and some extra planning.

Technorati Tags: SOA, Service Architecture

Google App Engine performance - Lots and Lots of threads

Just a quick one here. While Google App Engine's Python implementation limits you to a single thread it certainly isn't running a single thread and servicing requests from it. When running locally (where the performance per image is about the same) it certainly does appear to be single threaded as it takes an absolute age and logs are always one request after another. On the server however its a completely different case with multiple requests being served at the same time. This is what the Quota Breaking graphs seem to indicate as its servicing 90 requests a second which would seem to indicate that the App Engine just starts a new thread (or pulls from a thread pool) for each new request and is spreading theses requests across multiple CPUs. The reason I say that later piece is that the performance per image is pretty much the same all the time which indicates each one is getting dedicated time.

So lots of independent threads running on lots of different CPUs but probably sharing the same memory and storage space.

Technorati Tags: SOA, Service Architecture

Google App Engine, not for Web 2.0?

Redirect doesn't help on quota but something else does...



The block size has increased from 54x54 to 100x100 which still fits within the quota at the normal zoom (but is liable to break quota a bit as we zoom in). This moves the number of requests per image down from 625 to 225 which is a decent drop. Of course with the redirect we are at 450 but hopefully we'll be able to get that down with some more strategies.

The point here is that when you are looking to scale against quotas it is important to look at various things not simply the HTTP related elements. If you have a page view quota the easiest thing to do is shift bigger chunks less often.

One point that this does mean however is that Google App Engine isn't overly suited to Web 2.0 applications. It likes big pages rather than having a sexy Web 2.0 interface with lots and lots of requests back to the server. GMail for instance wouldn't be very good on App Engine as its interface is always going back to the server to get new adverts and checking for new emails.

So when looking at what sort of cloud works for you, do think about what sort of application you want to do. If you are doing lots of Web 2.0 small style AJAX requests then you are liable to come a cropper against the page view limit a lot earlier than you thought.

Technorati Tags: SOA, Service Architecture

Redirecting for caching - still not helping on quota

As I said, redirect isn't the solution to the problem but I thought I'd implement it anyway, after all when I do fix the problem its effectively a low cost option anyway.



What this does is shifts via a redirect (using 302 rather than 301 as I might decide on something else in future and let people render what ever they want) to the nearest "valid" box. Valid here is considered to be a box of size (width and height) of a power of 2 and based around a grid with a box starting at 0,0. So effectively we find the nearest power of 2 to the width then just move down from the current point to find the nearest one. Not exactly rocket science and its effectively doubling the number of hits.

Technorati Tags: SOA, Service Architecture

A generic VM that we can run VMs on

A call to arms for a generic VM was put up by Neil McCallister. Now part of this is the "aren't dynamic languages great" camp which really doesn't like operational costs to be factored in.

The other bit though is that this is just funny, the x86 CPU is effectively a generic VM, its what IBM called the microprocessor when they first created the because unlike dedicated previous approaches it could be many different machines just via programming. A "Virtual" Machine in fact.

Now we have the JVM and the CLR, they are both "generic" VMs and one has a much broader platform support and the other has more languages (still not seeing the research around language heterogenity being a good thing BTW). Sun is getting happy clappy with the scripting world and putting them on the JVM so we already sort of have that generic VM.

What a generic VM platform will lead to is people creating specific VMs running ontop of the generic VM for specific purposes. Not sure that this is a good thing but it will happen.

The JVM is just another machine, its a generic VM, all you have to do is write the language port. I knew this in January 1997 so it really isn't rocket science.

If you want your favourite scripting language to run on a "generic" VM then port it to the JVM or the CLR. What is all the fuss about?

Technorati Tags: SOA, Service Architecture

Redirects and the lack of precision

Okay the redirect plan is out of the window because the mathematical precision isn't good enough and we get into infinite redirect loops... Back to plan A which is to calculate a standard bottom left.

20.5 * 0.2 = 4.10000000000000005

No it ruddy doesn't it is 4.1

20.5 / 5 = 4.0999999999999996

Java is just as bad at this so its not a Python fault (although with Java at least its consistently bad, Python is inconsitently bad as it just defaults to the C implementation on the box).

Technorati Tags: SOA, Service Architecture

Caching strategies when redirect doesn't work

So the first challenge of cachability is solving the square problem. Simply put the created squares need to be repeatable when you zoom in and out.

So the calculation today is just find the click point and find the point halfway between that and the bottom left to give you the new bottom left.

The problem is that this is unique. So what we need to find is the right bottom left that is nearest to the new bottom left.


Now one way to do this would be to do a 301 redirect for all requests to the "right" position. This is a perfectly valid way of getting people to use the right resource and of limiting the total space of the resource set. What you are saying in effect is that a request for resource X is in fact a request for resource Y and you should look at the new place to get it. This works fine in this scenario but for one minor problem.

The challenge we have is page views and a 301 redirect counts as a page view, meaning that we'd be doubling the number of page views required to get to a given resource. Valid though this strategy is therefore it isn't the one that is going to work for this cloud application. We need something that will minimise the page views.

But as this is a test.... lets do it anyway!

Technorati Tags: SOA, Service Architecture

Google App Engine performance under heavy load - Part 4

Okay so the first time around the performance under relatively light load was pretty stable. Now given that we are at the Quota Denial Coral would this impact the performance?


First off look at the scale, its not as bad as it looks. We are still talking normally about a 10% range from Min to Max and even the worst case is 16% which is hardly a major issue. But is this a surprise?

No it isn't. The reason is that because of the quota that we are breaking (page views) we aren't actually stressing the CPU quota as much (although using several milli-seconds a second indicates that we are hitting it quite hard). That said however it is still pretty impressive that in a period of time where we are servicing around 90 requests a second the load behaviour is exactly the same, or arguably more stable as the min/max gap is more centred around the average, than when it is significantly lower.

So again the stability of performance of App Engine is looking pretty good independent of the overall load on the engine.

Technorati Tags: SOA, Service Architecture

HTTP Cache and the need for cachability

One of the often cited advantages of REST implemented in HTTP is the easy access to caching which can improve performance and reduce the load on the servers. Now with breaking app engine quota regularly around Mandel Map the obvious solution is to turn on caching. Which I just have by adding the line

self.response.headers['Cache-Control'] = 'max-age=2592000'

Which basically means "don't come and ask me again for a week". Now part of the problem is that hitting reload in a browser forces it to go back to a server anyway but there is a second and more important problem as you mess around with the Map. With the map, double click and zoom in... then hold shift and zoom out again.



Notice how it still re-draws when it zooms back out again? The reason for this is that the zoom in calculation just works around a given point and sets a new bottom left of the overall grid relative to that point. This means that every zoom in and out is pretty much unique (you've got a 1 in 2916 chance of getting back to the cached zoom out version after you have zoomed in).

So while the next time you see the map it will appear much quicker this doesn't actually help you in terms of it working quicker as it zooms in and out or in terms of reducing the server load for people who are mucking about with the Map on a regular basis. The challenge therefore is designing the application for cachability rather than just turning on HTTP Caching and expecting everything to magically work better.

The same principle applies when turning on server side caching (like memcache in Google App Engine). If every users gets a unique set of results then the caching will just burn memory rather than giving you better performance, indeed the performance will get slower as you will have a massively populated cache but have practically no successful hits from requests.

With this application it means that rather than simply do a basic calculation that forms the basis for the zoom it needs to do a calculation that forms a repeatable basis for the zoom. Effectively those 54x54 blocks need to be the same 54x54 blocks at a given zoom level for every request. This will make the "click" a bit less accurate (its not spot on now anyway) but will lead to an application which is much more effectively cachable than the current solution.

So HTTP Cache on its own doesn't make your application perform any better for end users or reduce the load on your servers. You have to design your application so the elements being returned are cachable in a way that will deliver performance improvements. For some applications its trivial, for others (like the Mandelbrot Map) its a little bit harder.

Technorati Tags: SOA, Service Architecture

Google App Engine - Quota breaking on a normal day

Okay after yesterday's quota smashing efforts I turned off the load testing and just let the normal website load go but with the "standard image" that I use for load profiling being requested every 30 seconds. That gives a load of less than 3000 requests in addition to the Mandel Map and gadget requests.

So its pretty clear that requests are WAY down from yesterday at a peak of under 8 requests a second which is down from a sustained load of around 90 requests a second. So how did this impact the quota? Well it appears that once you break the quota that you are going to get caught more often, almost like you get onto a watch list.
Interestingly though you'll note that again the denials don't match straight to demand. There is a whole period of requests where we have no denials and then it kicks in. This indicates that thresholds are being set for periods of time which are fixed rather than rolling, i.e. you have a 24 hour block that is measured and that then sets the quota for the next 24 hour block rather than it being a rolling 24 hour period (where we'd expect to see continual denails against a constant load).
Megacycles are again high on the demand graph but non-existant on the quota graph and the denails don't correspond directly to the highest CPU demand periods. So it does appear (to me) that the CPU piece isn't the issue here (even though its highlighting the number of 1.3x quota requests (that standard image)) but more testing will confirm that.

The last test was to determine whether the data measurement was actually working or not. Again we see the demand graph showing lots of data going forwards and backwards with nearly 4k a second being passed at peak. It takes about 3 Mandel Map requests to generate 1MB of data traffic so it certainly appears that right now Google aren't totting up on the Bandwidth or CPU fronts, its about the easy metrics of page requests and actual time. They are certainly capturing the information (that is what the demand graphs are) but they aren't tracking it as a moving total right now.

Next up I'll look at the performance of that standard image request to see if it fluctuates beyond its 350 - 400 milliseconds normal behaviour. But while I'm doing that I'll lob in some more requests by embedding another Mandel Map

Technorati Tags: SOA, Service Architecture

Google App Engine - breaking quota in a big way

Okay so yesterday's test was easy. Set four browsers running with the Reload plug-in set for every 10 seconds (with one browser set to 5 seconds). This meant that there would be 18,780 hits a minute. Now there are a bunch of quotas on Google App Engine and as I've noticed before its pretty much only the raw time one
that gets culled on an individual request.

So it scales for the cloud in terms of breaking down the problem but now we are running up against another quota. The 5,000,000 page views a month. This sounds like a lot, and it would be if each page was 1 request, but in this AJAX and Web 2.0 each page can be made of lots of small requests (625 images + the main page for starters). Now Google say that they throttle before your limit rather than just going up to it and stopping... and indeed they do.

That shows the requests coming in. Notice the two big troughs? That could be the test machines bandwidth dropping out for a second, or an issue on the App Engine side. More investigation required. That profile of usage soon hit the throttle

This looks like you can take a slashdotting for about 2 hours before the throttle limit kicks in. The throttle is also quickly released when the demand goes down. The issue however here is that it isn't clear how close I am to a quota and how much I have left, there isn't a monthly page count view and, as noted before, the bandwidth and cycles quotas don't appear to work at the moment
It still says I've used 0% of my CPU and bandwidth which is a little bit odd given this really does cane the CPU. Bug fixing required there I think!

So basically so far it appears that App Engine is running on two real quotas, one is the real time that a request takes and the other is related to the number of page views. If you are looking to scale on the cloud it is important to understand the metrics you need to really measure and those which are more informational. As Google become tighter on bandwidth and CPU then those will become real metrics but for now its all about the number of requests and the time those requests take.

Technorati Tags: SOA, Service Architecture

Google App Engine - How many megas in a giga cycle?

Well the Mandel Map testing is well underway to stress out the Google cloud. 12 hours in and there is something a bit odd going on....



Notice that the peak says that it was doing 4328 megacycles a second, and its generally been doing quite a bit with the 1000 megacycle a second barrier being broached on several occasions.

Now for the odd bit. According to the quota bit at the bottom I've used up 0.00 Gigacycles of my quota. Now the data one looks a little strange as well as it is firing back a lot of images and its not registering at all. So despite all of that load I've apparently not made a dent in the Google Apps measurements for CPU cycles or for data transfer. To my simple mind that one peak of 4328 megacycles should be around 4 Gigacycles however you do the maths. It really does seem to be a staggering amount of CPU and bandwidth that is available if all of this usage doesn't even make it to the 2nd decimal place of significance.

So here it is again to see if this helps rack up the numbers!

Technorati Tags: SOA, Service Architecture

I'm not sure if this depresses me or not...

Nanite is a Ruby thing. Apparently its new

Nanite is a new way of thinking about building cloud ready web applications. Having a scalable message queueing backend with all the discovery and dynamic load based dispatch that Nanite has is a very scalable way to construct web application backends.

Now to naive old me it sounds like something not really as good as Jini. Its difference appears to be that it dispatches work to "least loaded nodes" rather than a Jini/Javaspaces approach which would have workers determine who was the least loaded and then retrieve the work from the pool based on when they complete their previous task.

Having a scalable message queueing backends isn't new nor is it unusual to use these to do load-balancing. Some people have moved beyond this for cloud computing.

They say that those who don't learn from history are doomed to repeat it, it IT I think we can add the phrase "poorly".

I'm not that depressed because at least it is using a messaging infrastructure for the dispatch and not using a database as a sync point or something else strange. It would be nice though if the number of times we see "new" technologies that were not as complete as those we used in production 8 years ago was less than the number we see that are clearly better. Currently it appears to be 10 "old/new" inventions to every 1 new invention.... at best.

Now maybe I'm wrong about nanite (which would be nice) and it really is better than all of these older technologies and is a brand new way to think about the cloud, it can't just be an agent/dispatcher pattern can it? If it helps people great, if they learn from some of the other projects that have done this sort of thing before then super great, if we get some genuinely new advances around work distribution then fantastic. I just wish projects would not start with the assumption that they are re-defining how people work in a given area without reference to what they have improved upon.

Technorati Tags: SOA, Service Architecture

Killing a cloud - MandelMap

One of the things talked about with cloud computing is "horizontal scalability" which basically means being able to do lots of small things that are independent of each other. Now some tasks break down horizontally pretty well, others are a bit more of a chore but the challenge is normally how to re-combine those pieces into the overall set of results.

Horizontal scaling in the cloud also requires you to know when something is reaching its peak so you can add new servers. This could be done by the cloud provider, namely they see you are running out of space and they spin up another instance and ensure that it all runs across the same shared data store. This is fine, unless you are paying for those new instances and suddenly find yourself a few thousand dollars a minute down as a result of some coding problems.

Now the Google cloud forces you down this path even more as the last set of Performance metrics showed. Its focus is really short lived transactions which gave a big issue for the Mandelbrot set which is very computationally intensive. Even the little gadget on the right hand side is running at 1.3x quota.

So the solution is then to break the problem down further. Fortunately the Mandelbrot set is perfect for that as each individual point has a known value. So you can break the problem down as far as you want. I went for a series of 54x54 (the gadget is 200x200 so its about 1/13th the number of calculations) squares to create a bigger image. Then I got a little bit silly, thanks to GWT and decided to build a "Mandel Map" in other words use some of the infinite scrolling bits of Google Maps but applied to a Mandelbrot set.



Double click to zoom, shift and double click to zoom out. Drag to scroll around the map. Now the advantage of using the small bits becomes apparent, you can zoom in for miles and still remain within the quota set by Google but the actual effect is that you have a bigger image being displayed. Effectively here the browser is being used as a "late integration" approach to recombine the information at the point of display and act as the point of division for the problem into its smaller chunks. Now this is possible in part thanks to clearly parameterised URIs which can be used by clients to breakdown the problem. This puts a degree of control into the client but has the knock-on effect of more tightly coupling the two together.

Part of the reason for this is to see how Google copes with having a real horizontal hammering (the grid is 5x5x(5x5) so 625 requests for the basic image) and to provide a set of base data for the next challenge which is around how different types of caching can impact performance and how you actually need to design things with caching in mind. The test here is to see how well the standard load (that gadget at the side) copes while people are playing around with the map. This gives us an idea of what the peak scaling of a given image is and whether Google are doing something for us behind the scenes or not.

Yes its not perfect and it could be done in smaller chunks (etc etc) and maybe if I can be bothered I'll tune it. But its just to demonstrate a principle and took a whole couple of hours, if you want the full screen version, well here is MandelMap.

Technorati Tags: SOA, Service Architecture

Compromising for failure - anti-pattern

Description
This is the anti-pattern where people making compromises to, mainly, business stakeholders in order to get things accepted. Unfortunately this leads to a bloating of the programme and a lack of clarity in its goals.

Causes
The causes for this tend to be political intransigence and gaming. Someone will object in order to get their own pet project on the agenda, a political game will then ensue leading to the person getting their pet project in order to support the overall programme, this will be repeated over and over again leading to multiple pet projects being added to the cost, and timescales, of the overall programme.

Effects
A good example of this sort of anti-pattern is the US Congress where "pork" is regularly attached to bills in order to get certain senators or congressmen to vote in favour. The recent $700bn bailout bill for instance had a huge amount of added pork what this means of course is that things get more expensive than originally budgeted and become more complex to administer (more cost again). It also means in delivery projects that there becomes a lack of focus on the key goals of the programme and instead there becomes a focus on getting the pet projects done well to keep "Person X" happy.
Rapidly the programme descends into a fractured mess as the pet project sponsors care only about those elements and not at all about the over all objectives. Team members realising that success can be achieved via a pet project delivery also focus on those areas as its a more immediate return. The overall clarity of the project is lost and it loses direction and ultimately fails.

Resolution
The first part of the resolution is making sure you have a very high level sponsor who can drive over pet projects. To do this you need a clear, and concise, vision that a senior exec will sign off on and then you need to be clear in what does, and what doesn't, drive you towards that objective. The next bit is getting a backbone. Its very easy to give in and put a pet project in, its harder to stand up and say "no, it doesn't fit here" and the key to that is firstly doing it privately making clear your objections and then secondly doing it to the overall sponsor and finally that sponsor then doing it publicly. Rapidly people will stop trying to force in the pork if they find themselves accountable for trying to push it in.

The final bit is about culture, you need to establish a culture of "do one thing well" that focuses people around simple clear objectives and makes any pork pushing attempt ridiculously visible. The more concise your vision and objectives the harder it is for the pork to be added.

Technorati Tags: SOA, Service Architecture

Boiling the SOAcean - anti-pattern

Description
This is the SOA strategy that tries to much and sets its aspirations far too high. Often it starts as a little point problem before some one suggests that "we could do this as well" before long the SOA strategy is listing solving world hunger and the middle east peace process as secondary objectives that will come from meeting the now agreed two objectives of the project, interstellar space travel and universal harmony.

Causes
There are two basic causes of this problem. Firstly its powerpoint driven aspirational stuff with the people involved knowing they have no real worries about actually having to deliver the stuff they are writing about. Secondly its a belief that there is infinite budget available at least in the long term which is the key part of the this anti-pattern. Planning is for a future in which everything is fixed so you might as well get it down on paper today, this leads people to flights of fancy on trying to conceive of all possible outcomes and how SOA could be used to solve them. The centre to this anti-pattern is the lack of grounding that it has in reality, when combined with aspirational planning this leads to the sort of programme that is doomed to under achieve as it would be impossible to deliver.

Effects
The biggest effects are those of perception, firstly people will think that the programme is over stretching itself (they'd be right) and secondly will be disappointed when it fails to get close to its goals. The lack of focus in the aspirational planning often means that expensive investment is made in infrastructure for potential future requirements. These projects are often cancelled after making that infrastructure investment but not delivering the level of benefits expected for the spend.

Resolution
The key to resolving this problem is grounding it in reality. Push the programme into tight iterations (no more than 3 months) with each iteration delivering quantifiable business benefits this helps to move the focus away from infrastructure and towards the end game and also helps to shift the event horizon of the programme into something more manageable, around a year is ideal. The next bit is to adopt YAGNI as a core principle, infrastructure is procured no more than 1 iteration ahead (i.e. if you are on 1 month cycles then you procure at 2 months) with the first iteration being a selection process.

The final piece is making people get their hands dirty, aspirational planning comes from people who deliver to powerpoint rather than to live, make them responsible for go-lives and it will focus their mind on achievable objectives.

Technorati Tags: SOA, Service Architecture

We're all going on an architecture hunt

I'm not scared, we can't go over it, we can't go under it we'll have to go through it.

Duane talk about Forensic architecture. Its a great read on the lessons that he has learnt. The only thing I'll add is that I don't think we are CSI, we are more Bones, hell its pretty much Jurassic Park out there. Documenting architecture after the fact, as Duane says, isn't so much the exception, its the career.

Technorati Tags: SOA, Service Architecture

Pink Floyd meets Frank Sinatra - anti-pattern

Description
This anti-pattern is all about training and learning. Its the "we don't need no education" anti-pattern. It occurs when organisations want to "do everything themselves" and develop in isolation from other efforts and experiences. There can be some reading done but the predominant theme is that it will be done "My Way".

Effect
The effect of this is that the organisation starts creating its own definitions of terms. These are created in a "folksonomy" manner from local opinions rather than taking reference to external sources. Common elements are definitions of "services" that map to things that already exist (see Defensive SOA) rather than looking at a new approach. The other side is that when using technology the company will "learn themselves" how to do the implementation with very little involvement from either the vendor or experienced 3rd parties, the reason for this will be that the company wishes to "learn themselves" how to do SOA, the reality is that they are concerned that external inputs will demonstrate that their current approaches don't work.

The effect of this is that SOA becomes simply another label attached to the companies standard way of working and any changes are purely superficial. SOA rapidly becomes a "best practice we've done for years" and new technology projects fail to exploit the new technology successfully so little, or negative, improvement is seen in productivity.

Causes
The causes of this is an inward facing IT department which sees external companies, including vendors, as competitors rather than collaborators. Sometimes this is combined with a huge arrogance that the company knows best, but mostly its a mistaken belief that in order to remain in control you must do everything yourself. A large part of the issue comes from a fear of new ideas being demonstrated as measurably better than what went before and when tied up with a blame culture this results in a need to protect reputations over open collaboration and improvement.

Resolution
The first part of the resolution is to assess whether you really are that good. If you turn around and find that you are the favoured love child of Google and Amazon and Bill Joy, James Gosling and Donald Knuth are fit only to be junior project managers then carry on, you are right and you are the best. If however you only think that Bill Joy is rather smart, or even employ Bill Joy then remember his mantra at Sun "Most smart people work elsewhere". The second stage is to get rid of the Blame element. You've done an okay job and you want to do a better job. That needs to be seen as a good thing not a bad thing, it needs to be seen as the right behaviour. The next stage is a budgetary one, how will you measure the benefit of external spend and how will you justify it. You need to have clear cases on what you expect external spend to bring and how you will measure its impact. This way you keep in control and use external people to help you improve where you are either weak or where you don't want to waste your time.

Technorati Tags: SOA, Service Architecture

My way or the highway - anti-pattern

Description
This anti-pattern is about delivery processes, given a major goal of SOA is to enable business flexibility its amazing how organisations still insist on having a single delivery process for every type of delivery, or a best two approaches, one for package and one for the rest.

Effect
The effect here is that there is no link between the business drivers and requirements and the type of delivery process employed. No matter whether agility is required or a single compliance driven delivery its always a single process that is applied. This means seeing package projects capturing requirements rather than doing a best-fit analysis, it means seeing agile processes where a vanilla package implementation would be better and it means seeing software development projects doing waterfall because that is what is used on vanilla package implementations.

The end effect is that while the services might map well to the business objectives the delivery process prevents this being realised in a way that matches the way in which the business wants the SOA to be delivered. This gets SOA a reputation as either expensive (e.g. when packages get over customised) or slow (waterfall on a software development package).

Cause
The cause of this tends to be an over reliance on project management in the IT department and a fixed approach. The use of Waterfall across the board is often driven by the finance departments approval processes which mandate that everything must be known and priced before a project can start. Often these fixed processes are part of a broader "quality" initiative where the quality approval people find it easier to have a single process to audit than understanding and coping with different ways of working that actually deliver more quality

Resolution
The resolution is to understand the differing business and technical objectives that business services have and to tailor your process appropriately. This means increasing the ceremony where you have complex contracts and external interactions, it means doing waterfall when you have a vanilla package implementation and are going to change the business to fit and it means having co-located teams when you want true agility and pace of change.

This means convincing the quality people, or convincing the business that the quality people are getting in the way of quality. It also means shooting the sacred cows in the project management organisation and investing in the training and mentoring required. It also involves explaining to finance how different financial approval models make sense in different areas of the business. Its a whole lot of explaining but there is thankfully a whole lot of data out there on how single process approaches tend to destroy IT estates.

Technorati Tags: SOA, Service Architecture

DOA - antipattern

Description
The opposite end to the percolating process anti-pattern this is the one where people assume that data is the centre of the universe.

Effect
With this anti-pattern you see data being driven as the centre of everything, services are all related to data processing and the first set of services being defined are about "key" data objects in the organisation. The actual actions and capabilities aren't captured its all about the data so the Customer service has a series of CRUD methods on it, but nothing that captures how a new customer is really added to the organisation. The impact of this is that it becomes an abstraction above the database but can lead to data silos being created, it can also lead to very inflexible architectures especially where these data services try to include both the "live" and in-process data as well as the post transactional data.

These architectures match nicely to reporting systems but badly to active goal or process oriented areas of the business.

Cause
The cause is pretty simple, its because people have started to do SOA from a Data Oriented perspective and not noticed the difference between the letter S and the letter D.

Resolution
The resolution is pretty simple. Keep the data services as a post transactional base for data, effectively these become the reference data for your systems. Then do a proper mapping of the business services and use these data services as support services and to help communication between different areas of the business, for "live" data keep it local to the service doing the processing and only submit back to the support data services once processing has completed.

Technorati Tags: SOA, Service Architecture

Selling SOA

There is a conversation that I've had close on a hundred times now in the past 8 years about how to sell SOA to the business it goes like this

Business: Why do I want to do SOA? Isn't it just another IT acronym
Me: Do you understand how your current IT estate works?

B: No
M: Would you like it to look like the business?

B: Yes
M: Then that is what SOA is about, its about making the IT estate look like the business, evolve like the business and be costed in line with the business value it creates

Then you show them what the Business Service architecture piece is and how you will create a heatmap for them of where IT adds value and where you should be moving towards Utility (OpEx) spend and now you have a business person bought in and a goal on what the IT estate and its supporting organisation should look like.

Its not rocket science and it has nothing to do with technology.

Technorati Tags: SOA, Service Architecture

Yes Sir, No Sir, three service views full sir! - Anti-pattern

Description
This is where the time isn't spent to get a clear business service architecture and get the business bought into the approach and where IT doesn't have the confidence, or maybe the ability, to effectively negotiate with the business.
Effect
The effect is that the service definitions are changed on a regular basis, sometimes its just about a data field, sometimes a new capability and occasionally a complete re-writing of the service interfaces. Every meeting that happens results in changes at the interface and service level and the leads to a fracturing of the overall architecture around business silos. Eventually the whole SOA programme is canned because its become too complex to maintain and isn't delivering any benefits.
Cause
The Cause here is IT going it alone and defining a strawman and, unlike IT2B, not having the knowledge or confidence to enforce or structurally modify it. In IT2B the issue is the IT department thinking it knows best, here the issue is the IT department thinking it knows nothing and therefore must always say yes. Often this is driven by a lack of good communicators in IT and through a view from the business of IT as purely an "order taker" rather than a partner.
Resolution
Organisations that have this problem need to drive SOA as a cross-boundary business programme with IT providing support. IT need to invest in highly skilled communicators who can work with the business as partners and help bring together the differing perspectives. At the heart of this issue is one of confidence, communication and ownership, the solution is to make the business the owner and invest in the communication and confidence side of IT.

Technorati Tags: SOA, Service Architecture

ADHD SOA - Anti-pattern

Description

ADHD SOA is where, normally, a group of architects continually shift their mind about what "good" looks like from an SOA perspective.

Effect

This is all about continually "refreshing" the technology stack. On the technical side it means that rather than focusing on getting things into operations and industrialising their approach the architects instead concentrate on the upfront elements, especially the first few weeks of requirements and development and look for ways to continually shave fractions of effort or add additional features.

Both of these result in lots of unneeded work and an increase in the complexity of the IT estate under management, they slow down progression while generating activity.

Cause

The cause tends to be a focus on design and development time optimisations. Unlike the Shiny Nickel pattern which is always about the latest buzz the drive here is always to be different to the last project phrases like "we should try X out" ring around. The most common issue here however is a lack of measures, various different pieces are tried and compared based on personal preference and then combined together on the next project "lets try X with Y but not Z this time". A lot of this comes from vendor product upgrades, after all if you've paid for it then you should use the new features. This isn't driven by industry buzz-words but just pulled along by availability of options and a desire to try new things, often you will end up with bizarre cases where people are pushing using very old technologies along with the latest versions because "we know this worked in 2002" or similar justifications.

The basic cause here is a lack of focus on industrialisation of the delivery process, a lack of measures to demonstrate impact or improvement and a complete disregard for the testing, deployment and operations part of SOA.

Resolution
So how to fix it? Well first off make people really care about operations. Give the architects a target for operations. If the cost of operations doesn't go down then they aren't meeting their goals. Next up get some proper measures on how long it takes to get new people up to speed on a project and how much of the project is automated. Set the architects the task of improving these metrics. The ADHD approach tends to result in an increase in the time it takes to onboard people as its always new to everyone.

Technorati Tags: SOA, Service Architecture

SOA Anti-patterns revisted - Part 1

Back in 2006 I wrote an article, with some help, at InfoQ on SOA Anti-patterns and I thought it was about time to revisit and extend them.

The format again remains the same

Each anti-pattern presented in this article will follow the following format:

• Description - What it is?
• Effect - What you see?
• Cause - What behaviours led to that effect?
• Resolution - What can be done to solve the problem?

First off I'll revisit the old patterns to see if they are still valid.

Antipattern: The Shiny Nickel

Hell yes, people are still lobbing in new technologies without worrying about the consequences so they can get it on their CV.

Antipattern: The Technology Altar

Again its a Hell yes. Some of the REST preaching out there is right out of the technology altar book.

Antipattern: Percolating Process

Again its something I see over and over again. People are still saying things like "Process on top of Services" and I still see people mapping out processes and then calling activities "services" and claiming SOA.

Antipattern: Point to Point Web Services

Its still an anti-pattern but I'll admit that its one that is happening less and less. People are doing ESBs to avoid Point to Point and the REST crowd are claiming the dynamic nature of the URIs combined with DNS.

Antipattern: Splitting Hairs

Again its about splitting process and services, some people are even going further and advising you split further into data/service/process. Its still an anti-pattern and its still being done over and over again.

Antipattern: IT2B

Not really an SOA anti-pattern but an IT organisation pattern and its certainly still a problem.

Antipattern: DIY Transport

Its an anti-pattern but I've not seen someone do this recently. The ones who did this before are now leaping on REST as their transport, I'm still not sure that its any better than the custom XML-RPC approach of before as it still gives a completely unstandardised application interface. But the problem now probably isn't at the transport level. So if you are doing this you are seriously behind the times, its beyond an anti-pattern now and into the category of reasons for commital to an asylum.

Antipattern: Nobody Home

Are people building services without thinking about how they will be used? Oh yes they are. Some people have even talked about not being concerned about the interface design and that "you should at least start somewhere". Well you shouldn't start by making up services based on a set of perceived rather than actual needs.

Antipattern: Too many Cooks in the SOA

Yup still valid.

Antipattern: UBER service

I've not seen this one for 12 months or so, but its still valid.

Antipattern: A Million Services all in a row

Still seeing this, the focus at the edge, build up the volume and "they will come". Muppetry

Antipattern: Architectural Stovepipe

Big strategic views from Enterprise Architecture that result in hard to deliver solutions... almost a best practice in some places it appears. Its still an anti-pattern.

Antipattern: Defensive SOA

Getting worse this one. More and more companies are "already doing SOA" and when you look its the same thing they did before but with a software upgrade.


So to my mind they are all still valid but some of them are less of an issue than 2 years ago. Now what are the new SOA anti-patterns?

Technorati Tags: SOA, Service Architecture

DOA your SOA?

I like a lot of the stuff that Dave Linthicum writes but I don't agree with his latest post on why people should start with the data when doing SOA.

First off lets be clear, data is important its one of a few really important things in SOA

1. Services
2. Capabilities
3. Real World Effects
4. Data

Now I've deliberately but them in that order because that is the order I think is important. I've said before about SOA v POA and how I think you know that you are doing SOA. The key point on this is that a stack based view of

But the fact is you need to start with the data first, than work up to services, than the agile layer (process, orchestration, or composite). If you follow those basic steps you'll find that the solution is much easier to drive.

just doesn't work for me. It first of all implies a technology centric view of SOA that I don't agree with and secondly places services as just an intermediary between process and data, with the goal surely being to do more and more in the "agile" layer (although those of us who have done large BPM and composite projects probably aren't sure about the agile bit). This stack based view isn't going to progress IT, it might help with a single project but surely the goal of SOA is more than that?

To me this plays into the technology centric change view of much SOA advice out there. To me data is important but it is only important where it works this means understanding the services first.

Now if Dave is talking about having done the business services how he'd look to realise them with technology and that he'd have a single business governance piece over the service dealing with the data, process and other elements then I'm with him. But starting with data for your architecture.

To me that means you have a Data Oriented Architecture, and I can't see anyone selling to the business that the next project is going to be DOA and the goal is the whole IT estate to be DOA within 5 years.

So start with the business services, understand the business capabilities, understand the real world effects and then understand what data is being exchanged and managed to deliver that. Data is important, very important, but if you are taking a business view of modelling SOA then its just one of the technical artefacts, not the major purpose for existence.

Technorati Tags: SOA, Service Architecture

Vive la France?

Okay so recently I've read a bunch of articles about how France is becoming relaxed about the rise of English. It appears however this is just a front to make us relax so we won't see the real goal.

Unfortunately I've unearthed this plot via Apple, looking at the MacPro and going through the "HOW much could I spend on a PC?" test. I saw the following

See it?

What about now? Seriously its there in black and white

Yes you get a British keyboard, but you have to learn French to understand the instructions. This is a truly nefarious plot.

Now this actually has a good point from a globalised project perspective. You can't assume what language people are going to speak so you need to be very clear about what you are going to support (i.e. keyboard in English, but instructions in French) otherwise you can end up in a hell-hole of trying to support every language in existence.

Fortunately English in general remains the lingua franca ;)

Technorati Tags: SOA, Service Architecture