The 'Natural Platform' - Why people matter more than performance in picking hardware

One of the things that I get asked is 'what hardware should we run this on?'. I've said for years that I don't care about the tin and the tin is irrelevant from a differentiation perspective. Now before people leap up and say 'but X is 2x faster than Y' let me make a couple of points

1. Software tuning and performance will have miles more than a 2x impact
2. The software licenses will probably cost more than the hardware
3. The development will definitely cost more than the hardware
4. The software support and maintenance will definitely cost more

So what does that mean?  Well it means that when picking harder you should consider the people cost and the support costs much more than you could consider the performance.  What you should be considering is

 'what is the natural platform for this software?'

The natural platform is the one that the software is tested on.  That doesn't mean its the same as the hardware platform from the software vendor that they want you to buy, its the one that the developers are actually using to test that the software works.  Why do you do this?  Well when there is a bug, and lets face there are always bug, then instead of just having the support folks available to fix it you have all of the developers as well because they don't need to switch from their current environments.

Like I say this doesn't mean 'pick a hardware platform from the software vendor' it means pick the natural platform.  So if you think/know the developers are on Linux and deploying to Linux servers for test then you know there are more people able to help on Linux than anything else.  If they are developing on Windows and deploying to Linux then either of those platforms is natural.

As an example of what happens when you don't, let me take you back to 2000.  I was working on a project and we were using MQSeries, JMS and of course Java.  We developed it on Windows and deployed it to Linux for test.  For production however we'd been convinced to go for AIX to give us some major grunt.  We deployed the code into UAT and.... it broke.  Our assumption was that this was our fault because we didn't know AIX that well and clearly running IBM's AIX with IBM's Java Implementation, IBM's JMS implementation and IBM's MQSeries meant that it had all been tested, this was their flagship platform surely this was what it was meant to run on?

36 hours later we were talking directly to the product team who identified the problem as a memory issue that only occurs on AIX.  Making clear this meant that our configuration (pure IBM) had clearly not even been tested.

Working on another project where the database environment was different to that from the package provider and the hardware was a mainframe we had massive issues in just getting anyone who knew about our set-up in order to fix some problems.

These are normal problems and the key to them all is that its not about whether box X is faster than box Y they are about getting the best support and fixing problems quicker.  I'm not arguing that you shouldn't get an environment that scales, what I'm arguing is that when you look at the costs of tin then performance is a distant second to the people costs of fixing problems when they go wrong.

The problem is that normally the people buying tin are just buying tin.  In these days of virtualisation its about picking the right OS inside your virtualised server but its still important to think natural on platforms.

Pick the natural platform not the fastest shiny box.

Technorati Tags: SOA, Service Architecture

Improvised Explosive Consultants - neutralising a bad consultant

One of the challenges I often have is where a company employs a consultant to give 'independent' advice and the individual employed is a total snake-oil salesperson.  They've read a couple of books and websites and see their only job to lob in a 'bomb' in a meeting and sit back.  I categorise a bomb as a piece of input that is

• Completely and utterly wrong
• Contains a grain of truth that has been warped and distorted but can be made defensible 
• Said with conviction and patronisation as to how you can't see their point

So its the person who says 'Where are the WSDLs, this is an SOA project I expect to see WSDLs' and then follow up with an accusation that you don't have enough detail as you haven't done to WSDLs.  Or someone who in an MDM project asks 'why aren't you storing transactions in the Master, how are we expected to get them?' or questions to which answers are so obvious you don't bother putting them on the slide deck 'how on earth can you say you are doing a global implementation if you aren't using HTTP for the user interface'...'Err we are'.... 'Well you should have said, its a critical point'.  You know the sort of thing.  Dumb questions.  The reason they are dumb questions is that this person is meant to be an expert.  People who don't know can't ask dumb questions, they ask questions and you help them understand but when the person is put forwards as an expert then you have an issue.  I think of these people as Improvised Explosive Consultants as they are normally doing a specific thing, taking a small grain of knowledge and improvising it into a bomb to derail the project to show their value.

You see they are there to keep their fees up and to do that they need to demonstrate 'value'.  One of the easiest ways to do this is to undermine others and doing this in a technical area just means you have to seem smart to someone who doesn't understand the subject area.  One of the problems this causes is that you can't just call 'bullshit' on the bomb as often it sounds plausible to the layman while being rubbish and because doing so creates an antagonistic environment which is counter-productive and often leads to the IEC being seen as 'scoring a hit' with their insight.

This sort of consultancy often leads to bad decisions being made, the bomb is used to confuse and thus a confusing solution is then created.  The bomb maker, the IEC, is of course not going to be doing any of the actual delivery but can continue to stand at the side throwing bombs to demonstrate their 'value' and superiority.  So how do you defuse an IEC?

• Don't lose your temper - they are after this as it shows the bomb has been effective and makes you look uncertain, even though you've probably lost your temper because of the level of stupidity
• Get the specific objections down on paper - after the meeting where the bomb is thrown get the IEC to write a clear email listing their 3-5 points.
• Attack the facts not the person - don't debate the person keep on the facts.  Use terms like 'I'm a bit confused by number 3 because wouldn't that mean....' rather than '3 is just rubbish, don't you understand.
• When you finish knocking down the points send an email confirming that they accept that all of their 'challenges' have been addressed.
• Then send their reply of 'yes' (probably including some phrase like 'its important to check these things') to the whole group making clear that you've addressed all the points and the solution hasn't changed at all
• Repeat this every time. 
• When you get to the third bomb, add a statement to the reply that 'while you appreciate that IEC is just trying to get to the right solution, as we all are, I'm concerned that the team is being delayed addressing these concerns.  So far we've addressed 15 points and none have resulted in a modification to the solution.  Could I please ask that anyone who has specific challenges to the solution at this stage please email me (or add to the bug/CR/etc repository if you have one) before the meetings to speed up this process.

Hopefully during this process the IEC will quickly be identified with delay and your documented history of their comments producing nothing but additional work will result in them being defused.

If you are on the other side and think you might be employing an IEC there are a few key ways to check

1. Do they talk about their delivery successes or just their advising successes
2. Do they use patronising tones when people whose opinion you think is worth something explains stuff to them. 
3. Do you find that people with a history of successful delivery say they are 'confused' where these challenges are coming from.
4. Does the person talk about problems without out talking about simple clear solutions

If you've answered yes to 3 or more of the above then take a long hard look, you might have an IEC.

Technorati Tags: SOA, Service Architecture

NFC - What Apple does next? Buy Amex?

Chatting around on the iPhone 5 I have to say its all rather dull, yes they'll fix the antenna, yes the camera might get better and the processor faster. But really is that a big deal? So what could Apple announce either now or next year that would really blow people away.

I travel a lot, and one thing I see around Europe is the rise of NFC, so the use of 'smart'cards on the Underground, Metro, Dutch rail and elsewhere. The technology is basically the same everytime, Chip with some form of RFID or NFC is swiped over a sensor and your trip is paid for. Around the globe we are seeing MasterCard and Visa both include NFC to their cards so you can just wave the card around and get charged for things.

Now MasterCard and Visa get a percentage of the transaction, there are normally companies providing the Smartcard services for public transport. But what stops Apple creating or leveraging a 'standard' for NFC to cover these cases. Think 'Airplay' over six inches. Now Apple have said NFC won't be in iPhone 5, which indicates... well not much. But the key question here isn't so much when NFC will be included but the impact this will have on the market.

Lets say in the UK my Oyster card can be replaced with my iPhone and my iPhone can be linked to my Credit Card. Suddenly Apple are taking a slice not just of my iTunes transactions but of pretty much everything I do, and they have clarity via GPS on where I'm doing it. I can also charge via this system on the French, Dutch, etc networks without having to buy a ticket or going to a ticket office, think of the staff savings....

Now for fraud reasons it would be good to track the GPS so if I do an iPhone transaction in Utrecht and then there is a card transaction in London 10 minutes later then the odds are that one is fraudulent. This fraud protection would give Apple the reason to store and analyse this information, and of course provide it to the credit-card provider. All very open, if more than a little big brother.

The point here is that integrating NFC into the iPhone and driving a standard due to the massive market volumes that the iPhone has, something no-one else can currently do unless Google dictates hardware standards a bit more, would mean that governments and companies could massively reduce the timescales for adopting NFC and start decreasing their operating costs by relying on iPhone users to self-serve. This would have a knock-on effect of driving iPhone sales because with iPhone being the 'preferred' mechanism it becomes simpler to own one rather than not. Thus Apple provide to the credit card companies and other organisations a platform for NFC which gives Apple revenue and companies the leverage that a standardised market brings (think 802.11x against proprietary approaches).

That then got me thinking. Apple quite like being in control and have $76bn in cash, what could you buy for that which would really kick this up a level? How about VISA($63bn), Mastercard($44bn) or American Express($60bn) themselves? Apple could buy, for cash, anyone of these. So instead of having an open system for all of the providers it suddenly would become a closed system where you are best off having a specific card provider if you get an iPhone. Given the demographic target of Apple, and their larger charges over the competition, AMEX would seem to be a good fit.

Sometimes its scary when you join the dots.

Technorati Tags: SOA, Service Architecture

Java 7 SE approved... Meh

Hey Java SE 7 has been approved... now that is spectacularly quickly. You'd almost think that the normal Java Community Process had been ignored and instead the spec lead had taken an externally created spec straight to approval...

What is most depressing in reading the various Oracle (mainly ex-Sun employee) releases on this is that not a single one actually commented on the fact that of the people doing the approvals six of them expressed reservations about the licensing terms and the transparency of the process. All stated that they were approving it to get Java moving again and that there are issues they want to see addressed. I've said before that Java SE 7 is a nothing release for Java but its the approach and process that concerns me most. While I don't think the Java SE 6 approach taken by the Sun leads at the time was at all positive or constructive and the end result was not what was required at least there was a decent representation and debate. 18 companies spent 18 months and while the dumbest decisions remained (JAX-WS in JavaSE 6... how stupid does that look now?) at least there was a measure of debate.

The expert group for Java SE 7 was a sham, Five companies, five months and the first draft published days after the expert group was finally formed. The comments on the 'six companies who approved (+ Google who voted against) clearly indicate that there are significant changes required for Java to regain its position as the go-to platform for developers, enterprises and vendors.

I really hope that Java SE 8 actually does the radical things that the platform needs given the big shifts in the last 5 years since a decent release which didn't just dump cruft into the platform. Unfortunately I'm still concerned that the mentality is more to continue outside with a closed community which pretends to be open while actually pushing an already proven to fail line.

Java SE 7... I give it a 'Meh'.

Technorati Tags: SOA, Service Architecture

Dumb IT: We've already got some licenses for this

There are certain phrases that fill me with dread. 'We are using Agile so we don't need to have a vision, we'll just iterate', 'there are no data quality issues', 'We're the first people to use this' and 'The vendors roadmap says they'll do X in 2 years so it will be fine by the time we need that'. One however is completely variable in the fear it introduces because it comes in three clear flavours, one good and two bad.

'We've already got some licenses for this'

What this means is one of three things

1. We need to do something and a technology we know well is ideally suited where we fortunately don't have to buy anymore licenses
2. We bought an enterprise agreement when we bought product X and the vendor allows us to have a few licenses for Y and Z as well
3. I've spent a load of money on licenses for this and we are damned well going to use them

Now clear the first one is fine, this is for example where new functionality is being delivered for a website and the number of CPUs has to be increased.  The later is of course clearly dumb and often is the driver behind IT centric projects that burn money.

The middle-one though however is the most dangerous.  I've seen this over, and over and over again.  A company buys the flagship product from a market leader in a specific segment.  That market leader also has some other products which are either early to market, non-strategic or just plain a bit rubbish and to 'sweeten' the deal they include a bunch of licenses for them and offer this up as value.

A few weeks, months or even years later a project comes along which needs to do a specific thing.  Suddenly someone remembers, or most often pushes, that there are some licenses on the shelf that do that sort of thing so brilliantly money can be saved.

Let me recount a story of just such a decision....

In 2001 I was working with a company who had bought a enterprise package solution from one of the market leaders in the CRM space.  As part of this 'deal' the company was allowed to use up to 4 CPUs of any other product from that vendor.  We had to produce a website to enable consumer to interact directly with the package and this was well before .com front-ends were normal practice.

'Fortunately' the vendor had a new product to do just this, it was new, it was shiny and it was covered by the 4 CPUs.  The alternative was to spend about 6 months developing something custom with about 5 people and despite some heavy cautioning from me on adopting a brand new, unproven technology that looked rather rubbish when I investigated it the company decided that it would be cheaper because of those 4 CPU licenses....

18 months later and with an average team of around 15 people and much hacking, cursing and challenge the site went live with a fraction of the envisioned functionality.

So that 4 CPU 'saving' had in fact delivered a 20 man year cost increase and less functionality.

Want another?  How about the company who used some old EAI licenses and found out half-way through the project that the vendor was discontinuing the product?  How about the company who used the limited number of Web content management licenses and found that 10 years later it was a drain down which millions had been poured... seriously I could go on and on.

The point here is that lots of IT seem to account for license costs in a completely different way to people costs.  Something that saves [£€$]10 of license cost is good even if it delivers 10x that in additional people costs.

The solution is simple, when you look at a programme evaluate the total cost of ownership of the solution not just the immediate cost of buying licenses.  Cheap today is liable to be expensive tomorrow and potentially extortionate the week after that.  TCO is all that should count in these decisions but normally the lure of 'free licenses' outweighs the rationale of 'that isn't the right tool for the job'.

Now that really is Dumb IT

Technorati Tags: SOA, Service Architecture

Hackgate and what it teaches us about responsibility

The ongoing Hackgate scandal, we can call it that as people in the US are now interested rather than just the dull old 'phone hacking scandal', teaches us some very interesting lessons about corporate politics and the meaning of the term responsibility.

I see quite a few projects where the issue is that someone somewhere hasn't taken control or responsibility and therefore things have gone off the rails. A lot of the time this is about the personality types involved and those personalities massively impact how any recovery can be achieved.

In this scandal we've seen so far five completely different views of responsibility and each of these teaches us a lesson we can learn from

Responsibility without Action but with blame - David Cameron
First up is the Prime Minister David Cameron who has taken 'full responsibility' for hiring Andy Coulson.  What full responsibility means here is that he has stated that he takes that responsibility, but in reality actually nothing has changed or is in danger of actually being done.  I often see this on projects where a senior business sponsor has a pet project that they actually don't care that much about but just want to see it continue for power reasons.  As with this occasion this responsibility normally actually means shooting the project manager or some other individual so in reality what is being said is that the senior individual actually bears no real responsibility and that the failure is further down the chain of command.

This is very common in IT projects, often you will see a programme director being promoted during a project and then taking 'full responsibility' by firing the person who had to clean up their mess after they were promoted.  Its a very good career strategy as it implies that you are the sort of person who takes decisive action and has person integrity but in reality is classic blame farming.  This is one of the hardest situations to deal with when recovering a project as you often need the sponsor to change some of their behaviours and what you get instead is a continual statement that they have 'taken responsibility' but actually in reality done nothing to change the behaviours that led to them having to take responsibility for the failure. These people can be very useful when you recover a program as they are keen to be seen to be doing things and if you can leverage that into action it can be a positive thing.

Responsibility without responsibility - Rebecca Brooks
Rebecca Brooks (nee Wade) typifies another form of rogue sponsor when issues occur.  In this case the sponsor is often actively involved with the project and seen as more than simply a sponsor but actually a leader of the initiative.  Trouble occurs and the sponsor throws up their hands and claims that they had no idea at all what was going on and are appalled at how they were kept in the dark.  This is a very tricky act to play but if played well normally means that the entire project is about to get a huge kicking as literally nobody is going to protect the individuals and indeed the previous sponsor will often be the most vicious in terms of lashing out in order to protect their own reputation.

In IT programmes I regularly see this where a project within an IT Director's remit is failing and they see that their best chance of coming out well is to be seen as a 'strong' leader who has so many things to do that they can't be faulted for 'trusting' lieutenants who then turned out to be rubbish.  When trying to clean up a project these people are toxic as normally their only interest is ensuring that no blame ever attaches itself to themselves, Brooks' initial leading of the 'investigation' is a classic case of this where someone closely associated with the issues tries to ensure the 'correct' outcome by either directly or indirectly leading the investigation.  Sometimes in IT the phrase 'lets draw a line under it and move on' is used which can help if its meant as it means everyone can get on with fixing the problem rather than with worrying about the political issues.

Responsibility with personal accountability and exit - Sir Paul Stephenson 
Then we have the Met Police chief who has resigned for the faults within his organisation which pretty much no-one feels touch him personally.  His extremely barbed comment pointing out that his mistake was to employ someone who hadn't resigned in the original scandal, as opposed to David Cameron, clearly highlights what he feels is a double standard in how people talk about responsibility.  Now from one perspective having the overall sponsor take responsibility and leaving because of it demonstrates strong personal integrity and leadership, but in another it also means that the folks below are left without a clear leader and therefore there is uncertainty on who will clean things up.

In IT this often unfortunately occurs when the sponsor feels they can leap into another job somewhere else if they go before they are pushed.  The challenge to the project team remains who will clean up the mess and drive it forwards.

No responsibility but lots of finger pointing
What has been most common in this scandal is lots of people from the outside pointing fingers and suggesting how things should be cleaned up, with in the most part very little actual personal commitment in helping to clean things up.  MPs have condemned and moaned, but not promised to stop religiously courting the media.  Other scandal sheets have condemned and moaned about its impact on the reputation of 'good journalism', but certainly not offered up themselves to be investigated to clean their names.

This is really common in IT recovery programmes, lots of people standing on the sidelines with 'helpful' advice on how to improve or how to 'learn' from the mistakes but zero actual time commitment in cleaning things up.  Managing these people is central to recovering an IT programme.

No concept of responsibility
Paul McMullen has been the comedy turn of this scandal, a man so divorced from reality that he continues not just to excuse but positively to champion the sorts of behaviour that everyone is condemning around him.  Here is a man who both Hugh Grant and Steve Coogan have show up to be a total and utter muppet of the highest order.  At some stages I've really wondered if he isn't really a journalist but in fact a very hammy actor who is over playing the part.

In IT these are the folks who just don't get that there is an issue.  I once interviewed an architect shortly after Boo.com had failed.  He had been responsibly for lots of their key architectural decisions, several of which were behind the inability of people generally to use the site.  During a curtailed interview he continued to champion the approaches they had taken, which had failed, and even managed to support and promote the business model for Boo.com which had managed to burn through lottery winning cash in an amazingly short period of time (one founder entertainingly stated that they were 'too visionary', nope it was a bad idea, badly implemented).  People like this must be quickly exited from the programme as if they can't see any issues then they will be unable to fix them.


Responsibility with Action
What we haven't really seen so far is someone take responsibility with action in this crisis.  Sure the News of the World has been closed down but allegations by Jude Law against the Sun have been met with abuse rather than a culture of 'we are pretty sure we didn't, we take these allegations seriously, will will investigate and we are confident we will prove our innocence'.  The closure itself was simply an acceleration of a previously announced policy so there really hasn't been that active leadership yet in cleaning things up.

Responsibility with action in an IT failure is the person who stands up and says 'mistakes have been made, right lets fix them' and sets about driving change and leading the cultural shift that is normally required to recover a failed or failing project.  Responsibility with action is normally a quiet thing rather than a shouty thing,  its something that is done rather than talked about.  It isn't a committee to investigate  its an active approach to finding what went wrong and fixing it as quickly as possible.

Critically its not about blame in the sense of finding people to blame, its about finding problems and when these problems turn out to be people then those people are given the simple choice: change or leave.  It is about finding people who should have taken personal responsibility and ensuring that next time they do.

Recovering projects is normally one of the most thankless tasks that I do.  You enter into a scenario where someone else has screwed up and your end result is getting the project to a place where it should have been ages ago.  There is however something personally rewarding in changing the culture of individuals so that they are able to recognised the mistakes that were made and in exiting the toxic people from the process.  Crucially however there is one lesson that I've learnt doing this and that is that the first stage has to be recognise that there are systemic problems that need to be fixed.  If it turns out that actually its a localised issue then this is great, but the assumption must be that the rot is much broader and more general than the currently surfaced failure.  Normally there is a culture of poor sponsorship, leadership, management and clarity that leads to a general case of fail in which only the scale of the project fail stands out.

If you do see a project failing the first thing your should identify is not what went wrong in the weeds but how the sponsors and leaders will react.  Will they behave like a Brooks and deny everything?  Like a Cameron and take 'full responsibility' but actually blame farm?  Will they deny that there actually is an issue like McMullen? Will they fall on their sword and leave a vacuum or do you have someone with whom you can actually work to drive through the recovery?  Clarifying this 'top-cover' challenge is the first step in recovery,

Remember: Don't just people on whether they say they take or have responsibility but on what they do
 

Technorati Tags: SOA, Service Architecture