2010 the year of flexible packages

Lots of people make predictions but I'd like to make one that I haven't seen around.

For quite a few years there has been Package + Middleware but people have still been running it as middleware + package. So the middleware folks do CI and the package folks... well the package folks don't. This is a generalisation and a few, very few, people are doing full CI with a package infrastructure but its still two worlds.

Here I'm talking about SaaS or traditional package it really doesn't matter. What matters is that I'm beginning to see more and more people actually concerned about how to do things like CI and TDD in a package environment. Now what would help with this would be a few things

1. Package vendors to ship their unit tests
2. Standard VM building tools integrated into package suites
3. End-to-end deployment tools
4. OSS support for packages (i.e. Hudson, ant, etc)

The point is that SaaS and packages need to start having the development rigour that custom development packages have and they should have this more explicitly as they have already developed end-user functions.

So its a little question: Why don't vendors ship their Unit and SIT tests?

2010 is going to be a year when hopefully a bit more technical professionalism comes into package delivery and maybe the business professionalism will rub off on the middleware chaps.

Technorati Tags: SOA, Service Architecture

Supported doesn't mean you can get the same support

When you are down and dirty with a delivery project it becomes clear that there is a difference between when a vendors says something is support and the actual ability of that vendor to support you when you have a problem.

You often see this when a vendor supports "interoperability" with solutions from another vendor, even if they have a competing product. Sometimes this support is genuine and other times its what I'd call "sales supported" in other words you'll be able to get it running in the sales evaluation process but if you have a real nasty operational problem the level of support is going to go off a cliff.

So as a few "theoretical" examples.

Lets say Vendor A produces a product which is some sort of middleware thing and while they have their own rules engine product they also support products from a number of different vendors through "standard APIs". The demos work fine in sales and you choose an engine from Vendor B but in operation you start getting a really tricky problem

Vendor A says "its a rules engine problem, speak to Vendor B"
Vendor B says "its an integration problem, speak to Vendor A"

The problem is that you then waste a huge amount of time proving whose problem it actually is.

Next example would be where Vendor A produces a COTS package and says it runs on 20 different operating system environments. You have some older kit hanging around and its on the supported list so you go for it. Again the demo works fine and the first release is fine, you then try to upgrade and it goes horribly wrong.

Vendor A says "errr the support person for that is on holiday, he'll be back in a week"

The point here is that "supported" technically != "supported" operationally.

When evaluating products you need to be aware that there are preferred products that the vendor might not tell you about as they are worried about losing the sale, so they stand their saying "yes it will work" with a smile on their face.

Technically this is known as "grin fucking" as the smile is to cover the fact that they know in reality its going to go horribly wrong.

So when looking at product to product integrations and the level of operational support that you can get start asking questions

1. What proportion of the support staff are dedicated to this product to product interaction
2. How many people is that globally
3. How many people is that locally
4. Will you sign off on our operational SLAs around this supported product, including penalties

The objective here is to get to the stage where you have something that is as supported in operation as it is through the sales cycle.

do not believe supported product lists they are there to make you buy the product, what you want to know is what it takes to operate the product.

My preferred approach is to ask the following questions

1. What OS do you develop on
2. What products do you use to develop your product
3. What are the standard integrations that the product developers use as a normal part of their day job

These are the things that will really work and which have the largest number and highest quality of developers ready to fix your operational problem. The less gaps you have between the people who code your product and the runtime in operation the better.

The best support you will ever get is when the product development team can come straight in because your environment matches theirs. This means your future requirements are more likely to match their thinking and your current problems are easier for them to recreate.

So again, don't believe supported product lists and find out what the product developers are using.

Technorati Tags: products, vendors

Why the hell do vendors not use version control properly

This is an appeal to everyone out there who is writing a product that is meant for an enterprise customer, you know the people with budgets and it applies to companies big and small.

When doing a project you normally have a lot of things to control

• Word docs
• Powerpoints
• Visio
• Requirements
• Code
• Test Cases
• Data models
  
• etc

The point is that its a lot of different things. What makes it easier to control things is a version control system. What makes it easier is if we can use one version control system to manage all these elements. What this means is that your tool needs to

• Store things in lots of different files, every object or important piece of information should have its own file
• Delegate version control to an external tool

There are two very very simple principles that I'm fed up seeing people screw up. Having one massive XML file that your tool "helpfully" manages is just rubbish. Not only does this mean that people can't share sections of this but it also means that structurally I can't set up my version control structures so one area has all of the requirements, docs and interface specifications all managed together. This is a very bad thing and it means your product isn't helping my programme.

The worst thing about this is that its much easier for a tool vendor to just say "version control is done by another tool, if you want a free one then subversion is great otherwise we integrate with most of the major providers". Having this elementary dumb decisions of single files is just a very very bad design decision and completely ignores how enterprises need to manage and adopt technologies.

I really don't understand people who pitch up with a tool that fails on this basic test of

"see that individual important element in your tool, can I manage that as an individual element and do a release of that without dragging other crap in"

If I'm creating a Use Case I want to version at the Use Case level, not at the repository level. If I'm creating at the Process level I want to version at the individual process level, not at the repository level. If I'm modelling a database I'd like to be able to version at the entity level not the whole schema level.

Code works like this, it would be mental to have a code versioning system that worked on every file as a clump. Yet this is what lots of packages and other tools do.

Its 2009 folks and it appears people are still building tools that can't match the capabilities of RCS let alone CVS or heaven forbid something from this millenia. So a quick summary

1. Expose your things as files at the lowest level of sensible granularity, this is normally the blobs or icons in your tool
2. Delegate versioning to an external product that does this stuff better. If you feel you have to include it then ship a copy of subversion.

That is all. Its that simple.

Technorati Tags: SOA, Service Architecture

Requirements Landfill - the challenge of central services

Enterprise Services of any type from Business Services like procurement through to technical infrastructure services such as Identity management are all at risk from a threat of pollution. I call this threat the requirements landfill.

One of the key principles of SOA in operation has been the ability to have services which are used across the enterprise. These single services represent a single point of truth and a single functional boundary that ensures enterprise consistency, it is their goal to shift away from fragmented sytems towards Services which represent the single way of accomplishing a technical or business objective.

This is great and good but I've been seeing something more and more over the past few years. Its that these enterprise class services, particuarly when successful, suffer from being the easy place to add new requirements rather than always being the right place to add those requirements. The basic problem is simple

These services sit at the junction between multiple elements, some are nice new service programmes, some are hacky service programmes, some are old legacy systems and others are just systems owned by people who don't have much budget or are rather grumpy.

So what happens is that you get a big and complex requirement. About 50% of this requirement really does belong in the central service but the other 50% is a combination of elements

1. 20% requirements that should be done in the originating systems
2. 20% requirements that are around data storage and access that should be in another system
3. 10% requirements related to business process that are completely beyond the scope of the central service

Now the problem is that to implement this new set of requirements therefore requires a much broader set of changes if it is to be done properly. Initially the discussions might start out talking about doing it the right way but then someone will say the immortal line

It would be easier if we just put all of this is the enterprise service

While the enterprise service team will scream at the prospect this will make perfect sense to everyone else as they won't be doing the work. Unless there is a set of concrete governance processes around then the normal "democratic" decision process will result in the compromise decision being made. The first time this gets done its seen as a compromise that just had to be made. The second time onwards its just replicating the same approach as that is what we normally do. Rapidly therefore the enterprise service goes from being a single clearly defined entity with a defined business purpose to being the requirements landfill for all those requirements that people are either too lazy or too scared to try and do properly. This leads to the service getting ever slower to react to change and eventually being seen as a failure due to its transformation from a business service into a traditional old style application.

The point here is that governance and escalation are essential to maintain a clear set of enterprise wide services and to ensure that requirements are not simply dumped into areas due to them having budget, talented people or just because they are at a central point.

Most of the time I see requirements landfills its because IT owns and manages the services and thus they have no direct link to a specific business owner. This means that business people don't care about keeping them clean as it isn't their problem. One of the first steps in a solution therefore is to ensure that your enterprise services are clearly aligned to the business areas where they matter and thus ensure that they have an in-built desire to keep the services aligned to their area and not blurred across technical and organisational boundaries.

Favourites for requirements landfills are also centrally provisioned IT "enabling" solutions such as ESBs which almost stand up to the business and say "dump crap here and then point the finger of blame here when it goes wrong".

So have you got a decent policy to prevent requirements dumping? Do you have a refactoring and recycling programme to take dumped requirements out of the landfill and put them back where they should be? Or are you just hoping that you'll be able to convince people that its not your fault that you allowed the illegal dumping?

Technorati Tags: SOA, Service Architecture

Data Accuracy isn't always important

Now while Amex really should have understood the term minimum there are examples where it really isn't an issue if someone gets it wrong in displaying the information to you. Sometimes this indicates that a prediction has been incorrect or that "approximately" is good enough for this scenario.
Is a good example of this, the current temperature on the Sydney Morning Herald site is listed as one degree higher than the maximum for the day. Does this matter? Well no and for two reasons. Firstly a weather forecast is accepted as being approximate information, its a chaotic system and so by definition can't be predicted exactly. Secondly the Max number is only a prediction and the current temperature is indicating that it was an incorrect prediction. So by having an incorrect piece of information we actually have more information as it re-enforces the concept that weather forecasts cannot be 100% accurate.
Now when the next day rolls around then looking back you should clearly be recording the actual maximum achieved rather than the prediction. This is because the information has gone from being a record of a prediction into a record of fact. The only question therefore is at what point you should update the maximum. Do you change it dynamically or on a daily basis when reporting historical information. For the Sydney Morning Herald site the answer is simple changing the daily maximum as it increases during the day would defeat the purpose of the "max" level which is what the paper predicted it would be at the start of the day. Its a free news story if it goes well beyond the prediction "Sydney Weather was bonza today with max temperatures 5 degrees higher than expected".

So the point is that when you look at data do think about what level of accuracy is important. If reporting a bank balance then spot on is the only option, if reporting the number of customers who bought cheese with wine as a percentage of overall cheese buyers then you can probably get away with 1 decimal place or less. This sort of view applies even more when looking at forecasting and other predictive data sets, the effort of increasing accuracy by 1% might be pointless due to the extra time it takes.

Data Accuracy isn't an absolute, be clear about what matters to you.

Technorati Tags: SOA, Service Architecture

When you know it isn't a cloud

Following up on the previous post I thought I'd do the REALLY obvious ones that indicate it isn't a cloud. James' list wasn't nearly cynical enough in light of the things that are claimed to be a cloud.
So here goes

1. If its just a single website with no backups storing stuff to disk then its just a standard WEBSITE not a cloud (Hello Sidekick)
2. If its a physical box that is meant to help you manage a virtual environment... then its not a cloud (hello Cloudburst appliance)
3. Seriously if you are just a single website doing stuff with a database you aren't a cloud (hello Soonr)
   
4. No seriously if about buying a physical box it isn't a cloud (like HP's spin though that they are just cloud enabling... nice weasel room)

And I could go on. The point is that cloud is an infrastructure thing, it is IaaS in the "aaS" hierarchy. PaaS can have a go at being cloud but SaaS is "just" something that might be deployed to a cloud. Having a website (SaaS solution) that runs on Amazon doesn't make that SaaS solution "a cloud" it makes it a SaaS solution hosted on a cloud.

The hardware point is that making capital expenditure is exactly what a cloud isn't about and physicality is exactly what a cloud isn't about. You want virtual compute and storage that you pay as a utility. This is the economic model of cloud.

So in the words of Kryton. I know that strictly speaking I've only identified two things, but they were such big things I thought I'd say them twice.

Technorati Tags: SOA, Service Architecture