Software Development Wave 2 - the team developer

The problem with Wave 1 was that it didn't scale, I mean sure lots of the personal developers claimed it did scale, often laughing at large scale developments and going 'Me and four mates could do that in a couple of weeks' often they attempted to do that and suddenly realised that when you get a few people together it gets a bit more complicated and when that few gets over 20 it begins to get really, really complicated.  Back in the Java days this era saw the rise in the importance of design, but design focused on development rather than pictures, like with TogetherJ against the older school Rational products.

At the start of the 21st century this battle between the personal and the team developer was fought out on the floor of the JavaOne conference, '.com' champions of the personal style battled it out with a new breed of Java developers who were working for dull and boring corporations.  The noise and the hype in 2000, and indeed into 2001 given that not everyone had popped, was strongly on the side of the personal developers but the actual cash, rather than paper money, was on the side of the enterprise folks.

When the bubble had finally popped a new wave was about, sure Agile was kicking off but even in its most extreme of XP debates the talk was about quality, about testing, about making it maintainable.  Suddenly the battle wasn't between the personal and the team it was between what type of team dynamic was best.

Some organisations are beginning to hit this wave again, but I'd argue that it won't be the mainstream mentality for another couple of years.  What we'll see this time is a focus back on design but also a focus towards eliminating many of the things that made team dynamics so poor, things like deployment times, infrastructure management etc.  Cloud and now PaaS offers like CloudFoundry are changing the game here already but its still a jump for companies to move towards a next generation team development infrastructure.

This next generation will also see teams work between organisations, giving a whole new team dynamic challenge for us to overcome.  This means that the new generation is going to be less about processes and transactions (as the Java wave was) and more about information collaboration and digitization.   So while traditional design approaches such as UML might make a comeback I'd suggest that new approaches that concentrate around information-centric collaboration are going to be created and become more popular.

The challenge of the team developer era however was, and is, that its still for a minority.  The Java community between 2001 and 2003 was absolutely exploding but arguably the tools and techniques were still a bit above the majority.  The big question was how to roll out software development to everyone...

This is part of a series of posts on Why Software Development is the next big Wave and is preceded by Wave 1: The Personal Developer and followed by Wave 3: the Enterprise Developer and Wave 4: Back to the package

Software Development Wave 1: The Personal Developer

This is the wave we are in at the moment and its the wave that we last saw in the late 90s, this is where technologies enabled single people to build small specific things really quickly.  Java and its applets really were the peak of this first wave back then but now we are seeing people use technologies such as R, Python and others to create small solutions that offer really good point value.

Right now I see lots of this bubbling on the edges, just as I did in the late 90s.  The core of enterprises back then was about the SAP and other ERP projects, my work was in 'true' customer development (there isn't a package for Air Traffic Control) but as I moved into the enterprise space I saw a real shift away from rigour in software development towards more personal solutions where maintenance was considered less of an issue.  The Perl script that did something useful but only Dave could change, the C program that worked... but no-one had the source anymore.  The stored procedure that did everything you could ever want... as long as you wanted what was required 2 years ago.

The challenge with the personal developer was maintenance and evolution, people coded for themselves to support and for them to create on their own or in very small teams.  What we've seen already is that people are moving up the stack and using technologies like Hive and HAWQ to give older school interfaces on this data as that means more developers can work together... which brings its own challenges.

This is part of a series of posts on Why Software Development is the next big Wave and is followed by Wave 2: The team Developer, Wave 3: the Enterprise Developer and Wave 4: Back to the package

Musketeers Day - All Four One and One Four All

Okay in the spirit of brotherly love, helping people out and of course International Talk Like a Pirate Day I think we should declare April the 1st 2014 as Musketeers Day.  Why? Well for the vast Gregorian Calendar (non-US) part of the world the date will be 4/1/14 or All "four one and one four" All.

In honour of that day, and as it falls on April Fool's day to boot I declare the following:

1. All people should declare fealty to the French King
2. Each office should have an appointed D'Artagnan who must do most of the work while others take the credit
3. Wearing of swords is mandatory
4. As are floppy hats

Musketeers Day will not happen for another 100 years!

How British Airways failed to use the information they have

Having worked with companies a lot in the past few years on how to create a better customer experience and in-particular through MDM to help effectively identify the customer I know just what is possible even in very challenging environments.  The Airline industry is not one of those areas but today with British Airways I received another example of how a company might have all the information required to deliver a good customer service but chooses not to provide that information to where it counts.

I was flying into Heathrow, on a transatlantic flight I booked recently, I'd originally planned to head into London and from then on to Paris via train but checking the prices it turned out flying was cheaper and as I was already at Heathrow it seemed the smart and good corporate citizen thing to do, so I made another booking.

So lets see what BA knew about me at this stage....

• All of my passport information - the goldest of gold standards
• My frequent flier, with shiny gold card indicating I need to get out less
• All my flights with them in the last 10+ years
• How many times I've missed a flight with them due to my own fault (zero)
• How many times weather or their issues have caused me to miss a flight (was 3, now 4)

So in the text book of Consumer MDM they have the perfect set of source data to identify the individual (the passport) and a unique number the customer wants to give out (frequent flier ID).  It really couldn't be easier in a world where the ticket (which has the ID) and the gate (which scans the passport) to identify the individual.

They also had two bookings.

One that flew out on a Sunday and was due to land at 2pm on Monday, and a return leg for Wednesday (seriously I need to get out less. 

One that flew from Heathrow to Paris at 15:15 on Monday and returned on Wednesday

 Shared across these bookings are my details, indeed when my iPhone downloaded the two boarding passes it automatically paired them together as a single journey.

So what did BA do?  Well the flight into Heathrow was delayed, but I still had 45 minutes to make the connection.  There were a couple of others on the flight racing for the same flight... I reached the connections check-point first to be told:

You've been taken off the flight

The couple behind me (same inbound, same connection) were let through... this irritated me somewhat but it was only when I got to the rebooking desk that irritation turned into disbelief and outright annoyance.

BA: You missed the conformance check. 

Me: I was on the flight from Phoenix, you knew that

BA: We can't see that its on a separate booking

Me: With your airline, made with your website

BA: Yes but its a separate booking

Lets be clear, I like BA, I like the service from the individuals but this is a great example of how a company fails to leverage the information it has to deliver a decent customer service.  Its not giving its people the right information, information it has, to make the right decisions.

My iPhone automagically recognised that a flight taking off from an airport less than 2 hours after I landed from a previous one was part of a single journey, independent of how many booking IDs there were.  BAs systems have this information, I can see it in my Exec club profile, but at the front line they see only the booking.

Now the smart thing would be for BA to have a very simple check on bookings that says 'if bookings land and take off from the same airport within a set period, say 4 hours, we should consider them as one booking so our airport staff can see what is going on'.  At the very least the people at the airports should have access to all my flight bookings so before they kick a customer, and one they've marked out as a priority customer, off a flight the person can check 'wonder if there is an inbound they are on that we know about?'.

This is truly a great example of a company undermining its excellent customer service by not providing its staff with the information to deliver it.  This isn't about a lack of information, its not even about a lack of good quality information, its about the inability to integrate that information into the business processes where it is needed.

Disclaimer: I actually did a bunch of work for BA several years ago around BA.com and other customer facing parts of the business.  

NoSQL? No Thanks

There continues to be a disproportionate amount of hype around 'NoSQL' data stores.  By disproportionate I mean 'completely and utterly out of scale with the actual problems of the vast majority of companies'.  I wrote before about 'how NoSQL became more SQL'.  The point I made there is now more apparent the more I work with companies on Big Data challenges.

There are three worlds of data interaction developing

1. Traditional Reporting - its SQL, deal with it
2. Complex Analytics - its about the tools and languages, R, SAS, MADLib, etc
3. Embedding in applications

The point here is that getting all those reports, and more importantly all those people who write reports, re-written using a NoSQL approach makes no sense.  Sure Statistical languages and tools aren't SQL, but is it right to claim they are NoSQL approaches?  I'd argue not.  The use of a NoSQL database such as Hadoop or MongoDB is about the infrastructure behind it, its hidden from the users so while it make good technical sense to use such a data store it really doesn't change the way the users are working.

The point in these two areas is that its about the tools that people use to interact with information and supporting the languages they use to interact with that information.  The infrastructural question is simply one of abstraction and efficiency.  Like caring about whether your laptop is connecting over 802.11g or 802.11n, yes I know you care but that is because you are a techy.  The person using their iPad doesn't care as long as the videos stream from YouTube successfully.  Its the end user experience that counts not the infrastructure.

The final case is the world of developers, and here is another shock: business users couldn't care less what developers use as long as they deliver. If you can deliver better using SQL then use that, if you use NoSQL then use that, if you can deliver better by using a random number generator and the force then go for it.  Again however the business doesn't care if you use NoSQL or not and nor should they. What they care about is that it works, meets the business requirements and non-functionals and can be changed when they need it to.

Stop trying to force a technical approach onto the business, start hiding your technical infrastructure while giving them the tools and languages that they want.

Java and Analytics the next frontier

I've been pretty verbal about Java going down the wrong path and my view that what Java should do is start having a 'core' which is just the real basics of the VM and the language and then a few profiles which specify what needs to be loaded, with the rest coming in on-demand based on the requirements of a given project.  The old 'it needs to have everything so the browser/desktop/etc' is just rubbish these days with mobility, HTML 5 and generally lots of other ways to get that work done.

There is one area that Java really needs to step up to the mark, that is around Analytics.  I remember when Hibernate became the Java defacto standard for database access.  Brilliant thing it was to.

Now however plain of data access is just one of the problems.  Languages like R, technologies like MadLib are beginning to help move way beyond database access and into complex analytics.  Real-time analytics is another area.  Almost all of these approaches rely on Java in some way but its hard to see how the current approach of JavaSE 8 is actually supporting this new generation of challenges.

Java has a real opportunity to leverage its massive install base to offer real new opportunities through the growth of analytics.

Only if its leadership can start leading.