Big Data - are you the house or the played?

Coming back from the EMC World conference in Las Vegas I was looking at the people playing on the slots and making ridiculous bets at Craps and wondering 'don't these people know anything about statistics?'.  Lets be clear I get the idea of it being fun, but when you sit next to someone on a blackjack table who twists when the dealer is showing a six and they've got 15 is just depressing.

There is a business lesson for us here about the power of data and in particular the differentiation that Big Data will have between businesses.  Those that really leverage Big Data and Predictive analytics and integrate it back into business processes will have a significant 'house' advantage over their competition.  The retailer who knows what the next hot trend will be or who can negotiate prices based on a long term demand view will under-cut and out perform their rivals.  In the same way as Vegas knows the returns on the slots and table games and innovates to improve their odds so Big Data and Predictive Analytics will help some companies dominate their sector.

Things like Hadoop, R and the like are just tools in this journey and companies that focus on the technology are like gamblers who claim to have 'a system' for a game like roulette.  Sure they might get lucky a couple of times but over time they are just going to lose to the house.  The companies that win will be the ones like the casinos which run statistics and predictive models to a level that helps to assure their advantage.

So are you the house or the played?

Federated Caching in the world of the 64GB mobile

There is something that is beginning to irritate me, ok something else.  Its mobile applications that don't cache.  I'm fed up of travelling on a train or being on a plane and the end result being that my iPad or iPhone app doesn't work because I'm in an area that doesn't have reasonable network coverage.  I was using the App earlier when it had WiFi and it was all fine but the app requires me to be 'always connected' which I just feel is plain lazy.  My mobile phone has about 16GB of free space, my iPad has slightly less thanks to the videos but its still several GB of space.

I'm particularly talking here about BI tools and enterprise apps which have no concept of a variable network.  If I've approved a travel expense for someone via email then when I connect the email gets sent, if its on a mobile 'always on app' I can't do that.  Why?  I'm actually losing functionality over email.  If its a BI app and I'm looking at sales reports for a given geo then if I can't access it on the plane then I'm losing functionality over Excel.

Federated Caching is often a tough challenge but is one that we've solved over, and over, and over again in IT.  It shouldn't even be 'save to' on the device it should be a case that if you are doing the reports and have a cache size set on your device then the application should automatically pull the information locally.

The next generation of enterprise mobility will be about removing things like email and Excel or it will just be another pretty technology that does some stuff better but I'm still forced back to the old tools when I step outside the mobile view of a perfectly connected world.

Federated caching needs to be your starting point in a mobile world, you need to leverage the power of the device in a smart way not use it as just a browser and you need to deal with the reality that your users will not always be connected no matter what the adverts say.

Software Developers are you ready for the cage fight with the BI guys?

In all of my career to date in IT there really has been three clear worlds in IT, the software development guys who are the bespoke tailors, the package guys who deliver off the shelf and the BI guys.

I'm going to admit a prejudice here.  Until a couple of years ago my impression of BI guys was folks who were one step up from Excel guys.  It was dead-data and done in batches, sure it had to be done but it really wasn't gool.  The BI guys couldn't handle the operational requirements of the software developers and we really were not interested in Cubes, Star-Schemas and all of the stuff they did to generate what was basically an Excel spreadsheet on steroids.

Now having spent some more time on that side of the fence I've come to respect it more, but there is now an interesting evolution.  Contextual BI.  That is where BI guys talk about integrating BI and Analytics information back into operational processes.  In other words its BI guys talking about integrating to SAP and integrating into BPM and integrating into operational applications generally.

An IT turf war is about to break-out.  I'd argue that the software development guys have a bit of an advantage here, what is the current 'cool kids' BI tool?  Its Hadoop.  What do you need to know to really grok Hadoop?  Yup Java, a software development language.  Software Dev guys grok agile in a way that few BI guys do and are used to getting down into operational requirements.

But we suck at analytics and its massively expensive to develop the sorts of visualisations that BI guys get out of the box.  So this means its either a case of one side taking over the other, or we are going to have to co-operate... and I actually think this is the way forwards.  Software Development guys know how to do real-time scaling and operations, its not a trivial skill, and the BI guys know how to get us the right information into our processes and have the right visualisation tools.

So Software Developers out there its time to reach out and embrace your BI cousins and start looking at how you can work together to create an information fabric which spans the analytical models on historical data, supplements it with real-time analytics and then integrates that back into the operational process.  I know this collaboration is going to be hard and we've all got to get over prejudices to make this happen but the business is demanding it so we need to get it done.  Start thinking about information as a seemless infrastructure where delivery to the point of action is the challenge not delivery to a report.  Software Developers and BI guys working together can create this and deliver a real information fabric to the enterprise.

And don't worry Software Developers and BI guys will still have a common enemy, I'm not crazy, its still perfectly ok to not get on with the package guys.