Social Relationships don't count until they count

There is a game called "the Six Degrees of Kevin Bacon" which tries to link between any Kevin Bacon and any other actor in less than six steps.  This is a popular version of the "small world" thesis put forwards by Stanley Milgram.  In these days of Social Media and "relationships" there is a massive hype around farming these relationships with an implicit assumption that someone with lots of relationships is more valuable than someone who doesn't

The problem is that in reality this is all a version of the Travelling Salesman problem with everyone assuming that every link is of the same value.  The reality is that links have different values based on their strengths so understanding how individuals are actually related is significantly more complex than many social media "experts" would have you believe.

What do I mean by this?  Well my "Obama Number" is 4 as, via my wife, I can trace to Obama in 4 steps with each individual step being reasonably strong.  By reasonably strong I mean that each link has met the previous link several times and probably could put a name to the face.  Now the variability of strengths on these links is huge, from my wife (hopefully a strong link) to people who move in similar social circles and then into the political sphere where the connection to Obama is made.

I've a Myra Hindley number of 2 as I have a friend who met her more than once (before her conviction).

So for Republicans and Tea Party nut-jobs this means that its 6 steps max from Obama to a child killer.  Does this mean there is a relationship worth knowing or caring about?  Nope.

So how to weight relationships and how to weight each step within the graph?  Well this is actually pretty simple.  Lets say A has a relationship to B via a social network, lets call that a score of 0.0001.  Lets say that B (who is the person) has a score of 1.0.  So for each interaction between two individuals you then look at the strength from A to B.

1. How many times does A post to B?  If  > 10 then add 0.0001
2. How many times does B post to A?  If > 10 then add 0.001 (i.e. B connects to A, hence more likely to be mutual) for each multiple of 10
3. How many times does B indicate that they are at the same place as A? If > 10 then add 0.001 per 10
4. How many times does a voucher provided to A get used by B? If  > 10 then add 0.1 per 10
5. Are they directly related or married? If cousin or less then add 0.5
6. Do they work closely together? If within 1 reporting hop add 0.2
7. How many times have they met? If > 10 then add 0.05 per 10

What I'm saying is that its actually the interactions that matter to back up the social experience rather than the existence of a social link.

So while from Obama to me is 4 steps I'd say that overall its pretty weak (0.8 * 0.2 * 0.2 * 0.2 =  0.0064) a .64% link which really means I'm not worth lobbying to get influence over the US president.

This is where the combination of Big Data analytics could really deliver value, by understand the true weightings on individual relationships and from that determining the real genuine paths to the maximum possible market for the minimum effort.

Technorati Tags: SOA, Service Architecture

iCloud 2.0 - CloudApps

2 years ago I wrote a post on why Apple might dominate the cloud and how an integrated offline/cloud backup solution would both offer more value from Apple's cloud but also offer more of a lock-in. I do like it when a prediction comes pretty much spot on, even if they've only just started doing what I thought they would.

Now the iCloud 1.0 is just a pretty basic sync, and as predicted, it does provide a premium service that includes the ability to sync your whole library. It doesn't appear to do the interface suggestion I made of integrating it directly into the iPod player but instead requiring you to go via the iTunes application, but that really is a minor improvement (and not a difficult one to do either). So we can see that now they've added in the cloud backup for iOS it surely can't be long before its extended to include OS X, especially as its effectively including it for photos already.

What next for Apple and the cloud?

Well one thing they haven't done yet is automate some of this sync, so when I say in iTunes "last 5 un-played" it doesn't automatically do the update on your device but this is a minor piece really.

The bigger thing that isn't in there yet though is the idea of using processing on the cloud rather than simply storage. So doing things like fancy video effects rendered on the cloud would be a good way to extend the experience on both the desktop and the mobile world to include a whole new generation of Apps.

CloudApps

So you don't just have the back-up/sync and all of those other elements but once you have your information being exchanged in this way you open the world to more consumer focused applications, or cloud extensions to existing applications.

Microsoft already have a limited part of this with their cloud services, but they don't appear to have either the co-ordination, brand or vision to make it really happen. Google might have an opportunity with their services and Android but the control of the handset manufacturers and operators might stop them.

The other people who should be worried are Facebook. The point of CloudApps is going to be towards collaboration and multiple users, sharing and the like. So while Ping hasn't been a success this application centric cloud approach could give Apple just what it wants - control within the social media space.

Technorati Tags: SOA, Service Architecture

What REST needs to do to succeed in the enterprise

In the spirit of constructive criticism here is what REST needs to do in order to succeed in the enterprise and B2B markets, the sort of markets that make actual revenues and profits as opposed to hype markets with the stability of a bubble.

First off there is the mental change required, four steps here.

1. Focus on how people and especially teams work
2. Accept that HTTP isn't a functional API
3. Accept that enterprise integration, B2B and Machine to Machine require a new approach
4. Accept that the integration technology isn't the thing that delivers value

The point here is that REST won't move on and be successful beyond blogs and some very cool web sites and technologies unless it shifts away from technical purism and focuses instead on making the delivery of enterprise software solutions easier. This means helping disparate teams to work better together, and how do you do that.....

DEFINE A CONTRACT

Seriously its that easy. The reason why WSDL succeeded in the enterprise is that it gave a very simple way of doing just this. The interface contract needs to define a limited number of things

1. What is the function being invoked (for REST this could just be a description)
2. What data can be passed and will be returned
3. How to invoke it (that would be the URI and the method (POST, GET, PUT, DELETE))

This contractual definition should be a standard which is agreed, and adhered to, by core enterprise vendors and supported by tools. Now before people scream "but that is against what REST is about" well then you have a simple choice

1. REST remains a niche technology
2. REST becomes used in the enterprise

Now in order to become more used we need to also agree things like how you do user authentication, field level security & encryption, rules for reliability on non-idempotent requests, so you know whether your POST request really worked....

So what else does REST need to do? Well it needs to focus on tools because plumbing has zero value. Dynamism does happen but its measured in weeks and months not in days which means an agile release process can handle it perfectly well so all that dynamism and low level coding doesn't really add anything to enterprise development.

This is a key point, something I raised in 2006 (SOA v REST more pointless than vi v emacs) the value is what happens AFTER the call is made, focusing on making the calling "better" is just pointless, the aim is to make the calling as simple as possible.

So basically to succeed REST needs to copy the most successful part of SOAP... the WSDL, sorry folks but an "improved" WSDL based around REST and the associated tooling is required.

Or alternatively the REST crowd could just bury its head in the sand and pretend that its the fault of the enterprise that REST isn't being adopted.

And remember:

There is no value in integration only in what integration enables.

Technorati Tags: SOA, Service Architecture

REST isn't undead in the enterprise... its still born

Its always depressing to see fanbois bleating and moaning about their beloved technology or piece of bling not being universally liked. This is normally put down, in a wonderfully immature way, to the failure of "the other side" to see their point of view rather than any innate failings of their beloved approach. SOAP is not Dead - Its Undead is a classic of the genre.

Why hasn't REST succeeded in the enterprise? Not of course because it isn't actually any better than SOAP for enterprise scenarios and is indeed much, much worse in many. Nope.

But first there is the reason why REST is successful

His presentation showed that 73% of the APis on Programmable Web use REST. SOAP is far behind but is still represented in 17% of the APIs.

This is like going to France, doing a language survey and declaring that French is the most popular language in the US. So what would doing the same query on the likes of Oracle, SAP, IBM or Microsoft's enterprise technology stacks deliver? I assumed the number to beat would be huge and got ready for some serious searching.... but the number to beat is 2368... errr seriously? I've worked in single enterprises where they had more SOAP endpoints than that. When you include the libraries of WSDLs from SAP and Oracle and the Behemoth that is Oracle AIA has so many that Oracle don't boast about it as it might make it look complicated. Back in 2005 folks at Oracle boasted about over 3000 web services across their applications. Now before people bleat about this being proof of SOAP complexity... that just makes you a hypocrite if you on one hand use the ProgrammableWeb stats as "proof" of RESTs success but then try and use the massive volume of WSDLs out there as proof of SOAP's "complexity.

I'm also here not even into the global standards that use SOAP every day for B2B, people like SWIFT, Open Airlines... shall I go on and on? 2400 APIs is a success? SOAP isn't anywhere near that? Like I say its like going to France and claiming French is the most spoken language on the planet.

All this just proves what I've said for a long time. REST works for information traversal, but its not set up for the enterprise. So what is the issue with REST not displacing SOAP in the enterprise?

"All the tools, hires, licenses & codebase has been built around SOAP for a decade," Loveless wrote on Twitter. "Hard to turn on a dime."

Wow, the bare facedness of this statement is hard to beat. REST has been kicking on this door for over half of that time and some folks argue that in fact it predates SOAP. So it really is bullshit to claim that its all the fault of tools & codebase. SOAP replaced old EAI approaches in a couple of years in new enterprise projects. We went from a situation with everyone in about 1998 doing proprietary EAI integration, with occasional CORBA for RPC, to everyone by 2002 doing Web Services with some JMS. People in 2005 were telling me that REST was the future and REST would win, and now SIX YEARS LATER people are bleating about 10 years of SOAP adoption...

If an approach is better for integration in the enterprise it will be adopted. REST isn't better, yet, for enterprise integration because it fundamentally remains a developer approach not a professional enterprise approach. SOAP isn't complex, technically it might suck (hell my father said "great we've now got enough processing cycles to burn that ASCII-RPC has finally made it") but conceptually its simple, and when managing complex estates with lots of different people that conceptual simplicity on the head.

Michael Cote of Redmonk hits the nail partially on the head when he says

"As enterprise development teams start including cloud technologies in their applications, incompatible cloud platforms and APIs will be a huge road block," said Michael Cote, analyst at RedMonk. "We're already seeing a clamoring for tools and services that integrate this spaghetti bowl of end-points, and they're only going to become more important to realizing the benefits of cloud development."

In other words the lack of a formal contract and standard interface mechanism remains the real reason why REST isn't being adopted in the enterprise.

What SOAP did was solve a problem that the enterprise had. How do I describe integration interfaces so my systems on different technology stacks can communicate and do so in a way that enables my teams to work independently of each other. REST does not solve this problem in an effective way and bleating about "dynamic interfaces" being "better" misses the whole point of what has made B2B and Machine 2 Machine integration successful down the years, namely a focus on people-centric approaches rather than technical centric ones.

Unfortunately with REST there appears to be an active movement to stop this professionalism creeping into it and defining new standards that will actually make REST better for the enterprise.

REST needs a standard way to publish its API and for a way to notify changes in that API. This is a "solved" problem in IT but for some reason the REST community appears to prefer blaming others for the lack of enterprise success of their technology rather than facing up to the simple reality:

SOAP got some things right and the biggest thing it got right was a shareable and toolable contract (WSDL) which enabled interfaces to be published in a standard way which included by functional and data standards.

SOAP isn't undead, its very much living in the enterprise and indeed being the only real viable approach when integrating package solutions from a number of vendors (a massive piece of enterprise IT). REST however barely registers, less than 2500 APIs after all these years of development? Pathetic.

REST for the enterprise isn't undead... its been still-born for over five years.

Technorati Tags: SOA, Service Architecture

The three magic questions of business SOA (and IT Strategy)

Having a chat with someone today I was reminded of a presentation I gave a few years ago. I went to the client knowing that they'd spent a lot of money in the last few years on an IT "refresh" and the word SOA had been used rather a lot. The programme wasn't seen as being a success and the business were bitching that money had been wasted.

My opening point was that the business was sort of right and so I posed three questions

1. Do you have a clear vision of where you want your IT estate to be in 3 years
2. Do you want the IT estate to reflect that vision
3. Do you want the costs for IT to reflect the different values in that vision

The answers were "No, Yes, Yes" which wasn't a huge surprise but was my real point to both sides of what had turned into a political mudslinging competition.

IT had set off to try and do 2 and 3 without knowing 1 and the business had effectively said "we don't know what we want... but we know its not that". Neither of these was a good idea. So the focus therefore was on creating that vision (a business service driven vision) and doing the heatmap that meant IT could then align itself (number 2 and 3) to that vision.

Now I've said before that IT and Chairman Mao have a lot in common but there is another famous phrase that applies here

"A journey of a thousand miles begins with a single step"

Now that might be true, but its really important to know where you are going otherwise you look like a bit of a pillock in a thousand miles.

So can you answer the three magic questions of Business SOA and IT strategy?

Technorati Tags: SOA, Service Architecture

One year on: zero progress for Enterprise REST as Java races backwards

Just under a year ago I blogged about how REST and Sun had put Enterprise IT back five years so I thought it was about time to update that view and see what has happened in the last 12 months in the enterprise integration and governance space.

So on the REST front we've seen.... ummm.... struggling here.

Lets be clear, I'm talking here about REST as an enterprise integration approach, not as a way of exposing a Web API for content aggregation but as a functional integration approach for enterprises. Something to replace the "fundamentally flawed" WS-* that REST is so much better than. So what is the progress this year? Zip, zero, nada. Yup a few minor tweaks into enterprise stacks that say they can produce REST interfaces, but in reality most of them can't and the key problems of interface publication, versioning and testing remain unsolved.

Am I being harsh on REST? I don't think so. Its had more than enough time, and hype, to address the real problems of enterprise computing and step out of the niche, and in revenue terms it is a niche, of Web content aggregation. REST is great for that niche, but that wasn't the pitch made, the pitch made was that REST was great, WS-* sucked and REST would solve all the problems that existed with WS-*. This hype, and the smart people who followed it, has led to a stagnation in enterprise integration that is really hitting enterprise computing in real revenue terms. Its slowing the adoption of cloud computing and generally meaning that IT departments are less credible and less successful than they should be.

So one year on and REST continues to flatter to deceive.

What about Java? Well here the story is actually worse. I honestly believe that the REST crowd are a smart bunch of bunnies and are trying to solve problems, just not realising that most developers aren't as smart as them and that interface dynamism is actually a bad thing. With Java that is sadly not the case. With Mark Rheinhold now actively subverting the JCP and the debacle in the JCP generally over the last year its hard to think that the situation is doing anything other than getting worse.

What does this mean for Enterprise IT? Well a couple of things, it means that SAP, IBM and Oracle the three headed beasts of most IT estates don't have a clear future integration improvement roadmap, its all still based around WS-* as it was 4 to 5 years ago and they are all adding proprietary "tweaks" which "help" people when using their platforms. It also means that their core platform, Java, is stagnating and not getting some of the fundamental changes it needs to address the cloud and dynamic scalability. In particular the continuing "kitchen sink" approach of the Java dictatorship means that reduced profile VMs that are tuned to specific tasks (like Mobile potentially) just aren't being addressed which is leading directly to a fragmentation in the core operating platform.

What does then mean? Well with REST it means that enterprise IT shops are relying on WS-* for integration but still coming across things like vendors not having WS-Security support and with the decline of WS-I the number of "strange" defects is liable to be on the rise. Efforts around more formal contract approaches are dead. This means that the "spaghetti" of enterprise IT, which looked to improving in the first 6 years of the new millenium is actually now getting worse again. REST aimed at WS-* squarely and surely and has certainly hit it with a wounding shot, unfortunately it turns out that REST is incapable of being the replacement for WS-* it wanted to be. REST is Brutus, WS-* is Caesar.

And with Java? It means we are seeing language fragmentation and platform fragmentation which means that the support costs of IT estates are going to rise and so the on going challenge of reducing operating costs to enable investment in new solutions is swinging back against the business and towards entrenched IT estates.

I really can't see how a large scale enterprise IT programme is better off in 2011 than it was in 2005.

Technorati Tags: SOA, Service Architecture