Think in RPC develop in anything

Gregg Wonderly made a good comment on the Yahoo SOA list the other day

I think one of the still, largely unrecognized issues is that developers really should be designing services as RPC interfaces, always. Then, different service interface schemes, such as SOAP, HTTP (Rest et.al.), Jini, etc., can be more easily become a "deployment" technology introduction instead of a "foundation" technology implementation that greatly limits how and under what circumstances a service can be used. Programming Language/platform IDEs make it too easy to "just use" a single technology, and then everything melds into a pile of 'technology' instead of a 'service'.

The point here is that conceptually RPC is very easy for everyone to understand and at the highest levels it provides a consistent view. Now before people shriek that "But RPC sucks" I'll go through how it will work.

First off lets take a simple three service system where from an "RPC" perspective we have the following:

The Sales Service which has capabilities for "Buy Product" and "Get Forecast"

The Finance Service which has capabilities for "Report Sale" and "Make Forecast"

The Logistics Service which has capabilities for "Ship Product" and "Get Delivery Status"

There is also a customer who can "Receive Invoice"

Now we get into the conceptual design stage and we want to start talking through how these various services work and we use an "RPC language" to start working out how things happen.

RPC into Push
When we call "Make Forecast" on the Finance Service it needs to ask the Sales Service for its Forecast and therefore does a "Get Forecast" call on the Sales Service. We need the Forecast to be updated daily.

Now when we start working through this at a systems level we see that the mainframe solution of the Finance team is really old and creaky but it handles batch processing really well. Therefore given our requirement for a daily forecast what we do is take a nightly batch out of the CRM solution and Push it into the Mainframe. Conceptually we are still doing exactly what the RPC language says in that the data that the mainframe is processing has been obtained from the Sales area, but instead of making an RPC call to get that information we have decided in implementation to do it via Batch, FTP and ETL.

RPC into Events
The next piece that is looked at is the sales to invoice process Here the challenge is that historically there has been a real delay in getting invoices out to customers and it needs to be tightened up much more. Previously a batch has been sent at the end of each day to the logistics and finance departments and they've run their own processing. This has led to problems with customers being invoiced for products that aren't shipped and a 48 hour delay in getting invoices out.

The solution is to run an event based system where Sales sends out an event on a new Sale, this is received by both Finance and the Logistics department . The Logistics department then Ships the Product (Ship Product) after which it sends a "Product Shipped" event which results in the Finance department sending the invoice.

So while we have the conceptual view in RPC speak we have an implementation that is in Event Speak.

RPC into REST
The final piece is buying the products and getting the delivery status against an order. The decision was made to do this via REST on a shiny new website. Products are resources (of course), you add them to a shopping basket (by POSTing the URI of the product into the basket) this shopping basket then gets paid for and becomes an Order. The Order has a URI and you just simply GET to have the status.

So conceptually its RPC but we've implemented it using REST.

Conceptual v Delivery

The point here is that we can extend this approach of thinking about things in RPC terms through an architecture and people can talk to each other in this RPC language without having to worry about the specific implementation approach. By thinking of simply "Services" and "Capabilities" and mentally placing them as "Remote" calls from one service to another we can construct a consistent architectural model.

Once we've agreed on this model, that this is what we want to deliver, we are then able to design the services using the most appropriate technology approach. I'd contend that there really aren't any other conceptual models that work consistently. A Process Model assumes steps, a Data Model assumes some sort of entity relationship a REST model assumes its all resources and an Event model assumes its all events. Translating between these different conceptual models is much harder than jumping from a conceptual RPC model that just assumes Services and Capabilities with the Services "containing" the capabilities.

So the basic point is that architecture, and particularly business architecture, should always be RPC in flavour. Its conceptually easier to understand and its the easiest method to transcribe into different implementation approaches.

Technorati Tags: SOA, Service Architecture

SOA, REST, Packages, Web Services and Hamlet

In a few discussions recently where people have been looking across an enterprise I've had the same discussion in three different ways.

In the first one the future was all about packages, everything was going to be in a package, there would be no more software development as a package vendor did everything that they wanted. When I pointed out that this actually covered only the back-office functions and a minimal part of the sales function there was an awkward moment as they realised than in fact what they really meant was "we are going to standardise our back-office and sales processes around vendor X" this is a good objective to have but the point remained that one solution didn't fit all.

In the second one the world was going to be REST, EVERYTHING was going to be REST, the entire enterprise should be viewed as resources and the principles of the Web would be applied all over, most especially when dealing externally. I pointed out that not only did several suppliers already interact with them via Web Services but that they had a massive backend estate that had a reasonably successful integration strategy. After a couple of hours contemplating the refactoring of a huge number (think hundreds) of systems they decided that maybe REST would just be used around the Web part and for data when its appropriate, and then when there weren't already established approaches.

In the third one the future was all about Web Services, EVERYTHING should be a Web Service. Talking to a Mainframe? Use a Web Service. Talking within an application between one bit and another... use a Web Service. They were even looking at WS-TX. A similar discussion to the REST one then unfolded and now their B2B approach is looking at Web Services and their inter business domain comms will be Web Services.

This all put me in mind of Hamlet and the famous like

There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy

The point is that there is no single answer to a complex IT environment. I've always argued this around Business SOA. Business SOA gives a context not a complete solution. Technology approaches can only ever give part of the solution. Taking a fundamentalist approach to IT is always a bad idea and it does help progress IT forwards.

There are lots of bad ideas in IT, often they are good ideas which are applied in bad places.

Technorati Tags: SOA, Service Architecture

SOA v EDA - The business don't care and neither should you

The old EDA v SOA or SOA 2.0 view is still doing the rounds I note, which isn't surprising given the technology focus of lots of companies. But I thought I'd try very very briefly to explain what the difference is and why from a business perspective it isn't important what the implementation approach is.

Business Scenario

Okay so the scenario will be a standard supply chain where the invoice is sent out at the same time as dispatch from the warehouse. Using the SOA Methodology this gives us two "what's"
A who
and a few "why"s one of which goes between the two services.

So that is the business view. The next bit is to compare the Event based and request based implementation of that single why between the services. The direction, from a business perspective, is clearly from the warehousing to finance as that is what works in the business language for this scenario, this however doesn't matter to the technical implementation of the services. They key as with Christmas SOA is understanding how the business objectives can be met by technology, not trying to do a one-to-one match.

Request based

So what is the implementation in a request/reply world? Well clearly we have two services to build from above, but the question is what is the implementation approach for the request. The key with a request implementation is that the question is "I need to ask the other service to do something" hence its all about the destination service in the request. In plain old pseudo code this means

service Finance has

SendInvoice capability, which accepts a purchase order and customer details

This is the "traditional" Web Service way of thinking about things (although of course REST would be a post of an invoice as the code implementation). This is fine and dandy and clearly meetings the requirements of the overall business scenario.

Event based

In the event based world the question is about what has happened that will trigger a change. In this world its what the destination service is waiting to happen from the consumer. So in old pseudo code terms this means that we have two definitions.

service Warehouse has

DispatchOrder capability which sends an order dispatched event on completion

service Finance has

SendInvoice capability which listens for dispatched orders

This is the "traditional" Event way of thinking about things. Again this meets the business capability.

Pseudo Event - Polling for requests

There is a "third" (and in my opinion often technically poor) way of working which is to pretend you are event based but really its building an event model by using polling on requests. In this world the Finance service would continually put a request into Warehousing to ask "have any orders been dispatched" so back in pseudo code we have

service Warehouse has

DispatchedOrders capability which returns a list of the recently dispatched orders

In this case its sort of like Warehouse publishing an RSS feed of dispatched orders and Finance subscribing to that feed, or in old school speak its like Warehouse dumping a batch file at the end of the day to a shared disk and Finance uploading it. This again meets the business objective.

Summary

My point here is that whether doing Request/Reply, Eventing or Polling this is an implementation decision and is irrelevant from the business perspective. The key is to understand the best implementation method that meets the business objectives, this will be based on performance, timeliness and other "ilities" or just down to "how the current systems work". This separation of the business objectives from technology implementation is part of the key to making sure that IT moves in step with the business, you need to make IT deliver in business terms which means understanding the objectives rather than blindly implementing the requirements. Request/Reply, EDA, Polling, BDI and the like are all just technical implementation approaches to meet the business objectives. They are tools in the toolbox, and they are about design rather than architecture.

SOA, as in Business SOA, is about architecture, then choose the right tool for the implementation job.

Technorati Tags: SOA, EDA, Service Architecture

SOA v POA, EDA, GDA

First of all a warning... I have an SOA hammer and maybe everything does look like a nail.

SOA - Service Orientation - I tend to see SOA has the bit where I stand looking at the toolbox wondering what I should use to get the job done, its key aim isn't to do the detail, its to help select the right tool to do the detail. In this SOA attempts to define the broad areas and capabilities, not to focus on specifics at the start. So starts by understanding the "What" of the enterprise, the broad "what do we do" questions, it then worries about "who" do we interact with and finally "why" do all these things work together. It doesn't deal with the "How" of implementation it just creates the framework within which that can be done.

There are other options as to where you start of course, and I'd like to start by saying if it works for you then fine, but I'm not liable to be joining you.

POA - Process Orientation - This is IMO the most broken model of all, often building on business school work of the 1990s it suggests that organisations are oriented around their processes and that by understanding the end-to-end processes. The problem is that businesses aren't oriented like this, processes go across many areas and process based implementation tends to lead to brittle and inflexible solutions. Organisations aren't process oriented. POA also actively prevents SOA as its a bugger to try and retro-fit decent service granularity into a process model, and equally if you just have lots of small components managed by big processes then its just Visual Cobol with components.

EDA - Event Driven - Identify the events first says this approach and then you can understand the business. The problem I have here is that this is to me just the capabilities bit of SOA, you need the encompassing service for two reasons. Firstly to provide context as to what the event is about and what is its surrounding business environment, and secondly to provide a place into which new business events can be added. Viewing businesses based on their events quickly creates large lists of events and its pretty tough to add a structure, even if you do event chaining, because it is just single events calling other single events. While not as bad as POA this approach still takes the approach that organistions operate based around single events rather than taking a more holistic view.

GDA - Goal Driven - Identify the business goals and you will then be able to move onto how to deliver against them. This isn't that bad an approach as Goals are often complex composite things for a business and they do help you understand the key drivers and principles of an organistion. The trouble is, particularly if you are more extreme and gather goals in a "flat" way rather than concerning yourself with the domain within which the goal is managed or attained means that you again have a set of singletons interacting which can make creating simple architectural solutions a bit of a challenge.

The good news is that all of these things are great second steps if you take a good business service oriented view. If you have one service that is best modeled by thinking of it as processes then POA within that service is a good thing. If you have a reactive system based around events then EDA will work within that service. If you have a goal oriented area that is best described in terms of its outcomes then GDA is right within that service. if its best to model Beliefs, Drivers and Intents then BDI is liable to work within that service. You could even say that SOA models the services, EDA the capabilities and the POA how some of those capabilities are implemented, but either way its the service that needs to come first.

EDA, POA, GDA and BDI and even OO are great ways to model how a service should work, or more specifically how the capabilities on a service should work. This is the key, EDA, POA and GDA are all aiming to define the capabilities that an organisation, enterprise or project should have, SOA aims to provide the mechanism for containment and abstraction for those capabilities and this is why it needs (IMO) to come first.

You don't start modelling an Object Model by listing out all the methods and attributes then trying to fit them into classes, you start with the classes then wonder what the methods and attributes should be, then worry about how to model their implementation. Its no different with SOA but operating at a business rather than a technical level. The first question is what are the containers, the buckets, the blobs... the services, then and only then should we worry about the capabilities, and once we have them we should worry about how you implement them.

Like with Web 2.0 v SOA its not actually one or the other. SOA provides the context, the others worry about the details.

Technorati Tags: SOA, POA, EDA, GDA, Service Architecture