Saturday, January 09, 2010

Think in RPC develop in anything

Gregg Wonderly made a good comment on the Yahoo SOA list the other day
I think one of the still, largely unrecognized issues is that developers really should be designing services as RPC interfaces, always. Then, different service interface schemes, such as SOAP, HTTP (Rest, Jini, etc., can be more easily become a "deployment" technology introduction instead of a "foundation" technology implementation that greatly limits how and under what circumstances a service can be used. Programming Language/platform IDEs make it too easy to "just use" a single technology, and then everything melds into a pile of 'technology' instead of a 'service'.

The point here is that conceptually RPC is very easy for everyone to understand and at the highest levels it provides a consistent view. Now before people shriek that "But RPC sucks" I'll go through how it will work.

First off lets take a simple three service system where from an "RPC" perspective we have the following:

The Sales Service which has capabilities for "Buy Product" and "Get Forecast"

The Finance Service which has capabilities for "Report Sale" and "Make Forecast"

The Logistics Service which has capabilities for "Ship Product" and "Get Delivery Status"

There is also a customer who can "Receive Invoice"

Now we get into the conceptual design stage and we want to start talking through how these various services work and we use an "RPC language" to start working out how things happen.

RPC into Push
When we call "Make Forecast" on the Finance Service it needs to ask the Sales Service for its Forecast and therefore does a "Get Forecast" call on the Sales Service. We need the Forecast to be updated daily.

Now when we start working through this at a systems level we see that the mainframe solution of the Finance team is really old and creaky but it handles batch processing really well. Therefore given our requirement for a daily forecast what we do is take a nightly batch out of the CRM solution and Push it into the Mainframe. Conceptually we are still doing exactly what the RPC language says in that the data that the mainframe is processing has been obtained from the Sales area, but instead of making an RPC call to get that information we have decided in implementation to do it via Batch, FTP and ETL.

RPC into Events
The next piece that is looked at is the sales to invoice process Here the challenge is that historically there has been a real delay in getting invoices out to customers and it needs to be tightened up much more. Previously a batch has been sent at the end of each day to the logistics and finance departments and they've run their own processing. This has led to problems with customers being invoiced for products that aren't shipped and a 48 hour delay in getting invoices out.

The solution is to run an event based system where Sales sends out an event on a new Sale, this is received by both Finance and the Logistics department . The Logistics department then Ships the Product (Ship Product) after which it sends a "Product Shipped" event which results in the Finance department sending the invoice.

So while we have the conceptual view in RPC speak we have an implementation that is in Event Speak.

The final piece is buying the products and getting the delivery status against an order. The decision was made to do this via REST on a shiny new website. Products are resources (of course), you add them to a shopping basket (by POSTing the URI of the product into the basket) this shopping basket then gets paid for and becomes an Order. The Order has a URI and you just simply GET to have the status.

So conceptually its RPC but we've implemented it using REST.

Conceptual v Delivery

The point here is that we can extend this approach of thinking about things in RPC terms through an architecture and people can talk to each other in this RPC language without having to worry about the specific implementation approach. By thinking of simply "Services" and "Capabilities" and mentally placing them as "Remote" calls from one service to another we can construct a consistent architectural model.

Once we've agreed on this model, that this is what we want to deliver, we are then able to design the services using the most appropriate technology approach. I'd contend that there really aren't any other conceptual models that work consistently. A Process Model assumes steps, a Data Model assumes some sort of entity relationship a REST model assumes its all resources and an Event model assumes its all events. Translating between these different conceptual models is much harder than jumping from a conceptual RPC model that just assumes Services and Capabilities with the Services "containing" the capabilities.

So the basic point is that architecture, and particularly business architecture, should always be RPC in flavour. Its conceptually easier to understand and its the easiest method to transcribe into different implementation approaches.

Technorati Tags: ,


Doug said...

Seems like the same discussions we had with objects and then components. Conceptually, you have to agree on both the data and the behavior (the nouns and the verbs). Then you can decide on the implementation - doc style, RPC style, REST style, etc.

William said...

Sure. There is RPC as a family of invocation mechanisms versus RPC as a way to think about interactions. You may have been able to save yourself some grief by saying "UML sequence diagrams are useful and can then be mapped to any architecture" instead of appearing to defend the now-out-of-fashion RPC term. But maybe this mental dust-off is just what you are trying to do.

A nice and to-the-point post in any case.

John Scott said...

Rather than thinking in RPC and developing in anything, I would recommend thinking in terms of message exchanges and developing in anything. This would enable you to think in terms of MEPs and RPC becomes an implementatioon mechanism alongside web services, REST and all the others. This is how the UML represents communication between parts in a conceptual manner.

Steve Jones said...

Doug, William, yes it isn't really new its just making clear that actually the ways that work... still work. UML Sequence diagrams are (for me) the next stage in this and sometimes not always business friendly but its about separating the ARCHITECTURAL concerns from design and implementation.

Steve Jones said...


I couldn't agree less I'm afraid. Message Exchange is a very confusing way to think about complex systems without first thinking about the services and sending/receiving capabilities. If you have those first elements (which are RPCesque) then you can look at MEP as the next refinement but done within the context still of the services & capabilities. Looking _just_ at the message exchanges without that initial context is (IME) not a great way to go.

John Scott said...

I think RPC is too narrowing a concept from the outset. If I want a service from you I make a request and get a response. If I want to announce an event I produce a message which is consumed by interested parties. I don't want or need a response.

These could be implemented using anything from local calls, RPC, MOM to whatever depending on the physical requirements.

Steve Jones said...

But John you are saying here about design decisions not architectural ones. Your first scenario is RPC (request/response on a capability) and the second is a service with a notification capability which you choose to implement via events.

Events are not a "natural" language for most people and do not match well to business speak. It is extremely rare for the business to really not care about who listens to an event, they know that X does something and Y cares about it, hence conceptually "X calls Y" or "Y polls X until something happens" which are RPC pieces which we then choose to use EDA to implement.

Even in your language you used "services" as being central which means at the least you need to know what the services are before you know what the messages are, or are you proposing that you can have messages independent of services?

John Scott said...

I think RPC is just as unnatural. My previous point was that not all interactions are request-response, nor require both parties to be "present" at the same time, but I think to most people RPC implies that.

Your necessary explanations on how this RPC approach would work indicates to me that its not a natural way either.

I don't find the "X polls Y until something happens" models the real world well either. We don't employ people to do this kind of work. Business'es sometimes passively wait for events and a business process is the complete response the business makes to an event.

I probably chose terms for my examples from yours in a poor manner. I don't think services are the only way to solve problems, but a one-way message exchange is the smallest fundamental communication between parts on which all others can be built.

The trick might be finding effective names for the message exchange patterns, though I don't think RPC is one of them.

Steve Jones said...


While a message exchange is indeed the lowest point of an interaction this doesn't make it suitable for explaining the larger phenomena of the architecture. To use an analogy, message exchange is like quantum theory it explains things right down at the atomic level, can be very complicated and isn't helpful when trying to understand the much larger real world.

The point is whether you can construct an architecture in just MEP. I've really never seen that done successfully, I've seen it done very unsuccessfully and create very complicated networks of messages and exchange patterns which really helped no-one in understanding the overall purpose of the solution.

Polling does happen in the real world, for instance people regularly phone to check "Have my expenses been paid yet" as well as waiting for the "event" that tells them that they have. Even when looking at these events though the business condition is normally "We send an event to X" in otherwords its conceptually still just an RPC call. These events have emitters (services) and receivers (consumers) and far from being "fire and forget" its normally very important to ensure that an action follows the event being fired.

The point here is not RPC as the implementation or design element but RPC as the architectural and conceptual model.

Rob Eamon said...

Excellent thoughts here and on Y! groups. The "RPC as concept" works for me but clearly RPC as a TLA has lots of baggage.