The Blog based, RSS using, Yahoo pipe executed Carbon footprint calculator

I've had an idea for a while which started when I did the Geo ripping Wikipedia thing that created a web service. The idea was to have an easy way for me to calculate my carbon footprint, not the one for the house and those things but one that actually took into account how much I travel.

I also set myself another task and that was to achieve this end using communication technologies I hadn't used in that way before. So this ruled out using WS, and led me straight towards the world of POX... or is it REST, I'm not sure I can tell :)

Now taking my own advice and separating "Service" from invocation using a Facade to enable WS and REST on the same code base I quickly knocked up a servlet that takes in a place name (or a geo coord) and returns it as an RSS encoded entry. It also takes in a transport type (more of that later) which is basically to enable pass through. The Place search Pipe uses the same backend service as the WS service, with information ripped from Wikipedia.

Next up is to then have a source of information that will make this all useful. Welcome to Blogger as a data store: My trip blog is just test data (except the last destination for which I leave today). This gave me an input RSS feed that could then be consumed by another pipe, the next pipe is a bit more complex. Firstly it takes in a feed and then orders by date (the opposite way around to most feeds) and then it uses that information to call the first pipe to obtain a geo-tagged set of information. This Geotagging Pipe turns the blog feed into a set of points.

The final pipe in this stage is then one that is meant for cloning by other people if they want to do the same (thinking of the consumer's view) that one can be found here (NB you need to look at the RSS to get the real info).

Stage two of this exercise was then to have something that would take that RSS feed and do all the calculations. Again a quick bit of development followed by fronting using Rome and the next stage was ready. Using the JCoord library to do the distance calculations and taking the carbon numbers from The National Energy Foundation the next servlet was soon deployed. So back into Yahoo Pipes it was, this time taking the URI of the generated feed which is then passed to the new servlet which then returns its resuts as another RSS feed. The carbon calculation Pipe takes an input (as text because Yahoo pipes kept trying to be clever when I used a URL builder) of the Pipe generated RSS feed and then dumps it to the servlet. A Consumer focused (clonable) implementation for my feed is then available here.

Okay so there you have it a few hours of working and its all working, taking information geo ripped from Wikipedia, a blog feed and a bunch of servlets all done using RSS.

Now there is one slight issue here. I've got the carbon calculations for "train", "plane" and "car" but for some reason when you debug the feeds it shows the categories correctly but the deployed pipe is devoid of any category information, so at the moment this means all journeys are calculated based on flying. Other issues are that the database isn't complete (I need to update my regexp stuff to catch some more entries) and the search is currently "first" rather than "best" so London = London borough of Croydon.

Next step (obviously) is to take that feed and turn it into a GoogleMap.

Now one question I have is... did I "do" REST?

If I have time at the station I might lob up a picture of how it all works.

Technorati Tags: SOA, Yahoo Pipes, RSS

REST isn't about the enterprise.... its about fixing the world (apparently)

Now I've been playing around recently with REST, there are even some projects at work using it (mainly around front end stuff, WS tends to do the heavy lifting) and pretty much its working well for developer to developer constrained domains (i.e. not 3rd party integration) and I'm even thinking of applying for the REST JSR to make sure it doesn't go all fan boy and complex.

But in terms of sheer gobsmacking appeals the paper that Pete Lacey will be presenting at W3C next week surely wins the IT hubris award for the REST crowd. The majority of the abstract reads as a moan that "people don't get it" the three key points for Pete are

* The implied blessing of SOAP, WSDL, and other web service standards, by the W3C, thus leading people to believe that SOAP is the one true way to achieve a world wide web of services.
* Concordant with this, the lack of education, standardization efforts, or general evangelism by the W3C to further promote the proper use of REST.
* The tendency of enterprise software vendors and enterprises themselves to continually reinvent well understood enterprise middleware systems.

I'm loving the last one, particularly as Pete is actually asking W3C to champion REST to the exclusion of enterprise focused work. This to me is exactly what I expected to happen when I was writing REST v WS is pointless, all we have here is another camp claiming that their way of shifting documents between servers is the best and thinking that this technical implementation approach will deliver the benefits to the world. Now explicit in Pete's paper is the idea that Enterprise problems are not worthy of consideration by such an august body as the W3C.

Finally, the W3C should divorce itself from all efforts that further the technological needs of the enterprise at the expense of the world.

This is what really irks me about IT, the perception that "the world" really cares about REST v WS. What Pete means is "cool developers" rather than "employed professionals" and the gripe that Pete's chosen approach just isn't getting the focus it deserves and needs the W3C to stop worrying about all those big companies that actually fund the W3C and concentrate on an idealised view of what will help the world.

What will help the world is fixing the enterprise problems, this is where the money is generated and where the most people are employed. The problems in both the world and the enterprise are not going to be fixed by a different way of exchanging XML documents and to claim otherwise is just silly. REST should survive or die based on its support and its proponents should look to themselves if they feel that it isn't getting the visibility that they think it deserves. Personally I think this argument is just a huge waste of resources that could be better focused on understanding interaction models (which neither REST nor WS does) and on simple modelling approaches for difficult problems.

The final thing I'd like to say on this is that the "world" v "enterprise" argument was exactly the same one used by Mark Reinhold to justify JavaSE 6 and the inclusion of JAX-WS into the basic Java release. Claiming you are addressing the "world" rather than the enterprise is just bollocks, the reality is you are just avoiding the difficult questions to get your personal agenda accepted.

Technorati Tags: SOA, REST

Narcissistic IT, same old challenge, same old idiots

One of the wonders of the web is RSS, its one of those things where people used to say "how do you keep up to date" and the answer was I use a large amount of RSS feeds, but unfortunately they don't have the sophistication of Emacs' newsreader. Well today I saw that brilliant old IT challenge of "If I can do it in less characters, then it must be more effective" now there are only two problems with this argument, firstly its bollocks, and secondly its... bollocks. Now I know that strictly speaking this is only one problem, but its such a big one I thought I'd point it out twice (name that TV programme?).

I came across this challenge in two guises, the first "impressed me" that a library call in Ruby could do lots of things and the second went for the traditional Java sucks angle against scripting languages.

Now these would be wonderfully astute calls if they hadn't been made every year since I entered IT. The only problem is that the ability of the people arguing has gone massively down. When I first got into IT it was the LISP people arguing that they could do anything in less characters than anyone else, and they were right. This was because they used a functional programming language and really understood IT. They never ever thought that a scripting language was the best way, and indeed nobody did because a strongly typed and formal language had much better performance and that meant performance in project teams which is what is important after all.

IT suffers from a massive dumbing down, people make arguments based on out right stupidity, a stupidity based on little or no understanding of the principles of computer science. The complexity of IT is ever increasing but the average intelligence is plunging. Things that were plain dumb 10 years ago are now being promoted as "best practice", not because they have become good ideas but because there is a volume (as in barrel) of people who think that their mickey mouse experience is all that anyone ever needs to know.

This is one of the most depressing things about IT. When the LISP boys used to show off at least you could stare in wonder at what they'd done. Having a library that does time conversion isn't exactly the most impressive thing I've seen, its not even the most impressive thing I've see my dogs do today, and the "I can write code really quick in this scripting language" is a great lesson in the "I don't have a clue about TCO" problem that IT has always suffered from. Just for once it would be nice to see people shout out about "I used this and 5 years later people were still able to make changes" or "I wrote this code and the developer sitting next to me understood it".

Maintenance is the hardest part of IT, writing code that survives for 5, 10 or even 20 years should be an achievement, not writing something 20 seconds quicker than the bloke sitting next to you. It would be nice to see people laud languages because of the features they have that make support easier, facilities that make monitoring easier or just pieces that make deployment easier. There is something seriously wrong in IT when every new generation comes along and makes exactly the same mistakes that the muppets of the previous generation made, but worse than that where previously the muppets were small in number they now appear to be pretty much the majority of IT.

This is a major challenge for IT, and in particular the current challenges of SOA and participation based systems. Sure someone can knock up a glorified FTP site with a GUI (YouTube anyone?) but what about doing real, active, participation and creating real business value on existing systems?

As long as IT is dominated by people whose perspective is I can code in this quicker, like really cool dude then it will continue to be perceived as a profession dominated by an obsession of itself rather than an obsession with improving what it does.

Technorati Tags: PHP, Java, Ruby

Java SE 6 - How dumb was that JAX WS decision?

When the Java SE 6 JSR was going on I opposed time after time the inclusion of JAX-WS, I covered some of the reasons why JavaSE 6 wasn't enterprise grade and the obsession of the Sun spec lead with "Joe average" developer. This led to a proposal for JavaSE 7 that would basically put the control back in the hands of professionals and out of the hands of the fan-boys.

Well even before we've made it to the first JavaOne since JavaSE 6 was released there are already two cracking examples as to how dumb a decision the inclusion was. Firstly there is the problem with using JAX-WS 2.1 with JavaSE 6 which leads to amazing classloader abuse. This was raised on the group as an issue but dismissed by Mark Reinhold in a blinkered desire to get WS into the SE platform. This really demonstrates how dumb an idea it was, already people are having to work around the crap decision to get what they want done.

Following hot on the heels of that however is the announcement of the "REST" API for Java, which personally I think is quite a good idea if just because it will formalise some of the discussions and see if an "agreed" definition of a REST API and programming model can be created. This could be a good start for formalising other parts of the REST discussion, of course it could also turn into an API for shifting RSS docs about (ROME anyone?) which wouldn't be much use to man nor beast.

The idea that some developers won't want to use WS and would want to use REST was another objection on the group which was brushed over. It was almost "hell if its popular we'll add that one too", brilliant reasoning yet another choice that is only used occasionally. Last year I proposed a talk at JavaOne with the title "Coping with JavaSE 6" which was rejected, it sounds like this year Sun will have to present it during the keynote.

As a member of the group I've apologised before for the mess it has foisted upon IT. I just hope that Sun have learnt their lesson and that next time their "Chief Engineer" will focus more on smart engineering decisions and less on Slashdot marketing ones.

Technorati Tags: Java

Honest Marketing for SOA? Naah lets go cosmetic

Via Mr Creswell I picked up on a new set of posts around "Honest Marketing" and it made me think again about the best way to market a service. I still think its critical to market from the consumers perspective but quite clearly there are two choices here

1. Market honestly and say just what it does
2. Do Air Canada poor marketing
3. Go Cosmetic on the marketing

Now the later is best exemplified by the people at Clarins with their "Faraday Cage in a bottle" which is really just a water spray, but they couldn't charge £25 ($500) for it if they just said it did that.

Now the point here is that the problem that Air Canada have is that they are marketing something very direct, its real and its obvious, therefore its really hard to make up silly things and have them relevant. With the cosmetics industry however its all pretty much the same set of products so they need to make up for that with impressive marketing claims.

So which one will SOA fall into? We already know that Software vendors, particularly those in the Pacific North West, tend to make exaggerated claims of what their products can achieve, and I think that SOA is going to be the same.

You have a choice of two credit card clearing companies, one says "we clear credit card payments and do some minor fraud detection" the other says "using advanced einsteinium processors we analyse credit card payments for irregular heuristically defined patterns and give you the piece of mind that all credit card payments are processed without being tested on animals". Now if you are smart you'll just pick the first one, but how many people will do that? After all the interface is the same, the ingredients are the same, the result is the same... but one has a marketing budget, probably some pictures, and seems to promise a bit more.

IT is littered with these sorts of claims "the end of programming", "business people directly developing complex systems using a point and click interface", "Perl for all your enterprise needs", "Secure PHP", "Parallel computing made easy", "backed by our expert support team", "its all XML so there is no programming", "J2EE is dead", etc etc. There is really no reason to think that SOA and SaaS will change it.

Now this pessimism means that there is only one logical position to hold, namely that some other bugger will use the Clarins' marketing people, so you'd better get in there first....

SOA not only transforms your business and IT to be more efficient, more effective and more enjoyable, it also makes you feel younger, reduces your risk of heart disease, makes you more attractive and reduces your carbon emissions.

Honest marketing is a good idea in principle, but when has IT demonstrated any of those?

Technorati Tags: SOA, Service Architecture

The careful denial of reality - why muppets hate tools

Sometimes I come across posts that just nicely sum up attitudes in IT that I personally think are responsible for the issues we have in IT today. This is a great example it goes through what the writer in question wants to see and starts off with a cracker.

There is always a community of IT people fascinated by shiny code generators. IDE support for SOAP is just that. There is no more “architecture” built around these code generators than the CASE tools of the past.

Ahh yes the old hatred of "code generators" its a wonderful thing, Ruby on rails sucks for the same reason apparently.

The poster of the first article goes on to talk about "dynamic" programming styles and then trolls on about how "easy" it should be to do independent evolution of complex data centres... if only everyone would listen to their approach.

Genius.

Lets start with code generators suck. First off lets assume you aren't using binary to programme, because assembler gets compiled into binary so there is code generation there. Maybe you are using a language like Java which not only compiles (generates) the code, but worst of all it then actually hosts it in an environment where it does things you didn't ask it like garbage collection, memory allocation, talking to the operating system. The posters biggest complaint isn't about code generation its about static v dynamic interfaces and really its about the poster wanting to hand roll everything rather than actually just do the valuable work.

Code generation is a good thing, a very good thing. It makes tasks repeatable and enables you to automate parts of the cycle. Code generation for editable code has issues, I personally prefer the "magic" approach where you use facades and proxies to hide the generated pieces. The whole goal of IT should be to automate more and more parts of the process, and right now that goal should be focused on automating the communication between different systems.

The "code generation is bad" crowd are plain wrong, it reminds me of a question I asked at JavaOne in 2005 (IIRC), there was a Web Framework "Smackdown" with a JSF chap and a bunch of different OSS frameworks, all claiming theirs was the quickest to code in, while the JSF chap was claiming his was the quickest to deliver with tools.

My question was that given that GUI development should be WYSIWYG, therefore tools, weren't all the other frameworks pointless and shouldn't we just get behind the one that is designed to be tooled. One of the OSS boys flashed back with "you can't build serious GUI applications with tools"... a killing retort he clearly thought. My response was "I used to build Air Traffic Control GUIs using visual builders, and I'm pretty sure those would class as serious GUIs with anybody"... the OSS framework chappy then resorted to a classic five year old child argument of "sez you".

Tooling is a good idea, tooling is what everyone wants. People want to have pieces that are easy to assemble so they can get onto the value as quickly as possible.

There are four ways to get a new cupboard

1. Buy it ready made (package solution)
2. Buy it flat packed (assemble with simple tools)
3. Build it from raw materials (planks, nails, screws) with power tools (hammers, drills, saws) (smart custom build)
4. Build it from a tree (dumb custom build)
   

The goal of technical SOA is to make 2 happen more often, the goal of code generators and things like the JVM is to make 3 simpler. The goal of people who rail against this and what everything handcrafted is number 4.

It really annoys me when people rail against generators as a principle, and yet talk of dynamic languages or using compiled languages. It just shows that they fundamentally don't understand the environment they are working in, and yet they feel this ignorance gives them the ability to hand out advice.

Technorati Tags: SOA, Service Architecture

KISS SOA is rocket science

With the BSB concept and the general Business Service Architecture idea I'm a firm believer both that technology is not important and that the key to all of this is simplicity. Simplicity means simple from the users perspective, not from the perspective of the service or producer.

So I'd like to put forward a few simple rules about keeping things simple.

1. A clear picture of what the business wants
2. An organisation that is set up to deliver what the business wants
3. Technology that is focused on the value, not on the interface
1. Interfaces should be fixed
2. Interfaces should be verifiable
3. Interaction and consumption should be automated and tooled
4. XML is for 5% or less of the organisation, people things in simple traversal or object ideas. Sure marshaling is "suboptimal" but so are people, get used to it.
5. Rules that are dictatorial, flexibility is great at the edges but rigour and a big stick with nails in it binds everything together.
6. Assume people are average or worse, sure you get it, but would 80% of the organisation?
7. Don't fight the vendors, you might be smart, they are definitely rich, cope with the fact and use it.
8. Pick the most basic pieces that will be true next year, forget the big ideas and deal with reality
9. Think tooling, if it can't be tooled then its for the minority, good for you, crap for your career.
10. One size doesn't fit all. Understand where the rules apply, and when you are allowed to break them. 5% of people can break the rules, 95% don't need to know its possible.
    

I've seen too many EAI and SOA projects get into difficulties because clever people tried to be clever and average or worse people were asked to deliver it. For SOA to succeed it must succeed for the majority.

Or as Blaise Pascal said "I have made this letter longer than usual because I lack the time to make it shorter.". Make the time, get it simple, get it concise, that requires a much greater degree of talent than demonstrating how clever you are. KISS SOA is much harder than technology SOA, and its all the more valuable as a result.

Technorati Tags: SOA, Service Architecture

Will IT become a manufacturing industry?

In a few discussions recently I've been defending the idea that IT development will continue and that the business will continue to demand more complex functionality out of IT. Some have argued that the goal will be to create value out of the existing estate rather than new build being important.

I actually think that both sides are right and that this has some real implications on the skills that will be required in future in IT and the value that is placed upon them. There have been a bunch of studies recently that say the next few years will see demands on IT shift from being cost focused towards business value. What this means is that those in cost centric IT are going to find themselves more and more squeezed while those in value centric IT are going to find themselves in greater demand.

One of the bits around WS being a career choice is that as the exposure of services becomes more and more a commodity item the importance will be to understand what you do with those things. Understanding of the WS stack will be extremely important because it is exactly those things that you will have to understand what to do with. Sure the internet is cool and sexy, but the assets that you have are back inside the company, and this is where WS is rapidly becoming the only choice. The job of those in value centric IT will be to understand what to do with all these WS endpoints and of course how to ensure that what is exposed matches against the business service architecture.

Effectively the split between the heavy manufacturing part of IT, exposing services, driving cost out of non critical applications, the adoption of SaaS and BPO within an organisation, etc will be at that technical interface. Decisions will need to be made about which pieces are outsourced and which pieces are retained, and quite clearly the commoditisation and standardisation of those interfaces is going to be critical. For SaaS and BPO to really flourish then questions of security and data privacy are going to have to be solved in a standard way, it is no good having multiple different approaches as that will drive up the costs.

What this means is that the skills around assembly, composition and working with the business are going to be in greater demand. The skills around creating stand alone applications are going to be worthless and skills around creating interfaces on existing IT are going to be commodity.

To create this industry however and start creating production lines for IT means that there has to be a commoditisation on that interface between systems, and the critical interface for the next 5 years is going to be not the one externally (which is important, but new) but the interface on those existing applications. This means 100% that it needs to be WS-* based, it needs to be tooled and the interfaces need to including the QoS elements of security, reliability and more.

Once we have those applications wrapped and exposed in a standard way we can really start building on top of previous generations rather than just layering another load of cement over the dead bodies that the last project left behind. A major goal in SOA is to create a "use" culture within an organisation.

So a single standard way of describing a fixed interface and all of its SLAs is going to be critical. This mechanism needs to be tooled and it needs to be governed by policies and not by processes or bespoke implementation. There needs to be an acceptable definition of service and that definition needs to be an accepted commodity. The 802.11b of IT.

And here in lies the challenge. If IT "professionals" continue to look towards some perceived optimum solution and not to actually create a commodity in the ability of new applications to consume old ones then yet more time will go by with IT not progressing IT estates and just progressing technologies.

Being a manufacturing industry would be a good thing for IT, it would help us use ever more complex business services to deliver ever more demanding business applications. It would enable people to focus on delivering business value and not on their ability with a specific or latest fad technology. There is a great fear in certain parts of IT that if we commoditise the interface that their technical smarts will not be required, their smarts in effect will be replaced by robots. Personally I welcome that commoditisation because the people who I've worked with who are truly tech smart are also the people who are always focused on the end goal, and that end goal is never about technology.

Are we brave enough to make that choice? Or shall we just have another pointless debate about one method of shifting XML documents around over another?

Technorati Tags: SOA, Service Architecture

Vista and Firing 2.0

Many moons ago I did a presentation on the futures of user interfaces where I talk about why true immersive 3D environments (not like LookingGlass but true 3D) were unlikely to be useful for standard corporate users and why voice as a command mechanism is a bad idea. I had a slide back then with a cartoon on it, that unfortunately I cannot now find.

Basically the scenario is this, you fire someone, they are upset. All of your systems can be operated via voice commands... so what happens...


Yup the person runs down the corridor formatting all the hard drives that they can. Now back then I worked in a unix shop so the command was "rm -rf /" but I think format c: works just as well. My point was that voice commands are quite inefficient for lots of users and are also subject to office noise interference, the example was just a silly way to show what I meant when I talked about people wandering up to your desk and asking "did I send the email to you?" and the computer sending the email you hadn't finished.

Now it seems that Windows Vista might have made the silly a reality.

Welcome to Firing 2.0, like Firing 1.0 but with sound effects.

(with apologies to the excellent XKCD)

Technorati Tags: Vista, Voice