The promise and pitfalls of implementing a Services Oriented Architecture

This is a talk I gave on the afternoon of the 15th of March 2005, the first day of Ark Group’s “Planning and Implementing Service-Oriented Architecture” conference at the Sydney Marriott. I was Head of Architecture, Distribution at NAB at the time. It is reproduced from the PDF the organisers made from my PowerPoint five days before the conference, and that makes it necessarily brief: a deck is scaffolding for a person standing in front of it, and the argument lived in what was said between the slides. The PowerPoint itself has since surfaced in my backups and is archived below beside the PDF; all 101 lines of its text appear here word for word, and neither file holds any speaker notes — the deck never had them, so nothing fuller than this exists to reproduce.

The conference programme, printed on the 6th of January, bills me as Head of Architecture, Business Technology at MLC. That is out of date rather than wrong. MLC was where I had been until 2004, and by the time I actually spoke I had moved. The deck is the authority for how I was introduced. The organisers also dropped the h from my surname in the filename, which is why the file reads “Chris Tam”.

The deck is text throughout, so nothing here is a figure. Two things about the layout do not survive a plain reading of the file and have been put back. Slide 7 carries a highlighted banner reading “Is SOA in danger of being perceived as a Silver Bullet?” which sits directly under the title, though it is drawn last and so extracts last; it is set here where it appears on the slide, not where the file happens to store it. Slide 8 is two columns, “Armageddon” on the left and “Nirvana” on the right, and is set as a table so that the pairing survives. The bullets throughout were a small arrow in Wingdings, which is a typeface choice rather than content, so they are ordinary list bullets here. One repair to this page’s own history: the five phrases in slide 4’s definition that the deck sets in italics — its only character emphasis anywhere — were missed when this page was first transcribed from the PDF, and are restored from the PowerPoint.

The slides’ own slips stand as they were shown. Slide 8 has “Mutiple versions of each service”, slide 7 opens a quotation before “acceptance” and never closes it, and the same slide has “neither have them replaced older bullets” and “prophesized”. Nobody proofreads a deck, and these are what the room saw. The PowerPoint has every one of them too: they are mine, not the conversion’s.

The month after, Ark Group sent around its delegate-feedback report, which marked every session out of seven. This talk scored 5.2 for presentation and 5.5 for content — the fourth-highest content mark of the sixteen presentations scored — and of the five comments recorded under my name the one I have chosen to believe is “Good reality check. Cut through all the hype and told it like it was”. All five are in the Sources, and the report is archived below.

All ten slides are reproduced, at the deck’s own numbering. Five months later this deck grew into a case study of how MLC actually went about it, and sixteen months later I gave a longer and much more corporate talk on the same subject; this one is the version I got to be rude in.

Download the presentation (PDF, 129 KB)

Download the PowerPoint (PPT, 95 KB)

Download the delegate feedback report (PDF, 27 KB)

1 Title

The promise and pitfalls of implementing a Services Oriented Architecture

Presentation to Planning and Implementing Service-Oriented Architecture conference, Sydney, 15-17 March 2005

Chris Tham
Head of Architecture, Distribution
National Australia Bank

2 Agenda

• What makes SOA so attractive to an enterprise?
• The Myth of the “Silver Bullet” – Will SOA succumb?
• What will it take to REALLY implement SOA properly?
• Some potential problems from implementing SOA, and some ideas for avoiding them

3 Just what is “SOA” anyway?

a) “The best thing since sliced bread” – guaranteed to cure cancer, solve world hunger, and retire Third World debt

b) Something to do with SOAP and Web Services

c) What my vendor says will be supported in their “Next Generation” product

d) Yet another meaningless Three Letter Acronym, to be replaced by another fad in 3 years

e) It’s the same thing as (OOP, RPC, CORBA, CICS, …)

f) All of the above?

4 One definition (what my team came up with)

• A Service Oriented Architecture is an application architecture within which key business functions are implemented as re-useable services with well-defined, invocable, interfaces, which can be called in a defined sequence to form business processes.
• Note: the focus is on business processes, not object, functions, data, message … this differentiates SOA from previous paradigms such as object orientation

5 Why should we care? What makes SOA attractive to an Enterprise?

• Easy for the business to understand
  • The notion of structuring functionality as a set of “services” ties in very well with business process re-engineering, and product/services offerings, supply chain, etc.
• Relatively easy to implement
  • Doesn’t require complete rewrite of all systems, tools exist to “expose” functionality from legacy applications as services
• Broad (universal?) industry and cross-platform support
  • Who would have thought the likes of SAP, Oracle, Siebel, IBM, Microsoft would one day sing the same tune?
• Lowers (but does not eliminate) the barriers to heterogeneous interoperability
  • It’s (relatively) easy for a .NET client to invoke a J2EE web service, across multiple development and infrastructure environments
• Doesn’t require all existing applications to be thrown away or rewritten
  • There are techniques to expose functionality from legacy systems as services

6 The challenges of implementing an SOA

• “The difference between a vision and a fantasy lies in the execution”
• It doesn’t matter how “neat” or “clever” or “righteous” the concept is if no one understands or is willing to change
• At the end of the day, it’s about the people, not the technology
• Are you able to get the business to understand and support the concept?
• Can you get senior management buy in and commitment?
• Are you willing to spend the time to take everyone else along the journey?
• How do you address scepticism, the “not invented here” syndrome, and those who don’t see any need to change?
• Is it about revolution, or evolution?

7 The Myth of the “Silver Bullet”

[!NOTE] Is SOA in danger of being perceived as a Silver Bullet?

• A “Silver Bullet” is a concept/idea that promises to revolutionize IT (and cure cancer, … etc.)
• The IT industry is littered with the carcasses of “Silver Bullets” – none of these have become “pervasive” in the way their advocators originally prophesized, neither have them replaced older bullets:
  • Computer language: COBOL, FORTRAN, APL, Ada, Pascal, C, C++
  • Relational data modelling
  • Data warehousing
  • Object orientation
  • Message queuing, object brokers
• “Silver bullets” tend to fail because:
  • Hype and unrealistic expectations about potential impact and benefits
  • Utopian ideals are seldom realisable
  • Many silver bullets are based on an “all or nothing” philosophy – the old way of doing things is invalid, and all systems need to be re-engineered
  • Lack of true “buy in” and “acceptance, lack of change management
• Failures of execution are followed by a tendency to blame the concept (“I never thought it was a good idea in the first place”) and as a result the Silver Bullet is tarnished
• Everyone then moves on to the next Silver Bullet
• As a result, there is deep scepticism at all levels whenever a new Silver Bullet is proposed. This can doom the concept even before it is implemented.

8 Two Possible Scenarios at opposite ends of the spectrum

“Armageddon”	“Nirvana”
There are hundreds, perhaps millions of web services in your organisation – nobody really knows.	The whole enterprise is implemented on a Services Oriented Architecture.
Mutiple versions of each service, but don’t know which ones are used, and by what.	All key enterprise business processes are modelled using BPEL.
Spaghetti Junction: No one understands the complexities of the relationship between systems.	These are implemented using well defined services, with agreed and known usage contracts/service levels.
No one understands the services that support key business processes.	Services are based on consistent XML definitions based on a common object model.
If a business process breaks, nobody even knows which application to fix.	Services are closely integrated with human workflow activities.
Everyone is afraid to change anything because it might break something else.	Business processes and services can be “automagically” and dynamically deployed and provisioned on virtualized hardware.
You are stuck on old versions of packaged applications because you are relying on outdated web services no longer supported by the vendor.	Full round trip engineering between service modelling and service implementation.
	All services are known and well defined.

9 How to avoid Armageddon and get closer to Nirvana

• Evangelise
  • Make sure the business and senior IT management understand SOA, and get buy in and commitment.
  • Even more importantly, make sure everyone else does!
• Governance
  • Think long and hard about balancing between over-policing and anarchy.
• End to end linkage
  • Work out how to link business process design to service definitions to implementation to deployment to operations.
• Understand what your key vendors are doing
  • What is their positioning on SOA? When and how are they going to implement it? Do they have any ideas/commitment on how to maintain/evolve service definitions across major releases?
• Enterprise Architecture suddenly becomes very real, and important
  • Who will maintain the models and linkages between them? Who maintains a catalogue/repository of services?
• Track evolution of standards
  • Don’t get left behind, but at the same time don’t be on the bleeding edge.
• Build a competency centre
  • Who maintains the standards? Who is responsible for providing guidance and mentoring to development teams? Who is responsible for the end to end view of key business processes?

10 Think of SOA as a “religion” (both in a good and bad sense)

• It requires enough people believe in it and have faith in it to make a difference.
• You need:
  • Messiahs – people who can evangelise the New Way and show the way to the Promised Land
  • Scholars/Monks – subject matter experts who can document good practices/knowledge and develop best practices
  • Priests – those who can lead others on the path to salvation …
  • Parishioners – a large body of people who believe and is willing to change their behaviour
  • A Church for the faithful
  • “Hell” and “purgatory” for sinners?
• A “jihad” or “crusade” may not be the best way to convert others, although in certain instances it may make a difference …

Sources

• The deck, 09 Chris Tam Presentation.PDF, 129 KB, ten slides laid out two to a page as a handout. Its PDF metadata names the source file “Tham”, gives Microsoft PowerPoint as the creator and Acrobat PDFWriter 4.05 as the producer, and records the conversion as having been done by the conference organisers on the 10th of March 2005, five days before I spoke.
• The PowerPoint, The promise and pitfalls of implementing SOA.ppt, 95 KB, the file the organisers rendered that PDF from, archived beside it. Its OLE metadata records creation on the 9th of March 2005, eight revisions and nearly six hours of editing ending in a last save on the 10th — the day the organisers dated their conversion — and 1,153 words on the NAG corporate template. Ten slides of text: no pictures, no embedded objects, no speaker notes, and one run of character emphasis in the whole deck, the five italic phrases in slide 4’s definition.
• The conference programme, 00 SOAFinal.pdf, produced on the 6th of January 2005. It sets out the two days and the two post-conference workshops, prices the conference at $2,495 plus GST, and lists my session for 3.15pm on day one under the same four questions the agenda slide uses. Several speakers it advertises did not in the end appear, and several who did are not in it, which the booking conditions allow for in as many words: “All speakers are correct at the time of printing, but are subject to variation without notice.”
• The delegate feedback report, Implementing SOA - Speaker Feedback.pdf, 27 KB, three pages, made by the organisers on the 4th of April 2005 from a Word original, with the same Acrobat PDFWriter 4.05 that made the deck PDF. It marks every session of the two days and both post-conference workshops out of seven for presentation and content — even the chairperson is marked, 5.9 for timekeeping — and records the delegates’ comments verbatim, typos and all. Mine, in full: “Knowledgeable, interesting”; “To much time spent on theory, wanted more on NAB experiences”; “Interesting talk. Down to earth”; “Good reality check. Cut through all the hype and told it like it was”; and “Very impressive lady”.
• The Microsoft Architect Council cut, this same deck’s second outing that August: twenty-eight further seconds of editing, the event line changed, and the religion slide dropped.
• Services Oriented Architecture in a Retail Bank, the talk I gave on the same subject at BankTech.06 in July 2006.