Thursday, July 28, 2011

Oh Dear, Here We Go Again!

So, here we go again. The BBC today report that "IT giants are 'ripping off' Whitehall, say MPs". As I presumably work for one of those "IT giants" I will attempt to comment on this in as impartial a way as is possible.
  • As long as we have 'IT projects' rather than 'business improvement' or 'business change' projects in government, or anywhere else come to that, we (and it is 'we' as tax payers) will continue to get 'ripped off'. Buying IT because it is 'sexy'  is always going to end in tears. IT is a tool that may or may not fix a business problem. Unless you understand the true nature of that business problem throwing IT at it is doomed to failure. This is what software architects need to focus on. I'm coming to the conclusion that the best architects are actually technophobes rather than technophiles.
  • It's not Whitehall that is being 'ripped off' here. It's you and me as tax payers (assuming you live in the UK and pay taxes to the UK government of course). Whether you work in IT or anywhere else this effects you.
  • It's not only understanding the requirements that is important, it's also challenging those requirements as well as the business case that led to them in the first place. I suspect that many, many projects have been dreamt up as someones fantasy, nice to have system rather than having any real business value.
  • Governments should be no different from anyone else when it comes to buying IT. If I'm in the market for a new laptop I usually spend a little time reading up on what other buyers think and generally make sure I'm not about to buy something that's not fit for purpose. One of the criticisms leveled at government in this report is the "lack of IT skills in government and over-reliance on contracting out". In other words there are not enough experienced architects who work in government that can challenge some of the assumptions and proposed solutions that come from vendors.
  • Both vendors and government departments need to learn how to make agile work on large projects. We have enough experience now to know that multi-year, multi-person, multi-million pound projects that aim to deliver 'big-bang' fashion just do not work. Bringing a more agile approach to the table, delivering a little but more often so users can verify and feedback on what they are getting for their money is surely the way to go. This approach depends on more trust between client and supplier as well as better and more continuous engagement throughout the project's life.  

Wednesday, July 27, 2011

The Innovation Conundrum and Why Architecture Matters

A number of items in the financial and business news last week set me thinking about why architecture matters to innovation. Both IBM and Apple announced their second quarter results. IBM's revenue for Q2 2011 was $26.7B, up 12% on the same quarter last year and Apples revenue for the same quarter was  $24.67B, an incredible 83% jump on the same quarter last year. As I'm sure everyone now knows IBM is 100 years old this year whereas Apple is a mere 35 years old. It looks like both Apple and IBM will become $100B companies this year if all goes to plan (IBM having missed joining the $100B club by a mere $0.1B in 2010).

Coincidentally a Forbes article also caught my eye. Forbes listed the top 100 innovative companies. Top of the list was, Apple were number 5 and IBM were, er, not in the top 100! So what's going on here? How can a company that pretty much invented the mainframe and personal computer, helped put a man on the moon, invented the scanning electron microscope and scratched the letters IBM onto a nickel crystal one atom at a time, and, most recently, took artificial intelligence a giant leap forward with Watson not be classed as innovative?

Perhaps the clue is in what the measure of innovation is. The Forbes article measures innovation by an "innovation premium" which it defines as:

"A measure of how much investors have bid up the stock price of a company above the value of its existing business based on expectations of future innovative results (new products, services and markets)".

So it would appear that, going by this definition of innovation, investors don't think IBM is expected to be bringing any innovative products or services to market whereas the world will no doubt be inundated with all sorts of shiny iThingys over the course of the next year or so. But is that really all there is to being innovative? I would venture not.

The final article that caught my eye was about Apples cash reserves. Depending on which source you read this is around $60B and as anyone who has any cash to invest knows, sitting on it is not the best way of getting good returns! Companies generally have a few options with what to do when they amass so much cash, pay out higher dividends to shareholders, buy back their own shares, invest more in R&D or go on a buying spree and buy some companies that fill holes in their portfolio. Whilst this is a good way of quickly entering into markets companies may not be active in it tends to backfire on the innovation premium as mergers and acquisitions (M&A) are not, at least initially, seen as bringing anything new to market. M&A's has been IBM's approach over the last decade or so. As well as the big software brands like Lotus, Rational and Tivoli IBM has more recently bought lots of smaller software companies such as Cast Iron Systems, SPSS Statistics and Netezza.

A potential problem with this approach is that people don't want to buy a "bag of bits" and have to assemble their own solutions Lego style. What they want are business solutions that address the very real and complex (wicked, even) problems they face today. This is where the software architect comes into his or her own. The role of the software architect is to "take existing components and assemble them in interesting and important ways". To that I would add innovative ways as well. Companies no longer want the same old solutions (ERP system, contact management system etc) but new and innovative systems that solve their business problems. This is why we have one of the more interesting jobs there is out there today!

Thursday, July 21, 2011

A Service Based Development Process - Part 4

The first three of these blog posts (here, here and here) have looked at the process behind developing business processes and services that could be deployed into an appropriate environment, including a cloud (private, public or hybrid). In this final post I'll take a look at how to make this 'real' by describing an architecture that could be used for developing and deploying services, together with some software products for realising that architecture.

The diagram below shows both the development time and also run-time logical level architecture of a system that could be used for both developing and deploying business processes and services. This has been created using the sketching capability of Rational Software Architect.

Here's a brief summary of what each of the logical components in this architecture sketch do (i.e. their responsibilities):
  • SDLC Repository - The description of the SDLC goes here. That is the work breakdown structure, a description of all the phases, activities and tasks as well as the work products to be created by each task and also the roles used to create them. This would be created and modified by the actor Method Author using a SDLC Developer tool. The repository would typically include guidance (examples, templates, guidelines etc) that show how the SDLC is to be used and how to create work products.
  • SDLC Developer - The tool used by the Method Author to compose new or modify existing processes. This tool published the SDLC into the SDLC Repository.
  • Development Artefacts Repository - This is where the work products that are created on an actual project (i.e. 'instances' of the work products described in the SDLC) get placed.
  • Business Process Developer - The tool used to create and modify business processes.
  • IT Service Developer - The tool used to create and modify services.
  • Development Repository - This is where 'code' level artefacts get stored during development. This could be a subset of the Development Artefacts Repository.
  • Runtime Services Repository -Services get published hereonce they have been certified and can be released for general use.
  • Process Engine - Executes the business process.
  • Enterprise Service Bus - Runs the services and provides adapters to external or legacy systems.
Having described the logical components the next step is to show how these can be realised using one or more vendors products. No surprise that I am going to show how these map to products from IBM's portfolio however clearly your own particular requirements (including whose on your preferred vendor list of course) may dictate that you choose other vendors products. Nearly all the IBM product links allow you to download trial versions that you can use to try out this approach.
  • Rational Method Composer - This enables you to manage, author, evolve, measure and deploy effective processes (SDLCs) tailored to your project needs. It is based on Eclipse. Rational Method Composer allows publishing to a web site so effectively covers the needs of both the SDLC Repository and SDLC Developer components.
  • IBM Business Process Manager - This is the latest name for IBM's combined development and runtime business process server. As well as a business process runtime, ESB and BPM repository it also includes design tools for building processes and services.  The Process Designer allows business authors to build fully executable BPMN processes that include user interfaces for human interaction. The Integration Designer enables IT developers to develop services that easily plug into processes to provide integration and routing logic, data transformation and straight-through BPEL subprocesses. See this whitepaper for more information or click here for the IBM edition of the book BPM for Dummies. IBM Business Process Manager realises the components: Business Process Developer, IT Service Developer, Development Repository, Process Engine and Enterprise Service Bus.
  • WebSphere Service Registry and Repository - Catalogs, and organizes assets and services allowing customers to get a handle on what assets they have, making it easy to locate or distribute. Also enables policy management across the SOA lifecycle, spanning various domains of policies including runtime policies as well as service governance policies. Included in the Advanced Lifecycle Edition is
    Rational Asset Manager which provides life cycle management capabilities to manage asset workflow from concept, development, build, deployment, and retirement as well as Build Forge integration. WebSphere Service Registry and Repository realises the Development Artfefacts Repository as well as the Runtime Services Repository.
So, there it is. An approach for developing services as well as an initial architecture allowing for the development and deployment of both business processes and services together with some actual products to get you started. Please feel free to comment here or in any of my links if you have anything you'd like to say.

Tuesday, July 12, 2011

A Service Based Development Process - Part 3

In Part 2 I discussed how activities in a delivery process could be created from capability patterns that are comprised of tasks and/or activities. Capability patterns are reusable patterns which can be applied across a range of delivery processes. Capability patterns have input/output work products which together define the ‘contractual boundary’ to which performers of the capability pattern must conform. In this post I'll take a look at what these work products are and how they form part of a contract between capability patterns.

The table below shows the set of work products that are used in the process I have been describing.

Work Product Description Notation
Service Catalogue Catalogue of all services owned or used by the enterprise. May be technical or business, legacy or new/to be provided. Usually implemented as a service registry. WSDL
Service Certificate Identifies a service that meets certain quality criteria. Text
Service Portfolio Plan Describes what services will be provided, when and by whom. Text
Service Portfolio Describes the entire collection of business services that an enterprise uses or plans to use. Text
Service Model Defines the complete hierarchy of business and technical services used or planned to be used by the enterprise. Can exist at a logical and physical level. Contains one or more Service Specifications. UML
Service Specification Defines a detailed specification for each service. Can exist at a logical and physical level. Text/WSDL
Deployed Service A service deployed in the service platform. Jave, MQ etc
Deployed Service Environment A deployed service platform. N/A

This diagram shows the relationships (finish to finish dependencies) between the above work products.

Work Product Dependency Diagram
In this next diagram we can see how  what I'm referring to as 'contractual boundaries' can be realised using these work products which are passed between two of the capability patterns I discussed last time.

Contractual Boundaries

Activities produce one or more work products which are consumed by other activities, either in the same capability pattern, or in a different one. This means that provided the input and output work products are agreed the capability patterns can run in parallel or, if needed, at different times. So, one capability pattern, the 'provide capability' one in the above example can contribute to the Service Model work product which another capability pattern, the 'manage service portfolio' one can contribute to at a later time if need be. Having a common set of agreed work products, which are shared in a repository, becomes the key to having an effective delivery process.

Thursday, July 7, 2011

A Service Based Development Process - Part 2

The basic Service Based Development Process I described previously described an end-to-end delivery process for creating services, whether these be cloud-based services or traditional ones running on a an enterprise service bus. The process consisted of a number of activities (where an activity groups together several tasks) as follows:
  • Business Modelling - Develop business process models to understand the business activities that need to take place. Probably a mix of as-is and to-be as well as manual and automated activities.
  • Solution Architecture and Design - Create the architecture for the solution, decides what services are to be used, what platform and delivery environment (public/private cloud, ESB etc) will be used.
  • Solution Assembly/Implementation - Assemble the services into an application and implement using the appropriate technology. Hopefully much of this will be done using tools that generate the appropriate process flows and pull in the right services.
  • Service Identification - Decide what new services need to be specified (i.e. not already in the Service Portfolio).
  • Service Specification - Specify services, their contracts (functional and service levels).
  • Existing Asset Analysis - What assets does the enterprise already have that could be service enabled (legacy code that could be wrappered etc).
  • Service Realisation - Decide what technology will be used to implement services.
  • Service Implementation - Implement services using the chosen technology.
  • Service Platform Design and Installation - Having decided the service runtime environment design and install it. For cloud-based services this means procuring the right cloud resources.
  • Service Operations and Management - Manage and run the service.
  • Service Portfolio Management - Ensure the Service Portfolio kept up to date.
In the example process we have seen so far each of these activities is composed together in such a way that a complete solution can be built out of a set of services that are deployed into a run-time environment. But that is not the only way these activities can be composed. A better way is to compose activities into capability patterns. Capability patterns are comprised of tasks and/or activities that can be composed into ‘higher-order’ capability patterns or delivery processes. They are reusable patterns which can applied across a range of delivery processes. Capability patterns have input/output work products which together define the ‘contractual boundary’ to which performers of the capability pattern must conform (more on this next time).

The delivery process described in part 1 is actually comprised of four capability patterns, each of which could be a process in its own right, or composed into different delivery processes. The four capability patterns are:
  • Provide Capability: Create a new or updated business process/capability based on user requirements. Uses existing services or specifies new ones where needed. Uses tasks mainly from the Consume discipline group.
  • Provide Service: Specify, design and build a new service according to business need. Certify, publish and deploy the service into the operational environment. Uses tasks mainly from the Provide discipline group.
  • Manage Service Portfolio: Create and maintain a service portfolio with an initial set of specified services. Ensure services are certified. Uses tasks from the Manage group.
  • Provide Environment: Design, build, test, install and run a new environment capable of running services. Uses tasks mainly from the Enable group.
Using the same diagrams as before here are the above four capability patterns laid out in terms of disciplines and phases from OpenUP.
Provide Capability
Provide Service
Manage Service Portfolio
Provide Environment
In the third part I'll look at the work products which together define the ‘contractual boundary’ to which performers of each of the capability patterns must conform.

The ideas in this blog were jointly developed with my ex-IBM colleague Bruce Anderson.