Mapping Applications to the Cloud

by Darryl Chantry

Summary: As economic pressure builds globally, many organizations are starting to look at Cloud Computing as a potential choice to reduce the total cost of ownership for IT. In searching for ways to use Cloud Computing technologies, enterprises have to ask what applications make good candidates for moving to the Cloud and which do not, such as "Does the nature of the business itself allow for Cloud Computing to even be considered?"
This article provides a broad overview into what Cloud Computing is and discusses an approach to mapping enterprise applications to Cloud Computing platforms to assist in determining whether your applications or your business model are a good fit for the Cloud.

 

 

Which Came First: The Cloud or Cloud Computing?

Cloud computing has fired the imaginations of many informationtechnology (IT) professionals around the world, whether they are small independent software vendors (ISVs), Silicon Valley startups, or large corporations that are looking to cut costs. There seems to be an everincreasing number of people who look to the Cloud to hit upon the magic bullet that will solve any IT problem.

One interesting aspect of the hype that surrounds cloud computing is the lack of a clear definition as to what cloud computing is and, just as relevant, what it is not. If you were to ask 100 people to define the Cloud and what they believe cloud computing is, you would probably get 150 different answers (some people tend to answer twice, with the first answer contradicting the second). With this in mind, it seems only fitting to begin this article by discussing a general definition for cloud computing.

The Cloud (or the Internet, if you prefer) has been around for some time now, about 25 years; so without a doubt, the Cloud came first, right? Well, one could argue that the first servers on the Internet were really storage devices for data and applications to be shared and run globally; or to put it another way, to provide cloud computing resources in multiple locations globally with almost infinite scalability. Contrast that to today’s cloud-computing initiatives that are pretty much there to provide data, applications, and computing power globally with almost infinite scalability, and you quickly see the difference… Or is there really a difference?

The difference is that we are using new technologies to put a new spin on old ideas. Cloud computing is more about evolution than revolution, with technology allowing price points that take these thoughts and make them available to all people — regardless of budget size — via a utilitybased, pay-for-what-you-use model.

Utility Computing

Utility computing refers to using computing resources (infrastructure, storage, core services) in the same way you would use electricity or water; that is, as a metered service in which you only pay for what you use. The utility can eliminate the need to purchase, run, and maintain hardware, server, and application platforms, and to develop core services — for example, billing or security services. Consider the following scenario.

A Web-based ISV that wants to makes components available for Facebook or MySpace faces the following dilemma: The components they create could be adopted by thousands, or could struggle to find acceptance in any form. Most ISVs have limited capital, so they need to balance the expenditure between developing their application and providing infrastructure to support their software.

Such balancing acts can lead to poor applications with good platform support, or great applications that are rarely accessible due to poor platform support. Neither scenario is a path to success; this is where utilitybased cloud platforms can help. Cloud utility platforms can provide a low-cost alternative that can easily scale to meet the demand for the ISV’s application, which allows them to commit practically all of their resources toward building a great application.

As cloud services are essentially available as a utility offering, should the product fail, the ISV can simply shut down the services and stop all costs associated with the software.

The utility model also allows organizations to offset some of the costs of running private data centers by providing additional infrastructure resources to manage peak loads; this is also known as cloud bursting .

Traditionally, to handle peak loading, organizations would often design data centers that had the processing power to manage peak loads; this meant that for the majority of the time the data center was underutilized. By using cloud bursting, an organization can build a data center to the specifications that will allow the entity to run all normal day-to-day workloads within their environment, and then use cloud providers to provide additional resources to manage peak loads.

Utility computing is often associated to some form of virtualized platform that allows for an almost infinite amount of storage and/ or computing power to be made available to the platforms users through larger data centers. The evolution of cloud computing is now broadening the definition of utility computing to include service beyond those of pure infrastructure.

 

 

Will All Applications Move to the Cloud?

Will all applications run in the Cloud? Should you attempt to port all of your existing applications to the Cloud? Should all your new applications be developed in the Cloud? What is this Cloud thing, anyway? These are a few of the questions that arise whenever you start thinking about using cloud services.

Some applications will be ideal candidates to be ported to a cloud platform, developed on a cloud platform, or hosted on a cloud infrastructure, while other applications will be poor cloud candidates. In this case, the standard architectural answer “it depends” can be applied to all of the preceding questions. Practically every application potentially could exist either partially or fully in the Cloud; the only caveat to this are the trade-offs in an application’s attributes — and, possibly, functionality — that you might be willing to make to move it to the Cloud.

The following pages discuss a few ideas for decomposing an application into its basic attributes, and decomposing the Cloud into its basic attributes, to help make decisions as to whether running your specific application in the Cloud is practical.

Mapping an Application to the Cloud

Every application is designed to fulfill some purpose, whether an order management system, flight reservation system, CRM application, or something else. To implement the function of the application, certain attributes need to be present: For example, with an order management system, transaction and locking support might be critical to the application. This would mean that cloud storage might not be suitable as a data store for such a purpose. Determining the key attributes of any application or subsystem of a larger application is a key step in determining whether or not an application is suitable for the Cloud.

 

Figure 1. Attributes map of an application

 

Figure 1 shows a number of key high-level attributes (blue column) that could be relevant to any application. The potential number of attributes for any given application does not need to be recorded; what you are attempting to determine is which attributes are the critical ones for your application. This will likely produce a manageable list of attributes that can then be mapped to the Cloud. Selecting Data Management, for example, presents a list of secondary attributes that provide more details for the high-level attributes. Selecting Access then allows you to specify if you want either online, offline, or both online and offline access to your data source.

Building on the Data Access example, you can start to see how this attribute could affect the choice as to whether or not to use a cloud provider for data storage. Should the application in question need purely online data, cloud storage would be an excellent choice; however, if offline data is all that is required, this could be a key indicator that the application is not suited for the Cloud. And if you decide that the application requires both an online and an offline model, the cost of developing the application to synchronize data between the application and the Cloud would need to be considered.

Choosing to support both an offline and online experience for the end users will add additional cost to the project; however, should another attribute, such as high scalability, be identified, the advantages that the Cloud provides in this area easily could offset the cost of developing an offline experience. (See Appendix A, Sample Application-Mapping Attributes, page 7.)

What Makes Up the Cloud?

After you have decomposed an application and determined its key attributes, you can begin work on a similar exercise for the Cloud — specifically, for cloud service providers. Splitting cloud attributes into broad categories can simplify the mapping process. The categories used in this example are cloud infrastructure, cloud storage, cloud platform, cloud applications, and core cloud services.

You could map any application attributes to cloud attributes in one or more categories, as depicted in Figure 2.

 

Figure 2. Mapping application attributes to cloud attributes

 

Cloud Infrastructure

Cloud infrastructure is infrastructure, or more commonly, virtual servers in the Cloud. Infrastructure offerings are the horsepower behind large-scale processes or applications. For large-scale applications, think Facebook or MySpace; for large-scale processing, think a high-performance infrastructure cluster that is running engineering stress-test simulations for aircraft or automobile manufacturing.

The primary vehicle for cloud infrastructure is virtualization; more specifically, running virtual servers in large data centers, thereby removing the need to buy and maintain expensive hardware, and taking advantages of economies of scale by sharing Infrastructure resources. Virtualization platforms are typically either full virtualization or para-virtualization environments. (See Appendix B for a more detailed explanation of virtualization, page 8.)

Cloud Storage

Cloud storage refers to any type of data storage that resides in the Cloud, including: services that provide database-like functionality; unstructured data services (file storage of digital media, for example); data synchronization services; or Network Attached Storage (NAS) services. Data services are often consumed in a pay-as-you-go model or, in this case, a payper- GB model (including both stored and transferred data).

Cloud storage offers a number of benefits, such as the ability to store and retrieve large amounts of data in any location at any time. Data storage services are fast, inexpensive, and almost infinitely scalable; however, reliability can be an issue, as even the best services do sometimes fail. Transaction support is also an issue with cloud-based storage systems, a significant problem that needs to be addressed for storage services to be widely used in the enterprise.

Cloud Platform

A cloud platform is really the ability to build, test, deploy, run, and manage applications in the Cloud. Cloud platforms offer alternatives to these actions; for example, the build experience might be online only, offline only, or a combination of the two, while tools for testing applications might be nonexistent on some platforms, yet superb on others.

Cloud platforms as a general rule are low-cost, highly scalable hosting/development environments for Web-based applications and services. It is feasible (although an oversimplification) to consider cloud platforms as an advanced form of Web hosting, with more scalability and availability than the average Web host. There are pros and cons for any technology, and a con in the cloud platform world is portability. As soon as an application is developed to run on a specific platform, moving it to another cloud platform or back to a traditional hosting environment is not really an option.

Cloud Applications

A cloud application exists either partially or fully within the Cloud, and uses cloud services to implement core features within the application. The architecture of cloud applications can differ significantly from traditional application models and, as such, implementing cloud applications can require a fundamental shift in application-design thought processes.

Cloud applications can often eliminate the need to install and run the application locally, thereby reducing the expenditure required for software maintenance, deployment, management, and support. This type of application would be considered a Software as a Service (SaaS) application.

An alternative to this would be the Software plus Services (S+S) model. This is the hybrid between traditional application development and a full SaaS implementation. S+S applications typically use rich client applications that are installed on a client’s PC as an interface into externally hosted services. S+S often includes the ability to interact with an application in an offline mode, and sync back to a central service when required.

Core Cloud Services

Core cloud services are services that support cloud-based solutions, such as identity management, service-to-service integration, mapping, billing/ payment systems, search, messaging, business process management, workflow, and so on. Core cloud services can be consumed directly by an individual, or indirectly through system-to-system integration.

The evolution of core cloud services potentially will mimic that of the telecommunications industry, with many services falling under the categories of Business Support Systems (BSS) or Operational Support Systems (OSS).

BSS services manage the interactions, with customers typically handling tasks such as:

• Taking orders
• Processing bills
• Collecting payments.

OSS services manage the service itself and are responsible for items such as:

• Service monitoring
• Service provisioning
• Service configuration.

Attributes Map for Cloud Services

By using the five cloud categories, we can now develop a set of attributes for each of the categories. These attributes can be used in two ways:

• Mapping your application’s attributes to cloud attributes to validate whether cloud services are suitable for your application, and identifying which types of services to use
• Evaluating cloud service providers as possible candidates for hosting your applications, identifying which types of services are available from your chosen provider(s), and then determining specific implementation attributes of the services offered.

 

Figure 3. Five cloud categories and attributes for cloud storage

 

Figure 3 shows the five cloud categories and a list of attributes for the cloud-storage category. Each cloud provider implements its cloud services in a slightly different way, with companies like Microsoft offering a number of different storage alternatives that developers can choose to use, depending on the required features, for a specific application.

Just like application attributes, cloud attributes must be weighed carefully when determining whether a cloud provider’s services are a good fit for your needs, as you will have to factor in implementation cost for each decision you make. (See Appendix A, Sample Cloud- Mapping Attributes, page 8.)

Overlaying the Cloud and Applications

Now that you have a complete understanding of the application and of what cloud services you could use to implement a solution, you can now start to make decisions about what the eventual architecture could be. Should you find that the Cloud is a viable and cost-effective alternative to traditional application architecture, the next step would be to choose the most suitable cloud provider(s) for the application.

It is quite possible that no single vendor would match completely with your requirements; however, you might find that you can obtain all of the services that your application requires from multiple vendors.

 

Figure 4. Single application that uses multiple cloud services and vendors

 

Figure 4 depicts an application that uses a number of cloud services, from multiple cloud providers. The preceding example could represent an application that is built in ASP.NET and is running on the Azure platform (cloud platform); however, the application also needs components with full trust, which means that the components can run only in a full virtual environment (cloud infrastructure). Data is stored in a Microsoft cloud (cloud storage), with services such as Workflow and Identity (core cloud services) also provided via Azure. The last requirement for the application could be a billing/payment service (core cloud services), which could be provided by another cloud provider.

Although this scenario is feasible, the costs that are associated with having accounts with multiple providers, using a number of APIs, and then integrating all of the services into an application could be impractical. The likely solution would be to find a single vendor that delivers the majority of the services that are required by your application, and use this as the base platform for a hybrid solution.

 

 

One CloudOne Cloud to Rule Them All

Is there one cloud, or are there multiples clouds? This is a debate that I have heard a number of times already. One side of this argument is public versus private clouds. Can a private implementation of cloud technologies be called a cloud at all, or is it something else? Are all of the public cloud offerings the same? And what about applications or systems that span both private and public clouds in a hybrid model? Where do these fit?

The honest answer is that the argument is irrelevant. Whether you subscribe to one theory or another, the desired outcome is the same: Build the most cost-effective system that you can that works. The previous section looked at ideas to help you make decisions; now, we will take a quick look at potential applications that could exist in the Cloud or as part of a hybrid cloud solution.

Architecting Solutions in the Cloud

This section describes three application scenarios for which solutions could be implemented by using cloud services. The following scenarios are by no means an extensive list of possible solutions that are suitable for cloud services; they are only indications of applications that could be feasible.

Ticketing System

When discussing the benefits of a cloud infrastructure, there seems to be a consensus that a concert ticketing system would make an ideal candidate for a cloud scenario (Figure 5). On the surface, this type of application looks like a viable candidate; ticketing systems are often subject to high demand over a short period of time (people scrambling to buy tickets for a concert or sporting event that will sell out rapidly). This is often followed by long periods of low to moderate activity.

 

Figure 5. Using cloud infrastructure for ticketing system

 

Ticketing applications are often overloaded during the periods of high demand, when the need for computing resources is extremely high. The ability to run up instances of virtual machines to cover such periods would be beneficial. There are, however, a number of issues that must be taken into account before architecting such a solution:

• Ticketing systems are data intensive and highly transactional. Transactions can be required for the payments system as well as to reserve specific seats for a given event.
• Personally identifying information is almost certain to be collected, with many customers having an account with the ticketing company or, potentially, wanting to create an account with the organization.
• Validating credit-card payments can be time consuming, and is a potential source of bottlenecks in the ticketing process.
• Some virtual cloud-server platforms cannot save state, which means that once a server image is shut down, all changes or additions to data that is stored within the image are lost.
• Existing ticketing companies already will have significant investment in infrastructure and data management.
• Depending on the cloud service that is chosen, virtual cloud-server images that are not able to save state will need to be recreated for every event, which could well result in a significant amount of work being required to prepare an environment for each new instance.

With all of this in mind, there are a number of ways to use a cloud service to reduce the demand on an existing system during a peak loading time:

• Duplicate the internal system completely. This would require the most amount of work; once the application is ready to use, it still would need to be synchronized with the current system for every new instance. Permanently leaving the system on (even in a reduced capacity) could be expensive due to the cost of using a services platform.
• Split the workload between internal systems and a cloud service in real time. This would involve splitting the process of selecting seats and purchasing tickets across the two environments; for example, the transaction could begin on the internal ticketing system where customers log into their account, select the event they wish to attend, select the seats, and then are passed off to a cloud service for final processing of payment. This would mean creating a virtual cloud server that simply completes the final stage of processing and, as such, would not need to be synchronized with the main system — effectively being a stateless processing engine. Only minimal data would need to be transferred to the cloud service, and credit-card information could be collected on the external system for single use and then deleted.
• Split the workload between internal systems using batch processing. Similarly to the preceding processing method, this method would differ in that all personal information would be collected on the internal system, including Cc: details. This information then could be placed in a process queue and shipped in batches to the cloud service for processing. This would mean that should payment fail, a secondary process for contacting the person who is attempting to purchase the tickets would need to be implemented if it does not already exist.

The preceding solutions are examples of how a ticketing system could split processing between a cloud service and a company’s internal systems during periods of heavy use.

Photo/Video Processing

This example shows how you can combine multiple services (infrastructure, storage, and queuing) to provide a solution for data processing (Figure 6). In this scenario there are a chain of photo processing stores that make use of the cloud service to render or reformat digital media files.

 

Figure 6. Using cloud infrastructure for photo/video processing

 

The photo chain has a number of stores spread across the U.S. and wishes to centralize large image and video processing to reduce two aspects of the system: the amount of hardware in each store; and the complexity of maintaining and supporting the hardware.

When a customer comes into a store with a video that needs to be converted to a different format, the video file is first uploaded to a storage service, and then a message is placed in a queue service that a file is on the storage platform and needs to be converted to a different format. An application controller that is running computer instances receives the message from the queue, and then either uses an existing instance of a virtual machine, or creates a new instance, to handle the reformatting of the video. As soon as this process is complete, the controller places a message in the queue to notify the store that the project is complete.

The preceding scenario easily can be converted to an online experience, so that customers could upload files for processing without having to go to a physical location.

Web Site Peak Loading

The final example that I will use is that of a Web site that has an extremely high amount of traffic on an irregular basis, which makes it impractical to build out the hosting infrastructure to support such peaks (Figure 7). Such sites could be news sites with breaking stories, game sites announcing a new game, or movie sites showing trailers of the next blockbuster.

 

Figure 7. Using cloud infrastructure for peak load coverage

 

The solution to this scenario involves creating a complete copy of the company’s Web site, or the part of the Web site that will experience the heavy traffic, on a cloud-service infrastructure service. The copy of the site would be a static instance running across a number of Web servers that could be configured as either a load-balanced set of servers or as a cluster. You can make any changes you need to on the original Web site, and then synchronize them back to the cloud servers. This would create latency, but would greatly reduce the effort needed to maintain the Web servers and Web sites, and would eliminate the problem of maintaining state between the internally and externally hosted Web sites.

There are many ways to architect solutions for the preceding scenarios, as there are many more scenarios in which you can use a cloud service. The goal of this article is merely to highlight a few of the alternatives and uses for the services that are emerging.

 

 

Conclusion

The fascination of the Cloud and cloud technologies is driving many developers, ISVs, start-ups, and enterprises to scrutinize cloud services and assess their suitability for adoption. The promises of lower cost of ownership — and of almost limitless scalability, in both storage and infrastructure power — are hard to ignore. The promise of the Cloud definitely warrants inspection; however, you must manage the adoption of cloud services carefully and realize that not all applications are suited for the Cloud. Many applications will work in the Cloud; however, hidden costs of hosting some solutions in the Cloud could see projects being delivered with much higher development and running costs than would be true of more traditional and well-defined architectures and technologies.

 

 

Appendix A

 

 

Appendix B

Cloud-Infrastructure Platforms and Virtualization Types

One of the key enabling-technologies for cloud-computing platforms is virtualization, which is the ability to provide an abstraction of computing resources. When we look at cloud-infrastructure platforms as they stand today, they predominately come in two flavors: fully virtualized or paravirtualized environments.

There are many more variations to virtualization than the two that I have just mentioned; so, for this post, I thought that I would discuss some of the virtualization methods that exist and that could well find their way into a cloud-infrastructure offering.

Emulation

In this type of virtualization, the virtual environment emulates a hardware architecture that an unmodified guest operating system (OS) requires. One of the common instances in which you encounter emulated hardware is with mobile devices. Application developers will use an emulated environment to test applications that are designed to run on smart phones or on PDAs, for example. (See Figure 8.)

 

Figure 8. Emulated-virtualization environment

 

Pros: Simulates a hardware environment, which is completely different from the underlying hardware. An example of this would be a mobile device such as a smart phone emulated on a desktop PC.

Cons: Poor performance and high resource usage.

Full Virtualization

In full virtualization, an image of a complete unmodified guest OS is made and run within a virtualized environment. The difference between full virtualization and emulation is that all of the virtualized guests run on the same hardware architecture. All of the guests support the same hardware, which allows the guest to execute many instructions directly on the hardware—thereby, providing improved performance. (See Figure 9.)

 

Figure 9. Full-virtualization environment

 

Pros: The ability to run multiple OS versions from multiple vendors: Microsoft Windows Server 2003, Windows Server 2008, Linux, and UNIX, for example.

Cons: Virtualized images are complete OS installations and can be extremely large files. Significant performance hits can occur (particularly on commodity hardware), and input/output operationintensive applications can be adversely effected in such environments.

Para-Virtualization

In para-virtualization, a hypervisor exports a modified copy of the physical hardware. The exported layer has the same architecture as the server hardware. However, specific modifications are made to this layer that allow the guest OS to perform at near-native speeds. To take advantage of these modified calls, the guest OS is required to have small modifications made to it. For example, you might modify the guest OS to use a hypercall that provides the same functionality that you would expect from the physical hardware. However, by using the hypercall, the guest is significantly more efficient when it is run in a virtualized environment. (See Figure 10.)

 

Figure 10. Para-virtualization environment

 

Pros: Lightweight and fast. Image sizes are significantly smaller, and performance can reach near-native speeds. Allows for the virtualization of architectures that would not normally support full virtualization.

Cons: Requires modifications to the guest OS, which allows the OS to support hypercalls over native functions.

OS-Level Virtualization

In OS virtualization, there is no virtual machine; the virtualization is done completely within a single OS. The guest systems share common features and drivers of the underlying OS, while looking and feeling like completely separate computers. Each guest instance will have its own file system, IP address, and server configuration, and will run completely different applications. (See Figure 11.)

 

Figure 11. OS-virtualization environment

 

Pros: Fast, lightweight, and efficient, with the ability to support a large number of virtual instances.

Cons: Isolation of instances and security concerns around data are significant issues. All virtual instances must support the same OS.

Application Virtualization

Application virtualization, as with any other type of virtualization, requires a virtualization layer to be present. The virtualization layer intercepts all calls that are made by the virtualized application to the underlying file systems, redirecting calls to a virtual location. The application is completely abstracted from the physical platform and interacts only with the virtualization layer. This allows applications that are incompatible with each other to be run side by side: Microsoft Internet Information Services 4.0, 5.0, and 6.0 all could run side-by-side, for example. This would also improve the portability of applications by allowing them to run seamlessly on an OS for which they were not designed. (See Figure 12.)

 

Figure 12. Application-virtualization environment

 

Pros: Improves the portability of applications, allowing them to run in different operating environments. Allows incompatible applications to run side by side. Allows accelerated application deployment through on-demand application streaming.

Cons: Overhead of supporting a virtual machine can lead to much slower execution of applications, in both run-time and native environments. Not all software can be virtualized, so is not a complete solution.

 

 

Describing Solution Architectures

Pragmatic Approach to Describing Solution Architectures

by Mike Walker

Summary: The return on investment of technical documentation is often not realized and documentation is frequently looked upon as a necessary evil. Creating the right architecture descriptions can help guide decision making at the various stages of the IT life cycle, however, there are limitations on formalized structures, information models, and intelligent tooling that takes the current architecture documentation to the next level of usefulness. In this article, we look at how we view, approach, and maintain architecture descriptions, and consider how this process can be improved.

 

 

Since the dawn of information technology (IT), engineers and architects have created reams of documents to describe their solutions. These have been published in three-ring binders and placed in obscure places on bookshelves, eventually to collect dust and be forgotten — something with which engineers and architects have had to deal as a fact of life.

As time has passed, documentation has evolved from a listing of detailed methods and procedures to the separation of multiple aspects of the solution from the detailed APIs and user-interface (UI) design to the architectural aspects. Integration with developmental and architectural processes furthered this activity and, eventually, became a required and common practice. By doing so, governance processes in many ways restricted forward progress. In many ways, governance has created both justified and unjustified restrictions to forward-moving progress on designs and coding, if a set of documentation had not been completed. This has led to frustration and productivity blockers. Current architecture documentation does not live up to the promise of providing return on investment in the design process. Often, the design that is articulated in architecture documentation is not realized in the final implemented design that is hosted in a production environment.

This pervasive growth of documents shows how critical the information is to the support of the software-development life cycle (SDLC). We now see the same with the support of architecture efforts, too. There are a number of challenges that occur with this paradigm.

 

Figure 1: Documentation, increasing at an accelerated rate

 

As Figure 1 shows, we are adding more and more documents to an already large portfolio of documents that must be completed. In the past, this was manageable; now, however, documents can be bloated, and there is a higher probability of information being duplicated. Often, this is a result of tightly coupling a document with a process step.

The goal is to solve the deficiencies and challenges with current architecture documentation while preserving the aspects that do work and have been assimilated into the daily lives of architects.

To do this, we will explore the following concepts:

• Alleviating challenges with current documents — Templates and new thought-provoking ideas will be introduced that challenge the existing ways of documenting architectures.
• Living architecture designs — Designs often are static and do not have a life outside of the development life cycle. We will introduce ways to change that.
• Enhancing decision support — Often, there are templates and checklists that give an architect a common way to think about problems. This is an accelerator to solving problems that have yet to be solved or identified.
• Deriving to solutions — Given how the human mind works, writing down a design to a specific problem in “black and white” often shows gaps in current thinking.
• Common means of collaboration — These provide a working information store to share and collaborate on architectures with team members.
• Supportability and maintainability — Documents provide important information on how a system was built. This provides support personnel with vital information for solving postproduction issues. For architects, the understanding of current system architecture will allow them to build out a set of strategies for the enterprise.

 

 

Rethinking the Traditional Architecture Document

The most common and pervasive interface for creating architecture descriptions is Microsoft Office Word. Office Word is part of the Microsoft Office suite of productivity tools. With documentation, there are many other tools that play a role in the documentation processes.

 

Figure 2: Microsoft Office Word reduces complexity and cost

 

As Figure 2 shows, Office provides a way to reduce complexity and costs significantly. The Office suite is easy to understand and most users already have them on their desktops. Augmenting the existing Office tools to make them fit the usage of architects is particularly ideal. It maintains a consumable amount of complexity while limiting the overall costs. However, Office Word is not the only tool that is used in the architectural processes. There are many other tools that have roles to play in how we document our architectures. These tools include the following:

• Office Visio

• Office PowerPoint

• Office Excel

• Office SharePoint

It is important to understand the context in which these tools interact.

 

Figure 3: Other productivity tools playing a role in the documentation process

 

Figure 3 shows an illustration of how other productivity tools play a part in architectural processes.

We can assert quickly that there is not just one tool that used in the documentation of architecture; myriad tools are used for collecting information from or publishing to. If we want to solve the challenges with the process, we should keep this in mind.

 

 

Optimizing Office Word to Describe Architectures

The underlying goal is to change the role of Office Word from simply a word processor to more of a UI for designing architectures. Applying structured UI concepts to Office Word provides many benefits to the architecture document, including the following:

• Structured content — Information can be better described in the document. We want to do this because of the challenges mentioned regarding how information does not integrate well with process or future design activities. One example is the process of importing a model into an architecture document. Often, we import a picture that represents a model of the specific viewpoint of an architecture. If we had an information model, we could specify that the model imported is indeed the logical model, instead of a generic image file.
• Extensible — With a little more structure, information has meaning and definition. This makes the information extensible to other processes downstream.
• Consumable — Ability to consume external content is also possible with a more structured interface. As an example, if you so choose, you could import external architecture information from other systems to automate your design efforts.

Through the use of the Office Word 2007 features that center on XML, we can truly start to extend this interface. Word does so by providing:

• Embedding XML documents — embed XML file into document for full data qualification.
• Office Word XML format — fully describes the formatting from the data through XML to have true separation of the formatting versus the information.
• Content controls — map most content controls to elements in XML data attached to a document.

Building the new UI for the architecture documents is easier in Office Word. A series of out-of-the-box functions are provided in the Office Word interface. For the architecture document, we will use the built-in tools that Office Word provides to create this new UI, which will enable us to describe our information in a meaningful way.

 

 

The Architecture of the System-Architecture Document

To change fundamentally how architects use a system-architecture document, we must look at what services Office Word provides to add additional capabilities that will automate and create rich metadata integration services.

For this solution, we will use a real world reference architecture called the Enterprise Architecture Toolkit (EATK). This toolkit will show how to alter Office Word from a word processor to a UI for describing architectures in a new way. The Microsoft Office environment provides rich extensibility that will allow developers to extend in a meaningful way. To do so, an architectural approach will have to be taken. By separating out layers and capabilities, approaches and technologies can be identified to derive to the right solution.

 

Figure 4: Architectural layers

 

As Figure 4 shows, there are four architectural layers that expose discrete services for this solution. These include the following:

• Platform — This is what the solution-architecture document and the integration services connect to. The platform components integrate with Office SharePoint Application Server environment for rich line-ofbusiness (LOB) integration.
• Tool — The mechanism of Office Word is referred to here. The tool provides extensibility into the UI that provides Microsoft Office ribbons to execute tasks easily with a level of context and Microsoft Office task panes that extend the UI with additional entry or listings of information.
• Document — The document provides the way in which a user can enter architecture descriptions. This is different in the EATK, as the document acts as the glue between the tool and the information itself. This is accomplished through an architecture-document template and the use of Office Word custom controls.
• Information — The information in the document is managed completely different from a traditional document. All of the information that is typed into the document is linked back to an XML node behind the scenes. This fully qualifies what is typed. Not only is the information rich, but it is extensible.

 

 

Platform Architecture

A great deal of work has been done in building componentized add-ins at the tooling level in Office Word, but this is not enough to change fundamentally the architecture document into a tool for designing architectures. The components that are applied to Office Word build upon a larger architecture canvas. The EATK provides a server-side Architecture Portal and Architecture Meta-Data Repository (AMR). This architecture interacts with the document-level activities when we create an architecture design.

 

Figure 5: Architecture behind EATK system-architecture document

 

Figure 5 shows a logical representation of the architecture that the EATK provides. It leverages a series of Microsoft-platform technologies, which include the following:

• Microsoft Office SharePoint (MOSS) 2007 — Used as an Architecture Portal, Document Management Services, and Workflow
• Windows Workflow Foundation (WF) — The hosted workflow on the portal that interacts with the desktop application and add-ins
• Microsoft Project Server — Can be used as the platform for interacting with project and portfolio data
• Microsoft SQL Server — Used as the database platform for the AMR that the Office Word add-ins Patterns Browser and Architect Lookup will use to get their information
• Microsoft IIS 7.0 — Used as the hosting platform for the AMR Services layer

The server components in the EATK play a part in how we change the role of an architecture document. All of the components on the server leverage platform capabilities of Office SharePoint, but change the context in which they can be used. The context is the architecture-development process. In doing so, we can take generic capabilities such as Enterprise Content Management (ECM) to automate how information is audited and versioned. The following are new interfaces and EATK components that were developed specifically for the architectural process and to interact directly with the system-architecture document:

• AMR Web Services — Services layer that allows for programmatic interaction with the AMR. It provides extensibility into not only the AMR, but also other related services such as PPM or Application Portfolio Management (APM).
• AMR Data Services — The base information services that delivers information such as patterns and existing IT assets to the Office Word task panes.
• Document-Management Services — An Office SharePoint–based set of services that are used to manage documents. It comprises functionality such as check-in, auditing, versioning, integrating with workflow, security, and archiving.
• Workflow Services — WF is used as the base of the workflow capabilities. Also, it is hosted on the server, which allows the architecting workflows that are applicable to the entire enterprise, instead of to just one architect.

 

 

Tool Architecture

This aspect of the solution is key to bridging the system-architecture document to both the platform for LOB integration and the document itself, which will be the interface in which architects will describe their solutions. All of the application logic is encapsulated in this layer.

Because this is the layer in which code is developed, this solution is dependent on Office Word API and other standard integration technologies.

 

Figure 6: Components of Office Word interface

 

Figure 6 shows the extended capabilities of Office Word. We have extended the following aspects of Office Word:

• Ribbons — We use this functionality as a launch pad for downstream activities, workflow triggering, collaboration, and information retrieval.
• Task panes — We can use this functionality for various aspects in which we want to create other interfaces from within Office Word.
• Properties — Assigning metadata to a document

 

 

System-Architecture Document Ribbons

Using the ribbon for the system-architecture document, we will have a series of functions that relate directly to architectural processes and information. As Figure 7 shows, there are four ribbon components in the EATK:

 

Figure 7: Ribbon components of system-architecture document

 

• Patterns Search — It displays patterns and existing IT assets that will help solve specific architecture-design questions and challenges.
• Patterns Browser — It allows for surfacing the right patterns in a more intuitive way for solving a business problem.
• Architect Lookup — Looking up an architect who is assigned to a project is streamlined with Architect Lookup by integrating collaboration via office communication server (ocs)
• Upload Doc — Automatically have a way to integrate architecture information into hosted workflow and consumed into a metadata repository.

 

 

System-Architecture Document Task Panes

Task panes can be very useful to architects, as they can surface meaningful information at a moment’s notice with a click of a ribbon component. This information is not only visible, but also interactive.

Tangible ways in which these task panes empower IT architects are the following:

• Systematic reuse — Many of the architectures that we build have repeatable patterns built within. By surfacing patterns in a composite manner, we can reuse the aspects and views of architectures in a systematic way.
• Decision support — By reviewing existing architectures, we can review how solutions were developed based on a set of drivers. Not only can you review, but also you can contrast decisions on these architectures with the challenges of the current architecture efforts.
• Automation — Not only can you view the patterns and assets in the task pane, but also you can apply them to your architecture design. By reusing models and architectural descriptions, you can automate the architectural process by eliminating unnecessary work.
• Traceability of technology choices — Not only can you surface this information and apply it to your architecture designs, but now you can create relationships between what was imported and the architecture that you are designing.

As one example of this is the Patterns Browser. The intent of the Patterns Browser is to surface pattern information into the design environment. Two types of information are displayed:

• Assets — Shows what has been built.
• Patterns — Shows what should be built.

As Figure 8 shows, patterns can be applied to the elements of an existing architecture. In the preceding case, it is the reuse of a logical architecture model. Appling the selected model is as easy as double-clicking the pattern in the Patterns Browser. The pattern then is applied to the element that is selected in the document.

 

Figure 8: Interacting with elements in document

 

Another innovative way the Task Pane is utilized is by introducing collaboration. The architectural process is no longer a onearchitect job. Typically multiple architects with specific roles design systems architecture. Examples of these roles include; Application, Information, Hardware, Security, or Infrastructure architect.

Other roles in review processes will have an impact on the validity, quality, and level of completeness of the designs. This affects the architecture in a significant way, too. These roles usually are part of an Architectural Review Board of sorts. In this function, it is critical that the architecture document give this group of individuals the information that it needs to make decisions on the architecture. By introducing the collaborative aspects to the architectural process, we can reduce the number of issues ahead of time, instead of at the end of the process — thus, changing the architecture-design process from its current form of a reactive process to a proactive process.

 

Figure 9: Architect Lookup task pane

 

Architect Lookup would return the names and photos of the Hardware, Security, and Information architects. As Figure 9 shows, the Architect Lookup task pane reveals the architect resources, based on the project that is assigned to the architecture work. The way in which Architect Lookup determines this is by using the assigned project ID. The project ID is found in the meta-data of the document. This metadata can be entered either manually or through automated means when the architecture-design request is sent.

Architect Lookup will take the project ID and perform a query against multiple systems to return a set of results. Project Server (or a custom information store) can be used to retrieve the list of architect resources, while Office SharePoint and Active Directory are used to take the resources in the PPM repository and correlate them with the images of the person, along with links to IM and personal portal sites.

Use of Architect Lookup eliminates the need to go to project plans, portals, or file shares to find this information manually. It is a simple click away to get instant access to other resources on the project, to get answers to questions.

 

Figure 10: Collaborative architectures

 

Each role has a part to play in the architectural process. Given the illustration in Figure 10, we see that an Application architect might start the process, but then might need the aid of other architects. In this case, the Application architect might be the owner of the document and be the primary contributor. The other roles that are shown — such as Hardware architect, Security architect, and Information architect — are some that would interact. These roles will validate, add, and modify architecture-design descriptions.

In this context, a sample of common questions to these roles would look like the following:

• How will the security model affect how I design my application?
• I know that I must design an external portal, but what does the servertier model look like for the DMZ?
• What will the network stack look like, and how will that affect performance or scalability?
• What is the right hardware?

 

 

Document-Template Architecture

Along with enhancements to the tool, the next layer of concern is the document template.

The document-template layer does not have a great deal of intelligence to it; however, it does allow interaction from Office Word to the document. One example is that the EATK allows information from an external source to be “clicked and dragged” into the document. This pulls data from a database and populates the document with architecture models.

The EATK provides not only the Office Word add-ins that are described in the previous sections of this article, but also a system-architecture document template, as Figure 11 shows:

 

Figure 11: System-architecture document template

 

The views that are listed in the table of contents are a set of common views. These views can be renamed or removed, and additional views can be added. These views provide a starting point for architects. Whereas you can use the system-architecture document template as your corporate standard in your company, it will be common practice to modify it.

With many documents that are surrounded by process, we want to create templates to ensure that they are used in a consistent way. This is the same case with the system-architecture document template. A template for the architecture document will allow for:

Everyone using the right version of the architecture document.

• Ensuring that the information models stay intact.
• Allowing for information to be generated by the system.
• Enabling information to be consumed by downstream processes.

 

 

Information Architecture

The last piece of the puzzle is the information itself. Having great add-ins and template is great; however, without a meaningful way to express and ensure that the information is long living in a connected process, it is not as useful as it could be. Without information architecture, it would be difficult to qualify architectures.

The information layer is the base of the solution; it is where the majority of the consideration is made. All layers interact with each other in some way, but the information layer is connected directly to all. It is consumed in workflows, information-entry processes, and automation through Microsoft Office ribbons and task panes.

By using industry-standard techniques to derive to the target information architecture. Two fundamental aspects of the EATK that we must explore are the following:

• Ontology — We want to define a set of terms that are understood commonly within the enterprise. By doing so, we can relate information properly and consistently.
• Taxonomy — This will allow you to correlate architecture information with other aspects of architecture.

The architecture document should use the terms in the proper usage to qualify what the information is. Publishing an online ontology mapping will be useful toward understanding the information within the document.

In the context of the system-architecture document, ontology provides agreed-upon terminology for architecture. As an example, it would define the meanings of a platform, system, subsystem, application, and components.

Defining what these architecture elements are, however, is one piece of the puzzle. How these elements relate to each other is the next logical step. We will use taxonomy for this. The EATK uses an industry standard from IEEE to solve this challenge. IEEE 1471 is used as the basis for the taxonomy and ontology of the system-architecture document.

IEEE 1471 is the first formalized standard for describing architectures and how they relate to other aspects of the software-development process. It leverages a scalable meta-model that shows the relationships between common elements in the architecture-development process.

 

Figure 12: IEEE framework for architectural description

 

In Figure 12, IEEE 1471 provides a meta-model that allows us to relate architectures with other aspects of the software-development process. The system-architecture document focuses on specific areas of the taxonomy, while other EATK components focus on other aspects of the standard.

The aspects that are implemented to support IEEE 1471 in the systemarchitecture document are the following:

• Structured content — An architecture schema that represents the information that we want to gather in our architecture document was created that has a viewpoint-based model applied to it.
• Qualifying information — Links between decisions that have been made and other systems or architecture descriptions are built into the schema. As a user enters information or applies patterns to the architecture document, it will correlate that information by unique ID.
• Publishing mechanisms — Provides facilities to store information from the schema, so that it can be related to other non-document–related information.
• Generating information — Provides a mechanism to rebuild or generate sections of an architecture document.

 

 

System-Architecture Markup XML

With an ontology and taxonomy for architecture, we now can look at what this means from an implementation perspective. The system-architecture document has an underlying XML structure that is used to describe the information in the form of XML.

The System Architecture Markup XML is a way to provide clear separation between the presentation markup and the information markup.

This greatly simplifies integration to hosted workflow, repositories, and third party tooling.

Conclusion

In this article, we reviewed how to change fundamentally the way in which we view, approach, and maintain architecture descriptions. There is no doubt that there could be significant value in capturing information about the designs of our architectures that will help guide decision makers through the processes of making the right architecture decisions. However, this is not always the outcome, as often no formalized structure, process, information model, or intelligent tooling is bundled with this process.

Key takeaways include the following:

• Architects who want to implement these concepts can use the EATK, which provides a set of templates, processes, and a solution accelerator that can be used to accelerate this implementation for architects.

• Architects can achieve significant productivity gains through reduction of manual activities and process automation.

• Architecture information can be used in a much more meaningful way by eliminating the document graveyard effect, by integrating architecture descriptions with an AMR.

• The quality of decision making can be improved by eliminating points of duplication; thus, information quality can be increased significantly.

• Wikis and modeling tools complement this implementation—or, in some cases, replace it.

• Solutions such as COTS and custom-developed applications can be integrated with this solution through standard integration technologies.

About the Author

Mike Walker is a principle architect who delivers strategy for the enterprise architecture space at Microsoft. He is responsible for creating, driving, and evangelizing Microsoft’s worldwide Enterprise 2.0 and Enterprise Architecture strategies. Specifically, Mike ensures that institutions around the world realize the full extent of Microsoft’s vision and value proposition. He has also evolved many of the concepts behind mainstream architectural approaches and styles. His works are realized through publication in books, trade publications, authoritative architecture guidance, articles, code delivery and standardized frameworks. Those frameworks include: the Enterprise Architecture Toolkit (EATK), Loan Origination Reference Architecture, and Financial Services Component Library. You can visit his blog a thttp://www.MikeTheArchitect.com

Software Architecture

A Language for Software Architecture

by J.D. Meier

Summary: One of the most important outcomes of the patterns & practices Application Architecture Guide 2.0 project is a language for the space: a language for application architecture. Building software applications involves a lot of important decisions. By organizing these decisions as a language and a set of mental models, we can simplify organizing and sharing information. By mapping out the architecture space, we can organize and share knowledge more effectively. By using this map as a backdrop, we can also overlay principles, patterns, technologies, and key solutions assets in meaningful and relevant ways. Rather than a sea of information, we can quickly browse hot spots for relevant solutions.

 

 

A Map of the Terrain

One of the most effective ways to deal with information overload is to frame a space. Just like you frame a picture, you can frame a problem to show it a certain way. When I started the patterns & practices Application Architecture Guide 2.0 project, the first thing I wanted to do was to frame out the space. Rather than provide step-by-step architectural guidance, I thought it would be far more valuable to first create a map of what’s important. We could then use this map to prioritize and focus our efforts. We could also use this map as a durable, evolvable backdrop for creating, organizing and sharing our patterns & practices work. Figure 1 shows the main map, the Architecture Frame, we created to help us organize and share principles, patterns, and practices in the application architecture space.

 

Figure 1: Architecture Frame

 

 

Mapping Out the Architecture Space

Creating the map was an iterative and incremental process. The first step was to break up application architecture into meaningful buckets. It first started when I created a project proposal for our management team. As part of the proposal, I created a demo to show how we might chunk up the architecture space in a meaningful way. In the demo, I included a list of key trends, a set of application types, a set of architectural styles, a frame for quality attributes, an application feature frame, a set of example deployment patterns, and a map of patterns & practices solution assets. I used examples where possible simply to illustrate the idea. It was well received, and it served as a strawman for the team.

Each week, our core Application Architecture Guide 2.0 project team met with our extended development team, which primarily included patterns & practices development team members. During this time, we worked through a set of application types, created a canonical application, analyzed layers and tiers, evaluated key trends, and created technology matrix trade-off charts. To create and share information rapidly, we created many mind maps and slides. The mind maps worked well. Rather than get lost in documents, we used the mind maps as backdrops for conversation and elaboration.

Key Mapping Exercises

We mapped out several things in parallel:

• Key trends. Although we didn’t focus on trends in the guide, we first mapped out key trends to help figure out what to pay attention to. We used a mind map and we organized key trends by application, infrastructure, and process. While there weren’t any major surprises, it was a healthy exercise getting everybody on the same page in terms of which trends mattered.

• Canonical application. The first thing we did was figure out the delta from the original architecture guide. There were a few key changes. For example, we found that today’s applications have a lot more clients and scenarios they serve. They’ve matured and they’ve been extended. We also found today’s applications have a lot more services, both in terms of exposing and in terms of consuming. We also noticed that some of today’s applications are flatter and have fewer layers. Beyond that, many things such as the types of components and the types of layers were fairly consistent with the original model.

• Layers and tiers. This was one of the more painful exercises. Early in the project, we met each week with our development team, along with other reviewers. The goal was to map out the common layers, tiers, and components. While there was a lot of consistency with the original application architecture guide, we wanted to reflect any learnings and changes since the original model. Once we had a working map of the layers, tiers, and components, we vetted the map with multiple customers.

• Application types. We originally explored organizing applications around business purposes or dominant functionality, customer feedback told us we were better off optimizing around technical types, such as Web application or mobile client. They were easy for customers to identify with. They also made it easy to overlay patterns, technologies, and key patterns & practices solution assets. The technical application types also made it easy to map out relevant technologies.

• Architectural styles. This is where we had a lot of debate. While we ultimately agreed that it was helpful to have a simple language for abstracting the shapes of applications and the underlying principles from the technology, it was difficult to create a map that everybody was happy with. Things got easier once we changed some of the terminology and we organized the architectural styles by common hot spots. It then became obvious that the architectural styles are simply named sets of principles. We could then have a higher level conversation around whether to go with object-based community or message-based and SOA, for example. It was also easy to describe deployments in terms of 2-tier, 3-tier, and N-tier.

• Hot spots for architecture. When you build applications, there’s a common set of challenges that show up again, such as caching, data access, exception management, logging, and so on. These are application infrastructure problems or cross-cutting concerns. You usually don’t want to make these decisions ad hoc on any significant application. Instead, you want to have a set of patterns and guidelines or ideally reusable code that the team can leverage throughout the application. What makes these hot spots is that they are actionable, key engineering decisions. You want to avoid do-overs where you can. Some do-overs are more expensive than others. One of the beauties of the architecture hot spots is that they helped show the backdrop behind Enterprise Library. For example, there’s a data access block, a caching block, a validation block, and so forth.

• Hot spots for application types. When you build certain classes of application, there’s recurring hot spots. For example, when you build a rich client, one of the common hot spots to figure out is how to handle occasionally disconnected scenarios. The collection of hot spots for architecture served as a baseline for finding hot spots in the other application types. For example, from the common set of hot spots, we could then figure out which ones are relevant for Web applications, or which additional hot spots would we need to include.

• Patterns. Mapping out patterns was a lengthy process. Ultimately, we probably ended up with more information in our workspace than made it into the guide. To map out the patterns, we created multiple mind maps of various pattern depots. We summarized patterns so that we could quickly map them from problems to solutions. We then used our architecture hot spots and our hot spots for application types as a filter to find the relevant patterns. We then vetted the patterns with customers to see if the mapping was useful. We cut any patterns that didn’t seem high enough priority. We also cut many of our pattern descriptions when they started to weight the guide down.

We figured we had plenty of material and insight to carve out future pattern guides and we didn’t want to overshadow the value of the main chapters in the guide. We decided the best move for now was to provide a Pattern Map at the end of each application chapter to show which patterns are relevant for key hot spots. Customers seemed to like this approach, and it kept things lightweight.

• Patterns & practices solution assets. This was the ultimate exercise in organizing our catalog. We actually have a large body of documented patterns. We also have several application blocks and factories, as well as guides. By using our architecture frame, it was easier to organize the catalog. For example, the factories and reference implementations mapped to the application types. The Enterprise Library blocks mapped to the architecture hot spots. Several of the guides mapped to the quality attributes frame. For more information, see Cheat Sheet – patterns & practices Catalog at a Glance posted to CodePlex athttp://blogs.msdn.com/jmeier/archive/2008/10/09/cheat-sheet-patterns-practices-catalog-at-a-glance-posted-to-codeplex.aspx.

• Microsoft platform. This was a challenge. It meant slicing and dicing the platform stack in a meaningful way as well as finding the right product team contacts. Once we had our application types in place, it got a lot easier. Depending on which type of application you were building — rich internet application (RIA), Web, mobile, for example — this quickly narrowed down relevant technology options. We created technology matrices for presentation technologies, integration technologies, workflow technologies, and data access technologies. Since the bulk of the guide is principle and pattern based, we kept these matrices in the appendix for fast lookups.

 

 

Key Components of the Application Architecture Map

Over the weeks and months of the project, a very definite map of the landscape emerged. We found ourselves consistently looking for the same frames to organize information. While we tuned and pruned specific hot spots in areas, the overall model of common frames was helping us move through the space quickly.

• Architecture frame. The architecture frame was the main organizing map. It brought together the context (scenarios, quality attributes, requirements/constraints), application types, architectural styles, and the application hot spots.

• Application types. For application types, we optimized around a simple, technical set that resonated with customers, such as Web application, RIA, and mobile.

• Quality attributes. We organized quality attributes by key hot spots: system, runtime, design-time, and user qualities.

• Architectural styles. We organized architectural styles by key hot spots: communication, deployment, domain, interaction, and structure.

• Requirements and constraints. We organized requirements by key types: functional, non-functional, technological. We thought of constraints in terms of industry and organizational constraints, as well as by which concern (for example, constraints for security or privacy).

• Application feature frame. The application feature frame became a solid backdrop for organizing many guidelines through the guide.

The hot spots resonated: caching, communication, concurrency and transactions, configuration management, coupling and cohesion, data access, exception management, layering, logging and instrumentation, state management, structure, validation and workflow.

• Application type frames. The application type frames are simply hot spots for key application types. We created frames for: Web applications, RIA, mobile applications, rich client applications and services.

Layered architecture reference model. The canonical application is actually a layered architecture reference model. It helps show the layers and components in context.

• Layers and tiers. We used layers to represent logical partitions and tiers for physical partitions (this precedent was set in the original guide.) We identified key components within the key layers: presentation layer, business layer, data layer, and service layer.

• Pattern maps. Pattern maps are simply overlays of key patterns on top of relevant hot spots. We created pattern maps for the application types.

• Product and technology maps. We created technology matrices for relevant products and technologies. To put the technologies in context, we used application types where relevant. We also used scenarios. To help make trade-off decisions, we included benefits and considerations for each technology.

 

 

User, Business, and System Perspective

One thing that helped early on was creating a Venn diagram of the three perspectives, user, business, and system, as shown in Figure 2.

 

Figure 2: User, Business, and System Perspectives

 

In application architecture, it’s easy to lose perspective. It helps to keep three perspectives in mind. By having a quick visual of the three perspectives, it was easy to remind ourselves that architecture is always a trade-off among these perspectives. It also helped remind us to be clear which perspective we’re talking about at any point in time. This also helped resolve many debates. The problem in architecture debates is that everybody is usually right, but only from their perspective. Once we showed people where their perspective fit in the bigger picture, debates quickly turned from conflict to collaboration. It was easy to move through user goals, business goals, and system goals once people knew the map.

 

 

Architecture Frame

The Architecture Frame is a simple way to organize the space (Figure 1). It’s a durable, evolvable backdrop. You can extend it to suit your needs. The strength of the frame is that it combines multiple lenses.

Here are the key lenses:

• Scenarios. This sets the context. You can’t evaluate architecture in a vacuum. You need a backdrop. Scenarios provide the backdrop for evaluation and relevancy.

• Quality attributes. This includes your system qualities, your runtime qualities, your design-time qualities and user qualities.

• Requirements / constraints. Requirements and constraints includes functional requirements, non-functional requirements, technological requirements, industry constraints and organizational constraints.

• Application types. This is an extensible set of common types of applications or clients. You can imagine extending for business types. You can imagine including just the types of applications your organization builds. Think of it as product-line engineering. When you know the types of applications you build, you can optimize it.

• Architectural styles. This is a flat list of common architectural styles. The list of architectural styles is flexible and most applications are a mash up of various styles. Architectural styles become more useful when they are organized by key decisions or concerns.

• Application feature frame. The application feature frame is a concise set of hot spots that show up time and again across applications. They reflect cross-cutting concerns and common application infrastructure challenges.

 

 

Application Types

We defined a simple set of technical application types:

• Web applications. Applications of this type typically support connected scenarios and can support different browsers running on a range of operating systems and platforms.

• RIA. applications of this type can be developed to support multiple platforms and multiple browsers, displaying rich media or graphical content. Rich Internet applications run in a browser sandbox that restricts access to some devices on the client.

• Mobile applications. Applications of this type can be developed as thin client or rich client applications. Rich client mobile applications can support disconnected or occasionally connected scenarios. Web or thin client applications support connected scenarios only. The device resources may prove to be a constraint when designing mobile applications.

• Rich client applications. Applications of this type are usually developed as stand-alone applications with a graphical user interface that displays data using a range of controls. Rich client applications can be designed for disconnected and occasionally connected scenarios because the applications run on the client machine.

• Services. Services expose complex functionality and allow clients to access them from local or remote machine. Service operations are called using messages, based on XML schemas, passed over a transport channel. The goal in this type of application is to achieve loose coupling between the client and the server.

 

 

Application Feature Frame

This is the set of hot spots for applications we defined:

• Authentication and authorization. Authentication and authorization allow you to identify the users of your application with confidence, and to determine the resources and operations to which they should have access.

• Caching and state. Caching improves performance, reduces server round trips, and can be used to maintain the state of your application.

• Communication. Communication strategies determine how you will communicate between layers and tiers, including protocol, security, and communication-style decisions.

• Composition. Composition strategies determine how you manage component dependencies and the interactions between components.

• Concurrency and transactions. Concurrency is concerned with the way that your application handles conflicts caused by multiple users creating, reading, updating, and deleting data at the same time. Transactions are used for important multi-step operations in order to treat them as though they were atomic, and to recover in the case of a failure or error.

• Configuration management. Configuration management defines how you configure your application after deployment, where you store configuration data, and how you protect the configuration data.

• Coupling and cohesion. Coupling and cohesion are strategies concerned with layering, separating application components and layers, and organizing your application trust and functionality boundaries.

• Data access. Data access strategies describe techniques for abstracting and accessing data in your data store. This includes data entity design, error management, and managing database connections.

• Exception management. Exception-management strategies describe techniques for handling errors, logging errors for auditing purposes, and notifying users of error conditions.

• Logging and instrumentation. Logging and instrumentation represents the strategies for logging key business events, security actions, and provision of an audit trail in the case of an attack or failure.

• User experience. User experience is the interaction between your users and your application. A good user experience can improve the efficiency and effectiveness of the application, while a poor user experience may deter users from using an otherwise well-designed application.

• Validation. Validation is the means by which your application checks and verifies input from all sources before trusting and processing it. A good input and data-validation strategy takes into account not only the source of the data, but also how the data will be used, when determining how to validate it.

• Workflow. Workflow is a system-assisted process that is divided into a series of execution steps, events, and conditions. The workflow may be an orchestration between a set of components and systems, or it may include human collaboration.

 

 

Architectural Styles

For architectural styles, we first framed the key concerns to organize the architectural styles, and then we defined some common architectural styles.

Organizing Architectural Styles

These are the hot spots we used to organize architectural styles:

• Communication. Service-Oriented Architecture(SOA) and/or Message Bus and/or Pipes and Filters.

• Deployment. Client/server or 3-Tier or N-Tier.

• Domain. Domain Model or Gateway.

• Interaction. Separated Presentation.

• Structure. Component-Based and/or Object-Oriented and/or Layered Architecture.

Architectural Style Frame

These are some commonly recognized architectural styles:

• Client-server. Segregates the system into two applications, where the client makes a service request to the server.

• Component-based architecture. Decomposes application design into reusable functional or logical components that are location-transparent and expose well-defined communication interfaces.

• Layered architecture. Partitions the concerns of the application into stacked groups (layers) such as presentation layer, business layer, data layer, and services layer.

• Message-bus. A software system that can receive and send messages that are based on a set of known formats, so that systems can communicate with each other without needing to know the actual recipient.

• N-tier/3-tier. Segregates functionality into separate segments in much the same way as the layered style, but with each segment being a tier located on a physically separate computer.

• Object-oriented. An architectural style based on division of tasks for an application or system into individual reusable and self-sufficient objects, each containing the data and the behavior relevant to the object.

• Separated presentation. Separates the logic for managing user interaction from the user interface (UI) view and from the data with which the user works.

• Service-oriented architecture. Refers to Applications that expose and consume functionality as a service using contracts and messages.

 

 

Quality Attributes

For quality attributes, we first framed the key categories to organize the quality attributes, and then we defined some common quality attributes.

Organizing Quality Attributes

Table 1 shows a simple way to organize and group quality attributes:

 

Table 1: Quality Attributes

 

Quality Attribute Frame

Some common quality attributes include:

• Availability. Availability is the proportion of time that the system is functional and working. It can be measured as a percentage of the total system downtime over a predefined period. Availability will be affected by system errors, infrastructure problems, malicious attacks, and system load.

• Conceptual integrity. Conceptual integrity is the consistency and coherence of the overall design. This includes the way that components or modules are designed, as well as factors such as coding style and variable naming.

• Flexibility. The ability of a system to adapt to varying environments and situations, and to cope with changes in business policies and rules. A flexible system is one that is easy to reconfigure or adapt in response to different user and system requirements.

• Interoperability. Interoperability is the ability of diverse components of a system or different systems to operate successfully by exchanging information, often by using services. An interoperable system makes it easier to exchange and reuse information internally as well as externally.

• Maintainability. Maintainability is the ability of a system to undergo changes to its components, services, features, and interfaces as may be required when adding or changing the functionality, fixing errors, and meeting new business requirements.

Manageability. Manageability is how easy it is to manage the application, usually through sufficient and useful instrumentation exposed for use in monitoring systems and for debugging and performance tuning.

• Performance. Performance is an indication of the responsiveness of a system to execute any action within a given time interval. It can be measured in terms of latency or throughput. Latency is the time taken to respond to any event. Throughput is the number of events that take place within a given amount of time.

• Reliability. Reliability is the ability of a system to remain operational over time. Reliability is measured as the probability that a system will not fail to perform its intended functions over a specified time interval.

• Reusability. Reusability is the capability for components and subsystems to be suitable for use in other applications and in other scenarios. Reusability minimizes the duplication of components and also the implementation time.

• Scalability . Scalability is the ability of a system to function well when there are changes to the load or demand. Typically, the system will be able to be extended over more powerful or more numerous servers as demand and load increase.

• Security. Security is the ways that a system is protected from disclosure or loss of information, and the possibility of a successful malicious attack. A secure system aims to protect assets and prevent unauthorized modification of information.

• Supportability. Supportability is how easy it is for operators, developers, and users to understand and use the application, and how easy it is to resolve errors when the system fails to work correctly.

• Testability. Testability is a measure of how easy it is to create test criteria for the system and its components, and to execute these tests in order to determine if the criteria are met. Good testability makes it more likely that faults in a system can be isolated in a timely and effective manner.

• Usability. Usability defines how well the application meets the requirements of the user and consumer by being intuitive, easy to localize and globalize, and able to provide good access for disabled users and a good overall user experience.

 

 

Layered Architecture Reference Model

Figure 3 shows our canonical application example. It’s a layered architecture showing the common components within each layer: The canonical application model helped us show how the various layers and components work together. It was an easy diagram to pull up and talk through when we were discussing various design trade-offs at the different layers.

 

Figure 3: Layered Architecture Model

 

Layers

We identified the following layers:

• Presentation layer

• Business layer

• Data layer

• Service layer

They are logical layers. The important thing about layers is that they help factor and group your logic. They are also fractal. For example, a service can have multiple types of layers within it. The following is a quick explanation of the key components within each layer.

Presentation Layer Components

• User interface (UI) components. UI components provide a way for users to interact with the application. They render and format data for users and acquire and validate data input by the user.

• User process components. To help synchronize and orchestrate these user interactions, it can be useful to drive the process by using separate user process components. This means that the process-flow and state-management logic is not hard-coded in the UI elements themselves, and the same basic user interaction patterns can be reused by multiple UIs.

Business Layer Components

• Application facade (optional). Use a façade to combine multiple business operations into a single message-based operation. You might access the application façade from the presentation layer by using different communication technologies.

• Business components. Business components implement the business logic of the application. Regardless of whether a business process consists of a single step or an orchestrated workflow, your application will probably require components that implement business rules and perform business tasks.

• Business entity components. Business entities are used to pass data between components. The data represents real-world business entities, such as products and orders. The business entities used internally in the application are usually data structures, such as DataSets, DataReaders, or Extensible Markup Language (XML) streams, but they can also be implemented by using custom object-oriented classes that represent the real-world entities your application has to work with, such as a product or an order.

• Business workflows. Many business processes involve multiple steps that must be performed in the correct order and orchestrated. Business workflows define and coordinate long-running, multi-step business processes, and can be implemented using business process management tools.

Data Layer Components

• Data access logic components. Data access components abstract the logic necessary to access your underlying data stores. Doing so centralizes data access functionality, and makes the process easier to configure and maintain.

• Data helpers / utility components. Helper functions and utilities assist in data manipulation, data transformation, and data access within the layer. They consist of specialized libraries and/or custom routines especially designed to maximize data access performance and reduce the development requirements of the logic components and the service agent parts of the layer.

• Service agents. Service agents isolate your application from the idiosyncrasies of calling diverse services from your application, and can provide additional services such as basic mapping between the format of the data exposed by the service and the format your application requires.

Service Layer Components

• Service interfaces. Services expose a service interface to which all inbound messages are sent. The definition of the set of messages that must be exchanged with a service, in order for the service to perform a specific business task, constitutes a contract. You can think of a service interface as a façade that exposes the business logic implemented in the service to potential consumers.

• Message types. When exchanging data across the service layer, data structures are wrapped by message structures that support different types of operations. For example, you might have a Command message, a Document message, or another type of message. These message types are the “message contracts” for communication between service consumers and providers.

 

 

Tiers

Tiers represent the physical separation of the presentation, business, services, and data functionality of your design across separate computers and systems. Some common tiered design patterns include two-tier, three-tier, and n-tier.

Two-Tier

The two-tier pattern represents a basic structure with two main components, a client and a server (Figure 4).

 

Figure 4: Two-Tier Deployment

 

Three-Tier

In a three-tier design, the client interacts with application software deployed on a separate server, and the application server interacts with a database that is also located on a separate server (Figure 5). This is a very common pattern for most Web applications and Web services.

 

Figure 5: Three-Tier Deployment

 

N-Tier

In this scenario, the Web server (which contains the presentation layer logic) is physically separated from the application server that implements the business logic (Figure 6).

 

Figure 6: N-Tier Deployment

 

 

Conclusion

It’s easier to find your way around when you have a map. By having a map, you know where the key hot spots are. The map helps you organize and share relevant information more effectively. More importantly, the map helps bring together archetypes, arch styles, and hot spots in a meaningful way. When you put it all together, you have a simple language for describing large classes of applications, as well as a common language for application architecture.

 

 

Switching IT Outsourcing Vendors

Organizations invariably spend a considerable amount of time and money before entering into contracts with IT service providers and the negotiations are typically long drawn. Having thus invested in the process of selecting vendors, many organizations shy away from ousting IT suppliers once the contract is underway. And in most cases, once the contract has terminated, the incumbent IT provider is awarded an extension so as to avoid the pains of choosing a new supplier.

However, in the recent past there have been a number of cases where customers have either terminated a contract or switched suppliers post-termination. Deals to the tune of over $1.5 bn have seen acrimonious splits over pricing issues and in another case a customer dropped its IT provider to replace it with the incumbent supplier of its newly acquired arm.

Needless to say these decisions to switch vendors have always been a well thought over one. These decisions need to incorporate the base understanding and benefit rationalization of switching to a new vendor. For this, the customers need to well aware of the risks involved in changing suppliers, either mid-contract or post-termination. Once the risks are identified, they can be tackled with a certain amount of visibility into the future in conjunction with a thorough analysis of the existing conditions which gives a clear picture to the customer, on the cause and effects of the switching decision.


Need for switching vendors

Customers have switched IT providers for reasons as varied as pricing issues to nonperformance
on service levels to cultural nonalignment of organizations. At a broad level, the key
factors that act as a catalyst and hasten the switching of IT providers can be clubbed into the
following categories

• Customer Landscape Changes: The customer may be ready to move up its value ladder and hence may have outgrown the vendor. In some cases, the customer could find the regional vendor unable to service their global needs. Or in some cases, customers may feel that handing off operations to a contractor makes it the outsourcing customer less nimbleand may seek to regain control of its operations.

• Vendor Market Changes: Once a contract iswell underway, customers may realize that the vendor market has expanded and there are more players now who can help them build competitive advantage.

• Vendor Rationalization: In most cases, as a means of diversifying risk, outsourcing customers divide their portfolio among multiple vendors. As a result, certain customers experience a loss of purchasing power. In a bid to regain some of this purchasing power, customers may look to reduce the number of vendors. This may necessitate transitioning certain outsourced functions from one vendor to another existing one.

• Relationship/Performance Issues: In many cases, the vendor may fall short of the
  expectations of the outsourcing engagement. A survey from Benchmark Research
  found that outsourcing clients have experienced significant problems in managing
  their post-contractual relationships. This can arise because of multiple causes:
  
  • Unrealistic expectation setting prior to before signing the contract
  • Vendor not meeting service level agreements that are part of the contract
  • The vendor may actually be delivering on the service agreements but the
  • client sees less value than expected.

For any of these reasons, the customer may choose to change the supplier and go with a new player. However, such customers need to be aware of the risks that may affect the entire process of transitioning to the new vendor.


Risks in switching vendors

Switching from one outsourcing vendor to another mid-contract involves the effort and risk of both terminating and entering into an outsourcing agreement. Switching vendors can be costly, considering the employee time and effort for transition, and the legal and consulting fees as well.
Some of the costs and risks in changing vendors are:

• Knowledge Transition A key risk in changing vendors is ineffective knowledge transition between the incumbent vendor and the new vendor(s). While this risk has a lower impact in the case of switching vendors post-contract termination, in the scenario of changing vendors mid-contract this can be a serious concern. Effort must be taken to ensure that complete and effective knowledge transition happens between all the vendor parties involved.

• Knowledge Stickiness The problem of stickiness is not peculiar to a scenario of changing vendors but is more a issue in knowledge transfer. However it merits separate mention because in the case of switching suppliers, the risk of stickiness is magnified.
  Stickiness is the core characteristic of specialized, personal and tacit knowledge that inhibits easy transfer. Some of the origins of stickiness are strained relationships, lack of motivation on the part of the source, lack of absorptive capacity of the recipient and the extent of knowledge that is tacit. All of these factors need to be dealt with when switching vendors midway during an engagement.

• Business Disruption Risk If the customer is seeking to change vendors at the end of a contract, they would factor in the time required to get a new vendor on board so as to prevent disruption of business. However in case of mid-contract termination, the customer has to ensure that moving from one vendor to the other does not disrupt the business. The transition between vendors, even in a hostile scenario, has to be managed in a way that lets the business proceed as usual.

• Organizational Alignment Risks Changing a vendor in the course of a project requires the customer organization to work with a new vendor. Such changes are usually met with resistance in many organizations and needs to be resolved by the customer organization and the new vendor working together.

• Costs of exiting the current sourcing contract In the absence of a reversibility clause, the legal and consultancy charges could turn out to be huge for the client and can be a bottleneck for a smooth transition to a new vendor. These termination charges and unwind costs are a key consideration before taking the call for a new vendor.

Once the customer is aware of the costs and risks involved in switching IT vendors, the customer can take a call on the approach that needs to be adopted in changing vendors.



Approaches to Switching Vendors

Depending upon the context in which the vendor replacement occurs, there can be the different approaches to switching vendors. At one extreme is the scenario of a hostile transition from one vendor to another in the course of an ongoing engagement entailing compressed timelines. At the other extreme is the case of end-of-contract mutually-agreedon transition from one supplier to the other. This would allow the luxury of detailed transition, over longer time lines. Depending on such scenarios the following are some of the approaches that may be chosen:

• The Big Bang Approach
  
  In case a hostile vendor transition seems a possibility, companies opt for a big bang approach to facilitate a quick transition, especially in business critical engagements.
  
  The project time line is charted out keeping in mind the lack of co-operation the existing vendor may exhibit. Since there are chances that the current vendor shall be reticent in participating in the transition process, it is critical that all key stakeholders of client are taken into confidence.
  
  After having explored the possibility of poaching resources from the existing vendor the changeover is done in a short time span, leaving minimal chance for any planned hostility by the existing vendor. The incumbent vendor may initiate ways and means to prevent a smooth transition by trying to dismiss or transfer key stakeholders and developers to another project which might lead to the loss of important information –the new vendor and the customer need to be prepared for such an eventuality.

• The Phased Approach
  
  In the phased approach of new vendor transition the most business critical processes
  are transitioned on a priority.
  
  The customer chooses to transition in a phase wise manner, in which controls from the existing vendor and deliverables of the same are gradually taken over by the new vendor. Initially only the base subset of necessary operations is transitioned to the new vendor and the rest are gradually taken over depending on the priority of the processes and their business impacts.
  
  Given the time frame that this transition requires, it is imperative that the transition
  requires co-operation between the outgoing vendor and the new vendor.

Given these different approaches, the customer needs to make the right choice as to which approach is best suited for its current situation. The choice of the approach depends not only on the extent of co-operation that may be possible between the customer, the incumbent vendor and the new vendor, but also on the strategic importance of the functions being transferred.

For instance, if the functions that need to be transferred are business critical and there is a possibility that the co-operation from the incumbent vendor may not be complete, the customer may be better off using the Big Bang approach rather than using a Phased approach. This stems from the reasoning that the customer may want to get the functions transferred immediately and avoid any business disruption – opting for a Phased approach may result in delays in implementation owing to lack of co-operation, thus affecting the business continuity.


Recommended Framework for Transitioning

When switching vendors, the customer has to first decide the approach that is to be adopted and once that decision is made, the customer along with the new vendor must embark on getting the knowledge transferred from the incumbent to the new vendor. This does not pose an issue in the case of a mutually agreed upon, end-of-contract vendor switching scenario where total co-operation of the incumbent vendor can be expected. However, in cases where the co-operation from the incumbent is unlikely to be complete, transitioning can be tricky. In such scenarios, the customer, in tandem with the new vendor, must ensure that all required know-how is obtained from the incumbent vendo before the vendor exits.

Customers may choose to go about this process in different ways, depending on the exact scenario that exists. Described below is a recommended, 5-step framework, Analyze-Build Portfolio-Architect-Transition-Execute (ABATE), for transitioning, especially suited for cases where full co-operation from the incumbent cannot be expected.

• Analyze
  The new vendor must thoroughly analyze the current scenario and the customer vendor dynamics. This helps the new vendor understand the context, and the situation of the customer as well as that of the existing vendor.
  
  The new vendor must understand the reason the customer cites for switching vendors in a mid-contract situation. Getting an insight into these reasons would help in drawing up a comprehensive contract with the customer so as to prevent similar distasteful situations later on, once the contract is in place.
  

• Build Portfolio
  With an understanding of the ground situation from the earlier stage, the new vendor, along with the customer, must build a portfolio of the functions that need to be transferred from the incumbent vendor. This is critical as it determines the scope of the new contract that will be inked. In cases where the switching of vendors is taken up mid-engagement, the new vendor would also need to capture the exact state in which the different projects of the engagement are.
  
  As part of the portfolio building exercise, the new vendor, in conjunction with the customer, must also prioritize the different functions that need to be transferred. This prioritization must take into account parameters like the business criticality of the function and the state of the project.
  
  At this stage, the customer and the new vendor must also create a repository of all the project related documentation that can be obtained from the incumbent as this will help reduce the time required for transition.
  
  These details, along with the results of the analysis stage, would contribute to the contract that would be created between the new vendor and the customer.

• Architect
  The next step in the process is to create a contract between the new supplier and the customer. This new contract must be exhaustive and must imbibe lessons from the customer’s prior experience with the incumbent. The new vendor must aid the customer in drawing up a contract that has mutually beneficial safe guards for both sides.

• Transition
  Once the contract is in place, the next step is the actual transitioning of the knowledge from the incumbent to the new supplier. Ideally this phase would require certain degree of co-operation from the incumbent. In cases where there is low cooperation from the incumbent, the onus is on the vendor and the customer to ensure that sufficient know-how is gathered from the incumbent that will allow the new vendor to take over the functions.
  
  The new vendor can consider measures like employing project members from the incumbent vendor team, shadowing customer SMEs, use of checklists, scientific
  knowledge transfer processes, setting up knowledge repositories and continuous monitoring to ensure that all required know how is obtained before the incumbent
  vendor exits.
  

• Execute
  The last step in the process is the execution phase where the business processes outsourced are transitioned from the existing vendor to the new vendor. Metrics are
  defined in conjunction with the customer and these are monitored against the service
  level agreements set up in step 3 to ensure that the engagement is on track.

Transition Checklist

As part of the entire switching process there are a series of practices that the new vendor and the customer can adopt to ensure successful transitioning. Some of these are as follows:

• “Poaching” – The customer must examine its contract with the incumbent and if this does not restrain it from hiring the vendor employees, the customer must identify key vendor resources and get them on board. In some cases, the customer may be inhibited by contractual agreements from directly hiring the incumbent supplier employees. In these cases, the new vendor can step in and employ few important team members from the incumbent supplier organization so as to ensure knowledge continuity

• Infrastructure Set Up – The customer has to ensure that the right infrastructure set up is in place to allow for a fast and efficient transition from the incumbent to the new vendor. Unavailability of critical requirements like network access or communication links at this stage can set the schedule back by a fair extent. In cases where the incumbent vendor is not fully supportive and its exit date is non-negotiable, such a delay can derail the entire transition effort.
  

• Data touch points and Documentation – Important documentation like the manuals, data on the client systems, as well as the architecture standards for the customer’s environment etc should be obtained from the incumbent. The customer must help to identify and catalogue any such documentation that may exist. The complete repository of such documentation and other such system interdependencies are extremely useful for a successful transition to the new vendor.

• Secure Incumbent buy-in – The customer must take all possible steps to secure cooperation from the incumbent vendor for the transition. This could involve different
  measures like ensuring incumbent participation through exit clauses in the contract or
  putting in place incentives for the incumbent resources to be part of the transition.

• Debriefing – The customer must help identify the key resources who worked with the
  incumbent vendor or on the vendor’s application so as to get full details of the functions outsourced. The new vendor can then have detailed discussions with these key resources so as to fill in any gaps in the knowledge transitioned.

• Scientific Inputs to selecting resources – Many a time, customers make the error of
  assuming that resources who have spent more time in a function or on an application are the SMEs for that function/application. This has been proven to be erroneous and the new vendor must educate the customer that long term experience does not always mean increasing expertise. Ignoring this, may result in the new vendor spending time
  with customer personnel who may not be the key resources.

• Address Learning Styles – The customer must ensure that the new vendor is using
  the available team effectively during the transition phase.
  Different team members have different learning styles and recognizing this can help channel their efforts better. For example, identifying the “Accommodators” in the team and putting them onto trial fixes and enhancements while using “Divergers” for Application Demos would be better than sending the first group for Demos and assigning trail runs to the second.

• Transferring Software licenses – The customer along with the new vendor must identify the software that are required to be in place for the new vendor to take up the outsourced function and the rights of these must be transferred from the incumbent supplier to the new vendor, if the contract allows for this.

The above mentioned is not an exhaustive list since every scenario involving a change of vendors is different & unique. Hence, for some transitions this checklist may not suffice or be mapped to directly. The customer and the new vendor must work jointly to mitigate all the risks that may be posed by the change of suppliers and by having in place processes and systems like the framework described above, the transition can be made seamless and smooth.



Symbyo Framework in action

Symbyo has worked on a number of engagements where we replaced incumbent vendors – both mid-contract and also post-contract termination. In all of these engagements Symbyo successfully transitioned from the incumbent with minimal disruption of services to the customer.

HOW SYMBYO USE THE ABATE FRAMEWORK
The entire transition from an outgoing vendor to symbyo is done in an accelerated manner to meet the project deadlines.

• Analyze
  Timelines: Day 1 – Day 2
  
  A team from Symbyo meet with the customer and over a series of discussions analyzed the ground scenario. The team also meet with key members of the outgoing team to get their perspective of the situation. The inputs from these meetings are instrumental in determining the course of the project.
  

• Build Portfolio
  Timelines: Day 1 – Day 3
  
  TDuring these two days, Symbyo in conjunction with the customer prepare a list of functions and services that have to be transitioned from the incumbent vendor within the 2 week time frame.
  
  A detailed list of documents and artifacts that would help in transition is prepared and the customer take on the responsibility of obtaining these from the incumbent.
  

• Architect
  Timelines: Day 4 – Day 5
  
  With inputs from the previous two phases, the customer and Symbyo prepare a contract for the new relationship. This contract has in place exit clauses and other practices that were not followed in the earlier contract the customer had with the incumbent.
  

• Transition
  Timelines: Day 6 – Day 10
  
  This phase covers both the planning for transition and the actual transition.
  
  A comprehensive transition plan, spanning the transition of all components, is developed by Symbyo in collaboration with the client. This plan also detailed the transition process, roles, resources, task break up and timelines set up in collaboration with the client. The KT planning stage covers the following:

• Discussing and defining training inventory and schedule
• Pre-requisite skill set for Symbyo’s resources
• Quality control measures to track the status of KT

• Once the planning was completed and the plan was signed off by the customer and Symbyo, the actual transition was kicked off. The following are key highlights of the transition phase

• Transition properly planned and divided into different phases with Checkpoints in each phase, and clear deliverables. The different phases included

• Environment Setup
• Business Overview
• Technical Overview
• Database Overview
  

• The different components of the KT are

• Technical Environment Training for familiarization to technical
  environment, inventory collection, tools, utilities, commands etc.
• Business Training which consisted of business overview and functionality discussions for each business functionality and interfaces, user procedures and product training.
• Techno-Business or Application Training where technical sessions for business concepts were conducted.

• Formal documented transition sessions are organized involving vendor and Symbyo team with pre-decided daily and weekly deliverables.

• The system documents were updated and validated to ensure accurate knowledge transfer.

• Reverse presentations were made to the client at the end of each day. This ensured that the data/info provided by the contractors was validated and the gaps were identified and addressed at very early stages.

• Execute
  Timelines: Day 10 – End of engagement
  
  Post-transition, Symbyo completely replaced the incumbent and took full ownership of the outsourced function. Metrics were continually collected and analyzed to ensure that the terms laid out in the SLA were met. Once steady state was achieved, Symbyinitiated process improvement measures that helped deliver greater benefits to the client.
  

In the final analysis:

Switching vendors is a reality in the IT services domain and therefore it deems fit that, sooner clients and vendors take of this fact and prepares to tackle it, the better.

Customers need to understand the different risks that can affect their business as a result of inefficient vendor switching. The customer must also decide on the best approach that needs to be taken to go about the transition. This decision must take into consideration the strategic importance of the functions in balance and the extent of co-operation that can be expected between the customer, the incumbent vendor and the new supplier. The new vendor has a key role to play in the entire process – right from helping the customer in exiting the relation with the incumbent to forming the new contract. The new vendor also has the responsibility of helping capture the know how from the incumbent, using the best practices for the same.

Customers need to understand that switching vendors is not an easy task; it is daunting and tiresome. But at the same time, they must realize that it is not an impossible task. With proper planning and analysis, and backed by a strong vendor, the customer can certainly exit from an engagement with the incumbent and partner with the new vendor with no impact on the business.

Application Outsourcing Services

In tough economic times, businesses have three choices: ignore the situation, accept defeat, or take control. The first two options—doing nothing and riding out the storm or accepting the pounding—can be paralyzing, with devastating consequences. However, by taking control, businesses can stay ahead of competitors, grow market share, and ensure viability in all types of market conditions.

One way to take control is to consider all your options when it comes to your people, your processes, and your technologies. How can you make your people more productive? How can you streamline your business processes to minimize mistakes and maximize efficiency? And what technologies can help you achieve those goals quickly and cost-effectively?

With the evolution of Application Outsourcing services, companies have new choices when it comes to technologies, along with unprecedented opportunities. Instead of buying and setting up an extensive IT infrastructure—servers, networks, security, and licensed software, along with a staff to manage it all—customers who choose to outsourcing there applications management function of their IT department to a Software Outsourcing Company like Symbyo . And instead of the headaches associated with upgrades and new releases, customers simply decide which new features they want to use and take care of all the details. It’s all part of the larger picture of cloud computing pioneered by consumer companies such as Amazon.com and Google, in which people use the computing resources of other companies for their own purposes.

Symbyo Technologies is a pioneer in making this business model available to business users as well.

One of the advantages of cloud computing is that companies can focus on what they do best and on what gives them the most value for the smallest investment. Taking advantage of the cloud is one way to take control because it supports the following basic approaches. These approaches are always important to business success—and in difficult economic times, they’re critical.
:: Executing on business goals in a way that achieves revenue growth, while lowering costs and minimizing risk.
:: A strong commitment to supporting existing customers to expand the user base, ensure loyalty, and turn customers into advocates.
:: Smart and rigorous financial investments that focus on getting up and running quickly, with solutions that are flexible, support managers in making critical decisions, and make life easy for end users.

Within these approaches, there are key strategies for doing more with less, for empowering customers and partners, and for increasing the effectiveness of internal resources such as IT, sales, marketing, and customer service. This article explores how can support these strategies and how companies like yours have used Application Outsourcing to not only survive, but to thrive.

Implementation of ITIL-based processes

Overview

Until recently, internal IT departments have been relatively insulated from the corporate concerns of staying ahead of the competition. Certainly, IT has always existed in support of maintaining a competitive advantage, but that support has been largely tactical, given that IT has historically been viewed as a cost center, not a strategic business unit.

Things have changed dramatically in recent years as more and more businesses are moving out of a cost-centric view of IT’s potential — and into a realization that IT can transform business processes. This phenomenon is largely driven by increased competition in the marketplace, and the understanding that moving toward an IT-driven, customer-centric business approach isn’t a luxury — it’s imperative.

IT operations are increasingly expected to operate as a business unit, and with this expectation comes a slew of new challenges: improving performance, reducing operational costs, driving effective organizational change (via new processes and technology) to support IT’s bid to succeed in this new role and demonstrating the business impact of the department.

Yet, how does a legacy IT department transform itself into a business unit capable of delivering proactive, responsive service management across the organization? How does it provide not only the technology, but also the services, budgeting forecasts and metrics required to support the business goals of the company?

Failure can mean IT remains a nonstrategic cost center and its leader — the CIO — has no voice in strategic business decisions, or as is the trend recently, IT is outsourced completely. It’s a case of evolve or die, and the stakes are high for both the department and the organization it supports.

This article discusses how IT can function more effectively as a business unit, by using asset and service management solutions to implement and support IT Infrastructure Library® (ITIL®)-based processes for managing:

• • Configuration.
• • Incidents.
• • Problems.
• • Change.
• • Releases.
• • Service levels.
• • Availability.

Establish ITIL processes to align IT with business objectives

ITIL provides a nonproprietary, concrete framework for implementing service management best practices that are aligned with overall business objectives. Basing IT processes on ITIL guidelines enables organizations to more effectively manage IT changes, assets, personnel and service levels — going beyond simple IT asset management and service desk applications to deliver proactive IT business improvement. A well-implemented service can help:

• • Reduce the occurrence of IT failures.
• • Improve service levels and customer satisfaction.
• • Reduce fixed and variable costs.

This helps IT to develop credibility, improve performance, reduce costs and achieve business effectiveness and efficiency in the use of information systems. Moving toward a service oriented IT model is daunting but possible — especially given the best-in-breed service management software tools that are available and specifically designed to facilitate ITIL processes.

However, most ITIL-related offerings fall short in two important areas: resource management and service costs. If a solution has built-in capabilities for detailed analysis of labor, materials, and asset and service provisioning costs related to ITIL process activities, IT managers would have the information they need to support both a more effective service delivery process and ongoing service delivery investment decisions.

IBM Maximo® asset and service management solutions are built on a single, unified platform to support key IT business processes — enabling different groups to work together more seamlessly, generally free of data conflicts or duplication.

Two core products generate a single comprehensive view

Combining two core products — IBM Maximo Asset Management for IT and IBM Tivoli® Service Desk — Maximo asset and service management solutions provide a comprehensive view that helps:

• • Optimize IT processes.
• • Maximize return on assets.
• • Reduce risks and costs.
• • Improve service levels.

One of the keys to more efficient management of IT assets is knowing what the organization has and where it’s located. That’s why Maximo asset and service management solutions integrate with autodiscovery solutions such as Maximo Discovery to help an organization build and maintain information on deployed IT assets more efficiently. By incorporating this information into Maximo asset and service management solutions, customers can make more prudent investment decisions regarding technology resources and capital.

Even better, Maximo asset and service management solutions can grow with the organization as the ITIL processes are phased in. Each organization can implement Maximo software to create a more complete asset and service management solution — or choose to establish individual components according to a phased ITIL service delivery implementation. However the organization chooses to use them, Maximo asset and service management solutions integrate with most business systems, allowing each customer to work the way he or she wants to work.

Integrate ITIL processes with the IT environment

As described in the following sections, Maximo asset and service management solutions integrate seven ITIL processes from the ITIL Service Support and Service Delivery groupings. Consequently, they help organizations bridge what is sometimes an enormous gap between business and technology — and develop a superior service delivery approach to better meet internal and external customers’ needs, at a justifiable cost.

Optimize ITIL service support processes

Configuration management
Configuration management is the process of identifying, recording and reporting on all IT components in your infrastructure. The key to a successful configuration management process is the ability to discover, identify, verify and record all configuration items (CIs) and their relationships in a central or federated configuration management database (CMDB) and use this
as the official database of record to help maintain an accurate picture of
your IT infrastructure.

CIs comprise all components of the IT infrastructure that currently exist, or will exist in the future, in the IT environment — such as PCs, servers and network devices, software and software license agreements. A CMDB not only contains the attributes and history of each CI but also the relationships between and among them. Organizations that actively engage in a configuration management process benefit from:

• • Accurate and detailed IT asset information.
• • Greater peace of mind regarding software license compliance.
• • Better understanding of the potential impact of changes and fewer problems as the result
• of changes.
• • Expanded knowledge of budget needs.
• Maximo Asset Management for IT supports asset tracking, asset reconciliation, compliance management, contract management and procurement, helping the organization to:
• • Track IT assets, locations and changes and understand the relationship of assets to services.
• • Record and manage all contracts for software licenses, leases, warranties and maintenance.
• • Create and enforce technology standards.
• • Provide a streamlined process for procuring and receiving IT assets.
• • Reconcile deployed assets against authorized assets (those purchased and under contract).
• • Support other key ITIL processes such as incident management, problem management, change management, release management and service level management via a comprehensive, integrated asset database.

With Maximo asset and service management products, CIs are stored in a central database that is accessible to all — which helps avoid costly integrations. The products present a logical, current picture of the organization’s infrastructure and services by identifying, controlling, maintaining and verifying each version of existing CIs, as well as their relationships with each other and the customers they support — helping to improve service management processes.

Incident management

Incident management is the process of restoring normal service operation as quickly as possible to help minimize an incident’s adverse impact on business operations. In ITIL terms, an incident is any deviation from the expected standard operation of a system or service. Best-practice incident management involves immediate service restoration utilizing standard processes of investigation, diagnosis, resolution and recovery.

Tivoli Service Desk documents incidents from end users, service technicians and network systems management applications. Streamlining the process further, it leverages ticket types and classifications with powerful visual workflow escalation and e-mail notifications for quicker resolution, helping to meet customer expectations and improve service desk efficiency. Consolidation of user communication across channels — including phone, e-mail, Web and fax — captures each incident, creating a searchable knowledgebase that can vastly reduce staff response time to anomalies or outages if similar incidents reoccur. Incidents can be linked with appropriate problems or changes, and are associated with their related CIs in the CMDB.
Problem management

A problem is the underlying error in the infrastructure that is the cause of one or more incidents. Problem management is the process of diagnosing the root cause of the error and arranging for a correction. Furthermore, it seeks to prevent recurrence of incidents related to these errors. Effective problem management depends on IT’s ability to quickly and accurately determine the root cause and turn an unknown error into a known error — that is, problems for which the root cause is determined and attributed to a specific CI.

With Tivoli Service Desk, IT operations can more readily identify and classify the root cause of problems, assisting staff to quickly recognize and resolve known errors with minimal downtime. Built-in, real-time dashboards provide insight into all levels of service desk operations, so that any support staff, manager or executive can monitor role-based key performance indicators (KPIs) in an intuitive, graphical display. Dashboards provide actionable information and can identify potential problem areas, enabling IT to take appropriate corrective actions in most cases before critical services are adversely affected. Tivoli Service Desk enables the creation of changes from identified problems and ties appropriate incidents to these problems.

Change management

Change management is the process of ensuring that standardized methods and procedures are used for efficient and prompt handling of all changes — to help minimize the risk of change-related incidents and improve day-to-day operations.

In ITIL terms, a change is any action that alters the form, fit or function of one or more CIs. Most often, an authorized individual initiates a change via an approved request for change (RFC), which details the proposed change and includes both a justification and authorization for the change. Change management is vital to any IT organization that wishes to provide the highest level of service delivery. A finely tuned process enables improved stability of the IT environment, provides a clear audit trail for compliance and helps to maximize the efficiency of IT staff. In addition, a true change management process helps decrease help-desk incidents generated by random, unapproved or unmapped changes.

Change management is an essential process in the overall service delivery approach because it arms IT staff with the ammunition to respond to change, and to more successfully support the organization’s business goals. Change management should offer a road map for significant alterations to the IT infrastructure, thereby helping to reduce operational risk and decrease the time and effort of implementing the alterations.

Maximo asset and service management solutions offer comprehensive change management capabilities within IBM Maximo Change Manager, which is available with both Maximo Asset Management for IT and Tivoli Service Desk. Maximo Change Manager helps minimize the often overwhelming breadth of a change management process by parceling its components into smaller, more manageable pieces:

• • Tasks
• • Labor
• • Materials
• • Services
• • Tools

Maximo Change Manager automates requests and approvals, leveraging powerful visual workflow and escalations, and provides proactive service to help reduce outages. Thanks to a shared central database and unified design of Maximo asset and service management, IT staff can invoke change management from problem management, thus proactively planning for changes as part of an overall IT asset management process. Changes are automatically updated, and notifications of scheduled changes can alert support staff to actions that could temporarily increase the number of incidents. Additionally, Maximo Change Manager can identify and classify RFCs, and its workflow utilizes predefined processes for review and approval according to ITIL guidelines.

Release management

Release management is the process of ensuring that all aspects of a release, both technical and nontechnical, are considered together in order to
optimally navigate the release, and bridge the gap between application development and operations.

An effective release management process depends on the ability to ensure that only authorized and correct versions of software, hardware and other related

assets (training materials and documentation, for example) are available for use. This requires two key elements: the Definitive Software Library (DSL) and the Definitive Hardware Store (DHS). The DSL is a physical location that holds all original software in use throughout the organization, whether third-party or in-house. Similarly, the DHS (which may comprise one or several locations) securely stores spare hardware that has been preconfigured to meet the operating standards within the live environment.

Maximo Change Manager includes release management capabilities and simplifies the release management process by making available, anytime, all available information about approved software and hardware. Maximo Change Manager draws the information from a CMDB that serves as a virtual complement to DSL and DHS, and as the basis from which all technology releases are defined and deployed.

Maximo Change Manager includes built-in capabilities to identify and classify releases, and to plan and schedule their rollouts. Capabilities include defining individual rollout tasks and identifying the required personnel resources, materials, services and tools.

And, since all releases are changes, Maximo Change Manager creates logical associations between the two processes. As a result, it enables an organization to bundle changes (rather than execute similar changes one at a time) to help increase change efficiency and cost efficiency. Once releases are deployed, CIs are automatically updated in the CMDB.

Help improve ITIL service delivery processes

Service level management
Service level management is the process of maintaining and improving IT service quality through a constant cycle of establishing agreements, then monitoring and reporting on them to meet the customers’ business objectives.

Successful service level management depends on planning and implementing service level agreements (SLAs), or contracts between IT and its customers that guarantee a service deliverable in quantitative terms. The building blocks of SLAs are:

• • Operational level agreements (OLAs) that document all goals and metrics agreed on by internal IT groups working toward a common goal.
• • Underpinning service contracts that capture the metrics agreed on by IT and any of its external vendors.

Once defined and agreed on, SLA metrics must be actively monitored by both IT and the customer to ensure the commitments are met and to verify that service quality is cost-justified and gradually improved.

Maximo asset and service management helps manage service level operations and provides advanced processes for creating, managing and monitoring SLAs. It enables increased communication between IT and its internal customers and helps to align service levels with business strategies. For example, the service catalog feature allows IT organizations to more clearly define the services they will provide the business. They can then link assets, locations, contracts and SLAs to these services. Users can proactively monitor service levels via predefined key performance metrics (KPMs). Escalation management capabilities help manage resources properly to more consistently achieve service level commitments.

While SLAs are most closely associated with the service desk, service level management capabilities enable an organization to tie SLAs to other ITIL processes, enabling tighter management of configurations, changes, releases, problems and incidents. For example, it can be used to establish target response and resolution dates in incidents, problems, changes and releases, allowing for more agile service support and greater reliability in
daily operations.

Finally, Maximo asset and service management solutions can be used to establish reliability, capacity and availability commitments for assets, locations and services, assisting users to more proactively deliver critical business services.

Availability management

Availability management is the process of optimizing the capabilities of the IT infrastructure, services and supporting organization to deliver a cost-effective and sustained level of availability, to help the business meet its objectives. “Availability” tends to be a catch-all term that encompasses system reliability and resilience, maintainability, serviceability and security.
Availability is defined and promised within individual SLAs, but availability management moves a step beyond service level management in that it requires a thorough understanding of the IT infrastructure’s capabilities to deliver, and a sound process improvement loop to help optimize performance.

Both Tivoli Service Desk and Maximo Asset Management for IT utilize KPIs to calculate the following availability metrics:

• • Total availability and unavailability
• • Mean time between failures (reliability)
• • Mean time to repair (maintainability)
• • Vendor responsiveness (serviceability)

Escalation and workflow capabilities monitor and proactively notify managers of availability shortfalls and flag opportunities to improve. Specific availability metrics — such as downtime — can be analyzed using operational availability data provided by integrations to third-party solutions.

Maximo asset and service management solutions deliver extended benefits
Most comparable software solutions offer tools and processes to jump-start ITIL implementation, but they fall short in two important areas: resource management and service costs. While a robust system to manage IT assets and services is an excellent start on the journey toward a service delivery model, it’s not enough. An organization must also understand the impact such a model has on its labor, materials, assets and ongoing service costs. An IT organization’s ability to implement each of the seven critical ITIL processes and its understanding of how to best deploy resources both impact corporate service delivery effectiveness. Similarly, the ongoing management of costs associated with service provisioning also influences corporate service delivery effectiveness. Only when an organization has all of these capabilities will it achieve true unification among the business processes in IT.

To this end, in addition to its embedded ITIL service delivery principles, Maximo asset and service management solutions include core work management capabilities that allow organizations to granularly track reported costs associated with:

• Resources (labor, equipment).
• Tools.
• Materials (spare parts, consumables).
• Time.

Work management capabilities in Maximo asset and service management solutions support both reactive and proactive work activities and support mature work management processes in the IT department. Managers can track costs and set priorities based on service levels; they can also match job tasks to available resources and resource requirements, estimate and obtain approval of costs, establish priorities and initiate actions across the enterprise using the following features:

• • Tracking tools enable detailed analysis of resource usage and costs — helping decrease both internal and external labor costs.
• • Graphical work assignment manager helps optimize schedules and labor utilization, and assign
• the right person with the right skills to the right job.
• • Standard procedures functionality enables IT to streamline known processes and verify quality
• of work.
• • Analysis tools and KPIs help IT make informed decisions about resource and skills investments, according to the requirements expected to meet service levels.
• • Operating agreements help improve organizational communication and can be used to verify that other internal or external providers support service level commitments appropriately.

Conclusion

IT organizations face a substantially higher burden of proof when demonstrating their organizational viability and justifying new IT investments than they did just a few years ago. To prove its value to the rest of the organization and protect its position within it, IT must strive to become a proactive, business-driven entity. Moving toward an ITIL-informed service delivery model can be used to enable this transformation in ways that enhance IT’s ability to function as a value-added business unit.

Maximo asset and service management solutions help provide a smooth implementation of ITIL best practices, directly out of the box. Designed on a single, modern and agile platform, its processes are designed to work together and help minimize the cost of ownership. Maximo asset and service management solutions are part of Symbyo Service Management, which helps align IT functions with business objectives.

For more information
To learn more about how Maximo asset and service management solutions can help your organization implement ITIL processes across your infrastructure, contact your Symbyo representative or visit www.Symbyo.com

Winning outsourcing strategies -- How to increase value and reduce risk

1. Introduction

With today’s tenuous economy forcing organisations
to cut costs, whilst at the same time increasing the
value that they gain from efficient, secure business
practices, one key area in which they can increase the
value of their offerings is in developing specialised
software applications or services that supplement the
more general capabilities of commercial off-the-shelf
packaged applications. Such custom applications can
add value in numerous ways, such as allowing greater
collaboration with business partners, or improving the
efficiency of specific billing systems. This is leading to
continued rapid growth in the proportion of software
applications used by organisations that are developed
as one-off projects.
However, maintaining a large application development
and testing staff is costly and requires that specialist
resources be hired and retained. Because of this, more
and more organisations are choosing to outsource the
development of software applications to specialist third
parties that have experienced resources available, and
can use their expertise to develop applications faster
and, generally, at lower overall cost. However,
outsourcing is not without risk and requires careful
planning and control to ensure that projects run
smoothly and fulfil the requirements set.
This report aims to show how 200 of the very largest
organisations in their industries in the UK and the US
that are outsourcing significant parts of their software
applications development needs are handling their
outsourcing projects. Based on interviews with those in
charge of the outsourcing projects, this research aims
to uncover the processes that they put in place to
ensure that outsourced software application projects
deliver value, and how they drive risk out of the
projects. Further to this, questions were asked about
other fast-emerging forms of outsourcing, including
cloud computing and Software as a Service (SaaS). In
many of these cases, the main focus will be on the
writing of code that acts as “glue” between existing or
hosted services, or in the creation of functional
components, rather than an entire application being
written from scratch. This means that security is an
issue that must be considered in these situations also.
2. Drivers for outsourcing
As Figure 1 shows, the primary drivers for outsourcing
software application development across all
respondents are to speed up the development of
projects and to reduce the costs involved, followed by
the need to augment staff resources through access to
the specialist resources that are available through
outsourcers. Also, as comparison with Figure 2 shows,
these factors are a key consideration among those that
outsource the most—rather than doing projects in a
piecemeal fashion where specific skill sets are not
available for developing a certain application. This is
something that financial services organisations, in
particular, could learn from.
However, some industries have been faster than others
to embrace outsourcing as a means of adding value and
reducing costs across their application portfolios. As
Figure 2 shows, the use of outsourcing for software
development is currently greatest among public sector
and retail organisations, whilst transport and financial
services organisations are comparative laggards.

As Figure 3 illustrates, those industries that are
leading, in terms of the level of outsourcing they are
undertaking, have seen the greatest growth in
outsourcing over the past couple of years. This is also
borne out by qualitative insights from interviewees, a
significant number of which among those outsourcing
more than three-quarters of their application
development projects indicated during their interviews
that they had recently made the move to outsourcing
100% of their software development needs.

Over the next couple of years, the level of outsourcing
is still expected to expand, as Figure 4 illustrates.
Although the level of increase is likely to be lower
than previously, at least 20% of respondents in every
industry will see increased outsourcing activity.

3. Outsourcing can be a risky
strategy
With any outsourcing project, an organisation must
place its trust in the hands of its chosen partner. This
means that the organisation must trust that secure
coding best practices have been followed and that
applications have been developed with adequate levels
of security built into them—for example, ensuring that
a programmer cannot have placed a backdoor into an
application that could allow them to access that
application after it has been delivered, which could
lead to them carrying out a security exploit. However,
as Figure 5 shows, organisations are outsourcing even
those applications that are used to process and transmit
the most sensitive data, such as financial and human
resources applications.
Figure 5 also shows that it is organisations in those
industries that outsource the most significant
percentage of the application development needs that
fully outsource the most sensitive applications—and
yet these are the very ones who we will encounter the
fewest issues with their outsourcing projects. Among
public sector and retail organisations, respondents are
confident enough to fully outsource development of
any type of application included in the chart whilst, in the financial services sector, just 17.5% are happy to
fully outsource the development of financial
applications.
Does this mean that those that are outsourcing the most
are putting themselves at greatest risk? On the
contrary; throughout this research, answers to
questions indicated that those organisations that are
outsourcing the highest proportion of their application
development needs are putting in place the tightest
safeguards—in terms of requirements set out in
contracts, in defining what security tools and
procedures should be used, and in the level of testing
of applications that they are demanding of their
outsourcers. This provides them with the confidence
that they need to trust their outsourcers with even the
most business-critical software applications.
As Figure 6 indicates, their trust in their outsourcers
appears to be well founded. Having taken the trouble
to clearly define requirements upfront, respondents
from those industries in which outsourcing is most
prevalent have encountered fewer problems with
outsourcing projects going wrong. For example, just
22.5% of financial services organisations report that
they have experienced no problems with outsourced
application development projects, compared to 77.5%
of retailers. Conversely, 30% of financial services
organisations have had to take legal action against an
outsourcer as a result of a failed project, compared to
just 7.5% of retailers. In total, 17.5% of projects
resulted in legal action being taken but, as may be
expected, organisations in the US took legal action in
twice as many cases as their counterparts in the UK.
Overall, projects running over budget are the most
common problem, experienced by a full 43.5% of
respondents, rising to 61% of those outsourcing less
than half of their application development needs. As
well as this, 32.5% of all respondents reported that
projects had been called off completely owing to
problems—rising to 46% in the US.
Yet Quocirca does not believe that the types of project
being outsourced are inherently different—retailers are
dealing with the financial details of customers, just as
financial institutions are. The supply chains of retailers
are far more complex than those of finance, and the transaction volumes are generally higher. Therefore,
project complexity does not seem to be a factor in this.
The fact that legal action has been taken in some cases
demonstrates that there was a problem with the
outsourcing contract drawn up—giving the
organisation nothing to fall back on when problems
occurred in terms of an agreed-upon resolution and
escalation route. Outsourcing contracts must contain
specific requirements detailing vulnerability measures,
remediation cost recovery, and specific thresholds for
acceptable risk, in order to protect organisations from
potential harm. This is especially important for those
organisations with the least experience of outsourcing,
in order to avoid the creation of overly complex, oneoff
contracts that could contain loopholes which could
be exploited should the dispute need to be settled in
court. For example, by defining acceptance criteria that
include a list of specific critical vulnerabilities,
vulnerability classes, or that mandate a maximum
vulnerability risk level, the organisation can describe
the conditions that will result in the application being
rejected and returned to the outsourcer for remediation.
By doing this, an organisation protects itself against
the most common threats to its software and systems,
giving it legal recourse should the outsourcer refuse to
fix those vulnerabilities. In addition, requiring artefacts
that attest to the application’s security level puts the
onus on the outsourcer to either engage a security
certification clearing house or automated source code
analysis tool to provide reliable information. These
best practices, stipulated up front, reduce the likelihood
of legal action greatly.
Figure 7 takes a different slant on the same data—
comparing problems experienced with outsourced
development projects by the level of outsourcing
undertaken. This clearly shows that outsourcing does
not need to be considered as a risk—so long as the
correct processes are put in place. It would seem that
experience matters.
As Figure 8 shows, this has ramifications for
outsourcing providers themselves. If any project
should not go according to plan, those organisations
with the least experience of outsourcing are likely to
impose the stiffest penalties, adding to the costs
involved in a project for an outsourcer and reducing
their profitability. Such “stick with little carrot” contracts do not tend to work in reality—a contract that
majors on how resolutions are to be reached between
the two parties will always be a better bet. Therefore, it
is in the best interests of outsourcing organisations, and
not just the companies that are outsourcing the
business to them, to ensure that best practices are
followed in clearly laying out exactly what the
requirements are at the start of the project. This is
especially true for new customers, but also for new
projects with existing clients, where contracts
developed previously can more easily be repurposed.
In addition to this, the fact that more than 50% of all
respondents would move to another outsourcer clearly
shows the risk of project failure for outsourcing
providers. This would indicate that levels of loyalty are
low and do not necessarily translate to repeat
engagements. In order to engender loyalty and
differentiate themselves, outsourcing providers should
chase a mix of best overall value coupled with special
deals to attract and retain customers. Part of this could
be done through additional differentiation, such as
adherence to ISO standards for code testing to provide
a layer of assurance that the provider follows best
practices.
4. The importance of getting the
contract right
Prior to the start of any outsourcing project, it is
essential that organisations take time upfront to define
their requirements for the application to be developed,
including determining how business critical it is and
what levels of safeguards need to be built in to ensure
that the application delivered is secure. These
requirements will form the basis of any contract and
will be reinforced through any service level agreement
put in place.
When taken across the board, respondents to this
survey gave mixed results when stating what they
considered to be the essential goals that should be built
into the contract. The main exception was for
requirements related to staff at the outsourcing partner,
for which most were in agreement that stringent
requirements should be defined. Among those
outsourcing less than half or half to three-quarters of
their application development needs, fewer than 20%
of respondents in each category defined the other
requirements mentioned as essential. However, far
more consistency and emphasis was seen from the
“guru” group (those outsourcing more than threequarters
of their application development needs).
Since this group has been shown to suffer the fewest
problems and rarely experiences project failure, it can
be inferred that Table 1 shows the best practices that
should be adopted in any code outsourcing project.
No less critical than identifying the baseline goals of
the software and the required experience of the
developers, organisations must also define what
requirements they have related to application security
and the use of security tools and techniques in the
development of software applications by outsourcers.
These processes should then be written into the
contract to provide remediation assurances should the
security of applications fail to live up to the standards
set. Again, fewer than 20% of respondents outsourcing
less than half or half to three-quarters of their
application development needs are taking the trouble to
define these requirements and to specify the standards
needed in the contract. For this reason, the best
practices related to application development and
security requirements that should be built into
outsourcing projects (Table 2) are drawn from those in
the “guru” group—those with the most experience of
outsourcing of application development.
However, it should be pointed out that attestation that
secure coding best practices were followed, while
important, is by itself not enough. What is actually
needed is some level of certification that such practices
were followed, including detailed results that prove
this. There are many ways of testing applications that
will give repeatable, reliable metrics for this, including
the use of source code analysis, penetration testing and
manual code review.
In order to ensure that requirements have been
followed, organisations should not only ensure that
applications delivered are audited, but also should
specify how they require this to be done in the
outsourcing contract for each project. However, a
similar pattern emerges here in that those for whom
outsourcing is not a clear strategy are paying only lip
service to these needs (Figure 9). This places them in a
poor position in terms of verifying that the applications
will perform as required, without serious security
vulnerabilities present. It also increases the risk for
these organisations that applications will fail to
perform as they should.




Not only is the risk increased that their organisation
could be compromised by a security attack against
weaknesses in software applications, but it is also
likely that costs will be increased as such flaws ultimately cost more to put right. Should the flaw not
be caught early and result in a security breach that is
brought into the public domain, this could have a
further detrimental impact in terms of the
organisation’s brand reputation and profile.
The efforts taken by those in the “guru” group can be
seen in Figure 10. By building stringent requirements
into contracts and enforcing standards through service
level agreements and data handling procedures, those
outsourcing the most do not need to worry about the
level of control handed over to outsourcing partners in
terms of what types of data they are comfortable with
outsourcers handling, as Figure 10 shows. This can be
done by detailing such requirements as encryption for
data at rest and in motion, unacceptable vulnerabilities
and/or vulnerability classes, and the use of synthetic
and anonymised data for testing purposes.
Obviously, agreements have to be in place to ensure
that the outsourcing company adheres to the legal
requirements for handling any data provided to them—
in accordance with the laws that organisations, as their
customers, are party to. Therefore, the outsourcing
company must understand and be able to demonstrate
adherence and compliance to such areas as data
protection, data leak prevention, and so on.
Having taken the trouble to define what is required in
the contract, no respondents in the “guru” group feel
that handing confidential information over to
outsourcing partners is a risk as the appropriate
safeguards have been built into the contracts. This
demonstrates a solid trust relationship between such
organisations and the outsourcing companies—
something that both sides must strive to maintain. One
breach of process or security from either side breaks
the relationship too easily and, as the research shows,
with loyalty being low within the code outsourcing
market, organisations can easily decide to go
elsewhere.
5. Reducing risk through use of
appropriate security tools
As well as specifying requirements in the contract for
an outsourcing project, organisations must define
processes for the tools and techniques that they require
their outsourcer’s development staff to use. The organisation must also ensure that applications are
thoroughly tested for security before those applications
are delivered. Table 3 outlines the most important tools
and techniques that should be used by outsourcing
partners in developing software applications, again
according to those in the “guru” group. However, in
stark contrast to the answers to questions regarding
contract development, there is much greater equality
being seen among all respondents in the use of security
tools and techniques.
Whilst this means that they are taking secure
development processes seriously, the use of these tools
and techniques should be contractually required—not
just be expected.
Not all vulnerabilities are of the same risk to an
organisation. It is the responsibility of the organisation
to prioritise vulnerability types as to the risk they pose
the organisation. This then defines the most critical
vulnerabilities that must not be present in code in order
to reduce risk to their organisation. Again, there is
greater equality here amongst organisations according
to their level of outsourcing, but those in the “guru”
group are still outperforming the rest (Figure 11).

The vulnerabilities shown in Figure 11 are some of the
most common and are considered to be amongst the
most critical in terms of security. However, each
application developed is different and its design and
components could lead to a vulnerability not generally
considered to be a serious risk in most situations
leading to severe problems owing to the configuration
of particular code.
New vulnerabilities, attack methods and vectors also
become known at various points during the lifetime of
an application, and the outsourcer must be able to
demonstrate that they can deal with this. To gauge the
level of risk posed by each type of vulnerability
according to how it could affect a particular
application, risk-rating systems can be used to
determine how serious any flaws encountered are,
incorporating the level of risk that an organisation
would face should those vulnerabilities be exploited
(Figure 12). This provides an automated way of
proving that the most severe vulnerabilities are not
present in applications before they are accepted and
put into use.

6. Testing applications for
security
Since security should be a key criterion for acceptance
of a software application developed by an outsourcer,
organisations must require that initial security testing is
done by outsourcers. This should not, however,
absolve organisations from the need to test the security
themselves or to have independent validations done.
For one thing, this denies an outsourcing provider the
chance to claim that the application was signed off by
the organisation to which it is delivered in the case of
dispute. As shown in Figure 9, above, it is considered
best practice to write into the contract that the
organisation has the right to audit the application
before acceptance.
As Figure 13 shows, requiring outsourcers to perform
initial testing is a best practice that is not lost on the
most experienced outsourcers. Those still developing
their outsourcing programmes would do well to
emulate this, since performing tests in-house is a more
expensive option and requires that the organisation has
sufficient qualified resources to conduct those tests. As a general rule of thumb, testing in general, not just for
security, eats up around 25% to 30% of an application
development project in terms of time, resources
required and cost of testing.
Organisations outsourcing the development of software
applications should also define the most serious errors
for which the outsourcer should ensure that tests are
conducted. As Figure 14 shows, this is a requirement
set among the majority of organisations, although the
“guru” group still performs the best, reducing overall
risk of failure of applications delivered.

In order to ensure that applications are tested
thoroughly, including at different stages of the
software development lifecycle, organisations should
specify the sorts of security testing techniques that
must be used by their outsourcers.
As Figure 15 shows, the majority of organisations
interviewed take the trouble to specify what testing
techniques should be used, with the greatest efforts
being taken by those in the “guru” group. However,
this also means that the organisation doing the
outsourcing must clearly understand the benefit of each
of these approaches and what value they bring to the
overall testing process in order to be clear about what
they require to be used.
Finally, it is one thing to require that applications are
tested for security defects before they are delivered,
but how can the organisation that is outsourcing
development of an application ensure that those tests
have been carried out?
As Figure 16 shows, the vast majority of organisations
either perform their own tests or require some form of
validation of the testing results. This is positive,
although those that have not taken the trouble to define
what security tools and procedures should be used in
development, as well as how applications should be
tested by outsourcers, may find themselves with a
delayed project and, potentially, cost overruns as
application flaws are uncovered just when
organisations thought the application was ready to be
put into productive use.

7. Outsourcing to external
service providers
A fast-growing strategy being seen among
organisations today is that of the hosting and
management of their software applications to external
services providers. Figure 17 shows that the use of
these services is greatest among those vertical
industries that are the most reluctant to outsource the
actual development of software applications—namely,
financial services and transport organisations.




In these vertical industries in particular, organisations
are most likely to develop their own applications for
hosting and management by outsourced providers, or
to use services such as Software as a Service (SaaS).
Where organisations use external service providers to
host in-house developed code, they will often rely on
those partners or other third parties to provide add-on
code and other services that extend the value of the
software. With SaaS, integration with other internal
systems must be achieved, often requiring some level
of customisation or add-ons to code that could impede
the performance of other applications.
This survey looked to uncover the rigour that is used in
gauging the security of the services provided by such
outsourced service providers. Figure 18 shows that
security is a key criterion for many organisations that
are outsourcing their application hosting to external
service providers.
Although the results do not vary much by industry,
organisations in the “guru” group are applying the
most stringent requirements to outsourcing contracts—
namely, those in the public sector and retail
organisations—and still slightly outperform those in
other industry sectors.



This same pattern can be seen in Figure 19, which
shows that those in the “guru” group require the most
strict security architectures from their outsourced
service providers. However, financial services
organisations could do more to learn from their peers
in other industries and should consider beefing up their
requirements for security.
8. Conclusions
As this report shows, successful outsourcing benefits
from an awareness of the likely challenges and risks
that outsourcing can pose. For those organisations with
the most experience, success is much more likely, and
their established best practices can provide a roadmap
for others to follow. The stark differences in many
places uncovered between this group of leaders, and
those with less experience in development outsourcing,
demonstrates a major gap between what can be done,
and what many are actually doing.
Making use of the best practices detailed in this report,
based on the analysis of the responses from the “guru”
group, should provide anyone looking at outsourcing
development with greater peace of mind. Creating
upfront security requirements and deliverables, as well
as specifying the means through which they will be
measured, will lower risk and result in a more secure
work product, delivered on a more predictable
schedule, that will also require less maintenance over
time.
With the proper safeguards built into outsourcing
projects, organisations will be able to achieve their
ambitions of increased speed of software application
development and reduced costs—at the same time as
they shield themselves from the risk of project failure.
This will also allow them to build repeatable processes
that can more easily be repurposed to ensure the
success of subsequent projects undertaken.
These safeguards include not only taking the time
upfront to ensure that all expectations and
responsibilities are built into a workable contract, but
also that all applications have security built in from the
ground up and that they are tested for security as a
requisite for acceptance.
As well as this, organisations outsourcing application
development must ensure that they have the right to
audit the application, or have it independently verified,
and that remediation processes are in place for dealing
with any flaws uncovered.
Dealing with these processes is not only becoming
more important as application development
outsourcing increases, but also as more organisations
undertake newer, fast-emerging types of application
outsourcing where applications, including the data they
contain, are hosted by third parties, or where service
providers write add-ons to applications, such as is
increasingly happening with delivery mechanisms such
as Software as a Service (SaaS). Best practices gleaned
from more traditional outsourcing projects will be of
help in deriving greater value and reducing risk in
these types of services as well.