Tuesday, April 28, 2009

Mapping Applications to the Cloud

by Darryl Chantry

Summary: As economic pressure builds globally, many organizations are starting to look at Cloud Computing as a potential choice to reduce the total cost of ownership for IT. In searching for ways to use Cloud Computing technologies, enterprises have to ask what applications make good candidates for moving to the Cloud and which do not, such as "Does the nature of the business itself allow for Cloud Computing to even be considered?"
This article provides a broad overview into what Cloud Computing is and discusses an approach to mapping enterprise applications to Cloud Computing platforms to assist in determining whether your applications or your business model are a good fit for the Cloud.

 

 

Which Came First: The Cloud or Cloud Computing?

Cloud computing has fired the imaginations of many informationtechnology (IT) professionals around the world, whether they are small independent software vendors (ISVs), Silicon Valley startups, or large corporations that are looking to cut costs. There seems to be an everincreasing number of people who look to the Cloud to hit upon the magic bullet that will solve any IT problem.

One interesting aspect of the hype that surrounds cloud computing is the lack of a clear definition as to what cloud computing is and, just as relevant, what it is not. If you were to ask 100 people to define the Cloud and what they believe cloud computing is, you would probably get 150 different answers (some people tend to answer twice, with the first answer contradicting the second). With this in mind, it seems only fitting to begin this article by discussing a general definition for cloud computing.

The Cloud (or the Internet, if you prefer) has been around for some time now, about 25 years; so without a doubt, the Cloud came first, right? Well, one could argue that the first servers on the Internet were really storage devices for data and applications to be shared and run globally; or to put it another way, to provide cloud computing resources in multiple locations globally with almost infinite scalability. Contrast that to today’s cloud-computing initiatives that are pretty much there to provide data, applications, and computing power globally with almost infinite scalability, and you quickly see the difference… Or is there really a difference?

The difference is that we are using new technologies to put a new spin on old ideas. Cloud computing is more about evolution than revolution, with technology allowing price points that take these thoughts and make them available to all people — regardless of budget size — via a utilitybased, pay-for-what-you-use model.

Utility Computing

Utility computing refers to using computing resources (infrastructure, storage, core services) in the same way you would use electricity or water; that is, as a metered service in which you only pay for what you use. The utility can eliminate the need to purchase, run, and maintain hardware, server, and application platforms, and to develop core services — for example, billing or security services. Consider the following scenario.

A Web-based ISV that wants to makes components available for Facebook or MySpace faces the following dilemma: The components they create could be adopted by thousands, or could struggle to find acceptance in any form. Most ISVs have limited capital, so they need to balance the expenditure between developing their application and providing infrastructure to support their software.

Such balancing acts can lead to poor applications with good platform support, or great applications that are rarely accessible due to poor platform support. Neither scenario is a path to success; this is where utilitybased cloud platforms can help. Cloud utility platforms can provide a low-cost alternative that can easily scale to meet the demand for the ISV’s application, which allows them to commit practically all of their resources toward building a great application.

As cloud services are essentially available as a utility offering, should the product fail, the ISV can simply shut down the services and stop all costs associated with the software.

The utility model also allows organizations to offset some of the costs of running private data centers by providing additional infrastructure resources to manage peak loads; this is also known as cloud bursting .

Traditionally, to handle peak loading, organizations would often design data centers that had the processing power to manage peak loads; this meant that for the majority of the time the data center was underutilized. By using cloud bursting, an organization can build a data center to the specifications that will allow the entity to run all normal day-to-day workloads within their environment, and then use cloud providers to provide additional resources to manage peak loads.

Utility computing is often associated to some form of virtualized platform that allows for an almost infinite amount of storage and/ or computing power to be made available to the platforms users through larger data centers. The evolution of cloud computing is now broadening the definition of utility computing to include service beyond those of pure infrastructure.

 

 

Will All Applications Move to the Cloud?

Will all applications run in the Cloud? Should you attempt to port all of your existing applications to the Cloud? Should all your new applications be developed in the Cloud? What is this Cloud thing, anyway? These are a few of the questions that arise whenever you start thinking about using cloud services.

Some applications will be ideal candidates to be ported to a cloud platform, developed on a cloud platform, or hosted on a cloud infrastructure, while other applications will be poor cloud candidates. In this case, the standard architectural answer “it depends” can be applied to all of the preceding questions. Practically every application potentially could exist either partially or fully in the Cloud; the only caveat to this are the trade-offs in an application’s attributes — and, possibly, functionality — that you might be willing to make to move it to the Cloud.

The following pages discuss a few ideas for decomposing an application into its basic attributes, and decomposing the Cloud into its basic attributes, to help make decisions as to whether running your specific application in the Cloud is practical.

Mapping an Application to the Cloud

Every application is designed to fulfill some purpose, whether an order management system, flight reservation system, CRM application, or something else. To implement the function of the application, certain attributes need to be present: For example, with an order management system, transaction and locking support might be critical to the application. This would mean that cloud storage might not be suitable as a data store for such a purpose. Determining the key attributes of any application or subsystem of a larger application is a key step in determining whether or not an application is suitable for the Cloud.

 

Figure 1. Attributes map of an application

 

Figure 1 shows a number of key high-level attributes (blue column) that could be relevant to any application. The potential number of attributes for any given application does not need to be recorded; what you are attempting to determine is which attributes are the critical ones for your application. This will likely produce a manageable list of attributes that can then be mapped to the Cloud. Selecting Data Management, for example, presents a list of secondary attributes that provide more details for the high-level attributes. Selecting Access then allows you to specify if you want either online, offline, or both online and offline access to your data source.

Building on the Data Access example, you can start to see how this attribute could affect the choice as to whether or not to use a cloud provider for data storage. Should the application in question need purely online data, cloud storage would be an excellent choice; however, if offline data is all that is required, this could be a key indicator that the application is not suited for the Cloud. And if you decide that the application requires both an online and an offline model, the cost of developing the application to synchronize data between the application and the Cloud would need to be considered.

Choosing to support both an offline and online experience for the end users will add additional cost to the project; however, should another attribute, such as high scalability, be identified, the advantages that the Cloud provides in this area easily could offset the cost of developing an offline experience. (See Appendix A, Sample Application-Mapping Attributes, page 7.)

What Makes Up the Cloud?

After you have decomposed an application and determined its key attributes, you can begin work on a similar exercise for the Cloud — specifically, for cloud service providers. Splitting cloud attributes into broad categories can simplify the mapping process. The categories used in this example are cloud infrastructure, cloud storage, cloud platform, cloud applications, and core cloud services.

You could map any application attributes to cloud attributes in one or more categories, as depicted in Figure 2.

 

Figure 2. Mapping application attributes to cloud attributes

 

Cloud Infrastructure

Cloud infrastructure is infrastructure, or more commonly, virtual servers in the Cloud. Infrastructure offerings are the horsepower behind large-scale processes or applications. For large-scale applications, think Facebook or MySpace; for large-scale processing, think a high-performance infrastructure cluster that is running engineering stress-test simulations for aircraft or automobile manufacturing.

The primary vehicle for cloud infrastructure is virtualization; more specifically, running virtual servers in large data centers, thereby removing the need to buy and maintain expensive hardware, and taking advantages of economies of scale by sharing Infrastructure resources. Virtualization platforms are typically either full virtualization or para-virtualization environments. (See Appendix B for a more detailed explanation of virtualization, page 8.)

Cloud Storage

Cloud storage refers to any type of data storage that resides in the Cloud, including: services that provide database-like functionality; unstructured data services (file storage of digital media, for example); data synchronization services; or Network Attached Storage (NAS) services. Data services are often consumed in a pay-as-you-go model or, in this case, a payper- GB model (including both stored and transferred data).

Cloud storage offers a number of benefits, such as the ability to store and retrieve large amounts of data in any location at any time. Data storage services are fast, inexpensive, and almost infinitely scalable; however, reliability can be an issue, as even the best services do sometimes fail. Transaction support is also an issue with cloud-based storage systems, a significant problem that needs to be addressed for storage services to be widely used in the enterprise.

Cloud Platform

cloud platform is really the ability to build, test, deploy, run, and manage applications in the Cloud. Cloud platforms offer alternatives to these actions; for example, the build experience might be online only, offline only, or a combination of the two, while tools for testing applications might be nonexistent on some platforms, yet superb on others.

Cloud platforms as a general rule are low-cost, highly scalable hosting/development environments for Web-based applications and services. It is feasible (although an oversimplification) to consider cloud platforms as an advanced form of Web hosting, with more scalability and availability than the average Web host. There are pros and cons for any technology, and a con in the cloud platform world is portability. As soon as an application is developed to run on a specific platform, moving it to another cloud platform or back to a traditional hosting environment is not really an option.

Cloud Applications

cloud application exists either partially or fully within the Cloud, and uses cloud services to implement core features within the application. The architecture of cloud applications can differ significantly from traditional application models and, as such, implementing cloud applications can require a fundamental shift in application-design thought processes.

Cloud applications can often eliminate the need to install and run the application locally, thereby reducing the expenditure required for software maintenance, deployment, management, and support. This type of application would be considered a Software as a Service (SaaS) application.

An alternative to this would be the Software plus Services (S+S) model. This is the hybrid between traditional application development and a full SaaS implementation. S+S applications typically use rich client applications that are installed on a client’s PC as an interface into externally hosted services. S+S often includes the ability to interact with an application in an offline mode, and sync back to a central service when required.

Core Cloud Services

Core cloud services are services that support cloud-based solutions, such as identity management, service-to-service integration, mapping, billing/ payment systems, search, messaging, business process management, workflow, and so on. Core cloud services can be consumed directly by an individual, or indirectly through system-to-system integration.

The evolution of core cloud services potentially will mimic that of the telecommunications industry, with many services falling under the categories of Business Support Systems (BSS) or Operational Support Systems (OSS).

BSS services manage the interactions, with customers typically handling tasks such as:

  • Taking orders
  • Processing bills
  • Collecting payments.

OSS services manage the service itself and are responsible for items such as:

  • Service monitoring
  • Service provisioning
  • Service configuration.

Attributes Map for Cloud Services

By using the five cloud categories, we can now develop a set of attributes for each of the categories. These attributes can be used in two ways:

  • Mapping your application’s attributes to cloud attributes to validate whether cloud services are suitable for your application, and identifying which types of services to use
  • Evaluating cloud service providers as possible candidates for hosting your applications, identifying which types of services are available from your chosen provider(s), and then determining specific implementation attributes of the services offered.

 

Figure 3. Five cloud categories and attributes for cloud storage

 

Figure 3 shows the five cloud categories and a list of attributes for the cloud-storage category. Each cloud provider implements its cloud services in a slightly different way, with companies like Microsoft offering a number of different storage alternatives that developers can choose to use, depending on the required features, for a specific application.

Just like application attributes, cloud attributes must be weighed carefully when determining whether a cloud provider’s services are a good fit for your needs, as you will have to factor in implementation cost for each decision you make. (See Appendix A, Sample Cloud- Mapping Attributes, page 8.)

Overlaying the Cloud and Applications

Now that you have a complete understanding of the application and of what cloud services you could use to implement a solution, you can now start to make decisions about what the eventual architecture could be. Should you find that the Cloud is a viable and cost-effective alternative to traditional application architecture, the next step would be to choose the most suitable cloud provider(s) for the application.

It is quite possible that no single vendor would match completely with your requirements; however, you might find that you can obtain all of the services that your application requires from multiple vendors.

 

Figure 4. Single application that uses multiple cloud services and vendors

 

Figure 4 depicts an application that uses a number of cloud services, from multiple cloud providers. The preceding example could represent an application that is built in ASP.NET and is running on the Azure platform (cloud platform); however, the application also needs components with full trust, which means that the components can run only in a full virtual environment (cloud infrastructure). Data is stored in a Microsoft cloud (cloud storage), with services such as Workflow and Identity (core cloud services) also provided via Azure. The last requirement for the application could be a billing/payment service (core cloud services), which could be provided by another cloud provider.

Although this scenario is feasible, the costs that are associated with having accounts with multiple providers, using a number of APIs, and then integrating all of the services into an application could be impractical. The likely solution would be to find a single vendor that delivers the majority of the services that are required by your application, and use this as the base platform for a hybrid solution.

 

 

One CloudOne Cloud to Rule Them All

Is there one cloud, or are there multiples clouds? This is a debate that I have heard a number of times already. One side of this argument is public versus private clouds. Can a private implementation of cloud technologies be called a cloud at all, or is it something else? Are all of the public cloud offerings the same? And what about applications or systems that span both private and public clouds in a hybrid model? Where do these fit?

The honest answer is that the argument is irrelevant. Whether you subscribe to one theory or another, the desired outcome is the same: Build the most cost-effective system that you can that works. The previous section looked at ideas to help you make decisions; now, we will take a quick look at potential applications that could exist in the Cloud or as part of a hybrid cloud solution.

Architecting Solutions in the Cloud

This section describes three application scenarios for which solutions could be implemented by using cloud services. The following scenarios are by no means an extensive list of possible solutions that are suitable for cloud services; they are only indications of applications that could be feasible.

Ticketing System

When discussing the benefits of a cloud infrastructure, there seems to be a consensus that a concert ticketing system would make an ideal candidate for a cloud scenario (Figure 5). On the surface, this type of application looks like a viable candidate; ticketing systems are often subject to high demand over a short period of time (people scrambling to buy tickets for a concert or sporting event that will sell out rapidly). This is often followed by long periods of low to moderate activity.

 

Figure 5. Using cloud infrastructure for ticketing system

 

Ticketing applications are often overloaded during the periods of high demand, when the need for computing resources is extremely high. The ability to run up instances of virtual machines to cover such periods would be beneficial. There are, however, a number of issues that must be taken into account before architecting such a solution:

  • Ticketing systems are data intensive and highly transactional. Transactions can be required for the payments system as well as to reserve specific seats for a given event.
  • Personally identifying information is almost certain to be collected, with many customers having an account with the ticketing company or, potentially, wanting to create an account with the organization.
  • Validating credit-card payments can be time consuming, and is a potential source of bottlenecks in the ticketing process.
  • Some virtual cloud-server platforms cannot save state, which means that once a server image is shut down, all changes or additions to data that is stored within the image are lost.
  • Existing ticketing companies already will have significant investment in infrastructure and data management.
  • Depending on the cloud service that is chosen, virtual cloud-server images that are not able to save state will need to be recreated for every event, which could well result in a significant amount of work being required to prepare an environment for each new instance.

With all of this in mind, there are a number of ways to use a cloud service to reduce the demand on an existing system during a peak loading time:

  • Duplicate the internal system completely. This would require the most amount of work; once the application is ready to use, it still would need to be synchronized with the current system for every new instance. Permanently leaving the system on (even in a reduced capacity) could be expensive due to the cost of using a services platform.
  • Split the workload between internal systems and a cloud service in real time. This would involve splitting the process of selecting seats and purchasing tickets across the two environments; for example, the transaction could begin on the internal ticketing system where customers log into their account, select the event they wish to attend, select the seats, and then are passed off to a cloud service for final processing of payment. This would mean creating a virtual cloud server that simply completes the final stage of processing and, as such, would not need to be synchronized with the main system — effectively being a stateless processing engine. Only minimal data would need to be transferred to the cloud service, and credit-card information could be collected on the external system for single use and then deleted.
  • Split the workload between internal systems using batch processing. Similarly to the preceding processing method, this method would differ in that all personal information would be collected on the internal system, including Cc: details. This information then could be placed in a process queue and shipped in batches to the cloud service for processing. This would mean that should payment fail, a secondary process for contacting the person who is attempting to purchase the tickets would need to be implemented if it does not already exist.

The preceding solutions are examples of how a ticketing system could split processing between a cloud service and a company’s internal systems during periods of heavy use.

Photo/Video Processing

This example shows how you can combine multiple services (infrastructure, storage, and queuing) to provide a solution for data processing (Figure 6). In this scenario there are a chain of photo processing stores that make use of the cloud service to render or reformat digital media files.

 

Figure 6. Using cloud infrastructure for photo/video processing

 

The photo chain has a number of stores spread across the U.S. and wishes to centralize large image and video processing to reduce two aspects of the system: the amount of hardware in each store; and the complexity of maintaining and supporting the hardware.

When a customer comes into a store with a video that needs to be converted to a different format, the video file is first uploaded to a storage service, and then a message is placed in a queue service that a file is on the storage platform and needs to be converted to a different format. An application controller that is running computer instances receives the message from the queue, and then either uses an existing instance of a virtual machine, or creates a new instance, to handle the reformatting of the video. As soon as this process is complete, the controller places a message in the queue to notify the store that the project is complete.

The preceding scenario easily can be converted to an online experience, so that customers could upload files for processing without having to go to a physical location.

Web Site Peak Loading

The final example that I will use is that of a Web site that has an extremely high amount of traffic on an irregular basis, which makes it impractical to build out the hosting infrastructure to support such peaks (Figure 7). Such sites could be news sites with breaking stories, game sites announcing a new game, or movie sites showing trailers of the next blockbuster.

 

Figure 7. Using cloud infrastructure for peak load coverage

 

The solution to this scenario involves creating a complete copy of the company’s Web site, or the part of the Web site that will experience the heavy traffic, on a cloud-service infrastructure service. The copy of the site would be a static instance running across a number of Web servers that could be configured as either a load-balanced set of servers or as a cluster. You can make any changes you need to on the original Web site, and then synchronize them back to the cloud servers. This would create latency, but would greatly reduce the effort needed to maintain the Web servers and Web sites, and would eliminate the problem of maintaining state between the internally and externally hosted Web sites.

There are many ways to architect solutions for the preceding scenarios, as there are many more scenarios in which you can use a cloud service. The goal of this article is merely to highlight a few of the alternatives and uses for the services that are emerging.

 

 

Conclusion

The fascination of the Cloud and cloud technologies is driving many developers, ISVs, start-ups, and enterprises to scrutinize cloud services and assess their suitability for adoption. The promises of lower cost of ownership — and of almost limitless scalability, in both storage and infrastructure power — are hard to ignore. The promise of the Cloud definitely warrants inspection; however, you must manage the adoption of cloud services carefully and realize that not all applications are suited for the Cloud. Many applications will work in the Cloud; however, hidden costs of hosting some solutions in the Cloud could see projects being delivered with much higher development and running costs than would be true of more traditional and well-defined architectures and technologies.

 

 

Appendix A


 

 

Appendix B

Cloud-Infrastructure Platforms and Virtualization Types

One of the key enabling-technologies for cloud-computing platforms is virtualization, which is the ability to provide an abstraction of computing resources. When we look at cloud-infrastructure platforms as they stand today, they predominately come in two flavors: fully virtualized or paravirtualized environments.

There are many more variations to virtualization than the two that I have just mentioned; so, for this post, I thought that I would discuss some of the virtualization methods that exist and that could well find their way into a cloud-infrastructure offering.

Emulation

In this type of virtualization, the virtual environment emulates a hardware architecture that an unmodified guest operating system (OS) requires. One of the common instances in which you encounter emulated hardware is with mobile devices. Application developers will use an emulated environment to test applications that are designed to run on smart phones or on PDAs, for example. (See Figure 8.)

 

Figure 8. Emulated-virtualization environment

 

Pros: Simulates a hardware environment, which is completely different from the underlying hardware. An example of this would be a mobile device such as a smart phone emulated on a desktop PC.

Cons: Poor performance and high resource usage.

Full Virtualization

In full virtualization, an image of a complete unmodified guest OS is made and run within a virtualized environment. The difference between full virtualization and emulation is that all of the virtualized guests run on the same hardware architecture. All of the guests support the same hardware, which allows the guest to execute many instructions directly on the hardware—thereby, providing improved performance. (See Figure 9.)

 

Figure 9. Full-virtualization environment

 

Pros: The ability to run multiple OS versions from multiple vendors: Microsoft Windows Server 2003, Windows Server 2008, Linux, and UNIX, for example.

Cons: Virtualized images are complete OS installations and can be extremely large files. Significant performance hits can occur (particularly on commodity hardware), and input/output operationintensive applications can be adversely effected in such environments.

Para-Virtualization

In para-virtualization, a hypervisor exports a modified copy of the physical hardware. The exported layer has the same architecture as the server hardware. However, specific modifications are made to this layer that allow the guest OS to perform at near-native speeds. To take advantage of these modified calls, the guest OS is required to have small modifications made to it. For example, you might modify the guest OS to use a hypercall that provides the same functionality that you would expect from the physical hardware. However, by using the hypercall, the guest is significantly more efficient when it is run in a virtualized environment. (See Figure 10.)

 

Figure 10. Para-virtualization environment

 

Pros: Lightweight and fast. Image sizes are significantly smaller, and performance can reach near-native speeds. Allows for the virtualization of architectures that would not normally support full virtualization.

Cons: Requires modifications to the guest OS, which allows the OS to support hypercalls over native functions.

OS-Level Virtualization

In OS virtualization, there is no virtual machine; the virtualization is done completely within a single OS. The guest systems share common features and drivers of the underlying OS, while looking and feeling like completely separate computers. Each guest instance will have its own file system, IP address, and server configuration, and will run completely different applications. (See Figure 11.)

 

OS-virtualization environment

Figure 11. OS-virtualization environment

 

Pros: Fast, lightweight, and efficient, with the ability to support a large number of virtual instances.

Cons: Isolation of instances and security concerns around data are significant issues. All virtual instances must support the same OS.

Application Virtualization

Application virtualization, as with any other type of virtualization, requires a virtualization layer to be present. The virtualization layer intercepts all calls that are made by the virtualized application to the underlying file systems, redirecting calls to a virtual location. The application is completely abstracted from the physical platform and interacts only with the virtualization layer. This allows applications that are incompatible with each other to be run side by side: Microsoft Internet Information Services 4.0, 5.0, and 6.0 all could run side-by-side, for example. This would also improve the portability of applications by allowing them to run seamlessly on an OS for which they were not designed. (See Figure 12.)

 

Figure 12. Application-virtualization environment

 

Pros: Improves the portability of applications, allowing them to run in different operating environments. Allows incompatible applications to run side by side. Allows accelerated application deployment through on-demand application streaming.

Cons: Overhead of supporting a virtual machine can lead to much slower execution of applications, in both run-time and native environments. Not all software can be virtualized, so is not a complete solution.

 

 

Describing Solution Architectures

Pragmatic Approach to Describing Solution Architectures

by Mike Walker

Summary: The return on investment of technical documentation is often not realized and documentation is frequently looked upon as a necessary evil. Creating the right architecture descriptions can help guide decision making at the various stages of the IT life cycle, however, there are limitations on formalized structures, information models, and intelligent tooling that takes the current architecture documentation to the next level of usefulness. In this article, we look at how we view, approach, and maintain architecture descriptions, and consider how this process can be improved.

 

 

Since the dawn of information technology (IT), engineers and architects have created reams of documents to describe their solutions. These have been published in three-ring binders and placed in obscure places on bookshelves, eventually to collect dust and be forgotten — something with which engineers and architects have had to deal as a fact of life.

As time has passed, documentation has evolved from a listing of detailed methods and procedures to the separation of multiple aspects of the solution from the detailed APIs and user-interface (UI) design to the architectural aspects. Integration with developmental and architectural processes furthered this activity and, eventually, became a required and common practice. By doing so, governance processes in many ways restricted forward progress. In many ways, governance has created both justified and unjustified restrictions to forward-moving progress on designs and coding, if a set of documentation had not been completed. This has led to frustration and productivity blockers. Current architecture documentation does not live up to the promise of providing return on investment in the design process. Often, the design that is articulated in architecture documentation is not realized in the final implemented design that is hosted in a production environment.

This pervasive growth of documents shows how critical the information is to the support of the software-development life cycle (SDLC). We now see the same with the support of architecture efforts, too. There are a number of challenges that occur with this paradigm.

 

Figure 1: Documentation, increasing at an accelerated rate

 

As Figure 1 shows, we are adding more and more documents to an already large portfolio of documents that must be completed. In the past, this was manageable; now, however, documents can be bloated, and there is a higher probability of information being duplicated. Often, this is a result of tightly coupling a document with a process step.

The goal is to solve the deficiencies and challenges with current architecture documentation while preserving the aspects that do work and have been assimilated into the daily lives of architects.

To do this, we will explore the following concepts:

  • Alleviating challenges with current documents — Templates and new thought-provoking ideas will be introduced that challenge the existing ways of documenting architectures.
  • Living architecture designs — Designs often are static and do not have a life outside of the development life cycle. We will introduce ways to change that.
  • Enhancing decision support — Often, there are templates and checklists that give an architect a common way to think about problems. This is an accelerator to solving problems that have yet to be solved or identified.
  • Deriving to solutions — Given how the human mind works, writing down a design to a specific problem in “black and white” often shows gaps in current thinking.
  • Common means of collaboration — These provide a working information store to share and collaborate on architectures with team members.
  • Supportability and maintainability — Documents provide important information on how a system was built. This provides support personnel with vital information for solving postproduction issues. For architects, the understanding of current system architecture will allow them to build out a set of strategies for the enterprise.

 

 

Rethinking the Traditional Architecture Document

The most common and pervasive interface for creating architecture descriptions is Microsoft Office Word. Office Word is part of the Microsoft Office suite of productivity tools. With documentation, there are many other tools that play a role in the documentation processes.

 

Figure 2: Microsoft Office Word reduces complexity and cost

 

As Figure 2 shows, Office provides a way to reduce complexity and costs significantly. The Office suite is easy to understand and most users already have them on their desktops. Augmenting the existing Office tools to make them fit the usage of architects is particularly ideal. It maintains a consumable amount of complexity while limiting the overall costs. However, Office Word is not the only tool that is used in the architectural processes. There are many other tools that have roles to play in how we document our architectures. These tools include the following:

• Office Visio

• Office PowerPoint

• Office Excel

• Office SharePoint

It is important to understand the context in which these tools interact.

 

Figure 3: Other productivity tools playing a role in the documentation process

 

Figure 3 shows an illustration of how other productivity tools play a part in architectural processes.

We can assert quickly that there is not just one tool that used in the documentation of architecture; myriad tools are used for collecting information from or publishing to. If we want to solve the challenges with the process, we should keep this in mind.

 

 

Optimizing Office Word to Describe Architectures

The underlying goal is to change the role of Office Word from simply a word processor to more of a UI for designing architectures. Applying structured UI concepts to Office Word provides many benefits to the architecture document, including the following:

  • Structured content — Information can be better described in the document. We want to do this because of the challenges mentioned regarding how information does not integrate well with process or future design activities. One example is the process of importing a model into an architecture document. Often, we import a picture that represents a model of the specific viewpoint of an architecture. If we had an information model, we could specify that the model imported is indeed the logical model, instead of a generic image file.
  • Extensible — With a little more structure, information has meaning and definition. This makes the information extensible to other processes downstream.
  • Consumable — Ability to consume external content is also possible with a more structured interface. As an example, if you so choose, you could import external architecture information from other systems to automate your design efforts.

Through the use of the Office Word 2007 features that center on XML, we can truly start to extend this interface. Word does so by providing:

  • Embedding XML documents — embed XML file into document for full data qualification.
  • Office Word XML format — fully describes the formatting from the data through XML to have true separation of the formatting versus the information.
  • Content controls — map most content controls to elements in XML data attached to a document.

Building the new UI for the architecture documents is easier in Office Word. A series of out-of-the-box functions are provided in the Office Word interface. For the architecture document, we will use the built-in tools that Office Word provides to create this new UI, which will enable us to describe our information in a meaningful way.

 

 

The Architecture of the System-Architecture Document

To change fundamentally how architects use a system-architecture document, we must look at what services Office Word provides to add additional capabilities that will automate and create rich metadata integration services.

For this solution, we will use a real world reference architecture called the Enterprise Architecture Toolkit (EATK). This toolkit will show how to alter Office Word from a word processor to a UI for describing architectures in a new way. The Microsoft Office environment provides rich extensibility that will allow developers to extend in a meaningful way. To do so, an architectural approach will have to be taken. By separating out layers and capabilities, approaches and technologies can be identified to derive to the right solution.

 

Figure 4: Architectural layers

 

As Figure 4 shows, there are four architectural layers that expose discrete services for this solution. These include the following:

  • Platform — This is what the solution-architecture document and the integration services connect to. The platform components integrate with Office SharePoint Application Server environment for rich line-ofbusiness (LOB) integration.
  • Tool — The mechanism of Office Word is referred to here. The tool provides extensibility into the UI that provides Microsoft Office ribbons to execute tasks easily with a level of context and Microsoft Office task panes that extend the UI with additional entry or listings of information.
  • Document — The document provides the way in which a user can enter architecture descriptions. This is different in the EATK, as the document acts as the glue between the tool and the information itself. This is accomplished through an architecture-document template and the use of Office Word custom controls.
  • Information — The information in the document is managed completely different from a traditional document. All of the information that is typed into the document is linked back to an XML node behind the scenes. This fully qualifies what is typed. Not only is the information rich, but it is extensible.

 

 

Platform Architecture

A great deal of work has been done in building componentized add-ins at the tooling level in Office Word, but this is not enough to change fundamentally the architecture document into a tool for designing architectures. The components that are applied to Office Word build upon a larger architecture canvas. The EATK provides a server-side Architecture Portal and Architecture Meta-Data Repository (AMR). This architecture interacts with the document-level activities when we create an architecture design.

 

Figure 5: Architecture behind EATK system-architecture document

 

Figure 5 shows a logical representation of the architecture that the EATK provides. It leverages a series of Microsoft-platform technologies, which include the following:

  • Microsoft Office SharePoint (MOSS) 2007 — Used as an Architecture Portal, Document Management Services, and Workflow
  • Windows Workflow Foundation (WF) — The hosted workflow on the portal that interacts with the desktop application and add-ins
  • Microsoft Project Server — Can be used as the platform for interacting with project and portfolio data
  • Microsoft SQL Server — Used as the database platform for the AMR that the Office Word add-ins Patterns Browser and Architect Lookup will use to get their information
  • Microsoft IIS 7.0 — Used as the hosting platform for the AMR Services layer

The server components in the EATK play a part in how we change the role of an architecture document. All of the components on the server leverage platform capabilities of Office SharePoint, but change the context in which they can be used. The context is the architecture-development process. In doing so, we can take generic capabilities such as Enterprise Content Management (ECM) to automate how information is audited and versioned. The following are new interfaces and EATK components that were developed specifically for the architectural process and to interact directly with the system-architecture document:

  • AMR Web Services — Services layer that allows for programmatic interaction with the AMR. It provides extensibility into not only the AMR, but also other related services such as PPM or Application Portfolio Management (APM).
  • AMR Data Services — The base information services that delivers information such as patterns and existing IT assets to the Office Word task panes.
  • Document-Management Services — An Office SharePoint–based set of services that are used to manage documents. It comprises functionality such as check-in, auditing, versioning, integrating with workflow, security, and archiving.
  • Workflow Services — WF is used as the base of the workflow capabilities. Also, it is hosted on the server, which allows the architecting workflows that are applicable to the entire enterprise, instead of to just one architect.

 

 

Tool Architecture

This aspect of the solution is key to bridging the system-architecture document to both the platform for LOB integration and the document itself, which will be the interface in which architects will describe their solutions. All of the application logic is encapsulated in this layer.

Because this is the layer in which code is developed, this solution is dependent on Office Word API and other standard integration technologies.

 

Figure 6: Components of Office Word interface

 

Figure 6 shows the extended capabilities of Office Word. We have extended the following aspects of Office Word:

  • Ribbons — We use this functionality as a launch pad for downstream activities, workflow triggering, collaboration, and information retrieval.
  • Task panes — We can use this functionality for various aspects in which we want to create other interfaces from within Office Word.
  • Properties — Assigning metadata to a document

 

 

System-Architecture Document Ribbons

Using the ribbon for the system-architecture document, we will have a series of functions that relate directly to architectural processes and information. As Figure 7 shows, there are four ribbon components in the EATK:

 

Figure 7: Ribbon components of system-architecture document

 

  • Patterns Search — It displays patterns and existing IT assets that will help solve specific architecture-design questions and challenges.
  • Patterns Browser — It allows for surfacing the right patterns in a more intuitive way for solving a business problem.
  • Architect Lookup — Looking up an architect who is assigned to a project is streamlined with Architect Lookup by integrating collaboration via office communication server (ocs)
  • Upload Doc — Automatically have a way to integrate architecture information into hosted workflow and consumed into a metadata repository.

 

 

System-Architecture Document Task Panes

Task panes can be very useful to architects, as they can surface meaningful information at a moment’s notice with a click of a ribbon component. This information is not only visible, but also interactive.

Tangible ways in which these task panes empower IT architects are the following:

  • Systematic reuse — Many of the architectures that we build have repeatable patterns built within. By surfacing patterns in a composite manner, we can reuse the aspects and views of architectures in a systematic way.
  • Decision support — By reviewing existing architectures, we can review how solutions were developed based on a set of drivers. Not only can you review, but also you can contrast decisions on these architectures with the challenges of the current architecture efforts.
  • Automation — Not only can you view the patterns and assets in the task pane, but also you can apply them to your architecture design. By reusing models and architectural descriptions, you can automate the architectural process by eliminating unnecessary work.
  • Traceability of technology choices — Not only can you surface this information and apply it to your architecture designs, but now you can create relationships between what was imported and the architecture that you are designing.

As one example of this is the Patterns Browser. The intent of the Patterns Browser is to surface pattern information into the design environment. Two types of information are displayed:

  • Assets — Shows what has been built.
  • Patterns — Shows what should be built.

As Figure 8 shows, patterns can be applied to the elements of an existing architecture. In the preceding case, it is the reuse of a logical architecture model. Appling the selected model is as easy as double-clicking the pattern in the Patterns Browser. The pattern then is applied to the element that is selected in the document.

 

Figure 8: Interacting with elements in document

 

Another innovative way the Task Pane is utilized is by introducing collaboration. The architectural process is no longer a onearchitect job. Typically multiple architects with specific roles design systems architecture. Examples of these roles include; Application, Information, Hardware, Security, or Infrastructure architect.

Other roles in review processes will have an impact on the validity, quality, and level of completeness of the designs. This affects the architecture in a significant way, too. These roles usually are part of an Architectural Review Board of sorts. In this function, it is critical that the architecture document give this group of individuals the information that it needs to make decisions on the architecture. By introducing the collaborative aspects to the architectural process, we can reduce the number of issues ahead of time, instead of at the end of the process — thus, changing the architecture-design process from its current form of a reactive process to a proactive process.

 

Figure 9: Architect Lookup task pane

 

Architect Lookup would return the names and photos of the Hardware, Security, and Information architects. As Figure 9 shows, the Architect Lookup task pane reveals the architect resources, based on the project that is assigned to the architecture work. The way in which Architect Lookup determines this is by using the assigned project ID. The project ID is found in the meta-data of the document. This metadata can be entered either manually or through automated means when the architecture-design request is sent.

Architect Lookup will take the project ID and perform a query against multiple systems to return a set of results. Project Server (or a custom information store) can be used to retrieve the list of architect resources, while Office SharePoint and Active Directory are used to take the resources in the PPM repository and correlate them with the images of the person, along with links to IM and personal portal sites.

Use of Architect Lookup eliminates the need to go to project plans, portals, or file shares to find this information manually. It is a simple click away to get instant access to other resources on the project, to get answers to questions.

 

Figure 10: Collaborative architectures

 

Each role has a part to play in the architectural process. Given the illustration in Figure 10, we see that an Application architect might start the process, but then might need the aid of other architects. In this case, the Application architect might be the owner of the document and be the primary contributor. The other roles that are shown — such as Hardware architect, Security architect, and Information architect — are some that would interact. These roles will validate, add, and modify architecture-design descriptions.

In this context, a sample of common questions to these roles would look like the following:

  • How will the security model affect how I design my application?
  • I know that I must design an external portal, but what does the servertier model look like for the DMZ?
  • What will the network stack look like, and how will that affect performance or scalability?
  • What is the right hardware?

 

 

Document-Template Architecture

Along with enhancements to the tool, the next layer of concern is the document template.

The document-template layer does not have a great deal of intelligence to it; however, it does allow interaction from Office Word to the document. One example is that the EATK allows information from an external source to be “clicked and dragged” into the document. This pulls data from a database and populates the document with architecture models.

The EATK provides not only the Office Word add-ins that are described in the previous sections of this article, but also a system-architecture document template, as Figure 11 shows:

 

Figure 11: System-architecture document template

 

The views that are listed in the table of contents are a set of common views. These views can be renamed or removed, and additional views can be added. These views provide a starting point for architects. Whereas you can use the system-architecture document template as your corporate standard in your company, it will be common practice to modify it.

With many documents that are surrounded by process, we want to create templates to ensure that they are used in a consistent way. This is the same case with the system-architecture document template. A template for the architecture document will allow for:

Everyone using the right version of the architecture document.

  • Ensuring that the information models stay intact.
  • Allowing for information to be generated by the system.
  • Enabling information to be consumed by downstream processes.

 

 

Information Architecture

The last piece of the puzzle is the information itself. Having great add-ins and template is great; however, without a meaningful way to express and ensure that the information is long living in a connected process, it is not as useful as it could be. Without information architecture, it would be difficult to qualify architectures.

The information layer is the base of the solution; it is where the majority of the consideration is made. All layers interact with each other in some way, but the information layer is connected directly to all. It is consumed in workflows, information-entry processes, and automation through Microsoft Office ribbons and task panes.

By using industry-standard techniques to derive to the target information architecture. Two fundamental aspects of the EATK that we must explore are the following:

  • Ontology — We want to define a set of terms that are understood commonly within the enterprise. By doing so, we can relate information properly and consistently.
  • Taxonomy — This will allow you to correlate architecture information with other aspects of architecture.

The architecture document should use the terms in the proper usage to qualify what the information is. Publishing an online ontology mapping will be useful toward understanding the information within the document.

In the context of the system-architecture document, ontology provides agreed-upon terminology for architecture. As an example, it would define the meanings of a platform, system, subsystem, application, and components.

Defining what these architecture elements are, however, is one piece of the puzzle. How these elements relate to each other is the next logical step. We will use taxonomy for this. The EATK uses an industry standard from IEEE to solve this challenge. IEEE 1471 is used as the basis for the taxonomy and ontology of the system-architecture document.

IEEE 1471 is the first formalized standard for describing architectures and how they relate to other aspects of the software-development process. It leverages a scalable meta-model that shows the relationships between common elements in the architecture-development process.

 

Figure 12: IEEE framework for architectural description

 

In Figure 12, IEEE 1471 provides a meta-model that allows us to relate architectures with other aspects of the software-development process. The system-architecture document focuses on specific areas of the taxonomy, while other EATK components focus on other aspects of the standard.

The aspects that are implemented to support IEEE 1471 in the systemarchitecture document are the following:

  • Structured content — An architecture schema that represents the information that we want to gather in our architecture document was created that has a viewpoint-based model applied to it.
  • Qualifying information — Links between decisions that have been made and other systems or architecture descriptions are built into the schema. As a user enters information or applies patterns to the architecture document, it will correlate that information by unique ID.
  • Publishing mechanisms — Provides facilities to store information from the schema, so that it can be related to other non-document–related information.
  • Generating information — Provides a mechanism to rebuild or generate sections of an architecture document.

 

 

System-Architecture Markup XML

With an ontology and taxonomy for architecture, we now can look at what this means from an implementation perspective. The system-architecture document has an underlying XML structure that is used to describe the information in the form of XML.

The System Architecture Markup XML is a way to provide clear separation between the presentation markup and the information markup.

This greatly simplifies integration to hosted workflow, repositories, and third party tooling.

Conclusion

In this article, we reviewed how to change fundamentally the way in which we view, approach, and maintain architecture descriptions. There is no doubt that there could be significant value in capturing information about the designs of our architectures that will help guide decision makers through the processes of making the right architecture decisions. However, this is not always the outcome, as often no formalized structure, process, information model, or intelligent tooling is bundled with this process.

Key takeaways include the following:

• Architects who want to implement these concepts can use the EATK, which provides a set of templates, processes, and a solution accelerator that can be used to accelerate this implementation for architects.

• Architects can achieve significant productivity gains through reduction of manual activities and process automation.

• Architecture information can be used in a much more meaningful way by eliminating the document graveyard effect, by integrating architecture descriptions with an AMR.

• The quality of decision making can be improved by eliminating points of duplication; thus, information quality can be increased significantly.

• Wikis and modeling tools complement this implementation—or, in some cases, replace it.

• Solutions such as COTS and custom-developed applications can be integrated with this solution through standard integration technologies.

About the Author

Mike Walker is a principle architect who delivers strategy for the enterprise architecture space at Microsoft. He is responsible for creating, driving, and evangelizing Microsoft’s worldwide Enterprise 2.0 and Enterprise Architecture strategies. Specifically, Mike ensures that institutions around the world realize the full extent of Microsoft’s vision and value proposition. He has also evolved many of the concepts behind mainstream architectural approaches and styles. His works are realized through publication in books, trade publications, authoritative architecture guidance, articles, code delivery and standardized frameworks. Those frameworks include: the Enterprise Architecture Toolkit (EATK), Loan Origination Reference Architecture, and Financial Services Component Library. You can visit his blog a thttp://www.MikeTheArchitect.com