Author: escapebusinesssolutions

Deploying highly available and secure cloud solutions

December 2012

footer left page.jpg Deploying highly available and secure cloud solutions

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.

This document is provided “as-is.” Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it.

Copyright © 2012 Microsoft Corporation. All rights reserved.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Authors and contributors

DAVID BILLS – Microsoft Trustworthy Computing

CHRIS HALLUM – Microsoft Windows

YALE LI – Microsoft IT

MARC LAURICELLA – MicrosoftTrustworthy Computing

ALAN MEEUS – Windows Phone

DARYL PECELJ – Microsoft IT

TIM RAINS – Microsoft Trustworthy Computing

FRANK SIMORJAY – Microsoft Trustworthy Computing

SIAN SUTHERS – Microsoft Trustworthy Computing

TONY URECHE – Microsoft Windows

footer left page.jpg Table of contents Executive summary ……………………………………………………………………………………………………………….. 1 Introduction ………………………………………………………………………………………………………………………….. 3 Measuring reliability and user expectations …………………………………………………………………….. 4 Service-oriented architecture ………………………………………………………………………………………… 4 Separation of function …………………………………………………………………………………………………….. 5 Automatic failover……………………………………………………………………………………………………………. 5 Fault tolerance …………………………………………………………………………………………………………………. 5 Disaster planning …………………………………………………………………………………………………………….. 6 Test and measure …………………………………………………………………………………………………………….. 6 Cloud provider ……………………………………………………………………………………………………………………… 7 Cloud provider expectation and responsibility ……………………………………………………………. 7 Cloud availability ……………………………………………………………………………………………………………… 8 Design for availability ……………………………………………………………………………………………………… 8 Organizational customer of the cloud ……………………………………………………………………………… 11 The organization’s responsibility ………………………………………………………………………………….. 11 Availability of sensitive information stored in the cloud ……………………………………………13 The user and the device used to access the cloud …………………………………………………………15 User expectation and feedback …………………………………………………………………………………….15 Design for test ………………………………………………………………………………………………………………….16 User device availability ……………………………………………………………………………………………………16 Conclusions …………………………………………………………………………………………………………………………..19 Additional reading ……………………………………………………………………………………………………………… 20 Executive summary

Many organizations today are focused on improving the flexibility and performance of cloud applications. Although flexibility and performance are important, cloud applications must also be available to users whenever they want to connect. This paper focuses on key methodologies that technical decision makers can use to ensure that your cloud services, whether public or private, remain available to your users.

At a high level, each cloud session consists of a customer using a computing device to connect to an organization’s cloud-based service that is hosted by an internal or external entity. When planning for a highly available cloud service, it’s important to consider the expectations and responsibilities of each of these parties. Your plan needs to acknowledge the real-world limitations of technology, and that failures can occur. You must then identify how good design can isolate and repair failures with minimal impact on the service’s availability to users.

footer left page.jpg This paper showcases examples for deploying robust cloud solutions to maintain highly available and secure client connections. In addition, it uses real-world examples to discuss scalability issues. The goal of this paper is to demonstrate techniques that mitigate the impact of failures, provide highly available services, and create an optimal overall user experience.

footer left page.jpg Introduction

Customers have high expectations for the reliability of computing infrastructure, and the same expectations apply to cloud services. Uptime, for example, is a commonly used reliability metric. Today, users expect service uptimes from 99.9% (often referred to as three nines) to 99.999% (five nines), which translates to nine hours of downtime per year (at 99.9%) to five minutes of downtime per year (at 99.999%). Service providers frequently distinguish between planned and unplanned outages, but, as IT managers well know, even a planned change can result in unexpected problems. A single unexpected problem can put even a 99.9% service commitment at risk.

Reliability is ultimately about customer satisfaction, which means that managing reliability is a more nuanced challenge than simply measuring uptime. For example, you can imagine a service that never goes down but that is really slow or that is difficult to use. Although maintaining high levels of customer satisfaction is a multifaceted challenge, reliability is the foundation upon which other aspects of customer satisfaction are built. Cloud-based services must be designed from the beginning with reliability in mind. The following principles of cloud service reliability are discussed in this paper:

. Use a service-oriented architecture . Implement separation of function . Design for failure . Automate testing and measurement . Understand service level agreements

footer left page.jpg Measuring reliability and user expectations

In addition to uptime, which was discussed earlier, other reliability metrics exist that should be considered. A common measurement of computer hardware reliability is mean time to failure (MTTF). If a component fails, the service it provides is unavailable for use until the component is repaired. However, MTTF only tells half of the story. To track the time between failure and repair, the industry created the mean time to repair (MTTR) measurement. To calculate an important metric of service reliability, we can use the equation of MTTF/MTTR. This equation shows that reducing the repair time by half will result in a doubling of the measured availability. For example, consider the situation of an online service that has historically demonstrated an MTTF of one year and an MTTR of one hour. In terms of measured availability, halving the MTTR to a half hour is equivalent to doubling the MTTF to two years.

By focusing on MTTR, you can mitigate the potential impact of failure incidents and seek to improve reliability by creating a set of standby servers with a sufficiently redundant design to hasten recovery from such incidents. You should always document these types of mitigations in a service level agreement (SLA) from the cloud provider. By documenting them you are implicitly acknowledging that some amount of failure is expected to occur, and that the best way to minimize the impact from failure is to increase the MTTF and reduce the MTTR.

The following sections detail some of the key architectural requirements of designing highly available cloud-based services.

Service-oriented architecture

Effective cloud technology adoption requires appropriate design patterns. In a service-oriented architecture, each component should have a well-designed

footer left page.jpg interface so that its implementation is independent of every other component and able to be used by new components as they are deployed. Designing architecture in this way helps reduce overall system downtime, because components that call into a failing component can properly handle such an event.

Separation of function

Separation of function, also known as separation of concerns, is a design pattern that states that each component will implement only one or a small set of closely related functions with no overlap and loose coupling to other components. The three-tier architecture shown in Figure 1 later in this paper is a classic example of separation of function. This approach allows functionality to be spread across different geographies and networks so that each function has the best chance to survive failures of specific servers. The figure depicts redundant front-end web servers, message queues, and storage, each of which could be separated geographically.

Automatic failover

If the component interfaces are registered with a uniform resource identifier (URI), failover to alternate service providers can be as simple as a DNS lookup. Using URIs instead of locations for services increases the likelihood that a functioning service can be located.

Fault tolerance

Also known as graceful degradation, fault tolerance depends on aggregating the building blocks of a service without creating unnecessary dependencies. If the user web interface is as simple as possible and decoupled from the business logic or back-end, the communications channel can survive failures of other components and maintain the organization’s link to the user. In addition, the web interface can be used to inform the user of the current status of each piece of the organization’s cloud-based service. This approach not only helps the user understand when to expect full restoration of services, but also improves user satisfaction.

footer left page.jpg Disaster planning

You should expect that services will fail from time to time. Hardware failure, software imperfections, and man-made or natural disasters can cause service failure. You should complete planning for routine problems before deployment to help troubleshooters know what to look for and how to respond. But even huge environmental disruptions, sometimes known as black swan events, will occur periodically; therefore, you need to consider such events during the planning process. The black swan theory posits that these unlikely events collectively play vastly larger roles than regular outages.

Test and measure

Two types of test and measurement of a running service are appropriate. The automated polling of the service by a test server can result in early detection and reporting of failure and thereby reduce the MTTR.

User research should be conducted either immediately before or after a deployment to understand how users react and identify unmet expectations. An easy way to obtain user feedback is to simply ask them for it, not every time you see them but occasionally. You should be able to get useful data, even with a very low response rate, and you will also obtain key performance indicators (KPIs) from users for monthly status reports.

footer left page.jpg Cloud provider

Two concepts from the 1970s have been realized by new technologies in today’s cloud offerings.

. Virtualization of computer hardware is a reality, with virtual computer images and virtual hard drives that can be remotely managed. . Fast scaling and agility are realities, with management tools that can control the power of physical and virtual hardware.

It’s important that IT professionals understand these concepts and their powerful capabilities. As stated in the “Executive summary” section, the concepts in this paper apply to both public and private clouds, each of which has a place in the toolbox of forward-looking IT departments. The focus from the start of any project should be on cloud design and management services that provide minimally disruptive service delivery to users.

Cloud provider expectation and responsibility

A natural shared responsibility exists between any organization and its chosen cloud provider. For custom applications, the cloud provider designs its service for reliability based on the use of certain features, such as failover and monitoring, by the developer. The developer must understand and use these features for reliability to be an achievable goal.

An organization that implements a solution on top of cloud-based infrastructure must ensure that the service is available as much as possible. Users expect these types of services to be as reliable as a telephone. Outages may occur, but they are rare, localized events. The organization’s ability to provide such assurance requires transparent communication with the provider about what to expect from the service and what must be supplied by the service consumer. Without such transparency, finger-pointing about responsibility can occur instead of automated recovery when service failures lead to outages for users.

footer left page.jpg The same consideration applies to private clouds. An IT organization can partition infrastructure responsibility to in-house experts who then create a private cloud. The cloud service is expected to provide a reliable platform on which the rest of the IT team can create innovative solutions to address the business needs of the organization. But just as for an outsourced cloud service, fully transparent operations and documentation of expectations will help avoid failures that result from vague or poorly defined areas of responsibility.

Cloud availability

Typically, public cloud services provide high availability by using a geographically distributed and professionally managed collection of server farms and network devices. Even very large enterprises that have private clouds for specific high-value content can profitably use public cloud services to host their application solutions. And large public clouds have effectively infinite capacity, because they can respond with more servers when demand is greater than anticipated. Offloading the excess server capacity has multiple benefits, the most prominent of which is that, in most public cloud models, excess capacity is not billed until it is used.

Many cloud providers offer built-in capabilities for increased availability and responsiveness, including:

. Round-robin DNS . Content distribution networks . Automated failover . Geographic availability zones

Design for availability

As stated earlier, the most effective way to increase availability is to shorten the MTTR. If geographic and network diversity is available from the cloud, ensure that load balancing automatically routes users away from failed components to working components. Even a relatively simple capability such as network load balancing can be affected by unexpected interaction between the organization

footer left page.jpg and the cloud provider or DNS. Such interactions have the potential to introduce instabilities in the service offering that have not been anticipated.

For example, distributed denial of service (DDoS) attacks are external attacks against availability that all cloud services need to mitigate. However, unless mitigation is carefully implemented, organizations with little security experience can unintentionally cause an application to become unavailable, which can result in as much downtime damage as a DDoS attack. DDoS mitigation is an example of a capability that is most effectively provided at scale—that is, by the cloud provider or the ISP.

Most major cloud vendors are certified for reliability and security, which they report in documents such as those contained in the Cloud Security Alliance (CSA) Security, Trust, and Assurance Registry (STAR).1 Although STAR itself is relatively new, it’s important for IT managers to consider that most in-house systems have not received such third-party vetting. Obtaining such assurance at a shared cost is another potential benefit of public cloud services.

1 Security, Trust and Assurance Registry (STAR), at https://cloudsecurityalliance.org/star/

When evaluating the benefits and challenges of creating a highly available cloud- based solution, it is important to ensure that your design includes a threat analysis of well-known problems such as those defined earlier, as well as any business- disrupting failures that are unique to the solution you want to deploy. Typically, only security attacks are considered in a threat analysis, but a well-designed cloud solution will consider other types of loss of availability and plans for mitigations as well.

The following figure shows a generic three-tier design with redundancy capability. Each request has more than one path to a component that can respond to it. Starting from the left is a user device, such as a laptop computer, from which the request is routed (again, consider the use of round-robin DNS and network load balancing features, if available). In tandem, an automated availability test service is exercising as much of the system as possible to assure that any failure is quickly reported so that necessary repairs can begin quickly.

The cloud service in the figure shows separation of function. This separation helps ensure that the network path to the user has no common components that result

footer left page.jpg in failure of both paths. Within each cloud service site, additional component redundancies are possible. In this scenario, the queue may be provided by the cloud service provider. If a common database needs to be shared between both sites, it is up to the organization’s application architecture to route traffic to the data in a way that will fail over to some other site running a mirrored copy of the database; however, many public cloud offerings have built-in data redundancy capabilities that should also be explored.

In this example solution, load balancing is split between the Internet at the front end, the cloud service in the middle tier, and the organization’s application at the back-end. Tests of disabling each of these components should be undertaken with live loads to assure that fail-over options work as planned.

Figure 1. Designing for availability

If the website front-end component is sufficiently simple and straightforward, users should always be able to see the enterprise brand image and status information, which will help them have confidence in the reliability of the enterprise itself. Simplicity in both components and connections is the key to the reliability and availability of the system as a whole. Complex and tightly interconnected systems are difficult to maintain and debug when something fails.

footer left page.jpg Organizational customer of the cloud

An organization that acquires cloud services from either private or public cloud providers needs to fully understand the responsibilities of the cloud provider as well as the limitations of those responsibilities. Similarly, the cloud provider must understand the availability and security requirements of the solution that it provides to users. A complete cloud solution requires a thoroughly reliable implementation and an ability by the cloud service provider to create a service that integrates its own capabilities with the organization’s requirements. The good news is that this integration is where the most innovation and value-add for the organization is generated.

The organization’s responsibility

When the responsibilities of the cloud provider are specified, well understood, and documented in a service level agreement, any unmitigated threats become the responsibility of the customer—the organization. A best practice for identifying potential unmitigated threats is to conduct a brainstorming session to identify all possible threats and then filter out those that are known to be the responsibility of the cloud provider. Threats that remain are the organization’s responsibility to mitigate. The following list of risks and responsibilities can be used as a guide for types of threats to consider:

. Apply access control locally and in the cloud. Although data loss incidents can occur for a variety of reasons, they most commonly occur when an attacker either spoofs the identity of a valid user or elevates their own privileges to acquire access that has not been authorized. Most organizations compile a directory of employees and partners that can be federated, or have their accounts mirrored, into a cloud environment; however, other users may need to be authenticated using different methods. For future flexibility, adopt

footer left page.jpg cloud services that support federation and accept identities from on-premises directories as well as from external identity providers. The trend is for providers to include access control services as a part of their service offering. A best practice is to avoid duplicating your account database, because doing so increases the attack surface of the information it contains, such as password data. . Protect data in transit. Data loss can occur if the data is not protected in storage or in transit. Protecting data in transit can be accomplished by using Transport Layer Security (TLS) to provide encryption between endpoints. Protecting data in storage is more of a challenge. Encryption can be provided in the cloud, but if the data is to be decrypted by apps that also run in the cloud, the encryption key needs special protections. It’s important to note that providing the cloud with access to the encryption keys as well as to the encrypted data is equivalent to storing the data unencrypted. . Protect trusted roles. Authorization to perform administrative functions or to access high-value data will likely be based on users’ roles within their organizations. Because roles will vary for each user while their identity remains constant, some mapping must exist between each user ID and a list of the ID’s authorized roles. If the cloud provider is trusted to control access, this list must be made available to the cloud and managed accordingly. A best practice is to use claims-based authorization technologies such as Security Assertion Markup Language (SAML). . Protect data on mobile devices. Protection of user credentials and other sensitive data on mobile devices is only feasible if security policy can be enforced. A best practice is to configure Microsoft Exchange ActiveSync mailbox policies. Although not all devices implement all of the ActiveSync policies, the market is responding to this need and organizations should seek to deploy and enforce a endpoint security solutions on all mobile devices. . Develop all code in accordance with SDL. Application code is likely to come from a combination of the cloud provider (for example, in the form of sample code), the cloud tenant organization, and third parties. A threat modeling process such as the one used as part of the Security Development Lifecycle (SDL), the software development security assurance process created by Microsoft, needs to consider this factor. One area in particular that needs to

footer left page.jpg be analyzed is the potential for conflict if more than one component controls related functionality, such as authorization. . Optimize for low MTTR. The threat modeling process also needs to consider and specify different types of expected failures to help ensure low MTTR. For each potential failure, specify the tools and technologies that are available for recovering functionality quickly.

Availability of sensitive information stored in the cloud

The cloud offers some interesting options for securing data within a highly available architecture. Consider the following example cloud solution, in which the functionality for both security and availability is split between the organization and the cloud. Sensitive organizational information is stored in the cloud, but the decryption keys are maintained within the organization so that no attack on the cloud can reveal the sensitive information. One option is to encrypt all data that is stored in the cloud to prevent data leakage. However, it is also possible to differentiate between types of data so that high-value data is protected by encryption and low-value data is protected only by access control. The design principles of separation of function and geographical distribution that were suggested earlier are used here to increase resiliency.

The following figure illustrates how the data protection scenario works. Data is entered and retrieved at a workstation that is attached to an organization’s network. The network uses a firewall to protect it from intrusions. The workstation connects to a service in the network that uses a key to encrypt and decrypt data. The key is obtained from the organization’s central directory, so all distinct local protected networks can access the data that is stored in the cloud.

The user is authenticated by the organization’s directory. Federation and SAML are used to provide centralized control of authentication and authorization, which takes advantage of the available existing account repository (such as Active Directory) in a distributed environment.

footer left page.jpg Figure 2. Data encrypted in the cloud

This example of encrypted data in the cloud can be used as a pattern for a variety of implementations. The protection of the data decryption key removes the threat of data leaks in the cloud and puts control of the plaintext data under the organization’s full control. The distributed architecture helps to ensure the availability of the service.

footer left page.jpg The user and the device used to access the cloud

Users will measure the availability of a cloud service solely in terms of their ability to complete their current task, which means that the cloud service as well as the device that they use to access the cloud must be functional.

User expectation and feedback

Users measure service availability based on their success at achieving their objectives. The following figure shows the typical steps in the process of a user obtaining access to a cloud resource. First, the user’s device must connect to the local network and be authenticated. Next, the device’s security disposition, or health, is checked and some sort of role-based authorization process is used to establish appropriate access for the user. Finally, the cloud resource itself must be available. The failure of any of these components will block the user’s ability to complete their task. Sometimes the blockage is desired for security purposes, but the user will always perceive it to be an impediment.

Figure 3. Availability blockers

When any one of the links shown in the figure fails, it is important to let the user know the nature of the problem and also what needs to be done to restore availability of the solution. Whenever user action is required, instructions need to be clear and concise.

footer left page.jpg Design for test

An automated test program is helpful for detecting solution failures. A best practice is to design the solution for online testing. All customer-facing services and webpages need to enable automated query programs that use near real-time reporting with automated escalation when significant failures occur. Online testing can provide valuable performance indicators of the availability of the solution services.

Some method of communicating users’ perception of availability should be implemented as well. For example:

. If an application provides users with access to the cloud, use it to generate statistics, such as time from login to acquisition of cloud data. . Periodically ask users when their session ends if they would take a short survey. . Send user experience researchers into the field to get user feedback.

User device availability

Devices that are used to access cloud solutions must be trusted not to leak high- value information. Because mobile devices are increasingly being used to access such information, some way to evaluate device security, or health, is required. A good cloud design is able to evaluate device health and verify user identity. This section describes how to provision device health assessment in such a way that the cloud solution can be available to users from anywhere.

This example solution addresses the need to establish secure access to an organization’s network from a user-owned device. This type of scenario is often referred to as bring your own device to work, or BYOD. For many years, IT departments were able to protect enterprise assets by quarantining all resources, including the client computers that accessed those resources, inside a protected perimeter. In BYOD scenarios users have commercially available devices that can access all of their personal data from anywhere, and they want to use those same devices to access the organization’s resources as well. This phenomenon is known as the consumerization of IT.

footer left page.jpg In such a scenario, the cloud service needs to establish the user’s identity, learn their preferences about how their personal information can be used, and obtain the user’s permission if the service (or the organization) wants to store the user’s personal information for future use or share it with others. In addition, the cloud service may have content that should only be released to user devices that are determined to be secure. Many users want to know that their privacy, identity, and assets are protected from malware, although most users are unwilling to be inconvenienced by security mechanisms.

The following figure shows a solution built to assess mobile device health from the cloud. The mobile device authenticates the user through a connection to an identity provider in the cloud. If the web service has highly confidential information, or is trying to obtain a provable indication of the user’s intent, it may elect to verify the security of the mobile device before proceeding. The user’s device will then receive a health attestation that can be sent with the user ID.

Figure 4. Secure mobile clients

Windows 8 devices can be protected from low-level rootkits and bootkits by using low-level hardware technologies such as secure boot and trusted boot.

footer left page.jpg Secure boot is a firmware validation process that helps prevent rootkit attacks; it is part of the Unified Extensible Firmware Interface (UEFI) specification. The intent of UEFI is to define a standard way for the operating system to communicate with modern hardware, which can perform faster, more efficient input/output (I/O) functions than older, software interrupt-driven BIOS systems.

Trusted boot creates a condition in which malware—even if it is able to tamper with the boot process, which is unlikely—can be detected, which prevents a health attestation from being granted. Secure boot also protects the antimalware software itself.

A Remote Attestation Service (RAS) agent can communicate measured boot data that is protected by a Trusted Platform Module (TPM). After the device successfully boots, boot process measurement (for example, measured boot in Windows 8) data is sent to a RAS agent that compares the measurements and conveys the health state of the device—a positive, negative or unknown state—by sending a health claim back to the device.

If the device is healthy, it passes that information to the web service so the organization’s access control policy can be invoked to grant access.

Depending on the requirements of the content provider, device health data can be combined with user identity information in the form of Security Assertion Markup Language (SAML) or open standard for authorization (OAuth) claims. The identity provider, for example Active Directory, may belong to the user’s employer, to the content provider, or to a social network such as Facebook. The data is evaluated by fraud detection services that are already in use at most commercial websites. Access to content is then authorized to the appropriate level of trust for what the health assertions, or claims, merit. These claims protocols are structured to allow additional requests from the content provider to the user’s device as needed by the user’s transaction requests of the provider. For example, if high-value data or funds transfers are requested, additional security state may need to be established by querying the user’s device before the transaction can be completed.

footer left page.jpg Conclusions

The preceding examples illustrate solutions that emphasize a secure, service- oriented architecture with separation of functions. The demonstrated architectural patterns divide the solution into components with loose coupling. This approach allows each component to fail over gracefully, even if other components fail catastrophically. The service as a whole may continue with some or all functionality no matter which individual component fails. The design can use hybrid solutions that include some on-premises functionality while providing other functionality, such as solution scaling, through an off-premises public cloud.

As a best practice, availability needs to be monitored by a service that operates in the same realm as the user. In addition, services should be designed and located in a way that makes them accessible and operable from geographically diverse locations to provide availability when calamities and natural disasters occur.

Cloud service developers and cloud service customers alike need to communicate and cooperate to anticipate, design, and test for failures at every point. Such communication and cooperation will have a direct impact on the success of the solution and the satisfaction of its users.

footer left page.jpg Additional reading

For more information about the scenarios and solutions detailed in this paper, see the following resources. These documents provide additional information to help you make the right design decisions for the availability of your cloud-based solutions.

. The Windows Azure Application Model https://www.windowsazure.com/en– us/develop/nodejs/fundamentals/application-model/ . Microsoft System Center http://microsoft.com/systemcenter (this website is now focused on Release Candidate 2012) . Cloud Computing: Achieving Control in the Hybrid Cloud http://technet.microsoft.com/en-us/magazine/hh389788.aspx . Cloud Security Alliance – Security, Trust & Assurance Registry (STAR) https://cloudsecurityalliance.org/star/ . Security Guidelines for SQL Azure http://social.technet.microsoft.com/wiki/contents/articles/1069.security– guidelines-for-sql-azure.aspx . Service-Oriented Architecture – Design Patterns www.soapatterns.org/masterlist_c.php . Cloud Insecurity: Not Enough Tools, Experience or Transparency www.technewsworld.com/story/74890.html . How the Cloud Looks from the Top: Achieving Competitive Advantage In the Age of Cloud Computing (PDF) http://download.microsoft.com/download/1/4/4/1442E796-00D2-4740-AC2D– 782D47EA3808/16700%20HBR%20Microsoft%20Report%20LONG%20webview.pdf

footer left page.jpg

One Microsoft Way

Redmond, WA 98052-6399

microsoft.com/twcnext

Oracle White Paper—Oracle VM Template Developer’s Guide

An Oracle Technical White Paper

February 2009

Oracle VM Template Developer’s Guide:

Creating Pre-Built VMs for

Rapid Software Deployment

Oracle White Paper—Oracle VM Template Developer’s Guide

Introduction …………………………………………………………………………….. 1

Oracle VM Templates: Concept and Usage…………………………………. 2

Template Creation, Overview…………………………………………………. 2

What You Need to Know About Licenses ………………………………… 5

Prerequisites for Oracle VM Template Creation………………………… 6

Creating an Oracle VM Template……………………………………………….. 6

Step-by-Step: Template Creation……………………………………………….. 7

1. Decide On Your Template Specification ………………………………. 7

2. Using JeOS kit, Create the Initial Template …………………………. 8

3. Set-up the Template Development Environment …………………. 11

4. Configure the Virtual Machine for Product Installation ………….. 12

5. Install the Product Software into the VM …………………………….. 12

6. Identify Actions for Product Software Configuration at Install … 14

7. Define the Product Specific Reconfiguration Actions …………… 15

8. Define the Product Specific Cleanup Scripts ………………………. 16

9. Remove Proprietary Files from OS Disk Image & Replace

Parameter Values by Placeholder Strings ……………………………… 17

10. Package the Template …………………………………………………… 17

APPENDIX A…………………………………………………………………………. 20

APPENDIX B…………………………………………………………………………. 26

Oracle White Paper—Oracle VM Template Developer’s Guide

1

Introduction

An Oracle VM Template is a virtual machine (VM), or group of VMs, containing a full

software stack that is pre-installed, pre-configured, and ready to use. Simply download

and import a Template into Oracle VM, and then deploy the template as a VM in order to

use the pre-configured software. This eliminates the steps of installing, patching, and

configuring complex software.

Oracle VM Templates support both Oracle and non-Oracle applications, and can be built

by anyone including Oracle, ISVs, and third party solution providers. By using an

operating system like Oracle Enterprise Linux, developers can build their application as a

full stack virtual machine on an enterprise-class operating system that is freely

redistributable without any special agreement, and the template can be backed by

enterprise-class support.

This technical white paper is for those interested in knowing how to package applications

in a standard, pre-configured way as an Oracle VM Template. It’s a fast and reliable way

to deploy complex enterprise applications.

For information on the broader context of Oracle VM Templates, their benefits, and how

they are deployed, customized, and used from Oracle VM Manager, refer to the “Creating

and Using Oracle VM Templates: The Fastest Way to Deploy Any Enterprise Software”

white paper, and visit oracle.com/virtualization for more information about Oracle VM.

Oracle White Paper—Oracle VM Template Developer’s Guide

2

Oracle VM Templates: Concept and Usage

The use of Oracle VM Templates for the deployment of applications in Oracle VM virtual

machines (VMs) eliminates the need for a user to install and configure the operating system and

applications. The templates can be simply downloaded and the virtual machine(s) can be brought

up either from the Oracle VM Manager browser interface or by using an xm command issued on

the Oracle VM server command line.

The OS and applications are configured at initial template boot time. The configuration can be

done based on certain default values and actions based on the user’s input. An example of a

default configuration is an Oracle Enterprise Linux (OEL) template using a DHCP assigned IP

address. No user input is required to configure the template. An example of boot time

reconfiguration is an OEL template with static IP. In this case, the, user would need to supply

values for the IP address, gateway, netmask, and DNS to the virtual machine before the boot

process can be completed.

If you wish the virtual machine to be configured with a static IP address and other non-default

settings, you need to develop the template configuration actions in the form of writing a

configuration script, which can process the user’s input and perform the corresponding

configuration steps upon initial boot of the VMs created from the template.

The remainder of this paper will focus on how to create and Oracle VM Template for any

software you wish to deploy as a complete, pre-built virtual machine.

Template Creation, Overview

An Oracle VM template consists of one or more binary files and a text file. The binary files are

the disk images taken from a fully configured and functional virtual machine. The text file is a

virtual machine configuration file. The files are shipped in one archive (or tar file). For example,

an archive (or tar file) of the Oracle 11g Database template contains 3 files in one directory:

oracle11g/system.img (disk image with OS)

/oracle11g.img (disk image with oracle software)

/vm.cfg (vm configuration file)

Oracle White Paper—Oracle VM Template Developer’s Guide

3

The tar file contents can also be a multi-VM template set. For example, the Oracle Enterprise

Manager Grid Control template contains two virtual machines: a middle tier management server

(OMS) VM and a backend Oracle Database repository VM. Consequently, the template archive

consists of two directories, each containing 3 files:

db/system.img (disk image with OS)

/db.img (disk image with oracle DB software)

/vm.cfg (vm configuration file)

gc/system.img (disk image with OS)

/gc.img (disk image with oracle GC software)

/vm.cfg (vm configuration file)

Note: Packaging a template that consists of several virtual machines in a single archive simplifies

the download for the end user because there is less chance that an inconsistent or wrong

combination of VMs or files will be downloaded.

A template is a copy of a pre-installed virtual machine. To create this copy, one needs first to

create the virtual machine itself running the desired OS, and then install and configure the target

software. The next step is to implement a script to perform dynamic reconfiguration of the OS

and application at initial boot-time, if required.

The virtual machine with the desired OS can be created from scratch using Oracle Enterprise

Linux, or you can begin with an existing Oracle VM OS Template available from Oracle’s EDelivery

website. Oracle has made available for free download a “Just enough OS” or JeOS

edition of the Oracle Enterprise Linux 4 and 5 operating systems to facilitate building an OS

instance with a bare minimum number of packages needed for your template. This helps reduce

the disk footprint by up to 2GB or more per VM, and also improves security and reliability of the

virtual machine. Of course, you can customize and add packages to the JeOS edition package list

to completely tailor to the needs of the application. The JeOS edition download also includes a

script to help configure parameters such as disk sizes, etc.

Details on JeOS, including a complete description of the included modifyjeos script can be found

on the Oracle Open Source Software website at oss.oracle.com/el5/docs/modifyjeos.html

The dynamic reconfiguration boot time actions can be implemented in product specific scripts as

described later in this paper. The scripts are then installed in the template and can be set to

automatically run when the VM is first booted.

Note: Throughout the rest of this white paper, unless indicated otherwise, the term “product”

will be used to refer to any software product(s) included in the template in addition to the OS.

Oracle White Paper—Oracle VM Template Developer’s Guide

4

Start VM from

Template

Reconfigure

Template?

Is template

product config

script available

Execute product

template

configuration script

Generate new

SSH host keys

and up2date UUID

Is product

image

available?

Mount Product

Image on /u01 Yes

Execute JeOS

reconfiguration

scripts: Obtain

network

configuration from

user at console

End

VM OS boots at

runlevel 3

End

No reconfiguration

required No Yes

No

Yes

No

Figure 1: The Virtual Machine Start-Up and Product Configuration Process. At the VM first boot, the VM OS (OEL) and application can be

configured with new IP address, hostname, and registered to the Unbreakable Linux Network for on-going support.

Oracle White Paper—Oracle VM Template Developer’s Guide

5

What You Need to Know About Licenses

An Oracle VM Template can include any software and can be built by anyone. However, a

template developer or distributor should be aware of potential license implications.

Oracle VM Server and Oracle Enterprise Linux licenses are agreed upon at the time of download

from the E-Delivery website. The accepted End User License Agreement (EULA) is on the root

of the installation media (ISOs) downloaded and also included in the installed ovs-release and

enterprise-release packages (respectively). This covers Oracle VM Server and Oracle Enterprise

Linux only. Modifications or redistributions with other products thereafter may be or may not

be possible, and it is up to the individuals or organizations to consult the licensing terms for all

software included to ensure that use or distribution rights are not violated. Including software in

an Oracle VM Template does not reduce or eliminate any licensing obligations associated with

the included software.

Oracle VM Server and Oracle Enterprise Linux are free to download, use, and re-distribute. So a

template that contains only OEL (e.g. a default install of the OS) can be re-distributed without

any special agreement from Oracle.

Ease of Operations Maintenance

Since Oracle VM Templates do not require any proprietary directory structures or metadata,

there is no reason why the software included in the template cannot be completely compatible

with any utilities for patching and support provided by software vendors.

However, in order to support this compatibility, you may need to consider whether any

additional packages or software needs to be included in your template to support any required

vendor utilities used for maintenance of the template. For example, the template once deployed

should be able to support OS and product standard operating procedures going forward with

regards to:

• Patching

• Upgrade

• Configuration changes

• Monitoring

• Tech Support

For example, you should consider if it is likely that a monitoring agent, or agent used for

patching will need to be installed. You can consider either including these agents directly or just

plan to make sure the appropriate packages are included to allow future installation in the guest

VMs.

Oracle White Paper—Oracle VM Template Developer’s Guide

6

Prerequisites for Oracle VM Template Creation

In order to get started, you will need access to the following items to develop a template. All of

the software referenced below can be downloaded free from eDelivery.oracle.com

• One physical server running Oracle VM Server software

• One Linux desktop or server running Oracle VM Manager

• Oracle Enterprise Linux JeOS software kit

• Root access to the dom0 on Oracle VM Server to package the template

For hardware requirements and other specifics on installing or using Oracle VM or Enterprise

Linux software, refer to the user documentation available at oracle.com/virtualization or

oracle.com/linux.

Creating an Oracle VM Template

There are 10 basic steps for creating an Oracle VM Template summarized below. Depending on

the type of software being included in the template, not every step will apply. Each step will be

described in detail, including an example based on the Oracle Enterprise Manager Agent

Template developed by Oracle. Overview of the basic template creation steps:

1. Decide on your spec for the template: operating system, product version, and product

configuration;

2. Using JeOS kit, create the initial template with OS disk image and placeholder disk

image for product installation;

3. Setup a development environment;

4. Configure the virtual machine for the product installation;

5. Install the product software (e.g. Oracle database, Enterprise Manager Agent, etc.);

6. Identify the product configuration actions normally performed product installation that

will need to be performed by a template configuration script (e.g. system files modified,

product configuration files that refer to used-at-install hostname, IP address);

7. Develop a script for the product specific reconfiguration actions (e.g. in product

configuration files, replacement of used-at-install IP address by actual IP address assigned to

the virtual machine at deployment);

8. Develop the product specific cleanup scripts;

9. Remove the proprietary files from OS disk image and replace the relevant real parameter

values by placeholders;

10. Package and archive the template.

Oracle White Paper—Oracle VM Template Developer’s Guide

7

Step-by-Step: Template Creation

1. Decide On Your Template Specification

The first step is to specify the technical requirements for the template:

• OS version to include

• Guest type: paravirtualized (PVM) or fully virtualized (also known as “hardware virtualized” or

“HVM”). Generally stated, PVMs are higher performance and more scalable and thus are generally

preferable. Consult Oracle VM user documentation and white papers for more discussion on

PVMs vs. HVMs

• Version of product(s) to install

• Product configuration

• Product install directory

• Product install partition size

• Number of virtual CPUs to be configured when the guest VM is created. Note that this value

can be edited later and even changed from the Oracle VM Manager GUI.

• Size of virtual memory. Note that, in Oracle VM, as with any virtualization product using the

Xen hypervisor component, the amount of memory consumed by all the VMs running on a

physical server at any one time cannot exceed the amount of physical memory in the server. This

may be a consideration if you plan on running multiple VMs on the same physical machine.

• Swap size: Depends on the application. For Linux operating systems, 512B-1024GB is usually

sufficient but more may be ideal for your application. For example, for Oracle Database, 2GB is

recommended.

• Free size of root partition desired after OS install

• Name for your product disk images

• Mount point for the product disk image(s)

• Template name (default name for the archive and virtual machine(s)). Keep in mind that

templates containing multiple VMs will create a separate directory based on the VM name for each

VM. To avoid later confusion or conflicts when you have many templates deployed in the same

Oracle VM instance, it is recommended that you develop a naming convention that will not only

contain the name of the individual VM but also the name of the template that it is associated with.

For example, if you have multiple templates that contain a database VM, do not name each VM

simply as “database” because this will create a directory naming conflict but also create confusion

Oracle White Paper—Oracle VM Template Developer’s Guide

8

later as to which overall template a given “database” belongs-to. Also keep in mind versioning as

you are likely to update the template over time with patched or updated software or settings.

• Dependencies, overall topology, or other things specific to the products you are including.

Note: Although Oracle developed templates generally include two virtual disk images, it is

technically optional to create a second disk for the application install and it may or may not be

required for a given template. But, often two disk images may be needed to separate products

with different license terms and conditions, e.g. to separate software with open licenses such as

GPL from software or files that have a proprietary license. Consult the software license authority

with any questions regarding licensing requirements.

Step 1 Example: Oracle Enterprise Manager Agent Template – Template Specification

In this example of an Oracle Enterprise Manager Agent template, the template specification is as

follows:

• OS version: Enterprise Linux 5 Update 2 for x86 platform

• Guest type: Paravirtulized (PVM)

• Product Version: EM Agent 10.2.0.4 for Linux x86

• Product configuration: standard install

• Product install directory: /u01.

• Product install disk size: 5 GB

• Number of virtual CPU: 1

• Size of virtual memory: 1GB

• Swap size: 2GB

• Free size of root partition after OS install: 2GB

• Name for your product disk images: Emagent.img

• Mount point for the product disk image: /u01

• Template name: OVM_EL5U2_X86_EMAGENT_PVM.

2. Using JeOS kit, Create the Initial Template

Oracle recommends using the Oracle Enterprise Linux JeOS kit for creating a tailored for your

product OS image and product disk.

oracle.com/technology/software/products/virtualization/vm_jeos.html

Oracle White Paper—Oracle VM Template Developer’s Guide

9

Download and install JeOS rpm to a system running OEL 5 Update 2 or later. It can be either a

bare metal system or a virtual machine. The documentation, which is part of the rpm, is a

comprehensive guide on how to develop with JeOS.

Tips:

-The JeOS edition of Enterprise Linux only includes English language support by default. If your

template needs to support additional languages, these packages should be added to your OS

build.

-If you have rpm-based software products you wish to include and can configure access to the

rpm repository for that software, you may provide those packages as input to modifyjeos

command. If the product(s) you wish to add are not rpm based, and need to run their own

installer, you can do so by running the installer in the VM before completion of your template

packaging as described later in this paper.

Step 2 Example: Oracle Enterprise Manager Agent Template – Initial Template Creation with OEL JeOS

As a root user, install JeOS kit on the virtual machine running OEL5.2:

ovm-modify-jeos-1.1.0-2.el5.noarch.rpm

ovm-el5u2-xvm-jeos-1.1.0-1.el5.i386.rpm

Create the working directory to be used for JeOS output files and enter

this directory:

# mkdir /<any_path>/emagent_work

# cd /<any_path>/emagent_work

To create the template, we need to know the disk size(s), number of

virtual CPUs and memory size. This information should have been included

as a part of the specification described in Step 1 above.

Identify the additional packages required for the product. Get these

packages using modifyjeos and specify either a DVD ISO mount point or a

YUM repository on the network as a repository location. It should be

local ISO (DVD) or YUM repository accessible via network. Refer to JeOS

documentation for more details.

We will use an YUM repository as the source for packages in this example:

http://public-yum.oracle.com/repo/EnterpriseLinux/EL5/2/base/i386

Oracle White Paper—Oracle VM Template Developer’s Guide

10

The addpkg.lst file used as input by JeOS contains a list of additional

packages to be added to the standard, base JeOS image. For an EMagent

install, create ‘addpkg.lst’ with the following lines (packages):

* oracle-validated

Note that this is a very special and helpful rpm if your template will

include the Oracle Database or other software that depends on some of the

Database libraries, as is the case with the EM Agent. Basically you can

think of this as a kind of “master rpm” that contains references to all

the dependencies / rpms that you need on top of the JeOS base to support

the Database. By specifying this package alone, it will pull-in and

install all the other packages recommended and required by Oracle for the

Database, eliminating the need to manually identify each individual

Database package required. Named after, and derived from Validated

Configurations program, oracle-validated also creates an oracle OS user

and an oinstall and dba group. Kernel parameters are also set properly,

ensuring that the Oracle Universal Installer will proceed without

complaints.

* nc – required, used in template reconfiguration script

* unzip – optional but recommended, may be used to unzip the Agent

software source

* wget – optional but recommended, may be used to download the Agent

software.

Next, run the modifyjoes command to add the packages and generate the

.img and vm.cfg files:

# modifyjeos -i EL52_i386_PVM_jeos \

-n OVM_EL5U2_X86_EMAGENT_PVM \

-mem 1024 -cpu 1 \

-r http:// public-yum.oracle.com

repo/EnterpriseLinux/EL5/2/base/i386 \

-p addpkg.lst \

-S 2048 -I 6096 -R 2048 \

-P Emagent.img 5120 /u01 \

-conf /u01/emagent-reconfig.sh \

-cln /u01/emagent-cleanup.sh

Oracle White Paper—Oracle VM Template Developer’s Guide

11

u01/emagent-reconfig.sh referenced above is the name of reconfiguration

script we’ll develop later in Step 7.

/u01/emagent-cleanup.sh referenced above is the name of cleanup script

we’ll develop later in Step 8.

The following files are then produced in the current directory:

·System.img: This is the OS system image file

·Emagent.img: This is the image file for the EM agent

·vm.cfg: This is the virtual machine configuration file describing the

parameters needed to create the guest VM.

A partition with ext3 file system is created on Emagent.img and the

partition will be mounted on /u01.

Copy these three files to the directory reachable via http or ftp, for

example to /var/www/html/emagent_templ.

Note: For a complete description of all the modifyjeos options, refer to

the README file hosted at http://oss.oracle.com/el5/docs/modifyjeos.html

3. Set-up the Template Development Environment

Install the Oracle VM Server and Oracle VM Manager software if you have not installed them

already.

Login to Oracle VM Manager and import the VM created by JeOS by supplying the http or ftp

location via the wizard initiated from the ‘Import’ button on the Virtual Machines page. Start the

virtual machine. Refer to Oracle VM Manager User’s Guide for complete details on importing

and starting VMs in Oracle VM Manager.

Once the virtual machine has powered-on, you can login to the virtual machine as user ‘root‘ via

Oracle VM Manager’s VNC ‘Console’ button using password ‘ovsroot’‘.

Step 3 Example: Oracle Enterprise Manager Agent Template – Set-up Development Environment

In our example, we have Oracle VM server installed already, and have

imported the initial VM (the disk images and vm.cfg created in step 2)

into Oracle VM Manager. We now start the virtual machine using Oracle VM

Oracle White Paper—Oracle VM Template Developer’s Guide

12

Manager and then open the virtual machine console using the ‘Console’

button on the ‘Virtual Machines’ page.

You’ll be prompted to choose DHCP configuration or specify the IP

address, gateway, netmask, and DNS to configure the virtual machine

network. Choose the option which works better for you. It will not have

an effect on the template eventually created.

Answer the interview questions, and login to running virtual machine as

user ‘root’ using password ‘ovsroot’.

4. Configure the Virtual Machine for Product Installation

You may need to do additional OS configuration for the included product software. This may

include additional OS parameter settings and disk configuration but is completely dependent on

the specifics of the products that are included.

JeOS already configures swap and the disk for product installation for you. By default, the ext3

filesystem is used for the product disk format. Unless you want to change the filesystem format

from ext3 or the disk size, you do not need change anything.

Next, if you need a raw partition, additional configuration will be needed. For example, if you

need a partition for the datafiles located on Oracle ASM (Automatic Storage Management)

devices, you could create a second “product disk” using the modifyjeos script. But instead of

using it as a filesystem, un-mount that device and use the partition (e.g. /dev/xvdc1) for the

ASM device.

If you need to modify the OS parameters manually, you may need to edit OS configuration files.

For example, to set the kernel parameters per product requirement, you may need to edit

/etc/sysctl.conf as root.

Step 4 Example: Oracle Enterprise Manager Agent Template – Configuring Source Virtual Machine

We used JeOS to install the required dependant packages, and configure

the second disk. The oracle-validated.rpm created all the required user

accounts and set all the required OS parameters.

Refer to Oracle Database Installation Guide for information about oraclevalidated.

rpm.

5. Install the Product Software into the VM

Install the software into the VM(s) and perform the required configuration (sizing, data

population). As previously mentioned, if your software installation is completely rpm-based, you

could add it to the list of packages to be installed in the image created by JeOS. But if your

Oracle White Paper—Oracle VM Template Developer’s Guide

13

product installation is not rpm-based, and you need to run your installer, you would run it as a

part of this step.

For the product install, use the disk mounted at the mount point specified during the spec phase

(Step 1). This disk may or may not be formatted with the file system depending on how you

created the template.

Step 5 Example: Oracle Enterprise Manager Agent Template – Install Product Software

We downloaded agent software from the Oracle Technology Network (OTN) and

did an install using silent installation mode (refer to the Oracle

Enterprise Manager Installation Guide).

1)On the product image disk mounted at /u01, create a directory owned by

user ‘oracle’ and group ‘dba’

# mkdir /u01/app

# chown oracle:dba /u01/app

2) Check and configure the hostname, both in file /etc/hosts, and by

using command “hostname”, as the agent installer will check the local

hostname

3)Extract the agent install media from downloaded zip file

Unzip it in <UNZIP_DIR> directory

4) Modify <UNZIP_DIR>/response/additional_agent.rsp, set

BASEDIR=u01/app/oracle/product

sl_OMSConnectInfo={“dummy”, “9999”}

These ‘dummy’ host name and ‘dummy’ port allows to install emagent

without registering to OMS server. This is convenient if you do not have

OMS server installed in your environment. You can specify the values for

real OMS hostname and port if you wish. In this example, we assume that

there is no OMS server available at the template development time.

5) Invoke OUI in silent mode:

# su – oracle

# <UNZIP_DIR>/linux/agent/runInstaller –silent –force -noConfig –

responseFile <UNZIP_DIR>/linux/response/additional_agent.rsp

In the above syntax, -force noConfig is required if you run installer

with ‘dummy’ OMS server.

Oracle White Paper—Oracle VM Template Developer’s Guide

14

6) as root user, run post install scripts

/u01/app/oraInventory/orainstRoot.sh

/u01/app/oracle/product/agent10g/root.sh

6. Identify Actions for Product Software Configuration at Install

Next you need to analyze all the product configuration steps that were required to support

network, storage, or user specific configuration. Specifically, you need to note any items that

would likely need to be reconfigured when the virtual machine is deployed in a new environment

since the networking or storage or hostnames, etc. might be different. Reconfiguration of these

items will need to be provided as a part of your OS or product reconfiguration scripts as

described in section 7 below. Those could include any of the following:

• Files with modified content

• File and directory names modified

• Data entries in the database

• Dependencies on other existing environments

• Other items specific to the products installed

Step 6 Example: Oracle Enterprise Manager Agent Template – Identify the Reconfiguration Actions

In the case of the Oracle EM Agent, each agent instance has to register

with the OMS (Oracle Management Service) of Enterprise Manager.

Normal EM Agent installation also requires the following data for each

new instance:

·Management Service hostname

·Management Service IP address (if hostname cannot be resolved)

·Management Service Port

·Agent Registration password

Those parameter values need to be obtained from the user at initial

template boot

The configuration file for the EM agent is:

·$AGENT_HOME/sysman/config/emd.properties

Oracle White Paper—Oracle VM Template Developer’s Guide

15

OMS port and OMS hostname values are embedded into URLs for OMS Wallet

Source and OMS repository parameters. Those values need to be replaced at

the template boot to reflect the deployment setting.

It may be helpful to create a table describing the parameter names, their

values, and where they are stored such as is shown below to help keep

track of everything that will need to be addressed in the script(s):

Value Stored in Where / When to

configure

OMS hostname emdWalletSrcUrl parameter emd.properties file

OMS port emdWalletSrcUrl parameter emd.properties file

OMS hostname REPOSITORY_URL parameter emd.properties file

OMS port REPOSITORY_URL parameter emd.properties file

Agent password OMS Database At agent registration

7. Define the Product Specific Reconfiguration Actions

Now that you have analyzed and collected the list of parameters that would need to be set

uniquely for each VM instance created from the template, you need to develop the scripts to

gather the information and set the values. The script should perform the following tasks:

• Collect from the user the values for configurable parameters

• Perform the software application reconfiguration

• Start the software application if applicable

Step 7 Example: Oracle Enterprise Manager Agent Template – Develop The Product Specific

Configuration Scripts

For this EM agent example, the script is called ’emagent-reconfig.sh’ and

should be copied into /u01/. We used this name emagent-reconfig.sh when

we ran the modifyjeos command above in Step 2.

Refer to Appendix A at the end of this paper for the full source code for

the template reconfiguration script for this example.

The script is executed at the first boot of the VM created from the

template. The actions performed by the script are based on data from

Step 6, and consist of three logical parts:

Oracle White Paper—Oracle VM Template Developer’s Guide

16

1. Interactively collect the user’s input using a VNC session brought up

at first boot. Again, the VM console can be accessed by selecting the VM

in Oracle VM Manager and clicking on the ‘Console’ button in the UI.

2. Replace the placeholder strings by real data for parameters defined

in the example in Step 6:

sed -i “/^emdWalletSrcUrl/s|%OMS_HOST%|$OMS_HOST|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^emdWalletSrcUrl/s|%OMS_PORT%|$OMS_PORT|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^REPOSITORY_URL/s|%OMS_HOST%|$OMS_HOST|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^REPOSITORY_URL/s|%OMS_PORT%|$OMS_PORT|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^EMD_URL/s|%HOSTNAME%|$(hostname)|”

$AGENT_HOME/sysman/config/emd.properties

Note, that to make this approach work, before you archive the template, you need to substitute the

development VM parameter values in configuration file by placeholders like ‘%OMS_HOST’. This

will be done in steps 8 and 9.

3. Start the EM agent and register it to OMS

After you finish the development and testing of the script, copy it to

/u01.

8. Define the Product Specific Cleanup Scripts

Based on analysis of all the product installation and configuration steps, determine the cleanup

tasks that need to be done. These may include any of the following:

• Deleting product installation and other log files

• Deleting runlevel startup scripts

• Replacing the product parameter values by placeholders in the product configuration scripts

Then, you need to develop the scripts to perform these tasks.

Step 8 Example: Oracle Enterprise Manager Agent Template – Product Specific Cleanup Scripts

Oracle White Paper—Oracle VM Template Developer’s Guide

17

For this EM agent example, the cleanup script is called ’emagentcleanup.

sh’. We introduced this script name, emagent-cleanup.sh, when we

ran the modifyjeos script in Step 2.

Develop the script, and copy it to /u01 directory.

Refer to Appendix B at the end of this paper for the source code for the

template clean up script for this example.

9. Remove Proprietary Files from OS Disk Image & Replace

Parameter Values by Placeholder Strings

If the product is up and running, shut it down cleanly.

Run the JeOS cleanup script

# oraclevm-template –cleanup

The JeOS cleanup script invokes the product specific script (emagent-cleanup.sh in our example).

Hence, all product specific cleanup actions will be done. For example, the used-at-product-install

parameter values will be replaced by placeholders in the product configuration files.

The JeOS script cleans up the OS log files in /var/log, removes the systemid from up2date

configuration file, cleans up the yum caches, DHCP client caches, the root user’s ssh

configuration file and bash history files, removes etc/resolv.conf, resets /etc/hostnames to

default, resets network configuration to DHCP, and, finally, shuts down network service.

Enable ‘oraclevm-template’ service

# oraclevm-template -enable

This will make the template ready for the first boot, i.e. will set the sysconfig flag

/RUN_TEMPLATE_CONF/ to /YES/.

Step 9 Example: Oracle Enterprise Manager Agent Template – Clean Up The Disk Image

Run

# oraclevm-template –cleanup

# oraclevm-template –enable

10. Package the Template

• Shutdown the virtual machine

Oracle White Paper—Oracle VM Template Developer’s Guide

18

• Copy the virtual machine directory from Oracle VM Server back to the machine where

JeOS kit is installed

• Replace the modified vm.cfg file by the original vm.cfg created by JeOS

• Zero out the free space in the image files

• Archive the template

Step 10 Example: Oracle Enterprise Manager Agent Template – Package the Template

To package the template, you need a root access to Oracle VM server

Shutdown the virtual machine via OVM Manager UI.

Do remote copy of the entire <VM ID>emagent directory content including

vm.cfg, vm.cfg.orig, System.img and Emagent.img back to environment where

JeOS kit is installed. In our example that was OEL 5.2 virtual machine.

From any directory in this virtual, issue

# scp -r root@<Oracle VM Server>:/OVS/running_pool/<VM ID>_emagent

OVM_EL5U2_X86_EMAGENT_PVM

Change the working directory to OVM_EL5U2_X86_EMAGENT_PVM

#cd OVM_EL5U2_X86_EMAGENT_PVM

Replace the vm.cfg file generated by OVM manager by the original vm.cfg

file created by JeOS. The vm.cfg generated by JeOS is the more generic

template vm.cfg file and thus does not contain the MAC address that is

specific to the VM you created for development purposes.

# cp /var/www/html/emagent_templ/vm.cfg vm.cfg

Oracle White Paper—Oracle VM Template Developer’s Guide

19

Zero out image file content:

#modifyjeos –f System.img –zero-out-all

This command will automatically mount all other images that are part of a

template

Convert image files to sparse image format

#cp –sparse=always System.img System.img.sparse

#mv System.img.sparse System.img

#cp –-sparse=always Emagent.img Emagent.img.sparse

#mv Emagent.img.sparse Emagent.img

#cd ..

Archive the template

# tar -czvSf OVM_EL5U2_X86_EMAGENT_PVM.tgz OVM_EL5U2_X86_EMAGENT_PVM

Conclusion

Packaging software as Oracle VM Templates simplifies the product deployment process, saves

time, and improves the end-user’s experience. With Oracle VM Templates, developers can

deliver best practices for software deployment while not restricting users to rigid configurations.

For a broader discussion of Oracle VM Templates, refer to the “Creating and Using Oracle VM

Templates: The Fastest Way to Deploy Any Enterprise Software” white paper.

Also, visit oracle.com/virtualization for more information about Oracle VM.

Oracle White Paper—Oracle VM Template Developer’s Guide

20

APPENDIX A

Source Code for Agent Template Reconfiguration Script, /u01/emagent-reconfig.sh

#!/bin/bash

case “$ORACLE_TRACE” in

T) set -x ;;

*) ;;

esac

# oms_connection function interactively collects user’s input for

# OMS hostname, ip address, password, and port numbers

oms_connection () {

local oms_hostname

local oms_ip

local oms_host

local oms_port

# ask for oms connection information.

echo

echo “Provide Management Service hostname”

while true; do

# oms hostname

while true; do

echo -n “Enter OMS hostname: ”

read oms_hostname

if [ -z $oms_hostname ]; then

continue

Oracle White Paper—Oracle VM Template Developer’s Guide

21

fi

break

done

# if can not resolve the hostname, then request IP address

if ! ping -c 3 $oms_hostname >/dev/null 2>&1; then

echo “*Can not resolve hostname $oms_hostname.”

while true; do

echo -n “Enter OMS IP address: ”

read oms_ip

if [ -z $oms_ip ]; then

continue

fi

break

done

fi

if [ ! -z $oms_ip ]; then

oms_host=$oms_ip

else

oms_host=$oms_hostname

fi

# oms port

echo -n “Enter OMS port: [4889] ”

read oms_port

if [ -z “$oms_port” ]; then

oms_port=4889

fi

# test connection

if ! nc -w 3 -z $oms_host $oms_port >/dev/null; then

Oracle White Paper—Oracle VM Template Developer’s Guide

22

echo “*Can NOT connect to $oms_host:$oms_port, please enter OMS information again.”

oms_ip=

continue

else

OMS_HOST=$oms_hostname

OMS_PORT=$oms_port

if [ ! -z “$oms_ip” ]; then

/bin/cp -f /etc/hosts /etc/hosts.orabak$$

sed -i “/^$oms_ip/d” /etc/hosts

echo “$oms_ip $OMS_HOST ${OMS_HOST%%.*}” >> /etc/hosts

fi

fi

break

done

# enter password

echo

echo “Provide the Agent Registration password so that the Management Agent can

communicate with Secure Management Service.”

while true; do

echo -n “Enter Agent Registration Password: ”

stty -echo

read secure_passwd

stty echo

echo

if [ -z $secure_passwd ]; then

continue

fi

break

Oracle White Paper—Oracle VM Template Developer’s Guide

23

done

SECURE_PASSWD=$secure_passwd

}

# source functions that are part of standard JeOS template

. /usr/lib/oraclevm-template/functions

# ovm_configure_network function is part of JeOS function library

# this function interactively collects user’s input for the

# virtual machine network configuration: IP address, hostname,

# gateway,netmask, DNS

ovm_configure_network

# Reconfigure agent

echo

echo “Reconfiguring Agent…”

# Parameters to be reconfigured

OMS_HOST=

OMS_PORT=

SECURE_PASSWD=

# Get parameter values from user input

oms_connection

AGENT_HOME=/u01/app/oracle/product/agent10g

# replace the placeholder values by the actual OMS hostname, port and virtual machine

hostname in emd.properties

Oracle White Paper—Oracle VM Template Developer’s Guide

24

sed -i “/^emdWalletSrcUrl/s|%OMS_HOST%|$OMS_HOST|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^emdWalletSrcUrl/s|%OMS_PORT%|$OMS_PORT|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^REPOSITORY_URL/s|%OMS_HOST%|$OMS_HOST|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^REPOSITORY_URL/s|%OMS_PORT%|$OMS_PORT|”

$AGENT_HOME/sysman/config/emd.properties

sed -i “/^EMD_URL/s|%HOSTNAME%|$(hostname)|”

$AGENT_HOME/sysman/config/emd.properties

# Reconfiguration of Oracle EM agent is done

# Open ports for Oracle EM Agent and ssh services

system-config-securitylevel-tui –quiet \

–port=3872 \

–port=ssh

# reconfigure and rediscover targets

echo

su oracle -c “$AGENT_HOME/bin/agentca -f -d -t”

# secure Oracle EM agent

echo

echo “Securing Agent…”

su oracle -c “$AGENT_HOME/bin/emctl secure agent $SECURE_PASSWD”

# Start Oracle EM agent

echo

echo “Starting Oracle EM Agent…”

Oracle White Paper—Oracle VM Template Developer’s Guide

25

/etc/init.d/gcstartup start

# create startup script at runlevel 3 5

ln -sf /etc/rc.d/init.d/gcstartup /etc/rc.d/rc3.d/S99gcstartup

ln -sf /etc/rc.d/init.d/gcstartup /etc/rc.d/rc5.d/S99gcstartup

# set env variable for user ‘oracle’

cat >> /home/oracle/.bash_profile <<-EOF

# set environment variables

AGENT_HOME=$AGENT_HOME

ORACLE_HOME=\$AGENT_HOME

JAVA_HOME=\$ORACLE_HOME/jdk

PATH=\$ORACLE_HOME/bin:\$ORACLE_HOME/OPatch:\$PATH

EM_SECURE_VERBOSE=1

export ORACLE_HOME AGENT_HOME JAVA_HOME EM_SECURE_VERBOSE PATH

alias cdo=’cd \$ORACLE_HOME’

EOF

echo

echo “Reconfiguration of OEL5.2 and EM Agent 10.2.0.4 completed”

press_anykey

Oracle White Paper—Oracle VM Template Developer’s Guide

26

APPENDIX B

Source Code for Agent Template Reconfiguration Script, /u01/emagent-cleanup.sh

#!/bin/bash

AGENT_HOME=/u01/app/oracle/product/agent10g/

# substitute the parameter values by place holders

sed -i \

-e

‘/^REPOSITORY_URL=/s|=.*|=http://%OMS_HOST%:%OMS_PORT%/em/upload|’ \

-e

‘/^emdWalletSrcUrl=/s|=.*|=http://%OMS_HOST%:%OMS_PORT%/em/wallets/emd|’ \

-e ‘/^EMD_URL/s|=.*|=https://%HOSTNAME%:3872/emd/main/|’ \

$AGENT_HOME/sysman/config/emd.properties

# remove log files

rm -f $AGENT_HOME/sysman/log/*

# remove agent’s targets data

rm -f $AGENT_HOME/sysman/emd/upload/*

rm -rf $AGENT_HOME/sysman/emd/state/*

rm -f $AGENT_HOME/sysman/emd/lastupld.xml

# remove the runlevel startup script

rm -f /etc/rc.d/rc3.d/S99gcstartup

rm -f /etc/rc.d/rc5.d/S99gcstartup

Oracle VM Template Developer’s Guide

February 2009

Author: Tatyana Bagerman

Contributing Authors: Wiekus Beukes, Frank

Deng, Van Okamura

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

oracle.com

Copyright © 2009, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and

the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other

warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or

fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are

formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any

means, electronic or mechanical, for any purpose, without our prior written permission.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective

owners.

0109