vRealize Automation with Multiple Cloud Endpoints

One of my customers had deployed true hybrid vRealize Automation with multiple cloud endpoints: vCloud Air and internal vCloud Director and AWS. I was called in to troubleshoot strange issue where sometimes deployment of a cloud multimachine blueprint (vApp) would work but most often it would fail with the following message:

VCloud Clone VM failed for machine: XXX100 [Workflow Instance Id=19026]
System.InvalidOperationException: Error occurred while getting vApp template with ID: urn:vcloud:vapptemplate:a21de50d-8b5e-41a6-81d1-acfd8ab8364b
…
INNER EXCEPTION: com.vmware.vcloud.sdk.utility.VCloudException: [ 8ae6fbca-e0d2-43e7-bc94-5bc9d776bf8d ] No access to entity “com.vmware.vcloud.entity.vapptemplate:a21de50d-8b5e-41a6-81d1-acfd8ab8364b”

Endpoint was properly configured, template existed, so what could be wrong? Why were we denied the access to the template?

It turns out that by design vRealize Automation does not match a template to a particular endpoint. It identifies it just by name. So in our case sometimes it would try to deploy the blueprint to wrong endpoint where the template of the particular name did not exist.

The fix is simple:

• Define reservation policies which would identify each endpoint.
• Assign them to the proper reservations
• Assign reservation policies to the Cloud vApp blueprint. This way there will never be confusion from which template to provision to which endpoint.

vCAC 6 – How To Generate Signed Certificates

This is the procedure I used to generate and import signed certificates for vCloud Automation Center 6.0.

Identity Appliance

• Generate private key and certificate signing request with OpenSSL. Common name is FQDN of the Identity Appliance.

openssl.exe req -newkey rsa:2048 -keyout sso.key -nodes -days 3650 -out sso.csr -sha256

Loading ‘screen’ into random state – done
Generating a 2048 bit RSA private key
……………………………………………….+++
………………………………………………+++
writing new private key to ‘sso.key’
—–
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter ‘.’, the field will be left blank.
—–
Country Name (2 letter code) [AU]:CZ
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:Prague
Organization Name (eg, company) [Internet Widgits Pty Ltd]:fojta.com
Organizational Unit Name (eg, section) []:vCAC Identity Appliance
Common Name (e.g. server FQDN or YOUR name) []:vcacsso.fojta.com
Email Address []:

Please enter the following ‘extra’ attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

• Sign the certificate signing request sso.csr with your CA. Download the signed certificate in Base 64 encoded format (sso.cer).
• In the SSO > SSL section of Identity Appliance VAMI interface (https://<Identity Appliance FQDN>:5480) choose action: Import PEM encoded Certificate.
• Paste the private key sso.key to the RSA Private Key field
• Paste the signed certificate sso.cer to the Certificate Chain section. Append CA root certificate as well.
• Click Replace Certificate.

vCAC Appliance

The process is identical – the only difference is the certificate Common Name and that we are using vCAC Appliance VAMI interface (http://<vCAC Appliance FQDN>:5480 for the import.

IaaS Components

In distributed architecture there can be multiple IaaS components: load balanced website components with Model Manager, Manager service with DEM Orchestrator (active/passive) and multiple Agents and DEM Workers. All those components are Windows based with identical procedure to create domain certificate.

• Open Microsoft Management Console (mmc.exe) and add Certificates Snap-In (manage Computer account, Local computer).
• Browse to the Personal Certificates folder and select action Request New Certificate.
• Request Active Directory Enrollment Policy > Web Server. In the Subject tab configure certificate properties (FullDN, Common Name, Country, etc.), in the General tab type friendly name and in the Private Key tab make private key exportable.
• Finish by clicking Enroll. Your Domain based CA should now issue the signed certificate.

See my older post that describes this in more detail with screenshots.

Integration of vCloud Automation Center with vCloud Director

Introduction

They used to be fierce competitors but now are good buddies. Who? vCloud Automation Center (vCAC) and vCloud Director. vCAC version 5.2 has just been released and it is the second release of what was before known as DynamicOps Cloud Automation Center but since the VMware acquisition of DynamicOps in July 2012 has been rebranded under the vCloud brand.

Why would you integrate these two products if they were competing with each other in the past and are perceived to do the same things?

This marchitecture slide below was used by Pat Gelsinger during his EMC World keynote and perfectly shows the relationship of the two.

vCAC is part of the Cloud Service Provisioning pillar and is a tool that can operate above heterogeneous set of infrastructure resources be it vSphere, other hypervisors, public or private clouds and physical servers. vCAC sits (among others) on top of vCloud Director which can be either private or public cloud and provides policy based (that’s the piece that controls What, Who, Where, Why, How much, How long, …) provisioning with nice, simple and extensible GUI.

The previous December 2012 vCAC 5.1 release had very little integration with vCloud Director, required vSphere access and therefore did not work with public clouds. That is not the case anymore and vCloud Director is treated as first class cloud citizen together with Amazon EC2.

Configuration

Here follows brief description how to set up a vCloud Director based cloud as an endpoint and how to create a provisioning blue print.

1. Create vCloud Director endpoint. vCAC Administrator > Endpoints > New Endpoint > Cloud > vApp (vCloud Director). In the public cloud scenario we would use Organization Administrator credentials, in private cloud we could use vCloud Administrator credentials.
   
2. Perform Data Collection of the newly created endpoint. One of the Distributed Execution Manager (DEM) Workers will by using vCloud API connect to the cloud and collect available inventory.
   
3. Once the data collection is finished we can create a new enterprise group which is basically a logical separation of the infrastructure resources. vCAC Administrator > Enterprise Group > New Enterprise Group. Name the group, add the account of the enterprise administrator and select the vCloud resources (OrgVDCs) that will belong here.
   
4. Now we can log in as the Enterprise Administrator and start managing the group. We have to create a provisioning group which is basically a set of users that will be able to provision VMs. Enterprise Administrator > Provisioning Group > New Provisioning Group. I called my provisioning group TestDev
5. Now we can create reservation of resources for the provisioning group.  Enterprise Administrator > Reservations > New Reservation > Cloud > vApp (vCloud Director)
   
   I have assigned 2606-Public OrgVDC to the TestDev provisioning group. In the Resources subtab we can select storage tiers and networks that will be available to this reservation and optionally limit memory and storage. Network profile (set of IP addresses) can be assigned to the networks as well.
   
6. The Who and Where is ready. Now we need to prepare the What. We will create (global) blueprint which will define the VMs that the users can provision. vCloud vApp blueprint consists of component blueprint which defines the actual VMs and a vApp blueprint that specifies policies for the whole vApp.
   So starting with the component blueprint: Enterprise Administrator > Global Blueprints > New Blueprint > Cloud > vApp Component (vCloud Director). In the Blueprint Information tab we assign provisioning groups that can use the blueprint, prefix for the naming of the VMs, optionally approval policy and costs.
   In the Build Information tab we specify the VM template and maximums for its configuration. 
7. Now we can create vApp blueprint: Enterprise Administrator > Global Blueprints > New Blueprint > Cloud > vApp (vCloud Director)
   In the Build Information tab we link the template to the component blueprint.
   
8. Now we can log in as a user from the Provisioning Group, Go to Self-Service and Request a machine from the blueprint.
   
9. After while the machine is ready to be consumed.
   

User Mapping

vCAC provisions and manages the vCloud Director vApps with the administrator account configured in the Cloud Endpoint. However as vApp is created it changes its ownership to the user who requested it. If the vCloud Organization does not contain user with an identical username as the vCAC VM requestor it will try to import the user from LDAP. This can obviously work only if LDAP is configured for the vCloud Organization which is realistic only in private vCloud Director deployments. In public vClouds you will therefore have to make sure that the user (either local or SAML imported) exists in the organization prior to vCAC provisioning.

Licensing

There is one difference when licensing private or public clouds. For private clouds it is possible to use CPU socket based licensing of vCloud Suite Enterprise. That obviously does not work with public clouds and therefore per VM licensing is needed.