vCloud Director: Online Migration of Virtual Data Center – Part II

About two years ago I have written a blog post describing how service provider (with cloud based on vCloud Director) can replace hardware without tenants actually noticing this. The provider can stand up new vSphere cluster with new hardware and migrate whole Provider VDC to it with running virtual machines. In my mind this is one of the distinguishing features from other cloud management systems.

Recently I had an opportunity to test this in large environment with vCloud Director 5.5 and would like to report how it is even more simplified compared to vCloud Director 5.1 which the former post was based on. No more API calls are needed and everything can be done from vCloud Administrator GUI in the following five steps:

  1. Prepare new cluster and create new provider VDC on it.
  2. Merge it with the old provider VDC. Disable its resource pool(s).
    Merge PVDCs
  3. Migrate all VMs from the old resource pool to the new one in vCloud Director (not in vSphere!!!)
    Migrate VMs
  4. Redeploy all Edge Gateways running on the old hardware.
  5. Detach the old resource pool.
    Detaching Resource Pool

While the first 4 steps are identical as in vCloud Director 5.1 the step 5 is where the simplification is. The last step takes care of both migration of catalog templates and deletion of shadow VMs.

As before this is possible only with Elastic VDCs (Pay-As-You-Go or Elastic Allocation Pool).

 

Automate ESXi Host VTEP Default Gateway

As discussed in my older article VXLAN routed transport network requires to set default gateway of vxlan stack on each ESXi host. While NSX has concept of IP Pools which allows automatic VTEP configuration (including gateway), older vCloud Network and Security (vShield) technology does not have this feature and VTEP IP address must be configure via DHCP or manually.

Following quick and dirty PowerCLI script shows how this can be automated at cluster level:


$hosts = Get-Cluster Cluster1 |Get-VMHost
foreach($vihost in $hosts){
$esxcli = get-vmhost $vihost | Get-EsxCli
$vihost.name
$result=$esxcli.network.ip.route.ipv4.add("10.40.0.1","vxlan","default")
$esxcli.network.ip.interface.ipv4.get("","vxlan")|format-list IPv4Address
$esxcli.network.ip.route.ipv4.list("vxlan")|Format-Table
}

The script sets vxlan stack default gateway to 10.40.0.1 on each host in the cluster ‘Cluster1‘ and displays each host name, VTEP IP address and vxlan routing table.

VXLAN Routing Script

Credit for esxcli to PowerCLI command conversion goes to Virten.net.

 

 

vCloud Director Portal Access over IPv6

I got interesting question from a colleague if vCloud Director portal can be accessed over IPv6. I suspected the answer is yes so I had little bit of fun and did a quick test.

With NSX load balancer in front of my two VCD cells I created IPv6 VIPs for HTTP, HTTPs and VMware Remote Console (TCP 443) traffic and used the existing IPv4 pools. I also added these IPv6 addresses to my DNS servers so name resolution and certificates would work and was ready to test.

Load Balancer Virtual IPs

 

As I terminate SSL session on the LB and insert client IP into the header with X-Insert-For-HTTP I could observe both IPv6 and IPv4 clients in the vCloud Director logs:

Client coming from IPv6 fd13:5905:f858:e502::20:

2015-01-16 19:06:06,431 | SECURITY | pool-eventPublishing-4-thread-1 | SyslogEventPublisher           | Event [id=6869f13c-0643-4afc-b083-982ecc920341, timestamp=1421431566380, type=com/vmware/vcloud/event/session/login, serviceNamespace=com.vmware.vcloud, properties={
...
currentContext.user.clientIpAddress=fd13%3A5905%3Af858%3Ae502%3A%3A20,
entity.name=administrator,
currentContext.user.proxyAddress=10.0.1.1,

Client coming from IPv4 10.0.2.104:


2015-01-16 19:29:46,879 | SECURITY | pool-eventPublishing-4-thread-1 | SyslogEventPublisher | Event [id=6a414e3f-19e7-45c2-83b7-5e0a7d90758b, timestamp=1421432986823, type=com/vmware/vcloud/event/session/login, serviceNamespace=com.vmware.vcloud, properties={
...
currentContext.user.clientIpAddress=10.0.2.104,
entity.name=administrator,
currentContext.user.proxyAddress=10.0.1.1,

Where 10.0.1.1 is load balancer internal interface. Remote Console proxy and OVF Tool also work.

Source NAT Rule for All Internal Networks in vCloud Director

In order to access external network resources from internal Org VDC networks Source NAT (SNAT) rule must be created on the Edge Gateway which translates internal IP address to a sub-allocated IP address of a particular external interface.

The internal source IP address can be entered in these formats:

  • Single IP address
  • Range of IP addresses
  • CIDR format

As you can see it is not possible to put ‘Any’ as it is with firewall rules configuration.

After investigating what would be the easiest option to use, this is what I found out:

In case where Edge Gateway is deployed by NSX Manager then it is possible to use following CIDR entry 0.0.0.0/0.

SNAT Rule

Unfortunately this is not working with Edge Gateway deployed by vShield Manager (vCNS) where Edge configuration fails with the following error:

…- java.util.concurrent.ExecutionException: com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (15012): Invalid IP Address input ‘0.0.0.0/0′ for field ‘rules.natRulesDtos[4].originalAddress’.
– com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (15012): Invalid IP Address input ‘0.0.0.0/0′ for field ‘rules.natRulesDtos[4].originalAddress’.
– VSM response error (15012): Invalid IP Address input ‘0.0.0.0/0′ for field ‘rules.natRulesDtos[4].originalAddress’.

The alternative is to use the following IP range: 0.0.0.1-255.255.255.253.

How to Generate SSL Certificates for vRealize Operations

vRealize Operations 6 (vRops) has different SSL certificate generation requirements than the older version. I have not found it publicly documented anywhere so here it is:

  1. Generate private key:
    openssl genrsa -out vrops.key 2048
  2. Create certificate signing request:
    openssl req -new -key vrops.key -out vrops.scr -days 365 -sha256
  3. Sign vrops.scr by your Certificate Authority
  4. Create PEM text file which contains signed cert from #2, private key from #1 and CA certificate (optionally intermediate certs as well)
  5. Go to vRops admin portal and click the certificate icon in the top right corner (next to admin). Install the new certificate by uploading the PEM file from #4.

Install CertificateWait a little bit and then re-login. Reboot is not necessary.