Deploy 500 Edge Gateways in 500 Seconds

I am working on a few vCloud and NSX API scripts that collect data about Edge Gateways and needed to test how they work at scale. With this little PowerShell script leveraging NSX API I created 500 Edges in 500 seconds. And yes – I cheated as they were in undeployed state.

get-date
$NumberOfEdges = 500
$Username = "admin"
$Password = "default"
$NSXManager="nsx01.fojta.com"


$auth = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Username + ":" + $Password))
$head = @{"Authorization"="Basic $auth"}
$Request = "https://$NSXManager/api/4.0/edges"


$count=0
DO
{
$count++
$body = "
<edge>
	<datacenterMoid>datacenter-2</datacenterMoid>
	<name>$count</name>
	<description>TEST</description>
	<vnics>
		<vnic>
			<label>vNic_0</label>
			<name>vnic0</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>0</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_1</label>
			<name>vnic1</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>1</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_2</label>
			<name>vnic2</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>2</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_3</label>
			<name>vnic3</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>3</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_4</label>
			<name>vnic4</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>4</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_5</label>
			<name>vnic5</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>5</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_6</label>
			<name>vnic6</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>6</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_7</label>
			<name>vnic7</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>7</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_8</label>
			<name>vnic8</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>8</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
		<vnic>
			<label>vNic_9</label>
			<name>vnic9</name>
			<addressGroups/>
			<mtu>1500</mtu>
			<type>internal</type>
			<isConnected>false</isConnected>
			<index>9</index>
			<enableProxyArp>false</enableProxyArp>
			<enableSendRedirects>true</enableSendRedirects>
		</vnic>
	</vnics>
	<appliances>
		<deployAppliances>false</deployAppliances>
	</appliances>
</edge>"

$Response = Invoke-WebRequest -Uri $Request -Method 'Post' -Headers $head -ContentType "application/xml" -Body $body
Write-Host -NoNewline "#";
} Until ($count -ge $NumberOfEdges)
Write-Host
Write-Host "I have created $count Edges"
get-date

500 Edges in 500s

Well and you probably do not want to delete them by hand either.

Get-Date
$Username = "admin"
$Password = "default"
$NSXManager="nsx01.fojta.com"

$auth = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Username + ":" + $Password))
$head = @{"Authorization"="Basic $auth"}
$Request = "https://$NSXManager/api/3.0/edges"

$r = Invoke-WebRequest -Uri ($Request+"?startIndex=0&pageSize=1000") -Headers $head -ContentType "application/xml" -ErrorAction:Stop

foreach ($Edge in ([xml]$r.content).pagedEdgeList.edgePage.edgeSummary)
{
$edgeId = $Edge.objectId
If ($Edge.description -eq "TEST")
	{
	$count++
	$Response = Invoke-WebRequest -Uri ($Request+"/"+$edgeId) -Method 'Delete' -Headers $head -ContentType "application/xml"
	Write-Host Deleting Edge $edgeId
	}
}
Write-Host
Write-Host "I have deleted $count Edges"
Get-Date

Note: You need at least PowerShell 3.0.

vCloud Director: Online Migration of Virtual Data Center – Part II

About two years ago I have written a blog post describing how service provider (with cloud based on vCloud Director) can replace hardware without tenants actually noticing this. The provider can stand up new vSphere cluster with new hardware and migrate whole Provider VDC to it with running virtual machines. In my mind this is one of the distinguishing features from other cloud management systems.

Recently I had an opportunity to test this in large environment with vCloud Director 5.5 and would like to report how it is even more simplified compared to vCloud Director 5.1 which the former post was based on. No more API calls are needed and everything can be done from vCloud Administrator GUI in the following five steps:

  1. Prepare new cluster and create new provider VDC on it.
  2. Merge it with the old provider VDC. Disable its resource pool(s).
    Merge PVDCs
  3. Migrate all VMs from the old resource pool to the new one in vCloud Director (not in vSphere!!!)
    Migrate VMs
  4. Redeploy all Edge Gateways running on the old hardware.
  5. Detach the old resource pool.
    Detaching Resource Pool

While the first 4 steps are identical as in vCloud Director 5.1 the step 5 is where the simplification is. The last step takes care of both migration of catalog templates and deletion of shadow VMs.

As before this is possible only with Elastic VDCs (Pay-As-You-Go or Elastic Allocation Pool).

 

Automate ESXi Host VTEP Default Gateway

As discussed in my older article VXLAN routed transport network requires to set default gateway of vxlan stack on each ESXi host. While NSX has concept of IP Pools which allows automatic VTEP configuration (including gateway), older vCloud Network and Security (vShield) technology does not have this feature and VTEP IP address must be configure via DHCP or manually.

Following quick and dirty PowerCLI script shows how this can be automated at cluster level:


$hosts = Get-Cluster Cluster1 |Get-VMHost
foreach($vihost in $hosts){
$esxcli = get-vmhost $vihost | Get-EsxCli
$vihost.name
$result=$esxcli.network.ip.route.ipv4.add("10.40.0.1","vxlan","default")
$esxcli.network.ip.interface.ipv4.get("","vxlan")|format-list IPv4Address
$esxcli.network.ip.route.ipv4.list("vxlan")|Format-Table
}

The script sets vxlan stack default gateway to 10.40.0.1 on each host in the cluster ‘Cluster1‘ and displays each host name, VTEP IP address and vxlan routing table.

VXLAN Routing Script

Credit for esxcli to PowerCLI command conversion goes to Virten.net.

 

 

vCloud Director Portal Access over IPv6

I got interesting question from a colleague if vCloud Director portal can be accessed over IPv6. I suspected the answer is yes so I had little bit of fun and did a quick test.

With NSX load balancer in front of my two VCD cells I created IPv6 VIPs for HTTP, HTTPs and VMware Remote Console (TCP 443) traffic and used the existing IPv4 pools. I also added these IPv6 addresses to my DNS servers so name resolution and certificates would work and was ready to test.

Load Balancer Virtual IPs

 

As I terminate SSL session on the LB and insert client IP into the header with X-Insert-For-HTTP I could observe both IPv6 and IPv4 clients in the vCloud Director logs:

Client coming from IPv6 fd13:5905:f858:e502::20:

2015-01-16 19:06:06,431 | SECURITY | pool-eventPublishing-4-thread-1 | SyslogEventPublisher           | Event [id=6869f13c-0643-4afc-b083-982ecc920341, timestamp=1421431566380, type=com/vmware/vcloud/event/session/login, serviceNamespace=com.vmware.vcloud, properties={
...
currentContext.user.clientIpAddress=fd13%3A5905%3Af858%3Ae502%3A%3A20,
entity.name=administrator,
currentContext.user.proxyAddress=10.0.1.1,

Client coming from IPv4 10.0.2.104:


2015-01-16 19:29:46,879 | SECURITY | pool-eventPublishing-4-thread-1 | SyslogEventPublisher | Event [id=6a414e3f-19e7-45c2-83b7-5e0a7d90758b, timestamp=1421432986823, type=com/vmware/vcloud/event/session/login, serviceNamespace=com.vmware.vcloud, properties={
...
currentContext.user.clientIpAddress=10.0.2.104,
entity.name=administrator,
currentContext.user.proxyAddress=10.0.1.1,

Where 10.0.1.1 is load balancer internal interface. Remote Console proxy and OVF Tool also work.

Source NAT Rule for All Internal Networks in vCloud Director

In order to access external network resources from internal Org VDC networks Source NAT (SNAT) rule must be created on the Edge Gateway which translates internal IP address to a sub-allocated IP address of a particular external interface.

The internal source IP address can be entered in these formats:

  • Single IP address
  • Range of IP addresses
  • CIDR format

As you can see it is not possible to put ‘Any’ as it is with firewall rules configuration.

After investigating what would be the easiest option to use, this is what I found out:

In case where Edge Gateway is deployed by NSX Manager then it is possible to use following CIDR entry 0.0.0.0/0.

SNAT Rule

Unfortunately this is not working with Edge Gateway deployed by vShield Manager (vCNS) where Edge configuration fails with the following error:

…- java.util.concurrent.ExecutionException: com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (15012): Invalid IP Address input ‘0.0.0.0/0′ for field ‘rules.natRulesDtos[4].originalAddress’.
– com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (15012): Invalid IP Address input ‘0.0.0.0/0′ for field ‘rules.natRulesDtos[4].originalAddress’.
– VSM response error (15012): Invalid IP Address input ‘0.0.0.0/0′ for field ‘rules.natRulesDtos[4].originalAddress’.

The alternative is to use the following IP range: 0.0.0.1-255.255.255.253.