Organization VDC Permissions in vCloud Director

As of vCloud Director 8.10 it is possible for the tenant Organization Administrator to control access to Organization VDCs. This enables the following use case:

  • In one Organization there are multiple Org VDCs belonging to different business units. Each Org VDC has its own Organization VDC administrator (from the business unit) who can manage Org VDC resources (networks, Edge Gateways) in his Organization VDC but does not see VDCs of other business unit.

The capability is currently available only with vCloud API. There are also four new related user rights that system administrator can use to create new roles.

  • Allow Access to All Organization VDCs
  • Edit Access Control List of Organization VDCs
  • View Access Control List of Organization VDCs
  • Implicitly Import User/Group from IdP while Editing VDC ACL

Org VDC Admin Role

Note that I have removed most of Organization and all User rights from the custom Org VDC Admin role which follows the described use case.

In the following example I have two Org VDCs – Production and Test in Organization ACME. The Organization Administrator (acmeadmin) created two Org VDC Admin users – acmeadminprod and acmeadmintest. Now he will create access for each to his corresponding Org VDC.

Users

As was mentioned above, this is done with vCloud API PUT request. First we need to find out Org VDC and user references.

GET https://vcloud.fojta.com/api/admin/org/02b433db-0b37-4304-b07b-0717255ec297

Accept: application/*+xml;version=20.0
x-vcloud-authorization: 3e131f9e3bc240269a7758fdb6c1bf7f</pre>

<?xml version="1.0" encoding="UTF-8"?>
<AdminOrg xmlns="http://www.vmware.com/vcloud/v1.5" name="ACME" id="urn:vcloud:org:02b433db-0b37-4304-b07b-0717255ec297" href="https://vcloud.fojta.com/api/admin/org/02b433db-0b37-4304-b07b-0717255ec297" type="application/vnd.vmware.admin.organization+xml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://vcloud.fojta.com/api/v1.5/schema/master.xsd">
    <Link rel="down" href="https://vcloud.fojta.com/api/tasksList/02b433db-0b37-4304-b07b-0717255ec297" type="application/vnd.vmware.vcloud.tasksList+xml"/>
    <Link rel="down" href="https://vcloud.fojta.com/api/admin/org/02b433db-0b37-4304-b07b-0717255ec297/metadata" type="application/vnd.vmware.vcloud.metadata+xml"/>
...
    <Users>
        <UserReference href="https://vcloud.fojta.com/api/admin/user/395b2a93-d5ef-4c55-a316-ab500ea4829c" name="acmeuser" type="application/vnd.vmware.admin.user+xml"/>
        <UserReference href="https://vcloud.fojta.com/api/admin/user/46f40e2c-ed07-428f-af82-e691329f3cba" name="acmeadmin" type="application/vnd.vmware.admin.user+xml"/>
        <UserReference href="https://vcloud.fojta.com/api/admin/user/8c1af691-baa9-49db-9bf4-a5ad0562f92b" name="acmeadmintest" type="application/vnd.vmware.admin.user+xml"/>
        <UserReference href="https://vcloud.fojta.com/api/admin/user/e20edd07-e426-4a72-8f49-718b37685da6" name="acmeadminprod" type="application/vnd.vmware.admin.user+xml"/>
    </Users>
...
    <Vdcs>
        <Vdc href="https://vcloud.fojta.com/api/vdc/18d1590d-e033-4618-8179-432f99e5c54a" name="Test" type="application/vnd.vmware.vcloud.vdc+xml"/>
        <Vdc href="https://vcloud.fojta.com/api/vdc/47564d52-9204-40b1-b315-a00d59945cfd" name="Production" type="application/vnd.vmware.vcloud.vdc+xml"/>
    </Vdcs>
...
</AdminOrg>

Now we can construct PUT request for each Org VDC to assign user access:

Test Org VDC

PUT https://vcloud.fojta.com/api/vdc/18d1590d-e033-4618-8179-432f99e5c54a/action/controlAccess

Accept: application/*+xml;version=20.0
x-vcloud-authorization: 3e131f9e3bc240269a7758fdb6c1bf7f
Content-type: application/vnd.vmware.vcloud.controlAccess+xml

<?xml version="1.0" encoding="UTF-8"?>
<ControlAccessParams xmlns="http://www.vmware.com/vcloud/v1.5">
	<IsSharedToEveryone>false</IsSharedToEveryone>
	<AccessSettings>
		<AccessSetting>
			<Subject href="https://vcloud.fojta.com/api/admin/user/8c1af691-baa9-49db-9bf4-a5ad0562f92b" name="acmeadmintest" type="application/vnd.vmware.admin.user+xml"/>
			<AccessLevel>ReadOnly</AccessLevel>
		</AccessSetting>
	</AccessSettings>
</ControlAccessParams>

Production Org VDC

PUT https://vcloud.fojta.com/api/vdc/47564d52-9204-40b1-b315-a00d59945cfd/action/controlAccess

Accept: application/*+xml;version=20.0
x-vcloud-authorization: 3e131f9e3bc240269a7758fdb6c1bf7f
Content-type: application/vnd.vmware.vcloud.controlAccess+xml

<?xml version="1.0" encoding="UTF-8"?>
<ControlAccessParams xmlns="http://www.vmware.com/vcloud/v1.5">
	<IsSharedToEveryone>false</IsSharedToEveryone>
	<AccessSettings>
		<AccessSetting>
			<Subject href="https://vcloud.fojta.com/api/admin/user/e20edd07-e426-4a72-8f49-718b37685da6" name="acmeadminprod" type="application/vnd.vmware.admin.user+xml"/>
			<AccessLevel>ReadOnly</AccessLevel>
		</AccessSetting>
	</AccessSettings>
</ControlAccessParams>

Now we can log in as each Org VDC Administrator and verify that we see only one Org VDC:

User acmeadminprod can see only Org VDC Production:

Prod Org VDC

User acmeadmintest can see only Org VDC Test:

Test Org VDC

As both Org VDCs were set as private the Organization Administrator will now have to explicitly enable access for regular users to each Org VDC with the same PUT request. There is maximum of 200 user/group references per Org VDC.

Query Guest OS Customization Status with vCloud API

Very useful new feature of vCloud Director 8.10 is the possibility to query with vCloud API guest OS customization status. Typical use case is when the tenant runs custom orchestration to deploy VM and install and configure application in it. When the VM is powered-on for the first time, the operating system boots up and vCloud Director runs customization scripts to set identity (hostname, SID), networking, administrator password, etc. Read Massimo’s blog for deep dive into guest cutomization.

Tenant’s custom orchestration then needs to wait for the customization to finish and then finally log into the VM and proceed with the application installation and configuration. The problem in the past was that there was no easy way to find out if the guest customization was finished. Not anymore.

With vCloud API you can easily query the vApp VM to get its customization status.

Example:

GET https://vcd-01a.corp.local/api/vApp/vm-22e51563-52a6-4a13-961a-d9dffa6aabf5/guestcustomizationstatus	
	
<?xml version="1.0" encoding="UTF-8"?>
<GuestCustomizationStatusSection xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://vcd-01a.corp.local/api/v1.5/schema/master.xsd">
	    <GuestCustStatus>GC_PENDING</GuestCustStatus>
</GuestCustomizationStatusSection>

The possible guest customization states are:

  • GC_PENDING: Guest Customization is either not started or in progress.
  • REBOOT_PENDING: Guest Customization is successful, but reboot is pending.
  • GC_FAILED: Guest Customization failed, error is logged.
  • POST_GC_PENDING: Reboot has happened, waiting for post customization script to complete
  • GC_COMPLETE: Guest customization is complete

Import Running VM to vCloud Director

Another nice little feature in the recently released vCloud Director 8.10 (read eight dot ten) is the ability to import running VM under vCloud Director management.

In the past the vCloud system administrator could import virtual machine running in vCloud Director managed vSphere environment, however that VM had to be turned off.

Import from vSphere

Now in vCloud DIrector 8.10 the VM can be running which enables new use cases:

  • You can take existing vSphere environment under management of vCloud Director without impacting the workloads. Example would be going from vSphere only managed services to self service with vSphere + vCloud Director
  • Migrations of running VMs from vSphere to vCloud Director or between vCloud Directors. Cross vCenter Server vMotion nicely complements this feature. Cross-Cloud vMotion.

There are currently some limitations and considerations:

  • It is API only feature. The GUI button (above) can be used to import only powered-off VMs.
  • VM must be running on resources that are available to the Org VDC where it is being imported. That includes compute (cluster), storage policies and network port group. Obviously the networking is the most complex to get it right during the migration and requires some thought.
  • Import of VM with IDE controller is not supported.
  • A new vApp is created for the imported VM. You cannot import VM to an existing vApp.
  • As an input to the API call MoRef ID of the VM must be provided. The existing vCloud API call to list vSphere VMs outside of vCloud Director management however does not list running VMs. Therefore the MoRef ID must be acquired with vSphere API.

Example:

POST https://vcd-01a.corp.local/api/admin/extension/vimServer/3702cc8f-c23f-4a66-b9f3-73fc4b58ba82/importVmAsVApp

Accept: application/*+xml;version=20.0
x-vcloud-authorization: 3e131f9e3bc240269a7758fdb6c1bf7f
Content-type: application/vnd.vmware.admin.importVmAsVAppParams+xml


<?xml version="1.0" encoding="UTF-8"?>
<ImportVmAsVAppParams xmlns="http://www.vmware.com/vcloud/extension/v1.5" name="ImportedTestVM" sourceMove="true">
   <VmMoRef>vm-86</VmMoRef>
   <Vdc href="https://vcd-01a.corp.local/api/vdc/a10bdf92-18dc-474b-aafc-42d31ba83207" />
</ImportVmAsVAppParams>

vCloud Director: Share Console Proxy IP with UI/API IP Address

New vCloud DIrector 8.10 (read eight dot ten) is out and with it some little neat features. Let me quickly talk about one of them – the ability to run vCloud Director cell with just 1 IP address.

In the past you always had to configure vCloud Director cell at least with two IP addresses. One for the web interface (providing UI and API) and the other for remote console proxy. The reason was that both services shared the same port 443. In vCloud Director 8.10 there is possibility to specify ports for each service and thus use just one IP address. This helps if your DMZ subnet is too small and you need to deploy more VMs into that network (more cells, databases, etc.).

Note that the configure script will not ask you for ports, instead you need to use unattended installation option or add port entries afterward in global.config file.

Unattended Installation

Here is the example of configure parameters that sets console proxy to the same IP address as http (10.0.1.60) and uses port 8443 instead of the standard 443:


/opt/vmware/vcloud-director/bin/configure" -cons 10.0.1.60 --console-proxy-port-https 8443 -ip 10.0.1.60 --primary-port-http 80 –-primary-port-https 443 -dbhost 10.0.4.195 -dbport 1433 -dbtype sqlserver -dbinstance MSSQLSERVER -dbname vcloud -dbuser vcloud -dbpassword 'VMware1!' -k /opt/vmware/vcloud-director/etc/certificates.ks -w 'passwd' -loghost 10.0.4.211 -logport 514 -g --enable-ceip true -unattended

Global Config

An alternative option is to edit the /opt/vmware/vcloud-director/etc/global.config file and add new port entries:

Before:


...
product.version = 8.10.0.3879706
product.build_date = 2016-05-12T20:32:07-0700
vcloud.cell.ip.primary = 10.0.1.60
consoleproxy.host.https = 10.0.1.61
...

After


...
product.version = 8.10.0.3879706
product.build_date = 2016-05-12T20:32:07-0700
vcloud.cell.ip.primary = 10.0.1.60
consoleproxy.host.https = 10.0.1.60
consoleproxy.port.https = 8443
vcloud.http.port.standard = 80
vcloud.http.port.ssl = 443
...

Do not forget to reconfigure your loadbalancer remote console pool to point to the new IP-port combination.

NSX L2 Bridging Options

I had recently multiple discussions about NSX and its Layer 2 bridging capabilities with various service providers. Let me summarize some important points and considerations when you would use which.

Why?

Let’s start with simple question – why would you need layer 2 bridging? Here are some use cases:

  • The end-user wants to burst their application to the cloud but wants to keep certain components on-site and because its legacy application it cannot be re-IP’d or requires single subnet communication.
  • The service provider is building new cloud offering next to legacy business (collocation, managed services) and wants to enable existing customers to migrate or extend their workloads to the cloud seamlessly (no IP address changes)

How?

NSX offers three ways how to bridge layer two networks.

Layer 2 VPN

This is proprietary VPN solution which enables to create encrypted tunnel across IP networks between Edge Service Gateways that stitches one or more L2 networks. These Edge Gateways can be deployed in different management domains and there is also option of deploying standalone Edge which does not require NSX license. This is great for the cloud bursting use case. I have blogged about L2VPN already in the past here.

While this option is very flexible it is also quite CPU intensive for both the L2 VPN Client and Server Edge VMs. This option provides up to 2Gb throughput.

NSX Native L2 Bridging

L2 bridge is created in the ESXi VMkernel hypervisor by deploying a Logical router control VM. The control VM is used only for the bridge configuration and its pinning to a particular ESXi host. As the bridging happens in the VMkernel it is possible to achieve impressive line rate (10 Gb) throughput.

It is possible to bridge only VXLAN based logical switch with VLAN based port group. The same physical uplink must be utilized so this means that the VLAN port group must be on the same vSphere Distributed Switch (vDS) that is prepared with the VXLAN VTEP and where the VXLAN logical switch portgroups are created.

L2Bridge

VLAN and VXLAN portgroups are on the same vDS

VLAN and VXLAN portgroups are on the same vDS

The above fact prohibits a scenario where you would have Edge Cluster with multiple pairs of uplinks connected to separate vDS switches. One for VLAN based traffic and the other for VXLAN traffic. You cannot create NSX native L2 bridge instance between two vDS switches.

This important especially for the collocation use case mentioned at the beginning. In order to use the L2 bridge the customer VLAN must be connected to the Edge Cluster Top of the Rack pair of switches.

If this is not possible, as a workaround the service provider can use the L2 VPN option – it is even possible to run both L2 VPN Server and Client Edges on the same host connected through a transit VXLAN network where one Edge is connected to trunk with VLAN networks from one vDS and the other to trunk with VXLAN networks on another vDS. Unfortunately this has performance impact (even if NULL-MD5 encryption is used) and should be used only for temporary migration use cases.

L2VPN interfaces

L2VPN

Hardware VTEP

The last bridging option discussed is a new feature of NSX  6.2. It is possible to extend the VXLAN logical switch all the way to a compatible hardware device (switch from a VMware partner) that acts as Layer 2 gateway and bridges the logical switch with a VLAN network. The device performing the function of hardware VTEP is managed from NSX via OVSDB protocol while the control plane is still managed by NSX Controllers. More details are in the following white paper.

As this option requires new dedicated and NSX compatible switching hardware it is more useful for the permanent use cases.