I have been playing with vCloud Connector 2.0 which is already a third release of the tool that enables VM (vApp) transfers between various VMware clouds based on vSphere or vCloud Director. Here follows a bunch of notes that I came up with that might help others. Note I expect that the reader is familiar with basic functionality and architecture of vCC.
vCloud Connector 2.0 (vCC) is backward compatible with vCloud Director 1.5, however some advanced feature like Content Sync will work in such deployments with limitations (modified templates are not removed, but added instead with timestamp in the name) or will not work at all (Stretch Deploy). vCloud Connector 1.5 does not work with vCloud Director 5.1.
Architecture and Network Flows
The architecture is similar to vCC 1.5, but the ports have changed (port 8443 not needed anymore). Here is the picture from the Installing and Configuring vCC 2.0 guide and let me dive deeper into some of the network flows (black circled numbers).
Although vCC Server has a web interface (VAMI) available at port 5480, the interface can be used only for appliance configuration and setting up connectivity to vCC Nodes. Internet Explorer is recommended to use here as Firefox or Chrome did not display some VAMI interface parts properly. The vCC Advanced Edition license key is entered here and SSL certificates related to the vCC Client – vCC Server communications can be enabled or generated and uploaded here as well. VAMI interface always uses https (on port 5480) with self signed certificate by default that cannot be replaced via the GUI, but if required it can be replaced in the following file on the appliance:
vCC end-users will use either vSphere client (the .NET version, as web client is not supported yet) or vcloud.vmware.com portal for actual management of the vApp transfers and other vCC features.
Although the picture shows port 80, if SSL is enabled, 443 will be used instead. If the replaced certificates do not use intermediate CA, the web interface cannot be used for their import and java keytool command must be used instead from appliance CLI as described in the installation guide. For some reason I was not able to create the private key with the GUI, but java keytool did the job.
vCC Server needs to be able to reach all vCC Nodes so the arrow should be also between vCC Server and the vCC Node in the destination cloud. Again the communication can go over port 80 or 443 depending on SSL configuration – this time on the vCC Nodes. The same caveat as with Server applies when replacing the certificates.
Note: Enabling SSL is recommended here as discussed below.
Flows 3 and 6/7
Prior attaching vCC Nodes to vCC Server, the Nodes need to be configured to connect to a cloud (vCenter or vCloud Director). SSL (port 443) is always used, but if you do not want to enable the Ignore SSL Cert checkbox, vCenter or vCloud need to have CA signed certificates. If you are using enterprise CA, you have to import the CA root certificate to a different keystore than the one available in the GUI as described in KB 2045007.
For the inter node communication, one node is designated as the Controller. The Controller initiates the connection. Which node is picked as the Controller depends on the Public checkbox setting in the vCC Node registration at vCC Server.
It is expected that the non Public vCC node is unreachable from outside and therefore has to be the initiator of the communication between nodes and is therefore the Controller. So the Controller Node works either in push or pull mode. If the other Node has SSL enabled (see Flow 2), https port 443 is used for the transfer between nodes. If the other Node does not have SSL enabled the transfer will fail.
This is a new feature for vCloud Director deployments when the provider can deploy shared (multitenant) node and connect it to provider’s cloud. The tenants then do not have to deploy their own nodes inside their organization VDCs with all the troubles of securing connectivity and appropriate transfer storage.
In highly secure public clouds it gets tricky where and how to deploy the shared node. It cannot be load balanced, but provider can deploy multiple shared nodes and give their IP addresses only to specific groups of tenants. The node should be as close as possible to the vCloud API somewhere in DMZ accessible from the internet but if Web Application Firewall is used it should still be in-between the node and API as it could be used as an attack vector to other organizations. The node does not keep any vCloud credentials for communication with vCloud API. Those are transferred by vCC Server so again SSL should be enabled (see Flow 2).
This is a new feature, great for maintenance of catalogs in various clouds or might be also used by the provider for management of public catalog directly from vSphere where a vSphere folder is automatically synced with a vCloud Director catalog. Note however that vCC Advanced Edition license is needed which currently cannot be obtain with provider VSPP license, but only through vCloud Suite bundles.
The default polling interval for synchronization is 6 hours. It can be changed but the change is unsupported. Following file can be edited on the vCC Server:
Look for <property name=”jobExecutionIntervalInMinutes” value=”360″ />.
Although the documentation states that ports 8443 and 8080 are used for Content Sync, my understanding is that they are used only internally on the vCC Server.
This is again a new but licensed feature which enables migration of VMs between clouds without needing to change their IPs or MAC addresses thanks to a VPN connection (not VXLAN as often confused) which is established between the original and destination cloud. The SSL VPN is established between Edges on the vApp networks so this is quite different from the Site-to-Site IPSec VPN between Edge Gateways even though it shows up in vShield Manager in the IPSec VPN section. Its configuration is not exposed in the vCloud Director GUI at all.
There is quite a long list of prerequisites to get this working and some of them are out of tenant’s controls as they relate to the provider’s vCloud Director architecture – e.g. vSphere and vCloud Network and Security versions and type of distributed virtual switch used. Stretch Deploy will not work with Cisco Nexus 1000V switch. The tenant most likely will not know which switch the provider is using until he will experience following error:
Unable to update network “Stretched_VM_network”.
java.util.concurrent.ExecutionException: com.vmware.vcloud.fabric.nsm.error.VsmException: VSM response error (100): A specified parameter was not correct.
The source and transferred destination VMs need to be connected to vSphere Distributed Switch 5.1 as the unclaimed traffic needs to be sent over the SSL tunnel and this is not currently supported with other virtual switches.
The Stretch Deploy process is relatively complicated with many actions that are happening in the background. This is all abstracted from the user as he can start the process easily from vCC GUI. However if you want to end the stretch deploy by removing the remote VM, or bring it home back this must be done manually.
Delete Stretch Deployed VM
- stop the remote vApp, this will terminate the VPN connection and destroy the vApp Edge
- delete the remote vApp
- delete the IPSec configuration in the vSphere cloud Edge in vShield Manager
- delete vCenter custom attributes of the VM which was stretched deployed (DatacenterExtendedEntityId, DatacenterExtensionRole)
Bring Home the Stretch Deployed VM
This must be done by running a script from the vCC Node managing the private cloud. So an access to the Node is needed, the script needs to be untared and quite a lot of information must be typed in when executed and their correctness is not verified until they are all typed in. This is definitely not task for an average user and I expect this part might be improved in later releases.