Embedding vCloud Availability Portal into vCloud Director UI

Some time ago I blogged about the possibility to link to vCloud Availability Portal directly from vCloud Director UI (here and here). This was done by inserting custom links into the vCloud Director Flex UI.

vCloud Director 9.x tenant HTML5 UI provides much richer possibilities to embed additional links, pages and full websites. My colleague Kelby Valenti wrote two whitepapers and one blog post how to do so.

Extending VMware vCloud Director User Interface Using Portal

ExtensibilityExtending VMware vCloud Director User Interface Using Portal Extensibility – Ticketing Example

Publishing vCloud Director User Interface Extensions

VMware also already released one service that integrates its UI into vCloud Director – vRealize Operations Tenant App.

In the below screenshot you can see VCD UI extended with five new sections that appear as additional menu options next to Datacenters, Libraries and Administration:

Stub Module – default example included in the UI Extensibility SDK providing static page example (Terms of Service, etc.).

Operations Manager – above mentioned vRealize Operations Tenant App

Blog – this blog embedded as iframe.

Documentation – Static page with links to vCloud Director documentation.

The last module is the vCloud Availability 2.0 portal – the subject of this article:

It is also embedded using iframe.

I am attaching the source files so you can download and adapt them for your purposes. You will also need the SDK and I recommend deployment automation created by Kelby as described in his blog post listed above.

Some notes:

  • The actual link to the portal is in the src/main/vcav.component.ts file. In my case it is https://portal.proxy.cpsbu.local so replace it with the correct link for your environment.
  • For security reasons the vCloud Availability portal prohibits being rendered in browser frame by setting  X-Frame-Options header to DENY. To work around this limitation I am replacing the header with X-Frame-Options: ALLOW-FROM <VCD-url> on the existing load balancer that is load balancing my two vCloud Availability Portal nodes as well as redirecting external port 443 to appliances’ port 8443. This is done with NSX Edge Gateway, SSL termination and the following application rule:
  • The link to the portal is also passing the vCloud Director session authentication token for Single Sign-On. Note that however in the current release (2.0.1) this functionality is broken.



vCloud Availability – Updated Whitepaper

I have updated my vCAT-SP vCloud Availability whitepaper to reflect changes that came with vCloud Availability 2.0 and vSphere 6.5/6.7.

It can be downloaded from the vCAT-SP site from the Storage and Availability section. The direct link to PDF is here. You will know you have the latest document if you see June 2018 date on the title page.

Edit highlights:

  • Installer Appliance section
  • Tenant and Provider portal sections
  • PSC section update
  • Supported Org VDC Topologies
  • Application Network Design
  • Network Bandwidth Requirements
  • Monitoring updates
  • Updates and Upgrades section
  • Monitoring with vRealize Operations

vCloud Availability – Cloud Proxy with Multiple NICs

Cloud Proxy is important component of vCloud Availability solution that sits in DMZ and tunnels replicated traffic in and out of the provider’s environment. For deep dive on the traffic flows see this older article. Cloud Proxy is very similar to vCloud Director cell, it runs on Linux VM, can be multihomed with internet and management facing interfaces.

By default, Cloud Proxy uses its primary network interface both for to-the-cloud (port 443) and from-the-cloud (port 31031) traffic. When multihoming is used, it might be beneficial to move the listener of the from-the-cloud traffic to the internal interface. This can be accomplished by adding the following line to the $VCLOUD_HOME/etc/global.properties file, with the IP address of the internal interface.

cloudproxy.fromcloudtunnel.host =

After restarting the cell, the listener will be moved the the new IP address.

Here is example from my lab:

Cloud Proxy with two NICs:

[root@vcd-01a ~]# ifconfig
eno16780032: flags=4163&lt;UP,BROADCAST,RUNNING,MULTICAST&gt; mtu 1500
inet netmask broadcast
inet6 fe80::250:56ff:fe3f:969 prefixlen 64 scopeid 0x20&lt;link&gt;
inet6 fdba:dd06:f00d:a400:250:56ff:fe3f:969 prefixlen 64 scopeid 0x0&lt;global&gt;
ether 00:50:56:3f:09:69 txqueuelen 1000 (Ethernet)
RX packets 45153159 bytes 11625785984 (10.8 GiB)
RX errors 0 dropped 1118 overruns 0 frame 0
TX packets 52432329 bytes 14266764397 (13.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens224: flags=4163&lt;UP,BROADCAST,RUNNING,MULTICAST&gt; mtu 1500
inet netmask broadcast
inet6 fe80::570a:1196:4322:521f prefixlen 64 scopeid 0x20&lt;link&gt;
inet6 fdba:dd06:f00d:a400:3495:c013:e72:cc58 prefixlen 64 scopeid 0x0&lt;global&gt;
ether 00:50:56:37:03:81 txqueuelen 1000 (Ethernet)
RX packets 4409 bytes 279816 (273.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 26 bytes 2691 (2.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Before the edit:

[root@vcd-01a ~]# netstat -an|grep 31031
tcp6 0 0 :::* LISTEN

After the edit and cell restart:

[root@vcd-01a ~]# netstat -an|grep 31031
tcp6 0 0 :::* LISTEN

What’s New in vCloud Availability 2.0.1

Minor patch of vCloud Availability 2.0.1 was released last week. Besides many bug fixes, improved documentation and support for Cassandra version 3.x I want to highlight two undocumented features and add upgrade comment.

Provider vSphere Web Client Plugin

This is a return from 1.0 version of an  experimental feature, where the provider can monitor state of vSphere Replication Manager Server, vSphere Replication Servers and all incoming and outgoing replications from inside the vSphere Web Client plugin in the particular (provider side) vCenter Server. This is especially useful for quick troubleshooting.

vRMS Status
vRS Status
Replication Status

Complex vSphere SSO Domain Support

Although it is not recommended to have multiple vCloud Director / vCloud Availability instances sharing the same vSphere SSO domain, it is now possible to accommodate such scenario. The reason why it is not recommended is, that it creates unnecessary dependency between the instances, limits upgradability and scale of each instance.

Upon startup vSphere Replication Cloud Service (vRCS) is querying SSO Lookup Service for Cassandra nodes and resource vCenter Servers. In order to limit the scope of such query to only those that belong to the particular vCloud Availability instance, create text file /opt/vmware/hms/conf/sites on all vRCS nodes with SSO site names that should be queried (one line per site).

Update 5/2/2018

There might be some confusion between vCenter SSO domain and vCenter SSO site. vCenter SSO domain name is usually vsphere.local and the domain is defined by the span of all replicated PCSs. Any single node contains all replicated data, such us IdP configurations, lookup service registrations, solution users, tags, license information, etc.

vCenter SSO domain can contain multiple SSO sites. The first site name is defined with the first PSC deployment (in vSphere 6.7 the default name is default-site).

The other SSO sites are created when joining an existing SSO domain.

So based on the example above, the /opt/vmware/hms/conf/sites file could have single line with text string default-site. vRCS will then ignore any other SSO sites in this vsphere.local domain.

Upgrade Options

You can upgrade to vCloud Availability 2.0.1 both from version 1.0.x and 2.0, however you need to use different upgrade ISO images for upgrading of the replication components (vRMS, vRCS and vRS). The installer and UI appliances are redeployed fresh as they are all stateless.


vSphere Replication Issue with ESXi 6.5U1

This is a quick post to highlight an issue vSphere Replication has with ESXi 6.5U1 for To-the-cloud replication.

Only customers that use vSphere Replication for DR or migrations to the cloud endpoints (e.g. vCloud Availability for vCloud Director) with ESXi 6.5U1 hosts are affected (ESXi 6.5 and older works fine). Also host-to-host replication is not affected.

The root cause is that ESXi 6.5U1 hosts are unable to retrieve from vSphere Replication Appliance vr2c-firewall.vib that is responsible for opening outgoing communication ports for replication traffic on the ESXi host firewall.

This results in inability to perform any to-the-cloud replications. To see the issue look into the host Firewall configuration in the Security Profile section. If you do not see Replication-to-Cloud Traffic section you are affected.

The picture below which traffic it is related to (red rectangle on the left):

If you would look into esxupdate.log on the host you will see error: [Errno 14] curl#56 – “Content-Length: in 200 response”.

Until a fix is going to be released here is a workaround:

  1. Download the vr2c-firewall.vib from the vSphere Replication Appliance: https://vSphere-Replication-Appliance-ip-or-fqdn:8043/vib/vr2c-firewall.vib.
  2. Upload the vib to a shared location (datastore)
  3. Install the vib to every host with the following command: esxcli software vib install -v /vmfs/volumes/<datastore>/vr2c-firewall.vib
  4. Verify the fix was installed properly with: esxcli software vib list | grep vr2c