Datastore Cluster Issue in vCloud Director 5.5.1

Just found out there is a known issue with removing datastores in and out of datastore clusters. In vCloud Director 5.1 this was working fine. vCloud Director refers to storage via storage policies (formerly profiles) so you could on-the fly change the datastore cluster structure (as long as all datastore inside had the same storage policy).

However in vCloud Director 5.5.1 if you move a datastore in or out of a datastore cluster, vCloud Director will loose it. The fix is described in KB 2075366 and involves clean up of vCloud Director database inventory data.

VXLAN as an External vCloud Director Network

I was asked by a customer how to use a VXLAN network as an external network in vCloud Director. I thought there was already written blog article about it bud did not find any. So writing the answer here will benefit hopefully others as well.

Why?

First questions would be why would you do it? Aren’t vCloud Director external networks supposed to be the way to connect internal vCloud networks (usually VXLAN based) to the external world via VLAN based networks through a Edge Gateway device? Yes, but there are a few use cases for use of VXLAN network as an external network.

  • Usage of different virtual edge router other than vShield Edge that supports needed features (IPv6, dynamic routing protocols). In picture below you see virtual Fortigate router in place of vShield Edge. The router is deployed manually and its internal interface is connected to a VXLAN network (again created manually) which acts as external network that is directly connected to OrgVDC network. This helps saving VLANs which are usually scarce resource in service provider environment.
    Virtual Router
  • Service network spanning multiple pods crossing L3 boundaries. Each pod (cluster) has its own L2 networking so VLAN cannot span all clusters. However VXLAN can. So service network (for example syslog or monitoring network) can be used by any VM in any rack. See this article how to secure such network in multitenant environment.
    Service Network

How?

Although you can easily manually create a VXLAN network directly in vShield Manager (or in vSphere Web Client if you use NSX) you will not see the VXLAN portgroup in vCloud Director GUI.

Service Network

External Networks 1

The fix is simple. vCloud Director is filtering out all portgroups that start with ‘vxw’ string. Rename the portgroup in vCenter Server (remove the string) and you will be able to select the portgroup as an External Network.

External Networks 2

How To Download vCloud VMRC Plugin

I had a question if there is a way to download vCloud Director VMware Remote Console plug-in without the need to actually log in into vCloud Director. For example in order to prepare desktop images for your vCloud users.

Yes, it is possible and here are the links for vCloud Director 5.5:

https://vcd-url/cloud/vmrc/VMware-ClientIntegrationPlugin-5.5.0.exe
https://vcd-url/cloud/vmrc/VMware-ClientIntegrationPlugin-5.5.0.i386.bundle
https://vcd-url/cloud/vmrc/VMware-ClientIntegrationPlugin-5.5.0.mac64.dmg
https://vcd-url/cloud/vmrc/VMware-ClientIntegrationPlugin-5.5.0.x86_64.bundle

where vcd-url is the vCloud Director endpoint address. For example in case of vCHS the link looks like:

https://XXX.vchs.vmware.com/cloud/vmrc/VMware-ClientIntegrationPlugin-5.5.0.exe (replace XXX with your particular cloud instance name).

Improve vSphere Web Client Performance

I was always bothered by slow performance of my lab installation of vSphere Web Client (version 5.5). Although my lab is small I have large number of plugins. I use the Windows installable version which is running together with vCenter, Inventory service, Update Manager and database server.

I noticed that the Java process was using over 1 GB of RAM. The fix was simple – add more memory to the VM and to Web Client Tomcat server:

Edit wrapper.conf file which is located in C:\Program Files\VMware\Infrastructure\vSphereWebClient\server\bin\service\conf.

and increase

wrapper.java.maxmemory=1024m

in the JVM Memory section. I increased the value to 3072m.

Shuttle PC – vSphere 5.5 White Box Gotchas

I love barebone Shuttle PCs for home lab purposes. They have very compact design, can fit 3 hard disks (great for VSAN), low power consumption, are quiet and can fit up to 32 GB of RAM.

SH87R6

I have two of them (SZ68R5 and SH87R6). I was recently reinstalling them to brand new vSphere 5.5 U1 to prep them up for VSAN and here are some problems I encountered.

On-board Realtek NIC

Although I always add dual NIC Intel Pro/1000 PT card, there is one on-board Realtek 8111G NIC as well. This card used to work with vSphere 5.1, however as of vSphere 5.5 the driver for it is no longer included. If you upgraded from vSphere 5.1 to 5.5 the card will still work, but brand new installation will not recognize it.

To solve it I have created custom image with the vSphere 5.1 Realtek 8168 driver. Here is ImageBuilder PowerCLI script I used (includes also Cisco and NetApp VIBs).

Add-EsxSoftwareDepot .\update-from-esxi5.5-5.5_update01.zip
Add-EsxSoftwareDepot .\VEM550-201401164104-BG-release.zip
Add-EsxSoftwareDepot .\NetAppNasPlugin.v20.zip
New-EsxImageProfile -CloneProfile ESXi-5.5.0-20140302001-standard -name ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp -vendor Fojta
Add-EsxSoftwareDepot https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
Add-EsxSoftwarePackage -ImageProfile ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp -SoftwarePackage net-r8168
Add-EsxSoftwarePackage -ImageProfile ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp -SoftwarePackage net-r8169
Add-EsxSoftwarePackage -ImageProfile ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp -SoftwarePackage cisco-vem-v164-esx
Add-EsxSoftwarePackage -ImageProfile ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp -SoftwarePackage NetAppNasPlugin
Export-EsxImageProfile -ImageProfile ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp -ExportToIso -filepath ESXi-5.5.0-20140302001-Cisco-Realtek-NetApp.iso

Thanks go to: Paul Braren and Erik Bussink for the hints.

USB Flash Disk

I am booting off of USB flash disk in order to preserve internal SDD and HDD for VSAN. I use 8 GB SanDisk Cruzer Fit which is so tiny you don’t even notice it in the USB slot.

I prepared the USB flash disk in Workstation then plug it and do the initial network configuration. The strange thing I was encountering was that I was able to boot from the disk, but the ESXi installation was stateless. Any configuration changes I did would be lost after reboot.

The reason was that although I could boot of the USB, ESXi did not recognize it and would not save the configuration. After some troubleshooting I found out that I need to use USB 2.0 ports (B6) and not USB 3.0 (B7).

Shuttle USB 2.0 and 3.0 Ports

Identical BIOS UUID

Both Shuttle boxes have identical BIOS UUID:

esxcfg-info | grep “BIOS UUID”

results in 03000200-0400-0500-0006-000700080009.

The is big problem for Nexus 1000V who identifies VEMs via the supposedly unique BIOS UUID. For now I removed one of the boxes from the Nexus switch. I would be grateful for any info how to flash BIOS with different UUID.

EDIT 3/22/2014:

Thanks to the comments below from RonTom42 here are the steps needed to change BIOS UUID:

1. Download AMIDMI.EXE tool from here.
2. Download FreeDOS and put it onto USB stick from here. I used FreeDOS-1.1-memstick-2-2048M.img with the Win32 Disk Imager option.
3. Copy AMIDMI.EXE to USB disk
4. Boot Shuttle from the USB disk. Enter the fdos option 4.
5. Run amidmi /u command
6. Reboot ESXi host and check BIOS UUID with: esxcfg-info | grep “BIOS UUID”