Cisco Nexus 1000V and vCloud Director 5.1

Three months ago I wrote a blog post about vCloud Director and VXLAN integration with Cisco Nexus 1000V. The article was based on vCloud Director version 1.5.1. The new version of vCloud Director 5.1 integrates VXLAN in quite different way so I want to dedicate this post to it.

Although there is an excellent article about VXLAN setup in vCloud Director 5.1 it describes only the native vSphere VXLAN implementation. Cisco will most likely release an updated whitepaper on the integration, however it will take at least couple of months. Fortunately the setup is not that difficult and is basically the combination of the two procedures and can be derived by understanding of the following two points.

• Cisco Nexus 1000V does not integrate VXLAN via the Network isolation-backed network pool ‘hack’ into vCloud Director as was the case in the old 1.5.1 version.
• vShield Manager now fully abstracts the VXLAN creation, so from vCloud Director perspective it makes no difference if Nexus or vSphere distributed switch is used.

The process:

1. Enable VXLAN on the Nexus 1000V switch (network segmentation manager and segmentation features must be enabled)
2. Create port-profile on Cisco VSM with VXLAN capability.
3. Create vmkernel vmknic on each ESX host that will participate in VXLAN networks. This will be the uplink for all VXLAN encapsulated traffic.
4. Enlarge MTU on the ethernet uplink to at least 1550 (1600 recommended) to support the header overhead of the UDP encapsulated packets.
5. As multicast is used to limit the broadcast traffic to only those ESX hosts that have VMs in given VXLAN segment, enable IGMP snooping on the upstream physical switches.
6. If VXLAN traverses layer 3 networks enable layer 3 multicast routing (PIM).
7. If VXLAN traverses layer 3 networks and the VXLAN vmknic is in different subnet the VEM (and uses different default gateway) enable proxy ARP on the upstream layer 3 switch/router.
8. Integrate Cisco Nexus 1000V switch with vShield Manager (vSM): In the left vSM tab select Settings and Reports, Configuration, Networking. Click Add Switch Provider and enter the switch name, Service API base URL (https://<cisco vsm address>/n1k/services/NSM and admin VSM credentials.
9. Prepare the hosts/clusters for VXLAN. In the left tab select the datacenter in which VXLAN participating hosts are residing, select Network Virtualization tab, Preparation, click Edit and chose the Nexus distributed switch.
10. Create a pool of segment IDs and multicast range that can be used for VXLAN networks: while still in the Network Virtualization tab, Preparation, click Segment ID, Edit and enter the Segment ID pool and Multicast addresses.

    

    Segment IDs

That’s all. Now vCloud Director can create VXLAN networks on Nexus 1000V switch. The VXLAN network pool is created automatically with the creation of provider vDC.

Network Pools

Note: It is not possible to use both vSphere distributed switch and Nexus switch as an VXLAN Tunnel Endpoint on the same host. Although it is possible to mix switch providers on the same VXLAN, VMware recommends to use a consistent switch type (vendor etc.) and version across a given network scope. Inconsistent switch types can lead to undefined behavior in your VXLAN virtual wire.