Private VLANs (PVLANs) are useful in multitenant environments where there is a need to provide access to single server for all customers while not allowing them to see each other and NAT or dedicated networks are not an option. Agent based backup with central backup server or VM monitoring are example of such use-cases.
There might be a constraint that does not allow the usage PVLAN. For example hardware that does not support it (Cisco UCS) or VXLAN logical network. The latter I used in one of my designs. There was a need for single backup network stretched over pods without L2 connectivity. VXLAN can overlay L3 fabric and thus create single network spanning the pods however VXLAN does not support PVLANs. If VMware NSX is used then distributed (in-kernel) firewall can be used instead. If vCloud Network and Security is providing the VXLAN networks then there is App Firewall (vShield App) which unfortunately provides significant performance and throughput hit as it is inspecting every packet/frame in user-space service VM. It also adds complexity to the solution.
Access Control Lists
As of vSphere 5.5 vSphere distributed switch (vDS) supports access control lists (ACL) at the portgroup level. The ACL configuration is available only via the vSphere Web Client in the vDS portgroup > Manage > Settings > Policies > Edit > Traffic filtering and marking section. The following configuration can be used to provide the similar behavior to private VLAN.
Rule 1: Allow VMs to Server
Action: Allow
Type: MAC
Protocol/Traffic type: any
VLAN ID: any
Source Address: any
Destination Address: MAC address of the promiscous server or router
Rule 2: Allow Server to VMs
Action: Allow
Type: MAC
Protocol/Traffic type: any
VLAN ID: any
Source Address: MAC address of the promiscous server or router
Destination Address: any
Rule 3: Allow ARP
Action: Allow
Type: MAC
Protocol/Traffic type: any
VLAN ID: any
Source Address: any
Destination Address: FF:FF:FF:FF:FF:FF
Rule 4: Drop all
Action: Drop
Type: MAC
Protocol/Traffic type: any
VLAN ID: any
Source Address: any
Destination Address: any
Screenshot of the final configuration:
Note: As with PVLANs there is still a security issue of the tenant misconfiguring VMs IP address and causing Denial-of-Service for another VM with the same IP address. There are ways to remediate it but out of scope of this article.
I’ve also ran into the same issues regarding UCS not supporting PVLANs, or at least not passing PVLANs correctly through to a VDS. We couldn’t actually do VXLAN at the time (we wanted to have our IGMP snooping/querier done at the fabric interconnect, but the version of UCS firmware on the chassis didn’t support it yet), and after discussions with Cisco, we found the only way to support that was by introducing a N1KV.
Not the best solution, but it’s another way to deal with the above scenario…albeit what you describe is far more desirable. (Sweet post, man :D)
Hello Tom
Regarding the note at the end of the post, have you addressed the ways to mitigate this risk of tenant using wrong IP address in another post or could you kindly share your thoughts on how to do this ?
thanks
NSX SpoofGuard or similar technology can be used to prevent assignment of wrong IP address.
Thanks Tomas. Have you yet seen some solutions automating this as well as integrating it with IPAM system having self service capabilities ?
vCloud Director blocking tasks + RabbitMQ + vRealize Orchestrator (or HP OO or similar orchestrator) + IPAM solution
Thanks a lot Tomas
Hi
Thanks for this post.
I have similar configuration but I discovered few not solved issues with it.
1. I would like to prevent ARP Spoofing and not have any MAC addresses in VMs ARP table except MAC address of Backup Server
2. I am thinking if this solution can be used when PVLAN couldn’t be implemented (due to the hardware version etc.)?
3. What are limitations of this solution versus Private VLANs?
Thanks for any information if you have it about above points. I would like to simply separate traffic (ARP & IP) between VMs in the same VLAN (dedicated for backup solution) and Backup Server. I have dedicate virtual NIC in Backup VLAN, the same on VM installed as Backup server. Do you know if there is possibility to use this filtering as replacement for PVLAN functionality?