Topic of promiscuous portgroup on virtual switch came up lately from different directions therefore I decided to summarize some information and also debunk one particular myth.
What is promiscuous port? This is what Wikipedia says:
In computer networking, promiscuous mode or promisc mode is a mode for a wired network interface controller (NIC) or wireless network interface controller (WNIC) that causes the controller to pass all traffic it receives to the central processing unit (CPU) rather than passing only the frames that the controller is intended to receive. This mode is normally used for packet sniffing that takes place on a router or on a computer connected to a hub (instead of a switch)…
So how can this be related to virtual environment and virtual switch? VMware KB article 1002934 sheds some light here:
By default, a guest operating system’s virtual network adapter only receives frames that are meant for it. Placing the guest’s network adapter in promiscuous mode causes it to receive all frames passed on the virtual switch that are allowed under the VLAN policy for the associated portgroup. This can be useful for intrusion detection monitoring or if a sniffer needs to analyze all traffic on the network segment.
So does this mean enabling promiscuous port on a vSwitch will make all vSwitch frames visible to VM connected to such port? So let’s step back and explain how VMware vSwitch works. The main difference from a physical switch is that it does not learn MAC addresses by observing passing traffic (and that is why you hear sometimes networking people saying it is not a real switch). It instead relies on the the information the hypervisor (VMkernel) provides about VM vNIC MAC addresses. It basically knows that all vSwitch non uplink ports are used only by VMs (with known MAC addresses). So a frame originating on a VM connected to vSwitch will be either delivered to the right port on the same host (if it is in the same VLAN and matches the destination MAC) or sent to uplink (usually trunk) port. There it is either flooded or switched to the right port by the physical switching infrastructure depending if it is unknown or known unicast and eventually delivered to the right host (if it belongs to a VM) or to a physical device.
So if you think about this behavior described above it should be clear that VM connected to a promiscuous port or portgroup will not see all the vSwitch traffic but only the traffic that is accessible on the host where the VM resides.
Let’s have a look at the following example with three virtual machines (VM A, VM B, VM C) each on different ESX host (Host A, Host B, Host C) connected via physical switch (Port A, Port B, Port C):
All VMs are in the same portgroup of VMware vSphere Distributed Switch in VLAN 100, while the port of VM A is set as promiscuous.
So which traffic between VM B and C will and will not VM A see?
When VM B (with MAC B) will try to talk to VM C it will send broadcast ARP packet to find VM C MAC address (MAC C). The physical switch will see the broadcast frame coming to Port B, will note the MAC B address on VLAN 100 is behind this port and will flood the frame to all other ports. vSwitch on host A will get this frame and forward it to all ports on VLAN 100 (it is a broadcast frame) thus to VM A.
VM B will get the ARP request and reply to it with unicast reply from MAC C to MAC B. Physical switch will enter the MAC C into its MAC table noting it is behind port C on VLAN 100 and switch the frame to already learned location of MAC B – to port B. vSwitch will then deliver the frame to VM B. As you can see no frame was delivered to host A and therefore VM A will not see the reply.
Now when communication between VM B and VM C has been established they can start talking with each other and physical switch knowing locations of MAC B and MAC C will switch the frames only between ports B and C of hosts B and C. VM A will see nothing from this unicast communication.
After while the MAC table on the physical switch will expire (if it has shorter timeout than VM B or C ARP cache). In such case it will forget the location of MAC B or C and will flood frame to B (or C) to all VLAN 100 ports and only then VM A will get the frame as the flooded frame reached host A as well.
Broadcast and possibly multicast traffic from VM B or C will reach host A and thus VM A as well.
This should debunk the myth that promiscuous port can be used for packet sniffing. For that you need port mirroring.
There are however use cases for promiscuous port and these are related to the (non-)learning behavior of vSwitch. If VM A would like to see traffic for additional MAC address D which is not hardcoded to its vNIC promiscuous port is requirement. Examples of such use case are nested VMs (VM A is virtual ESXi host) or floating MAC for highly available load balancing VMs (MAC masquarade). As the MAC D responds to ARP requests the physical switch will learn that MAC D is behind port A and will deliver the frame properly. vSwitch on host A will then flood the traffic to all promiscuous ports in the VLAN on the host as it does not know otherwise where to deliver it. Read William Lam’s article how to improve efficiency of this through VMware Fling (VMkernel vib plugin) that gives vSwitch learning ability.
Tom Fojta's Blog 