Interface Ordering on VMware (vMX / vSRX / vQFX)

Just a quick post on Interface ordering when using vmxnet3 on VMware. When you add more than 4 vmxnet3 data interfaces, on the addition of the 5th interface you may see that interface ordering is no longer sequential. This is not a vMX specific issue, it is due to the way the VMware balances PCI slot numbers to the guest PCI bus topology. More info here.

For vMX, interface mapping would initially look like this:

vnic1 – fxp0 (e1000)
vnic2 – internal link (e1000)
vnic3 – ge-0/0/0 (vmxnet3)
vnic4 – ge-0/0/1 (vmxnet3)
vnic5 – ge-0/0/2 (vmxnet3)
vnic6 – ge-0/0/3 (vmxnet3)

However on the addition of vnic7 – ge-0/0/4, the interface ordering changes to:

vnic1 – fxp0
vnic2 – internal link
vnic3 – ge-0/0/0
vnic4 – ge-0/0/2
vnic5 – ge-0/0/3
vnic6 – ge-0/0/4
vnic7 – ge-0/0/1

As you can see, the interface ordering is no longer sequential.

To workaround this issue, you should perform the following steps:

  1. Power off the VFP VM and open the VMware Edit Settings page.
  2. If you are using the vSphere Web Client, select the VM Options tab, click Advanced in the left frame, and click Edit Configuration at the bottom of the page.
    If you are using the vSphere desktop client, select General under Advanced in the left frame, and click Configuration Parameters at the bottom of the page.)
  3. Change the value of the following pciBridge settings from TRUE to FALSE:
    pciBridge5.present
    pciBridge6.present
    pciBridge7.present
    Do not make any changes to pciBridge0.present and pciBridge4.present
  4. Click OK to update the VM settings, and then add any further vmxnet3 network adapters as may be required.
  5. Now power up the VFP VM and check that the interface ordering is now correct.

Note, if you require more than 7 data interfaces, it may be necessary to make further changes. Repeat the steps above, however now change the pciBridge.preset from FALSE to TRUE and update the below parameters (add any that are missing):
pciBridge5.virtualDev = pcieRootPort
pciBridge5.functions = 8
pciBridge5.pciSlotNumber = 22

Leave a Reply

Your email address will not be published. Required fields are marked *