{"id":262,"date":"2012-01-19T22:48:15","date_gmt":"2012-01-19T21:48:15","guid":{"rendered":"http:\/\/www.bussink.ch\/?p=262"},"modified":"2018-05-23T18:00:17","modified_gmt":"2018-05-23T16:00:17","slug":"esxi-multi-nic-multi-vlan-on-ucs","status":"publish","type":"post","link":"https:\/\/www.bussink.ch\/?p=262","title":{"rendered":"ESXi Multi-NIC & Multi-VLAN vMotion on UCS"},"content":{"rendered":"

I’ve been deploying a Cisco UCS Chassis with multiple Cisco B230 M2 Blades. Yet the uplinks switches of the Fabric Interconnect are medium-Enterprise sized Switches, and not some Nexus 5K or better. In a vSphere 5.0 cluster designs you add one or more NICs to the vMotion interface. With the enhancements of Sphere 5.0<\/a> you can combine multiple 1G or 10G network cards for vMotion, and get better performance.<\/p>\n

Duncan Epping wrote<\/a> on the 14th December 2011 on his site
\n[quote]”I had a question last week about multi NIC vMotion. The question was if multi NIC vMotion was a multi initiator \/ multi target solution. Meaning that, if available, on both the source and the destination multiple NICs are used for the vMotion \/ migration of a VM. Yes it is!”[\/quote]<\/p>\n

I was a bit worried by having my ESXi 5.0 vMotion traffic go up the Fabric Interconnect from my source Blade, across the network switches and back down the Fabric Interconnect and the target Blade. I decided to create two vmkernel port for vMotion per ESXi, and segregate them in two VLAN. Each VLAN is only used inside one Fabric Interconnect.<\/p>\n

vNIC Interface eth4 for vMotion-A on Fabric A<\/strong> (VLAN 70)<\/p>\n

\"\"<\/a>

vNIC Interface eth4-vMotionA<\/p><\/div>\n

vNIC Interface eth5 for vMotion-B on Fabric B<\/strong> (VLAN 71)<\/p>\n

\"\"<\/a>

vNIC Interface eth5-vMotionB<\/p><\/div>\n

And now let’s try this nice configuration.<\/p>\n

The VM that would be used for testing purposes is a fat nested vESX with 32 vCPU and 64GB of memory (named esx21). It is vMotion’ed from esx12 (Source network stats in Red) towards esx11 (Target network stats in Blue).<\/p>\n

\"\"<\/a><\/p>\n

The screenshot speaks for itself… we see that the vMotion uses both NICs and VLANs<\/strong> to transfer the memory to esx11. It flies at a total speed of 7504MbTX\/s to 7369MbRX\/s in two streams. One stream cannot pass the 5400Mb\/s rate, because of the limitation of the Cisco 2104XP FEX and the 6120XP Fabric Interconnect. Each 10Gb link is used by two B230 M2 blades.<\/p>\n

If you want to learn how to setup Multi-NIC vMotion<\/a>, check out Duncan’s post on the topic.<\/p>\n

Thanks go to Duncan Epping (@duncanyb<\/a>) and Dave Alexander (@ucs_dave<\/a>) for their help.<\/p>\n","protected":false},"excerpt":{"rendered":"

ESXi Multi-NIC & Multi-VLAN vMotion on Cisco UCS. Each vMotion VLAN is located on one Cisco 6120xp Fabric Interconnect<\/p>\n","protected":false},"author":1,"featured_media":265,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[7,9],"tags":[214,25,23,24,27],"_links":{"self":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/262"}],"collection":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=262"}],"version-history":[{"count":45,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/262\/revisions"}],"predecessor-version":[{"id":326,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/262\/revisions\/326"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/media\/265"}],"wp:attachment":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=262"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=262"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=262"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}