{"id":1541,"date":"2014-05-03T19:20:06","date_gmt":"2014-05-03T18:20:06","guid":{"rendered":"http:\/\/www.bussink.ch\/?p=1541"},"modified":"2018-05-23T17:57:38","modified_gmt":"2018-05-23T15:57:38","slug":"upgrading-mellanox-connectx-firmware-using-esxi","status":"publish","type":"post","link":"https:\/\/www.bussink.ch\/?p=1541","title":{"rendered":"Upgrading Mellanox ConnectX firmware within ESXi"},"content":{"rendered":"

Last summer, while reading the ServeTheHome.com website, I saw a great link to Ebay for Mellanox ConnectX-3 VPI <\/a>cards (MCX354A-FCBT). These cards where selling at $299 on ebay. I took three of the awesome cards. These Mellanox ConnectX-3 VPI adapters where simply too good to be true… Dual FDR 56Gb\/s or 40\/56GbE using PCIe Generation 3 slots. Having three of these Host Card Adapters without a InfiniBand switch is limiting.<\/p>\n

With my new Homelab 2014 design<\/a>, I now have two vSphere hosts that have PCIe Generation 3 slots, and using a simple QSFP+ Fiber Cable<\/a>, I can create a direct point-to-point connection between the two vSphere hosts.<\/p>\n

The Mellanox Firmware Tools (MFT) that can run within the vSphere 5.5 and allow to check the state of the InfiniBand adapter and even update the firmware.<\/p>\n

\"MFT<\/a><\/p>\n

Installing the tools is very straight forward.<\/p>\n

# esxcli software vib install -d \/tmp\/mlx-fw\/MLNX-MFT-ESXi5.5-3.5.1.7.zip<\/strong><\/p>\n

\"Install<\/a><\/p>\n

Unfortunately it requires a reboot.<\/p>\n

The next steps going to be to start the MST service, check the status of the of the Mellanox devices and query them to check the current level of firmware.<\/p>\n

I don’t need to have the Mellanox MST driver running all the time, so I will simply start it using \/opt\/mellanox\/bin\/mst start<\/strong>.<\/p>\n

Next we will query the state of all Mellanox devices in the host using \/opt\/mellanox\/bin\/mst status -v <\/strong>from which we will get the path to the devices.<\/p>\n

We then use the flint<\/strong> tool to query the devices to get their stats.<\/p>\n

\/opt\/mellanox\/bin\/flint -d \/dev\/mt40099_pci_cr0 hw query<\/strong><\/p>\n

and<\/p>\n

\/opt\/mellanox\/bin\/flint -d \/dev\/mt40099_pci_cr0 query<\/strong><\/p>\n

which returns us the current Firmware version and the GUIDs and MACs for the host card adapters.<\/p>\n

\"Mellanox<\/a><\/p>\n

Well as I’m running only FW Version 2.10.700 its time to upgrade this firmware to release 2.30.8000<\/p>\n

\u00a0\/opt\/mellanox\/bin\/flint -d \/dev\/mt4099_pci_cr0 -i \/tmp\/mlx-fw\/fw-ConnectX3-rel-2_30_8000-MCX354A-FCB_A1-FlexBoot-3.4.151_VPI.bin burn<\/strong> does the trick.<\/p>\n

\"Mellanox<\/a><\/p>\n

And we can quickly check the new running firmware on the InfiniBand adapter.<\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Last summer, while reading the ServeTheHome.com website, I saw a great link to Ebay for Mellanox ConnectX-3 VPI cards (MCX354A-FCBT). These cards where selling at $299 on ebay. I took three of the awesome cards. These Mellanox ConnectX-3 VPI adapters where simply too good to be true… Dual FDR 56Gb\/s or 40\/56GbE using PCIe Generation […]<\/p>\n","protected":false},"author":1,"featured_media":1305,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[253,251,9],"tags":[179,25,168,104,111,181,180,83],"_links":{"self":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1541"}],"collection":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1541"}],"version-history":[{"count":5,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1541\/revisions"}],"predecessor-version":[{"id":1550,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1541\/revisions\/1550"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/media\/1305"}],"wp:attachment":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1541"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1541"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1541"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}