{"id":1572,"date":"2014-05-12T21:53:53","date_gmt":"2014-05-12T20:53:53","guid":{"rendered":"http:\/\/www.bussink.ch\/?p=1572"},"modified":"2018-05-23T17:57:37","modified_gmt":"2018-05-23T15:57:37","slug":"speed-testing-40g-ethernet-in-the-homelab","status":"publish","type":"post","link":"https:\/\/www.bussink.ch\/?p=1572","title":{"rendered":"Speed testing 40G Ethernet in the Homelab"},"content":{"rendered":"

In my previous post, I described the building of two Linux virtual machines to benchmark the network<\/a>. Here are the results.<\/p>\n

\"homelab_network_1g_10g_40g_iperf_testing\"<\/a><\/p>\n

 <\/p>\n

The first blip<\/strong>, is running iperf to the maximum speed between the two Linux VMs at 1Gbps<\/strong>, on separate hosts using Intel I350-T2 adapters<\/a>.<\/p>\n

The second spike<\/strong> (or vmnic0), is running iperf to the maximum speed between two Linux VMs at 10Gbps<\/strong>. The two ESXi hosts are using Intel X540-T2 adapters<\/a>.<\/p>\n

The third mountain<\/strong>\u00a0(or vmnic4) and most impressive result is running iperf between the Linux VMs using 40Gb Ethernet<\/strong>. The two ESXi hosts are using Mellanox ConnectX-3 VPI adapters<\/a>.<\/p>\n

The Homelab 2014 ESXi hosts<\/a>, uses a\u00a0Supermicro X9SRH-7TF <\/a>come with an embedded Intel X540-T2. We can more closely see the \u00a0results of the iperf test at 10Gbps in the following picture.<\/span><\/p>\n

\"homelab_network_10g_iperf_testing\"<\/a><\/p>\n

I also got last summer from Ebay, a set of Mellanox ConnectX-3 VPI<\/strong> Dual Adapters for $300. These cards support InfiniBand 40Gb\/s and 56Gb\/s, and Ethernet at 10Gb\/s and 40Gb\/s. By default, vSphere 5.5 recognizes these adapters as 40Gb Ethernet adapters. And I really wanted to test these adapters at 40Gb Ethernet… and the results are great. I can push upto 37.3 Gbits\/sec thru a single 40Gb Ethernet link, or 4299 MBytes\/sec. Just have a peak at the following screenshot.<\/span><\/p>\n

\"homelab_network_40g_iperf_testing\"<\/a><\/p>\n

I guess having 40Gb Ethernet for vMotion is too fast… \u00a0The vMotion of a 12GB VM takes 15-16 seconds, of which only 3 seconds are used for the memory transfer, the rest is the memory snapshot, processes freeze, cpu register cloning and the rest.<\/p>\n

All the test run at 10Gb Ethernet and 40Gb Ethernet where done with Jumbo Frames. For 40Gb Ethernet it makes real (x 2.5) difference in bandwidth.<\/p>\n

This was a fun piece to lab in the homelab.<\/p>\n","protected":false},"excerpt":{"rendered":"

In my previous post, I described the building of two Linux virtual machines to benchmark the network. Here are the results.   The first blip, is running iperf to the maximum speed between the two Linux VMs at 1Gbps, on separate hosts using Intel I350-T2 adapters. The second spike (or vmnic0), is running iperf to […]<\/p>\n","protected":false},"author":1,"featured_media":1578,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[253,251,9],"tags":[159,192,179,104,185,183,111,188,191],"_links":{"self":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1572"}],"collection":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1572"}],"version-history":[{"count":7,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1572\/revisions"}],"predecessor-version":[{"id":1583,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1572\/revisions\/1583"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/media\/1578"}],"wp:attachment":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1572"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}