{"id":1810,"date":"2016-08-21T11:56:28","date_gmt":"2016-08-21T09:56:28","guid":{"rendered":"http:\/\/www.bussink.ch\/?p=1810"},"modified":"2018-05-23T17:57:35","modified_gmt":"2018-05-23T15:57:35","slug":"network-core-switch-cisco-nexus-3064pq","status":"publish","type":"post","link":"https:\/\/www.bussink.ch\/?p=1810","title":{"rendered":"Network core switch Cisco Nexus 3064PQ"},"content":{"rendered":"<p>Here is my new network core switch for the Home Datacenter, a Cisco Nexus 3064PQ-10GE.<\/p>\n<div id=\"attachment_1818\" style=\"width: 1875px\" class=\"wp-caption alignnone\"><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/nexus_3064pq_front-1.jpg\"><img aria-describedby=\"caption-attachment-1818\" loading=\"lazy\" class=\"size-full wp-image-1818\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/nexus_3064pq_front-1.jpg\" alt=\"Cisco Nexus 3064PQ-10GE (48x SFP+ &amp; 4x QSFP+)\" width=\"1865\" height=\"459\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_front-1.jpg 1865w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_front-1-300x74.jpg 300w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_front-1-768x189.jpg 768w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_front-1-1024x252.jpg 1024w\" sizes=\"(max-width: 1865px) 100vw, 1865px\" \/><\/a><p id=\"caption-attachment-1818\" class=\"wp-caption-text\">Cisco Nexus 3064PQ-10GE (48x SFP+ &amp; 4x QSFP+)<\/p><\/div>\n<p>But before I speak more about the Cisco Nexus 3064PQ-10GE, let me just bring you back in time&#8230;\u00a0Two years ago, I purchased a <a href=\"http:\/\/www.cisco.com\/c\/en\/us\/support\/switches\/sg500xg-8f8t-16-port-10-gigabit-stackable-managed-switch\/model.html\" target=\"_blank\" rel=\"noopener noreferrer\">Cisco SG500XG-8F8T 16-port 10-Gigabit Stackable Managed Switch<\/a>. This was first described in my <a href=\"http:\/\/www.bussink.ch\/?p=1604\" target=\"_blank\" rel=\"noopener noreferrer\">Homelab 2014 build<\/a>. This was my most expensive networking investment I ever did. During the past two years, as the lab grew, I used the <a href=\"http:\/\/www.cisco.com\/c\/en\/us\/products\/collateral\/switches\/small-business-500-series-stackable-managed-switches\/c78-695646_data_sheet.html\" target=\"_blank\" rel=\"noopener noreferrer\">SG500XG and two SG500X-24<\/a>\u00a0for my networking stack.\u00a0This stack is still running on the 1.4.0.88 firmware.<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/sg500xg_stack.jpg\"><img loading=\"lazy\" class=\"size-full wp-image-1812 aligncenter\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/sg500xg_stack.jpg\" alt=\"sg500xg_stack\" width=\"766\" height=\"387\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/sg500xg_stack.jpg 766w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/sg500xg_stack-300x152.jpg 300w\" sizes=\"(max-width: 766px) 100vw, 766px\" \/><\/a><\/p>\n<p>During these past two years, I have learned the hard way that network chipsets for 10GbE using RJ-45 cabling was outputting so much more heat than the SFP+ chipset. My initial Virtual SAN Hybrid implementation using a cluster of three ESXi host with Supermicro X9SRH-7TF (Network chipset is Intel X540-AT2) crashed more than once, when the network chipset became so hot that I lost my 10G connectivity, but the ESXi host kept on running. Only a powerdown &amp; cool off of the motherboard, would allow my host to restart with the 10G connectivity. This also lead me to expand the VSAN Hybrid cluster from three to four hosts and to have a closer look at the heating issues when running 10G over RJ45.<\/p>\n<p>Small business network switches with 10GBase-T connectivity are more expensive than the more enterprise oriented SFP+ switch, but they also output so much more heat (<a href=\"https:\/\/www.google.ch\/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwja_OTDjdLOAhVGNxQKHREIB3UQFggfMAA&amp;url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FBritish_thermal_unit&amp;usg=AFQjCNECFCYVq_PDLzi8QDvdv9A0RQB1Uw&amp;sig2=MgyjJ1QHiFXuOOVMtPtJ4g\" target=\"_blank\" rel=\"noopener noreferrer\">Measured in BTU\/hr<\/a>). Sure once the 10GBase-T switch is purchased, the cost of Category 6A cables is cheaper than getting the Passive Copper cables, who are limited to 7 meters.<\/p>\n<p>The Cisco SG500XG-8F8T is a great switch as it allows me to connect using both RJ-45 and SFP+ cables.<\/p>\n<p>As the lab expanded, I started to ensure that my new hosts have either no 10GBase-T adapters on the motherboard, or use the SFP+ adapter (Like my recent <a href=\"http:\/\/www.bussink.ch\/?p=1766\">X10SDV-4C-7TP4F ESXi host<\/a>). I have started using the Intel X710 Dual SFP+ adapters on some of my host. I like this Intel network adapter, as the network chipset gives out less heat than previous generations chipset, and has a firmware update function that can done from the command prompt inside of vSphere 6.0.<\/p>\n<p>This brings me to the fact that I was starting to run out of SFP+ ports as the labs expands. I have found on ebay some older Cisco Nexus switch, and the one that caught my eye for it&#8217;s amount of ports, it price and it&#8217;s capabilities is the <a href=\"http:\/\/www.cisco.com\/c\/en\/us\/support\/switches\/nexus-3064-switch\/model.html\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Cisco Nexus 3064PQ-10GE<\/strong><\/a>. These babies are going for about $1200-$1500 on ebay now.<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/3064pq_on_ebay.jpg\"><img loading=\"lazy\" class=\" wp-image-1813 aligncenter\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/3064pq_on_ebay.jpg\" alt=\"3064pq_on_ebay\" width=\"412\" height=\"522\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064pq_on_ebay.jpg 801w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064pq_on_ebay-237x300.jpg 237w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064pq_on_ebay-768x972.jpg 768w\" sizes=\"(max-width: 412px) 100vw, 412px\" \/><\/a><\/p>\n<p>The switch comes with 48-ports SFP+ and 4-ports in QSFP+ format. These four ports can be configured in either 16x10G using fan-out cables or 4x40G. This is a software command that can be put on the switch to change from one mode to the other.<\/p>\n<p>Here is my switch with the interface output. I&#8217;m using a <a href=\"http:\/\/www.get-console.com\/airconsole\/\" target=\"_blank\" rel=\"noopener noreferrer\">Get-Console Airconsole<\/a> to extend the console port to my iPad over Bluetooth.<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/nexus_3064pq_10g_40g-1.jpg\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-1814\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/nexus_3064pq_10g_40g-1.jpg\" alt=\"nexus_3064pq_10g_40g-1\" width=\"1785\" height=\"1280\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_10g_40g-1.jpg 1785w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_10g_40g-1-300x215.jpg 300w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_10g_40g-1-768x551.jpg 768w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_10g_40g-1-1024x734.jpg 1024w\" sizes=\"(max-width: 1785px) 100vw, 1785px\" \/><\/a><\/p>\n<p>My vSphere 6.0 host is now connected to the switch using an <a href=\"http:\/\/ark.intel.com\/products\/83967\/Intel-Ethernet-Converged-Network-Adapter-XL710-QDA2\" target=\"_blank\" rel=\"noopener noreferrer\">Intel XL710-QDA2<\/a> 40GbE network adapter and a QSFP+ copper cable.<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/esxi_40G.jpg\"><img loading=\"lazy\" class=\"size-full wp-image-1815 aligncenter\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/esxi_40G.jpg\" alt=\"esxi_40G\" width=\"725\" height=\"779\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/esxi_40G.jpg 725w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/esxi_40G-279x300.jpg 279w\" sizes=\"(max-width: 725px) 100vw, 725px\" \/><\/a><\/p>\n<p>I&#8217;m going to use the four QSFP+ connectors on the Cisco Nexus 3064PQ-10GE to connect my Compute cluster with NSX and VSAN All-Flash.<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/3064_10g_40g_show_int.jpg\"><img loading=\"lazy\" class=\"wp-image-1822 aligncenter\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/3064_10g_40g_show_int.jpg\" alt=\"3064_10g_40g_show_int\" width=\"768\" height=\"455\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064_10g_40g_show_int.jpg 1694w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064_10g_40g_show_int-300x178.jpg 300w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064_10g_40g_show_int-768x455.jpg 768w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3064_10g_40g_show_int-1024x606.jpg 1024w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>The switch came with NX-OS 5.0(3)U5(1f).<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/3068_nx-os.jpg\"><img loading=\"lazy\" class=\"wp-image-1821 aligncenter\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/3068_nx-os.jpg\" alt=\"3068_nx-os\" width=\"736\" height=\"350\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3068_nx-os.jpg 1512w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3068_nx-os-300x142.jpg 300w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3068_nx-os-768x365.jpg 768w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/3068_nx-os-1024x486.jpg 1024w\" sizes=\"(max-width: 736px) 100vw, 736px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>Concerning the heat output of the Cisco Nexus 3064PQ-10GE (<a href=\"http:\/\/www.cisco.com\/c\/en\/us\/products\/collateral\/switches\/nexus-3000-series-switches\/data_sheet_c78-651097.html\" target=\"_blank\" rel=\"noopener noreferrer\">datasheet<\/a>) I was pleasantly surprised to note that it&#8217;s output is rather small at 488 BTU\/hr when all 48 SFP+ are used. I also noted that the noise level of the fans was linked to the fan speed and the charge of the switch. Going from 59 dBA at 40% duty cycle to 66 dBA at 60% duty cycle to 71 dBA when at 100% duty cycle.<\/p>\n<p>Here is the back of the Cisco Nexus 3064PQ-10GE. I did purchase the switch with a DC power (top of switch to the right), because the switch I wanted had both the LAN_BASE_SERVICES and the LAN_ENTERPRISE_SERVICES license. I sourced two N2200-PAC-400W-B power supply from another place.<\/p>\n<p><a href=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/nexus_3064pq_back-1.jpg\"><img loading=\"lazy\" class=\"alignnone size-full wp-image-1819\" src=\"http:\/\/www.bussink.ch\/wp-content\/uploads\/\/2016\/08\/nexus_3064pq_back-1.jpg\" alt=\"nexus_3064pq_back-1\" width=\"1862\" height=\"735\" srcset=\"https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_back-1.jpg 1862w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_back-1-300x118.jpg 300w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_back-1-768x303.jpg 768w, https:\/\/www.bussink.ch\/wp-content\/uploads\/2016\/08\/nexus_3064pq_back-1-1024x404.jpg 1024w\" sizes=\"(max-width: 1862px) 100vw, 1862px\" \/><\/a><\/p>\n<p>Link to the <a href=\"https:\/\/people.ucsc.edu\/~warner\/Bufs\/cisco-3064PQ-architecture.pdf\" target=\"_blank\" rel=\"noopener noreferrer\">Cisco Nexus 3064PQ Architecture<\/a>.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Here is my new network core switch for the Home Datacenter, a Cisco Nexus 3064PQ-10GE. But before I speak more about the Cisco Nexus 3064PQ-10GE, let me just bring you back in time&#8230;\u00a0Two years ago, I purchased a Cisco SG500XG-8F8T 16-port 10-Gigabit Stackable Managed Switch. This was first described in my Homelab 2014 build. This [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[11,253,251],"tags":[159,192,200,237,240,239,238,84,99],"_links":{"self":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1810"}],"collection":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1810"}],"version-history":[{"count":8,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1810\/revisions"}],"predecessor-version":[{"id":1865,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=\/wp\/v2\/posts\/1810\/revisions\/1865"}],"wp:attachment":[{"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1810"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1810"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bussink.ch\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}