Homelab with vSphere 5.5 and VSAN

This is to give you a quick insight in my 2013 Homelab that is running vSphere 5.5 and has a running version of Virtual SAN (VSAN) in Beta code. I have been quiet on the blog for a while, as I’ve been doing some tests with vSphere 5.5 and VSAN, but the NDA has limited my communications.

This is probably the smallest VSAN implementation you can do without going with a Nested VSAN (Awesome design by William Lam) or with three Mac Mini’s.

This VSAN is only 24cm x 22cm x 20cm (High x Depth x Width), and runs with 130 Watts total consumption. On the following screenshot you see the Homelab next to an old Synology DS1010+

Homelab running vSphere 5.5 with VSAN

Homelab running vSphere 5.5 with VSAN

 

It is composed of three Shuttle XH61v with a quad-core i7-3770s processor (65W) and 16GB of memory (Two 8GB Kingston SO-DIMM). The Shuttle XH61v also comes with two Gigabit network cards. Each Shuttle XH61v has the following storage

  1. Kingston USB 3.0 DataTraveller 16GB Key to boot vSphere 5.5
  2. Intel mSATA 525 SSD 120GB which is used by vFlash
  3. Intel 530 SSD 240GB 2.5″ or Samsung 840 Pro 256GB
  4. Seagate Momentus HD 2.5″ 750GB 7200rpm

That is a lot of storage in such a small case, but it works, and you don’t even hear the ventilator (for Wife Acceptance Factor approval).

I’m not going to cover in this article how you need to create a VMkernel interface for VSAN, and that you need to disable HA before turning VSAN on. This article “VSAN How to Configure” by David Hill does an excellent job, and his follow-up post “Configure disk redundancy in VSAN” adds more information.

From the vSphere Web Client, this is the configuration of my VSAN after I enabled it.

VSAN Disk Management

VSAN Disk Management

So once you enable the VSAN with three hosts that each have an empty SSD and HD (in my case 240GB SSD and 750GB HD) you get the following.

VSAN Datastore created

VSAN Datastore created

Another great functionality of the VSAN, is that if you take another ESXi host and configure it’s VSAN VMkernel interface and add it to the VSAN Resource, it automaticaly mounts the VSAN Datastore. This will greatly simplify the provisioning of storage in a vSphere Cluster. The VSAN Datastore is also the first implementation of Virtual Volumes (VVOL) that I have seen. Cormac Hogan has a great Virtual Volume (VVOL) Tech Preview article.

The Virtual SAN from VMware should be available in Beta for a wider audience very soon, so go over to VMware VSAN Beta Register.

 

  • Concerning the Shuttle XH61v, it’s only down side is the two SO-DIMM slots of memory. There is no current capacity to increase beyond the 16GB the memory of a Mini-ITX motherboard.
  • The Shuttle XH61v cannot boot in USB3 mode from the USB key, you need to modify the BIOS and downgrade the USB3 to USB2 mode.

 

  • Nice and compact! Cool … Now I’m stuck with 3 ATX boxes, but it could be my future cluster. But WAF dependent -:).

  • Pingback: Expanding the lab and network reflexions | Erik Bussink()

  • James Harless

    I’m running a very similar lab setup. But, the XH61V’s aren’t working properly as a VSAN environment for me. Have you had any issues with drives getting degraded?

    • On one of the H61XV I had a mSATA going down a few times, but when i replaced it was fine. VSAN with the 5.5 GA has been more stable than the 5.5 RC code. No more degraded VSAN since this week.

      I had to increase the VSAN network to two vmkernel-vsan
      networks with non-overlapping IP rangess. I have a vmkernel-vsan-0 on
      nic0 and vmkernel-vsan-1 on the nic1. This gives a better network
      backbone.

      I’ve installed Log Insight on VSAN and it’s holding it’s own right now.

  • John

    I’ve got a 3 host10Gbps deployment coming this week…

    http://vexpert.me/gV

  • Alexandru Covaliov

    Nice setup. I will try to do the same on my home lab. I’m stuck as Vladan with 2 hosts. WAF accepts everything I want, but family budget is too limited to allow any additional purchase. =)

  • Pingback: VSAN homelab on Shuttle XH61v | VSAN Cookbook()

  • Anders

    Just a quick question, have you done any tweaks to the esxi5.5 for better performance or are you running it as it was installed?
    Btw, thanks for the tip on the shuttle, just installed 2 for a tiny office installation :)

    • Anders

      Maybe not the right Place to post the querstion but I couldnt fint any pure 5.5 installation page, sorry ::

      • I’m not sure what you mean by Pure 5.5 install.

  • bondernaut

    I have just purchased an XH61V and I can load 5.5 after inserting the proper driver for the NIC’s but when it tries to start it gets stuck at Loading ACPI. Has anyone else run into that issue with these?

    • I ran into issues install vSphere 5.5 Beta 2, but nothing since then.

  • Bob Bobson

    Excellent setup. I’m only really playing with ESXi for the first time but have the XV61V setup with a 512GB OCZ Vertex4 and 500GB Seagate Momentus XT with v5.5 of ESXi.
    Everything is working great so far as I used your guide to get the Realtek network drivers injected into 5.5 (thanks for that). The only thing I can’t seem to get working is any device plugged into a USB 3 port. USB 2 is fine but 3 doesn’t seem to work for some reason. Have you seen this?

    • Yes, I had to config the BIOS to downgrade the USB3 to USB2 to be able to boot from a USB key.

      • Bob Bobson

        I see, I’ll have a play with that thanks. Have you ever managed to configure any monitoring like SMART in ESXi on these boards?

        • No, there is nothing much in these boxes to push the monitoring that far. You will need boxes with IPMI management if you want some more in-depth details. But those cost a lot more.
          Sorry.

  • Ian Stewart

    Erik,
    I have a question in regards to your setup here. How were you able to install the DC and vCenter before the vSAN network was setup since you only have 3 hosts and the host that participate in vSAN have to have unclaimed disk space?

  • John

    Erik, I plan a similar setup for my home lab. One of the pre-requisites listed by David Hill is a passthru RAID controller. I cannot see that in your build, is it not needed? Also, does this setup require an ESXi license, or can it be done with a free license?

  • Stephen

    Hi Erik, I’ve got an question: Did the Synology DS1010+ work fine with ESXi 5.5 U2? Because it’s not on VMware compat. list. Is it connected via nfs or iscsi (im interessted in nfs…) But if it works, I’d like to use this for a lab… Kind regards