Using the VSAN Observer in vCenter 5.5

VSAN observer is an experimental feature. It can be used to understand VSAN performance characteristics and as such is a tool intended for customers who desire deeper insight into VSAN as well as by VMware Support to analyze performance issues encountered in the field.”  This is the tool any tester of VSAN can use to monitor his hosts, disks, VMs and see the distribution across hosts.

Rawlinson (@PunchingClouds) has created two very interesting articles on the VSAN Observer, which I’ve been hearing about for a few weeks. In his posts, Rawlinson shows how to use the VSAN observer that comes with the vCenter Appliance Using RVC VSAN Observer Pt1 and Using RVC VSAN Observer Pt2. I will show you here how to use the one that comes with the Windows implementation of vCenter 5.5

The VSAN Observer runs on the Ruby vSphere Console (RVC). Ruby vSphere Console (RVC) is a Linux console UI for vSphere, built on the RbVmomi bindings to the vSphere API. The vSphere object graph is presented as a virtual filesystem, allowing you to navigate and run commands against managed entities using familiar shell syntax.Your vCenter 5.5 ships with RVC installed.

Starting your own VSAN Observer

In the vCenter 5.5 server under the path C:\Program Files\VMware\Infrastructure\VirtualCenter Server\support\rvc you will find the rvc.bat file. Edit the rvc.bat file with notepad or notepad++ and jump at the end of the line to change the name of the user that will connect to the vCenter and the name of the vCenter. That can be seen from the output below in the first orange box.

  • Remember that the Ruby vSphere Console and the VSAN Observer tool are an experimental feature. There is no user authentication to the VSAN Observer website, and I’ve found out that the VSAN Observer process dies after a few hours.

Once you launch the RVC tool and enter the password for your vCenter account, you can use RVC commands. You can use ls to list objects, or cd <number> to drill down in an object. William Lam (@lamw) has some interesting articles about RVC (RVC 1.6 released)

But the command you want is to launch the vsan.observer program that will launch a webserver to which you can connect on port 8010 (Second orange box).

vsan.observer <vcenter-hostname>/<Datacenter-name>/computers/<Cluster-Name>/ –run-webserver –force

or for me

vsan.observer vcenter01.bussink.org/Home/computers/Management\ Cluster/ –run-webserver –force

VSAN Observer on Windows 01

To stop the vsan.observer process you can stop it with a double Ctrl+C.

VSAN Observer Web interface

So now that you have your vsan.observer running, let’s connect to it with a browser on port 8010. This is the About section  that will list your VSAN hosts.

VSAN Observer About

But you can get some very interesting information about your Hosts such as VSAN Disks (per-host).

VSAN Observer VSAN Disks per-host

Here is the VSAN Disk (deep-dive) to see the performance of the SSD caching in front of the magnetic disk. Here the vCenter Log Insight appliance kept on the VSAN, had a peak during a reboot.

VSAN Observer VSAN Disks deep-dive

You can also drill deep with the Full graphs to get more details of the write operations on the SSD.

VSAN Observer VSAN Disks deep-dive SSD 01

VSAN Observer VSAN Disks deep-dive SSD 02

These charts are not always the easiest to read. But you will find great stuff here.

VM VSAN Stats with Backing Storage.

The is the most interesting charts I’ve found. This is where you can see the different component of the storage backing your VM. My Storage Policu for the vCenter Log Insight is placed in the vCenter with a VSAN Redundancy policy (Number of failures to tolerate = 1).

I recommend you see this picture in full size, to better see the various details.

VSAN Observer VMs vCenter Log Backing

This below is the original view you get with the vSphere Web Client view from the Monitor, Virtual SAN and on the VM.

vSphere Web Client vCenter Log Insight VSAN Redundancy

 

After having played a bit with the RVC VSAN Observer in the last 24 hours. I think this will be an interesting tool for Storage IO analysis. I really hope this makes it into a Fling or a full plugin for the vCenter server.

 

VSAN Observer Firewall rule

If your vCenter Server 5.5 is running on a Windows hosts with the integrated firewall activated. Here is the rule to open the port on your system to check the VSAN Observer, from another machine.

netsh advfirewall firewall add rule name = “VMware RVC VSAN Observer” dir = in protocol = tcp action = allow localport = 8010 remoteip = localsubnet profile = DOMAIN

 

 

  • Pingback: VSAN Observer showing Degraded status… | Erik Bussink()

  • Pingback: How to configure the Virtual SAN observer for monitoring()

  • Duco

    Hi Erik, great post. Got a little tired of having to start the observer manually each time, so created a batch file to start it from the desktop :-) See http://blog.vconsult.nl/2014/02/14/batch-file-to-start-vsan-observer-on-windows-vcenter-server/

    • Rizul Khanna

      Hello Erik,

      I saw ur post on Vmware virtual san observer here.

      I am trying to connect to my windows based vcenter server(running vsan cluster) from a secondary vcenter server appliance, that has been created dedicatedly for this purpose. I run the vsan.observer command using that, the command keeps running and loads infor. for 2 hours, but nothing comes up if I try to connect to 8010 port on the vcenter server.

      I read in a post that it might be due to nokogiri -v 1.5.6. My vcenter server is on windows server 2008 R2 datacenter(http://vinfrastructure.it/2014/05/vmware-virtual-san-observer/)

      When I try running this command directly on vcenter server by navigating to the rvc folder in programfiles amd running the RVC command, I am unable to navigate to the Cluster, I get this error: RuntimeError: unknown VMODL type AnyType

      Now I just upgraded my vcenter server to 5.5 U1b from 5.5 U 1a, still the issue persists and even after successful start of the webserver, nothing comes up on 8010 port. It is mentioned in release notes of 5.5 U1b that this issue for unable to navigate has been fixed in this release.

      I would be highly obliged for your help and patience. Please help me.

      regards,

      Rizul Khanna(rizulkhanna@gmail.com, India)

      • Rizul,

        I highly recommnend that you move to vCenter 5.5 Update 2. Also once you have it installed or try it now, here is a great script to automate the VSAN Observer launching from Windows.

        It’s at Duco Jaspars’s website

        http://blog.vconsult.nl/2014/02/14/batch-file-to-start-vsan-observer-on-windows-vcenter-server/

        Hope it helps.

        • a_brams

          hello Mr. Erik Bussink .I Need Your Help please, I’m sorry to bump this Topics.

          Thank You before for Your tuts, i have googling how to use vsan observer without installing vcenter appliance and lastly i found your super precious tuts, but i get this error (please see attchment)
          i have use Duco’s .bat but i still dont know how to use it, maybe he’s busy and not give some clues :p, just kidding

          on another comment You said that You have an issue with the self-signed SSL cert on Your vCenter, maybe its the same problems with Yours. Would You mind to share it?

          again, Thank You Very much

          sorry for my english,
          regards,
          bram

          • Try adding a –no-https in your command.

          • a_brams

            Thank You very much Mr.@ErikBussink:disqus It Works !!! May God Bless You.
            Now I can step moving forward to analyze My project, I’ll learn about how to use Your shortcuts methods.

            I’m using it for my final projects, designing / planning vsan to store virtual multimedia server

            again Thank You very much…

            :cheers and :thumbsup

          • I’m also using shortcuts in rvc now. Once I start rvc I paste the following first
            mark

            mark cluster vcsa-01.bussink.org/DC/computers/Management_Cluster/
            mark ds vcsa-01.bussink.org/DC/datastores/vsanDatastore/
            mark vm vcsa-01.bussink.org/DC/vms

            And now you can simplify your VSAN Observer command and use the ~cluster variable.

            vsan.observer ~cluster –run-webserver –no-https –force

    • You’re script is great !!! I added the –no-https as I was having an issue with the self-signed SSL cert on my vCenter.

  • Pingback: VSAN Observer Windows Server 2012 R2 | VMFocus()