vSphere 5.0 on HP ML110 G7

Last friday, I came across this very interesting deal, Two HP ProLiant ML110 G7 with Xeon E3-1220 (Quad-Core @3.1Ghz) for the price of one. So Two HP ML110 G7 for $960 seemed a great bargain to me. I got some extra Kingston memory and I should have some decent lab servers.

But when I started installing VMware ESXi 5.0.0 Build 504980 on the HP ML110 G7 it kernel dumped.

HP ML110 G7 crashing during ESXi 5.0 Build 504890 startup

After having filmed the crash, the last thing that came up before the crash was ACPI.

I looked up the Performance Best Practices for VMware vSphere 5.0 PDF for specific ACPI settings and Power States. It does have some specific tuning tips on page 14/15

  • In order to allow ESXi to control CPU power-saving features, set power management in the BIOS to “OS Controlled Mode” or equivalent. Even if you don’t intend to use these power-saving features, ESXi  provides a convenient way to manage them.
  • NOTE Some systems have Processor Clocking Control (PCC) technology, which allows ESXi to manage power on the host system even if its BIOS settings do not specify “OS Controlled mode.” With this technology, ESXi does not manage P-states directly, but instead cooperates with the BIOS to determine the processor clock rate. On HP systems that support this technology, it’s called Cooperative Power Management in the BIOS settings and is enabled by default. This feature is fully supported by ESXi and we therefore recommend enabling it (or leaving it enabled) in the BIOS.
  • Availability of the C1E halt state typically provides a reduction in power consumption with little or no impact on performance. When “Turbo Boost” is enabled, the availability of C1E can sometimes even increase the performance of certain single-threaded workloads. We therefore recommend that you enable  C1E in BIOS.
  • However, for a very few workloads that are highly sensitive to I/O latency, especially those with low CPU  utilization, C1E can reduce performance. In these cases, you might obtain better performance by disabling C1E in BIOS, if that option is available
  • C-states deeper than C1/C1E (i.e., C3, C6) allow further power savings, though with an increased chance of performance impacts. We recommend, however, that you enable all C-states in BIOS, then use ESXi host power management to control their use

So I modified the Power Management settings in the HP ML110 G7 BIOS.

[box]

HP Power Profile: Custom

HP Power Regulator: OS Control Mode

Advanced Power Management Options \ Minimum Processor Idle Power State: C6 States[/box]

Just changing the No C-States to the C6 States will allow you to install and run ESXi 5.0 on the HP ML110 G7.

ML110 G7 BIOS Advanced Power Management Options C6 States

And here is the beautifully screenshot of the ML110 G7 in the vCenter

ESXi 5.0 on ML110 G7

And a closer look at the Power Management Settings tab from vCenter 5.0. You can now change the power settings without having to reboot and modify the BIOS.

ESXi 5.0 Power Management with ML110 G7

I hope this will be usefull to other people in preparing their VCP5 Certification and for a great home lab equipment.

And for those that want to test further, the ML110 G7 supports Intel VT-d.

  • Bj

    Just tested it, it worked.

    Thanks,
    B.

  • King Tid

    Erik you are a star, I have spent hours trying to get this to work, but was looking at the PF exception 14 error and suspected driver memory related issues.

  • Info

    it works fine

  • Bialy_rb

    Great post. Thanks for sharing your findings. So much time and stress saved.

    Regards,

  • Seng

    have you tried to boot from usb drive?

    • I’m booting from the Internal USB connector. Just make sure to modify the BIOS setting so that USB connect takes precedences over the external ones.

  • Hiya Seng,

    Yes it boots great from an USB drive. I used the internal USB connector.

  • Franco

    thanks for the info. I just wanted to ask about FT and VMDirectPath support. Any things that might not be supported when purchasing these?

  • Rmcdonald

    Hello there, Could you tell me how the performance is on this server and how many VMs have you been able to run on this server? I am trying to figure out if an 1 CPU or 2 CPU server will be best for my needs. I am looking at these servers as well as Dell servers. My goal is to do the normal vSphere 5 stuff, along with View, SRM, vCloud Director etc. I am planning on running the HP P4000 VSA on this server as well. Could you please tell me if you are using a RAID controller in this server and if so, what model?

    • Performance wise, I think the E3-1220 with 16GB is very decent. It’s really NOT about the CPU resource, but memory. I’m now running 5 VMs per ML110 G7. But I have four of them in my “Sandy Bridge” Cluster (with EVC). The latest VMs I’m running on the lab, are actually getting pretty large. I have two SQL Server 2008 R2 with 6GB of memory, and I’m running vCenter Operations Manager 5 which as a large vApp (2 VMs with 4 vCPU and 16GB of memory, which I trimmed down to 14GB).
      The four ML110 G7 I got in my kit have the SmartArray P212/256 but my servers did not have any disks at all. I believe that the SmartArray P212/256 is a nice entry level RAID card, but might be lacking in power.
      If you trully want to have a nice test & validation lab, I think you need at least 2 ML110 G7 or better 3 with a shared storage solution. Then you can use HA & DRS, FT, DPM, vMotion, svMotion, SIOC and NetIOC.
      I’m also running two vCloud Director cells, and the vShield Manager is protected with VMware FT.

    • Performance wise, I think the E3-1220 with 16GB is very decent. It’s really NOT about the CPU resource, but memory. I’m now running 5 VMs per ML110 G7. But I have four of them in my “Sandy Bridge” Cluster (with EVC). The latest VMs I’m running on the lab, are actually getting pretty large. I have two SQL Server 2008 R2 with 6GB of memory, and I’m running vCenter Operations Manager 5 which as a large vApp (2 VMs with 4 vCPU and 16GB of memory, which I trimmed down to 14GB).
      The four ML110 G7 I got in my kit have the SmartArray P212/256 but my servers did not have any disks at all. I believe that the SmartArray P212/256 is a nice entry level RAID card, but might be lacking in power.
      If you trully want to have a nice test & validation lab, I think you need at least 2 ML110 G7 or better 3 with a shared storage solution. Then you can use HA & DRS, FT, DPM, vMotion, svMotion, SIOC and NetIOC.
      I’m also running two vCloud Director cells, and the vShield Manager is protected with VMware FT.

      • Søren Jensen

        At my work we are also using the P212 controller, depending on disks (SAS, SATA or SSD) we use it with or without cache.
        The server seems to run very fast and smooth with E3-1220 (IIRC), 12+GB RAM, P212 controller (no cache) and Intel X310/X320 SSD’s.
        I also agree with Erik – it comes more to RAM than CPU.
        Not that it is easy to compare, but at my old work we ran 20-30 VMs on some IBM x3650 server with Intel E53xx CPU. They lacked RAM more than anything.

        I recall a few years ago I read that somebody said that 4-5 VMs/core, depending on usage etc. I assume 4-5VMs/core refers to 4-5vCPUs/physical core.

        But either way, my experience is that unless you run very CPU hungry VMs, you don’t have to care about CPU at all.

        On my own private stand-alone server I run 22 VMs (mixed Linux, FreeBSD, Windows 2003/2008), some acceptable IBM controller with cache+battery but with NL SAS disks only. CPU is usually at 15-30% (2xE5520). Just for the record, that is not some ML110 G7.

        But with the price of the ML110 G7 + P212 controller, just about anybody should be able to afford that one with free ESXi license.

        • Thanks S0ren for your excellent input on the P212 and SSDs.
          I’ve also got an older ML150 G5 with a Xeon 5410 with 16GB that was running 15 VMs.

  • Søren Jensen

    Just tested here as well – works perfectly.
    Thanks a lot! I had been struggling with this for weeks. I have 2 servers that worked out of the box, and suddenly it wouldn’t install on a new server. I even tried buying a new ML110G7, same problem.
    Maybe HP changed the defaults, as I am quite sure I didn’t change any of these settings on the two first servers.
    Same BIOS/iLO.

    But once again – thanks a lot!

    • Hiya S0ren. I purchased 4 of the ML110 G7. The two first ones I unboxed had the C6 state disabled in the BIOS so I had to figure it out, while the 3rd & 4th had the C6 state already set. So I guess they came from different batches. Looking back, I can now see why some people said that vSphere 5 was working fine on their ML110 G7 and others where just not having any success.
      At least we now have the proper setting for this.

    • Hiya S0ren. I purchased 4 of the ML110 G7. The two first ones I unboxed had the C6 state disabled in the BIOS so I had to figure it out, while the 3rd & 4th had the C6 state already set. So I guess they came from different batches. Looking back, I can now see why some people said that vSphere 5 was working fine on their ML110 G7 and others where just not having any success.
      At least we now have the proper setting for this.

      • Søren Jensen

        Aha, so it isn’t just me. The first two I used was bought some time during 2011 like September-November, while the 2 newer ones was bought within the last month.
        I find it strange tho, that they ship with “random” BIOS settings.
        The one I tested your settings at had C6, but not the Custom+OS Control Mode.

        • The Custom OS power setting is not critical, but it allows you to control the settings from within the vCenter. It’s only the C6 state that stops people from installing ESXi 5 on the ML110 G7.

          • Søren Jensen

            Okay, strange, because this server I had problems with had the C6 state setting set, but none of the two others, and it PSOD when running the installation.
            Unless the state might change when altering the two others (I started from top to bottom, and the state is the last of the three)

  • Fahad Zaman

    Erik,

    I know that the SATA controller is not supported by ESXi, but I am wondering if the HP NC112i (NIC) are supported by ESXi. I am about to buy 2 ML110 G7 myself, but dont want to spend extra money on Intel NIC.

    • Søren Jensen

      No idea if it is officially supported, but it works out just fine for us using the onboard one

    • The HP NC112i uses Intel chipset. It is supported by ESXi.

  • Olemissjim

    Have you tried using 8GB dimm’s? Intel says that chipset supports 32GB, but I noticed HP says you can only use 4 x 4GB (16GB). Even unsuported, with the E3 quad core,iLO3, dual NIC’s, dual hot plug PS’s, 4 hot-plug 3.5″ drive bays, and 32GB that is a really smoking lab or SMB ESX server.

    • Hiya Jim. Unfortunately neither the HP nor the Kingston website where talking about 8GB DIMMs when I purchased all my equipment. So no I did not try 4x 8GB to see if it was working. That be some uber news, as I know there are Supermicro cards with 32GB on E3-1220 CPUs. If you get any news on this please do not hesitate to communicate it to us.

  • David Fields

    Erik, I took your advice on the HP ML110-G7s and they are working great for me. However, I’m trying to add some quad-port NICs to these systems and have run into a snag. Do you know or have you actually tested the Intel Pro/1000 PT Quad Port NICs (EXPI9404PT) using the Intel 82571GB controller in the HP ML110-G7s? They are in VMware’s HCL. So I have two quad-port NICs and two HP ML110-G7s, neither NIC is seen by either HP system, no matter which slot I place them in. I came across a BIOS setting on the HP’s to force PCI Generation 1 support, but this did not help either. I even disabled the onboard NICs on the HPs, but no luck. Any help would be so much appreciated!

    Thanks in advance and great work!
    David

    • David Fields

      Erik,

      The BIOS setting to force the PCI Generation to Version 1 support was the final fix. I had been trying so many things last night including taking the quad-port card out and putting it in other servers and I didn’t realize I hadn’t put it back into the HP system I was testing the BIOS setting out on. So now its working just fine with the quad-port NIC as long as the following setting is in place:

      Power Management Options -> Advanced Power Management Options -> PCI Express Generation 2.0 Support -> Force PCI-E Generation 1

      Thanks again for the great work you are doing – it is helping me a lot!

      David

      • Thanks for the follow-up. I’m using myself some Supermicro AOC-SG-4i cards that come with Quad Intel 82576 chipset. I find the Supermicro equipment to be top notch, and much cheaper than Intel cards.

      • faisal

        Dear David ,
        I am planning to buy two HP ML 110- G7 server for my vmware and cisco ( gns3 ) lab , Could you please tell which quad port nic is support in HP ML 110 G7 servers . and how many quad port will support ???

  • Ade

    Hi Erik,

    Great work figuring this out and sharing.

    I am expecting a pair of ML110 G7s and have found lots of conflicting information about ESXi5 support for the integrated B110i storage controller. I know it will not support RAID because it is software RAID, but could you please confirm if ESXi detected and installed drivers for the controller in your setup?

    • My ML110 G7 came with the SmartArray P212. but I didn’t even use them. I’m not sure if the B110i will show the disks to the system as a JBOD. But otherwise you could connect your disks on the SATA ports directly on the motherboard. Sorry I can’t help more on this topic.

      • Ade

        Thanks for the response. I got my servers and the install went great with your instructions. I can also confirm that the B110i was detected and worked perfectly fine with individual SATA disks. I hope this helps someone else who needs to confirm this. Thanks once again.

  • Andux

    Hey Guys, thanks for help, it works.

  • Trevor Smith

    Brilliant! Thanks

  • Jhurst

    Really great job. Thanks for taking the time to post.

  • Onur AYDIN

    Hi,
    hp ML110 and hp P212 g7 have hardware raid card. I installed a free version of ESXi 5. but I’m having disk performance issue. takes 2-3 minutes, for example the formation of a disk 100 GB. i also going slow on Windwos Server 2008 R2 setup. Even 20 minutes after the installation I try to install vmware server close takes a toll. I use raid 1. I wonder am I doing something wrong or after the bios settings. vmware a warning about opening the second line memory is orange. in the solution and can not read fast. I wonder on the ML110 is the ideal G7 or you know a document describing the installation of vmware Can you help.
    Note: translated with google translation.

    • Onur, the P212 is a Entry-Level RAID card without cache. I’m trying to find out which RAID processor is used by the P212.
      Why don’t you put one of your HDD directly on the SATA controller on the motherboard and compare the performance.

  • Pingback: Server is binnen | esxilab()

  • JR

    Thanks for the informative blog post. I’ve just bought 2 x ML110 G7 for my vSphere lab and your post was very useful in confirming everything would work before I ordered it!

  • VictorCarmichael

    Excellent post, saved me a lot of time. I found that all 3 settings had to be configured before the server would boot to the installer. Set the C6 states setting first but no success until the other 2 were set. Cheers

  • Soren

    Hi Erik

    Thinking of buying the ML110 for running ESXi5. I’ve read that the B110i isn’t supported by ESXi. I have a HP P400 somewhere.. Could that work? 🙂

    • Go for the P400 it’s way (50x) better than the P212 or B110i.

  • Pingback: Virtualized Brain (vBrain) » HP Microserver N40L Home Lab Server / ML110 G6/7 Comparision()

  • Fredy

    Onur AYDIN wrote about performance issues with die integraded RAID.
    I do have the same problem. An ISO Transfer of 200MB takes over 2 Minutes on my 1Gbit Network. Do I copy this ISO to my desktop, it takes only seconds.

    My XP VM takes over 20 Minutes to boot.

    I do have 2 RAID 1 configured. First two slots with 2x 1TB, last two slots with 2x 2TB.

    What do I wrong?

    Thanks for any help.

    • The P212 is a Entry-Level RAID card without cache. I’m trying to find out which RAID processor is used by the P212.
      Why don’t you put one of your HDD directly on the SATA controller on the motherboard and compare the performance.

  • This also works for HP Proliant DL120 G7…you can even use RAID 1+0 using P212 controller…..been struggling for weeks to install ESXi5.0….thanks a lot for these thread….

  • A_bagha

    Thanks,thanks,thanks for great idea thanks its work perfectly thanks

  • Riaan Rai

    Hi

    Thanks this worked perfect. Had two identical HP ML110G7 servers with P212/ZM controllers , ESXI installed perfectly on one and kept on crashing on the other. Have an SMB customer using this server in production.

  • I think I may be having performance issues too, I copied a 64GB file from one datastore to another and it took half the day! additionally if I create a VM (300GB) with thick zero it takes ages too, almost 45 minutes later and its only 24%!

    Any ideas?

    • How are your datastores connected on the ML110G7 ? SATA backend, P212 ? B110i ?
      Thick Eager Zeroed will take ages, because it needs to write 300GB of 0000 on the disk. Format a 300GB VM with Thick Lazy Zeroed, it will go much faster. Here your local storage RAID card will make a huge difference.
      If you do want some performance from storage, try a P400/P410 series.

  • Anjay

    Thank you so much for this!

  • Guest

    Thanks

  • gast

    you’re my hero,
    just bought one and thought the money was wasted, when i found the thread on the vmware forums 🙂

  • DaveG

    Well done on getting this working. Any luck with the hardware CIM providers? Would be nice to be able to be able to see any underlying hard disk failures when using a P410.

    • If you use vCenter Update Manager, there is a way to download updated CIM providers, Firmware and Drivers from HP that you can then apply to your ESXi.

      In VUM Add a new provider for the HP vibsdepot with http://vibsdepot.hp.com/index.xml, then you create a Dynamic Baseline named HP, select the drivers for ESXi 5.0, ESXi 5.1 or Any that you want in your baseline, and add it to the Hosts or Clusters that you want to keep updated.

      Here is an offical HP document “Deploying and updating VMware vSphere 5.0 on HP ProLiant Servers”
      http://h20000.www2.hp.com/bc/docs/support/SupportManual/c03285134/c03285134.pdf

  • zaki abdi

    Thanks for sharing. Does it have draw backs, if not I will buy 2 of servers, plz let me know

    thanks.

  • great post. But which CPU with VT-D is the cheapest? or should I just take the Xeon E31200?

    • I would stick with the E3-1230 to ensure you have VT-d and Hyperthreading enabled.

  • Huseyin YILDIZ

    Hi Erik
    thank you very much for great job.

  • Alex T.

    Hi Erik

    I have another problem on ml110 G7 server, the problem is that I can not enable hyperthreading on my intel XENON E31220 CPU. Do yuo have solution for this?

    thx in advance

  • Jorge Ortiz

    Muito Obrigado!!!!!

    Funcionou perfeitamente!!!!

    Abraços

    Jorge Ortiz – Brazil

  • Keamas

    Hi does the RAID Work on the HP DL110 G7 with the ESXi ?

  • Bytehorde

    HP say that max. RAM is 16 GB.
    Can’t I add 32 GB ? 4x 8 GB ?

    • Søren Jensen

      No, 16 is max as HP says…

  • Søren Jensen

    ML110 is no more in Denmark – it has been replaced with other models.

  • Dannyzts

    Have you tried ESXi 5.1 ???

  • Gr8..Hp is known for its performance ad its accessories and products like hp cad workstation,hp proliant micro server etc

  • Pingback: Welcome to vSphere-land! » Home Lab Links()

  • Mohammed NA

    thenks

  • guest

    article you write is very useful for obat keputihan gatal berbau and me personally.

  • Henri Aghaei

    I still have that crappy pinkish page! 🙁

  • Pingback: HP Proliant ML110 G7 12gb Memory Upgrade & Microserver N36L 8GB Memory Upgrade | calmdownpony()