HP ML110 G7 and VT-d DirectPath I/O Configuration and VMware FT

I’ve had quiet a few questions over the past days about the HP ProLiant ML110 G7. And here are some screenshots aboutabout using Intel VT-d or DirectPath I/O and VMware FT.

To use the Intel VT-d DirectPath I/O you need to make sure that Intel VT-d is enabled in the BIOS of the ML110 G7. Then you can assign any non used PCIe card that supports it for DirectPath I/O configuration. Assigning a PCIe device for DirectPath I/O configuration requires the ESXi to reboot.

ML110G7 and VT-d DirectPath IO Configuration

Here is a close up on the SmartArray P212 select for Passthrough mode.

SmartArray P212 in Passthrough mode

The second question I got a few time is can I also use VMware FT. Yes you can, theIntel E3 Xeon CPU are recent enough to support the Lockstep process of VMware FT. Here is a screenshot of my vShield Manager appliance running with VMware FT protection between two HP Proliant ML110 G7.

VMware FT on ML110G7

 

So Just to resume, the HP ProliantML110 G7 is an awesome system for ESXi 5.0 and allows you to use VMware HA & DRS, VMware FT, DirecPath I/O mode (VT-d), VMware Distributed Power Management via the HP iLO3 module.

 

 

Using vCenter Update Manager for HP ESXi installations

This post will explain how to use the vCenter Update Manager to create a custom Hewlett-Packard Extensions baseline, so that you can install the HP Drivers on your ESXi install and the HP CIM Management Tools.

Having just purchased a set of HP Proliant ML110 G7, I found out that HP has release at least two sets of drivers for ESXi 5.0.

The first one is the VMware ESXi 5.0 Driver CD for the HP SmartArray version 5.0.0-24.0 released on 2011/08/22. I recommend that you download this driver from the VMware website on your vCenter and extract it in a convenient place, as we will need the hpsa-500-5.0.0-offline_bundle-537239.zip file. We will come back to this file later.

The second one is the HP ESXi 5.0 Offline Bundle now in version 1.1 since December 2011. This bundle contains multiple drivers such as the HP Common Information Model (CIM) Providers, HP Integrated Lights-Out (iLO) driver and HP Compaq ROM Utility (CRU) driver. Download this file but don’t extract it. We will use the file hp-esxi5.0uX-bundle-1.1-37.zip as is.

On your vCenter, jump to the Update Manager Administration pane, and select the Import Patches option.

We first import the HP SmartArray Driver

Import hpsa-500-5.0.0-offline_bundle

Importing HP SmartArray Driver for ESX

We then import the HP ESXi 5.0 Offline Bundle

Import HP-ESXi5.0-bundle-1.1.37

Import HP-ESXi5.0-bundle-1.1.37

And we now see both offline bundles in the Patch Repository

Patch Repository

We will now create a new Baseline Extension for these offline patches so we can apply them to our HP servers.

Create a new Baseline - Host Extension

Add the HP Drivers and Tools to the new Host Extension

Add HP Drivers to Host Extension

And save the New Baseline

Save new Baseline

Lets attach this new Baseline Host Extension to our HP ML110 G7 Cluster and Scan the Cluster.

Attach new Baseline to Cluster and Scan

We can now Remediate our HP Proliant ML110 G7 host with the new Host Extension. Please note that you cannot remediate VMware Patches and the Host Extensions at the same time. You will need to do this in two passes.

Here is the Hardware Status of an HP ML110 G7 before applying the HP Host Extension patches

ML110G7 Prior to HP Drivers and Tools

and after having the remediation.

ML110G7 with Storage Information & SmartArray Driver

Thanks to these drivers, we could now see the HP SmartArray Array Status if there where any disks attached to it.

 

HP ML110 G7 and Power Management (iLO3 & vSphere DPM)

The HP ML110 G7 works great with VMware vSphere 5.0 once the C-States has been modified.

In the following screenshot we can have a glimpse of the Hardware Status of the ML110 G7. We can clearly see that this ML110 G7 with it’s integrate iLO3 controller gives the same information as the more expensive HP Proliant servers.

ML110 G7 Hardware Status and ILO3

And such as it’s bigger brothers, if we access the ILO3 Administration Console

ML110 G7 ILO3 Overview

and create a dedicate vmware user with Power Controls.

ML110 G7 ILO3 User Administration

Once this is done we just need to add in the IPMI/ILO Settings for Power Management

IPMI/iLO Settings for Power Management

And to test the functionality. Now we can use Distributed Power Management on the ML110 G7 Cluster.

Distributed Power Management

The more I test the ML110 G7, the more I like it.

vSphere 5.0 on HP ML110 G7

Last friday, I came across this very interesting deal, Two HP ProLiant ML110 G7 with Xeon E3-1220 (Quad-Core @3.1Ghz) for the price of one. So Two HP ML110 G7 for $960 seemed a great bargain to me. I got some extra Kingston memory and I should have some decent lab servers.

But when I started installing VMware ESXi 5.0.0 Build 504980 on the HP ML110 G7 it kernel dumped.

HP ML110 G7 crashing during ESXi 5.0 Build 504890 startup

After having filmed the crash, the last thing that came up before the crash was ACPI.

I looked up the Performance Best Practices for VMware vSphere 5.0 PDF for specific ACPI settings and Power States. It does have some specific tuning tips on page 14/15

  • In order to allow ESXi to control CPU power-saving features, set power management in the BIOS to “OS Controlled Mode” or equivalent. Even if you don’t intend to use these power-saving features, ESXi  provides a convenient way to manage them.
  • NOTE Some systems have Processor Clocking Control (PCC) technology, which allows ESXi to manage power on the host system even if its BIOS settings do not specify “OS Controlled mode.” With this technology, ESXi does not manage P-states directly, but instead cooperates with the BIOS to determine the processor clock rate. On HP systems that support this technology, it’s called Cooperative Power Management in the BIOS settings and is enabled by default. This feature is fully supported by ESXi and we therefore recommend enabling it (or leaving it enabled) in the BIOS.
  • Availability of the C1E halt state typically provides a reduction in power consumption with little or no impact on performance. When “Turbo Boost” is enabled, the availability of C1E can sometimes even increase the performance of certain single-threaded workloads. We therefore recommend that you enable  C1E in BIOS.
  • However, for a very few workloads that are highly sensitive to I/O latency, especially those with low CPU  utilization, C1E can reduce performance. In these cases, you might obtain better performance by disabling C1E in BIOS, if that option is available
  • C-states deeper than C1/C1E (i.e., C3, C6) allow further power savings, though with an increased chance of performance impacts. We recommend, however, that you enable all C-states in BIOS, then use ESXi host power management to control their use

So I modified the Power Management settings in the HP ML110 G7 BIOS.

[box]

HP Power Profile: Custom

HP Power Regulator: OS Control Mode

Advanced Power Management Options \ Minimum Processor Idle Power State: C6 States[/box]

Just changing the No C-States to the C6 States will allow you to install and run ESXi 5.0 on the HP ML110 G7.

ML110 G7 BIOS Advanced Power Management Options C6 States

And here is the beautifully screenshot of the ML110 G7 in the vCenter

ESXi 5.0 on ML110 G7

And a closer look at the Power Management Settings tab from vCenter 5.0. You can now change the power settings without having to reboot and modify the BIOS.

ESXi 5.0 Power Management with ML110 G7

I hope this will be usefull to other people in preparing their VCP5 Certification and for a great home lab equipment.

And for those that want to test further, the ML110 G7 supports Intel VT-d.