What is this?

This is basically where I write down stuff that I work with at my job as a GIS Technical Analyst (previously system administrator). I do it because it's practical for documentation purposes (although, I remove stuff that might be a security breach) and I hope it can be of use to someone out there. I frequently search the net for help myself, and this is my way of contributing.

Saturday, February 20, 2010

HP BladeSystem C7000 Onboard Administrator lost contact with ILO2 on a Proliant bl460c blade

A while ago we upgraded our HP BladeSystem C7000 Onboard Administrator (OA) firmware to the latest 2.60. A few weeks later we upgraded the ILO2 firmware on the blades to 1.80. All but one worked fine. The last blade would not communicate properly with OA. Nothing serious I suppose, but I'm always suspicious - thinking that small issues could lead to something horrible if not dealt with properly. Plus - it's annoying.

Symptoms:
  • I could not initiate a ILO remote control session from the OA web interface (it would time out and say "Unable to perform autologon").
  • The only way I could connect to ILO was adding a local user with HP Lights-Out online configuration utility (hponcfg) from Windows running locally on the blade.
  • The configured ILO DNS-name would not register with OA.
  • OA / Device Bay Summary would report "iLO name: [Acquiring]" and "iLO DVD Status: unknown".
  • IML Log for the blade was not available from OA.
I tried all tricks I could think of:
  • Ran a emulated eject/insert of the blade ("reset server bay XX" from OA CLI).
  • Reset all settings on the ILO card to default from hponcfg.
  • Rebooted the ILO card from hponcfg.
  • Upgraded ILO2 firmware to 1.81
  • Restarted OA.
However - what finally solved it was simple enough. I simply reapplied the 2.60 firmware upgrade again. Fair enough.