// HP BladeSystem - Onboard supervisor chip reverse engineering

In a previous post I presented way to power up blade switch module, without having a chassis. It was a pure hardware approach, by enforcing logic states on particular IC's pins. Now, I'll present internal onboard protocol eavesdroping which lead to preparing custom firmware.

 Eavesdropped I²C communication during boot

Preface

During boot sequence switch complains about communication issues with chassis interface:

Error: in system startup, communication failure with on board micro-controller
Failed to version in row 0
Error: in system startup, communication failure with on board micro-controller
Failed to read GPIO data in row 1
Error: in system startup, communication failure with on board micro-controller
Failed to read ISR reason block row 2
Error: in system startup, communication failure with on board micro-controller
Failed to read the Bay Number in row 12

This error doesn't appear, when LPC chip is not resetted, so it would be great to keep it on, and only change state of one output. Flash on LPC ARM controllers can be programmed and read via serial port. PCB analysis shows, that RXD0 (receive) and TXD0 (transmit) ports are routed to J10, an unpopulated pads for 4-pin connector. So it shouldn't be complicated to write own firmware or even modify existing, by adding one LDR and STR operation in compiled code.

 Unpopulated J10 connector, with serial port signals

Further reconnaissance showed, that ARM microcontroller provides a simple interface for chassis manager to remote control of device. Serial console is available on its UART0 port, 9600 baud 8N1 3.3V TTL signals. Protocol is human-friendly:

HP BladeSystem ISMIC
Version 1.02
(c) Copyright 2006 Hewlett-Packard Development Company, L.P.
All rights reserved
> help [ok]
PORT	Enable or disable port
RESET	Reset the chip or ISMIC
POWER	Set power level to switch
LINK	Enable/disable inter-switch link
UID	Enable/disable UID
KLED	Enable/disable keying LED
SET	Set Index Value
GET	Get Index Value
GPIO	Show GPIO values
READ	read  <#bytes>
WRITE	write  
HELP	Show help messages
PS	Show running threads
CLS	Clear the screen
VER	Show version of firmware
MEM	Memory Usage
UPTIME	Show up time
[done]
> 
CTRL-A Z for help | 9600 8N1 | NOR | Minicom 2.7 | VT102 | Offline | ttyUSB0

Et voilà! System is up.

> power on [ok]
[done]
> power [ok]
POWER state is on
[done]
CTRL-A Z for help | 9600 8N1 | NOR | Minicom 2.7 | VT102 | Offline | ttyUSB0

Getting original firmware

Before any software modification it would be great to have backup of original firmware. First thought was to fetch the code via programming interface (ISP), but I didn't expect that the IC's security bits can be not set. Fortunately, one of the commands listed above can be used to dump given memory area.

>  read 0 16 [ok]
 Data:
  0x18[*] 0xF0[*] 0x9F[*] 0xE5[*] 0x18[*] 0xF0[*] 0x9F[*] 0xE5[*]
  0x18[*] 0xF0[*] 0x9F[*] 0xE5[*] 0x18[*] 0xF0[*] 0x9F[*] 0xE5[*]
  ..  

Familiarized with LPC's memory map, I've downloaded content of code area. On-chip non-volatile memory starts at 0x00000000 and ends at 0x00007777. This region could be fetched without issues, but attempt to fetch single byte from SRAM or GPIO APB causes CPU to hang.

I converted text output to binary with simple Python script.

ismic_text_to_bin.py
#!/usr/bin/env python
 
import sys
 
for line in sys.stdin:
    line = line.replace("[ ]", "[X]") #spaces in decoded data can interfere with field delimiter
    ch = line.strip().split(" ")
    for i in ch:
        i=i[:4]
        sys.stdout.write(chr(eval(i)))

I examined text strings in firmware, but nothing interesting has been found, all commands seems to be documented in help to ISMIC's CLI. Then I disassembled code, with objdump. Output was a big mess due to code and data being mixed, but I've found some references to GPIO address area (base addr: 0xE0028000). Unfortunately, I haven't found appearance of copy constant value to P0.6. Ultimately, I've decided to write own software.

LPC ISP

Overwriting on-chip flash with own firmware is irreversible, and it is the only device I have so I decided to make some reconnaisance in LPC's ISP features and Linux tools. lpc21isp looks mature and does the work well with downloading into IC, but upload and download terms are not distinguished and documented enough. I haven't any other LPC chip close at hand to try, so I can't afford to mistake. ISP protocol1) is quite simple, so I decided to fetch firmware manually. ISP bootloader gives possibility to read/write random memory area. Data are sent/received in UUencoded format, easily handled under Linux.

Synchronized

Synchronized
OK

25000
OK

K
0

11

2

R 0 65..23768
12

R 0 32768
0

M-$"?Y0)0H.,`4(3E`U"@XP10A.4<()_E`#"@XY,``N$H((+BDQ`"X0
CTRL-A Z for help | 9600 8N1 | NOR | Minicom 2.7 | VT102 | Offline | ttyUSB0

On-board communication

The ISMIC subsystem communicates with Cisco CPU using I²C protocol. ISMIC takes function of remote memory, data are organized in banks identified by integer number. ISMIC acts as slave device. During boot sequence, CPU fetchs content some part of registers, and fills content of another.

 Eavesdropped I²C communication during boot

Base address of ISMIC controller is 0x7E. It is combined with less significant R/W bit to make 8-bit device address used on bus transmission, so it gives decimal 252 on write, and 253 on read command. Operation is similar to serial I²C EEPROM memories (like AT24xx series). In request to read data from slot, master sends slot number to write address. Next, master sends read address and clocks SCL line until slave doesn't acknowledge (NAK sequence). Original ISMIC sends one byte more, than slot size, so it seems to be a form of CRC sum. Thanks to few eavesdropped packets it was clear, that checksum algorithm is very simple, consecutive bytes are substracted from remainder, with initial value equal zero, cast to unsigned char.

Salae Logic decoder has eye candy and feature rich interface, but can't decode protocols to text in realtime. For further analysis I used sigrok decoder. It prints decoded protocol in realtime, with some changes in source it can also prints timestamps.

$ sigrok-cli -d fx2lafw -c samplerate=1M  --continuous -P i2c:sda=0:scl=1
Start
Address write: 7E
ACK
Data write: 04
ACK
Start repeat
Address read: 7E
ACK
Data read: 44
ACK
Data read: 65
ACK
Data read: 66
ACK
Data read: 61
ACK
Data read: 75
ACK
Data read: 6C
ACK
Data read: 74
ACK
Data read: 20
ACK
Data read: 43
ACK
Data read: 68
ACK
Data read: 61
ACK
Data read: 73
ACK
Data read: 73
ACK
Data read: 69
ACK
Data read: 73
ACK
Data read: 20
ACK
Data read: 4E
ACK
Data read: 61
ACK
Data read: 6D
ACK
Data read: 65
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: 00
ACK
Data read: AC
NACK
Stop
Start
Address write: 48
ACK
Data write: 00
ACK
Start repeat
Address read: 48
ACK
Data read: 1D
ACK
Data read: 00
NACK
Stop

The temperature sensor (with address 48) shares the same bus, and is polled by CPU with 30s interval, one of queries has been caught above. This fact will be used to control fans.

Custom implementation

New firmware provides:

  • automatic power on,
  • front LED control,
  • basic support for data communication with main CPU,
  • fan speed controller, based on temperature.

Only three GPIO pins are available on J10 connector, so PWM used to fan control is shared with TXD of USART0.

1) UM10120 LPC2131/2/4/6/8 User manual, section 20.8 ISP commands

Comments

David Clapp
No. 1 @ 10-05-2014 23:14

Nice work!

Thanks, David

Leave a comment…




ESI If you can't read the letters on the image, download this .wav file to get them read to you.
  • E-Mail address will not be published.
  • Formatting:
    //italic//  __underlined__
    **bold**  ''preformatted''
  • Links:
    [[http://example.com]]
    [[http://example.com|Link Text]]
  • Quotation:
    > This is a quote. Don't forget the space in front of the text: "> "
  • Code:
    <code>This is unspecific source code</code>
    <code [lang]>This is specifc [lang] code</code>
    <code php><?php echo 'example'; ?></code>
    Available: html, css, javascript, bash, cpp, …
  • Lists:
    Indent your text by two spaces and use a * for
    each unordered list item or a - for ordered ones.
Hello World!

Tomasz Głuch
Hi! I'm Tomasz Głuch, sysadmin - IT specialist and electronics enthusiast from Kraków, PL . You'll find here articles about electronics, Linux. Welcome to my site.

QR Code: URL of current page