# OpenBMC on FVP Base and Arm FVP platform ## Table of Contents * [Communication Channels](#communication-channels) + [Redfish Host Interface](#redfish-host-interface) * [Build](#build) * [Setup PoC demo](#setup-poc-demo) * [Usage](#usage) + [BMC Out-of-Band Communication](#bmc-out-of-band-communication) - [Redfish APIs](#redfish-apis) - [SSH](#ssh) + [Side-Band Communication](#side-band-communication) - [PLDM Daemon](#pldm-daemon) - [PLDM Sensor Reading](#pldm-sensor-reading) - [PLDM CPER Event](#pldm-cper-event) + [In-band Communication (EDK2)](#in-band-communication-edk2) - [Sample EDK2 IPMI DXE driver and IPMI Commandline Application](#sample-edk2-ipmi-dxe-driver-and-ipmi-commandline-application) - [Sample EDK2 RedfishPkg and Redfish Commandline Application](#sample-edk2-redfishpkg-and-redfish-commandline-application) + [In-band Communication (Operating System)](#in-band-communication-operating-system) - [Ipmitool Application](#ipmitool-application) - [Redfish-finder and Dmidecode Application](#redfish-finder-and-dmidecode-application) * [Other PoC Showcases](#other-poc-showcases) + [OpenBMC on AST2600 EVK and Arm FVP RD-V3-R1](#openbmc-on-ast2600-evk-and-arm-fvp-rd-v3-r1) + [MCTP over PCC](#mctp-over-pcc) This repository provides scripts and documentation for reproducing a proof of concept utilising OpenBMC and Arm FVP platform to run IPMI and PLDM/MCTP communication. These scripts have been tested with Ubuntu 22.04 Jammy (LTS) and details instructions to build and run OpenBMC and Arm FVP platform. The demo uses two separate software stacks to verify the communication channels. In this example, RD-V3-R1 acts as our Reference Platform and Base-FVP as the BMC. And we will showcase below communication channels. * IPMI Communication * PLDM/MCTP Communication (Sensor Reading and CPER Event) * Redfish Out-of-Band Communication * Redfish Host Interface Communication Host machine recommended hardware configuration: * 500GB SSD drive * 64GB RAM ## Communication Channels Two communication channels, In-Band and Side-Band interface, between OpenBMC and FVP RDV3R1 and the channels communication are listed below. * In-band communication uses the IPMI over Serial protocol and Redfish Host Interface over virtual ethernet between the Application Processors (AP) and the BMC. * Side-Band Communication uses the PLDM and MCTP over Serial protocol between the Management Control Processor (MCP) and the BMC. ``` AP debug console | +--------------------------+ +-----------------------------+ | | | | | Base FVP | | Neoverse RD-V3-R1 FVP | | | Redfish Host Interface | | | | over | _______________ | | | virtual ethernet | | | | | Virtio Net |----------------------------------|------| | | | | | | | | | | | | | | SOL--|------------ /dev/ttyAMA3 |------- Host Serial Console ------|------| | | | | (terminal_3) (terminal_ns_uart0) | | AP | | | | | | | | | | | | | | | /dev/ttyAMA2 |--------- IPMI over UART ---------|------| | | | | (terminal_2) (terminal_3) | |_______________| | | | | | | | | | | | PLDM over | | | | MCTP over +-------+ +-------+ Redfish--| /dev/ttyAMA1 |-------------- UART --------------| MCP | | SCP | +--------------------------+ (terminal_1) (terminal_2) +-------+-------------+-------+ | | | FVP debug console | debug console (terminal_0) | (terminal_uart_scp) debug console (terminal_uart_mcp) ``` ### Redfish Host Interface Redfish is a flexible system management tool that can be successfully applied to various system architectures. It defines an industry standard protocol and provides a RESTful interface for the remote management of servers, storage, networking, and converged infrastructure. Redfish Host Interface is an in-band communication channel that enables host software to access the Redfish provided by a management controller without relying on external networking. In this PoC, the host is simulated by the RD-V3-R1 FVP and the BMC is simulated by the Base FVP. A virtual ethernet interface supports the Redfish Host Interface to provide in-band communication. The host (EDK2) constructs the RedfishPkg by querying BMC information using IPMI over Serial interface. The host then populates SMBIOS type 42 record for host software discovery. As a result, end users (EDK2 and the operating system) can manage the computer system through the Redfish Host Interface. ## Build Run the "build.sh" scripts with parameter to automatically fetch the sources and then build the images for FVP RDV3R1 and OpenBMC. Need to run "./build.sh setup" to fetch the source code first, then to run "./build.sh build" to build the images. ```bash ubuntu-user:~$ ./build.sh -h Setup and build PLDM Sensor and Event PoC on Arm FVP Usage: build.sh [parameters] parameters: setup Download the required source code and apply the patch sets build bmcfvp Build the required host image and BMC image for BASEFVP build ast2600 Build the required host image and BMC image for AST2600 clean Clean the build folders ``` ```bash ubuntu-user:~$ ./build.sh setup ubuntu-user:~$ ./build.sh build bmcfvp ``` **Note:** If you run into out-of-memory errors, you can restrict the number of build threads by adding the following settings to `./openbmc/build/fvp/local.conf`: ``` BB_NUMBER_THREADS = "16" PARALLEL_MAKE = "-j16" ``` ## Setup PoC demo This PoC utilizes TUN/TAP virtual network devices. You need to install additional packages on the host machine and configure TAP interface before running the demo. This command installs additional packages on the host machine, the user is expected to have sufficient privileges on the host machine. See detail in https://neoverse-reference-design.docs.arm.com/en/latest/user_guides/getting_started.html?highlight=tap#enable-network-for-fvp-s-optional ```bash # Host Dependencies $> sudo apt update $> sudo apt install qemu-kvm libvirt-daemon-system iproute2 # Ensure that libvirtd service is running $> sudo systemctl start libvirtd # Example uses `virbr0` for the bridge name $> sudo ip link add name virbr0 type bridge $> sudo ip link set dev virbr0 up # Create TAP interface for Host NIC and attached to `virbr0` $> sudo ip tuntap add dev tap0 mode tap user $(whoami) $> sudo ip link set tap0 promisc on $> sudo ip addr add 0.0.0.0 dev tap0 $> sudo ip link set tap0 up $> sudo ip link set tap0 master virbr0 # Create TAP interface for BMC NIC and attached to `virbr0` $> sudo ip tuntap add dev RedfishHI mode tap user $(whoami) $> sudo ip link set RedfishHI promisc on $> sudo ip addr add 0.0.0.0 dev RedfishHI $> sudo ip link set RedfishHI up $> sudo ip link set RedfishHI master virbr0 ``` Execute the `run.sh` script with the full path of FVP model to launch the OpenBMC and FVP RDV3R1 demo. This will open several console windows including the OpenBMC FVP console, the terminal_uart_mcp console, and the terminal_uart_ns_uart0 (AP) console. Once everything is up and running, you'll see the message `MCTP discovery success` that indicates you're ready to start testing the communication channels. To exit the PoC demo by pressing Ctrl+c to exit the PoC demo. If you encounter the message `MCTP discovery failed`, then log into the OpenBMC console and run `systemctl restart mctp-local` to retry the MCTP discovery process. ```bash ubuntu-user:~$ ./run.sh Usage: ./run.sh [-m ] [-ast -bmc_ip -uart_to_ap -uart_to_mcp ] [-h] -h show this help -m FVP_MODEL_PATH (required) -ast Use ASPEED as BMC platform (optional) -bmc_ip BMC IP address (required when using -ast) -uart_to_ap UART for BMC and AP communication (required when using -ast) -uart_to_mcp UART for BMC and MCP communication (required when using -ast) ``` ```bash ubuntu-user:~$ ./run.sh -m /home/workstation/FVP_RD_V3_R1/models/Linux64_GCC-9.3/FVP_RD_V3_R1 ``` ![](demo.png) Find more detial for manually setup * https://github.com/openbmc/openbmc/blob/master/meta-evb/meta-evb-arm/meta-evb-fvp-base/README.md * https://neoverse-reference-design.docs.arm.com/en/latest/features/bmc.html **Note:** - There are two terminal_uart_mcp windows available. Use the one repeatedly prints the PLDM/MCTP logs. - OpenBMC default username is `root` and password is `0penBmc`. ## Usage ### BMC Out-of-Band Communication #### Redfish APIs HTTPs server runs in OpenBMC is port number 4223. To access Redfish APIs, executing below command on the Ubuntu machine terminal. ```bash ubuntu-user:~$ curl -k -u root:0penBmc -X GET https://127.0.0.1:4223/redfish/v1/ ``` #### SSH SSH server runs in OpenBMC is port number 4222. To SSH into the OpenBMC, executing below command on the Ubuntu terminal. ```bash ubuntu-user:~$ ssh -p 4222 root@localhost ``` ### Side-Band Communication #### PLDM Daemon Log in to the OpenBMC terminal with username `root` and password `0penBmc`, then run the command below to list all sensors that PLDM can access: ```bash root@fvp:~# busctl tree xyz.openbmc_project.PLDM `- /xyz `- /xyz/openbmc_project |- /xyz/openbmc_project/inventory | `- /xyz/openbmc_project/inventory/system | `- /xyz/openbmc_project/inventory/system/board | `- /xyz/openbmc_project/inventory/system/board/SatMC | `- /xyz/openbmc_project/inventory/system/board/SatMC/SatMC_CoreTemp |- /xyz/openbmc_project/pldm |- /xyz/openbmc_project/sensors | `- /xyz/openbmc_project/sensors/temperature | `- /xyz/openbmc_project/sensors/temperature/SatMC_CoreTemp `- /xyz/openbmc_project/software ``` **Note:** If you didn't see any detected sensor, then run `systemctl restart mctp-local` to restart the MCTP discovery process. #### PLDM Sensor Reading A sample temperature sensor is implemented in the MCP firmware. When PLDM initializes between the platform and the BMC, the sensor's PDR (Platform Descriptor Record) is sent to the BMC, which then polls the sensor reading at the interval specified in `update_interval`. The sensor value will increment everytime the BMC polls for the sensor value. To retrieve the sensor reading with Redfish APIs, use the following Redfish command: ``` ubuntu-user:~$ curl -k -u root:0penBmc -X GET https://localhost:4223/redfish/v1/Chassis/SatMC/Sensors/temperature_SatMC_CoreTemp { "@odata.id": "/redfish/v1/Chassis/SatMC/Sensors/temperature_SatMC_CoreTemp", "@odata.type": "#Sensor.v1_2_0.Sensor", "Id": "temperature_SatMC_CoreTemp", "Name": "SatMC CoreTemp", "Reading": 17.0, "ReadingRangeMax": 255.0, "ReadingRangeMin": 0.0, "ReadingType": "Temperature", "ReadingUnits": "Cel", "Status": { "Health": "OK", "State": "Enabled" } } ``` #### PLDM CPER Event A sample CPER record can be sent from the MCP firmware to the BMC. To do this, execute the below command in the MCP terminal window terminal_uart_mcp that repeatedly prints the PLDM/MCTP logs. **Note:** To enter MCP Debug Prompt by pressing Ctrl+e on MCP debug console. To exit MCP Debug Prompt by pressing Ctrl+d on MCP debug console. ``` [CLI_DEBUGGER_MODULE] Entering CLI > pldm cper [PLDM FW] Send Platform Event Message - CPER ... > ``` To retrieve the CPER event information and download CPER event logs with Redfish APIs, use the following Redfish command: ``` ubuntu-user:~$ curl -k -u root:0penBmc -X GET https://localhost:4223/redfish/v1/Managers/bmc/LogServices/FaultLog/Entries/1 { "@odata.id": "/redfish/v1/Managers/bmc/LogServices/FaultLog/Entries/1", "@odata.type": "#LogEntry.v1_11_0.LogEntry", "AdditionalDataSizeBytes": 104, "AdditionalDataURI": "/redfish/v1/Managers/bmc/LogServices/FaultLog/Entries/1/attachment", "Created": "2024-12-20T00:46:04.280600+00:00", "DiagnosticDataType": "CPER", "EntryType": "Event", "Id": "1", "Name": "FaultLog Dump Entry" } ``` ``` ubuntu-user:~$ curl -k -u root:0penBmc -X GET https://localhost:4223/redfish/v1/Managers/bmc/LogServices/FaultLog/Entries/1/attachment | base64 -d | hexdump -C % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 140 100 140 0 0 208 0 --:--:-- --:--:-- --:--:-- 208 00000000 48 00 00 00 20 00 00 00 00 01 00 00 00 00 00 00 |H... ...........| 00000010 16 3d 9e e1 11 bc e4 11 9c aa c2 05 1d 5d 46 b0 |.=...........]F.| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000030 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000040 00 00 00 00 00 00 00 00 02 02 00 00 00 00 00 00 |................| 00000050 00 00 00 00 00 00 00 00 78 56 34 12 cd ab ba dc |........xV4.....| 00000060 01 02 03 04 05 06 07 08 |........| 00000068 ``` ### In-band Communication (EDK2) #### Sample EDK2 IPMI DXE driver and IPMI Commandline Application A sample dxe driver is included in the edk2-platforms to demonstrate the usage of IPMI in an edk2 driver. It gets the IP address and Subnet mask from the BMC and prints the below log during the boot. ``` Loading driver at 0x000F2135000 EntryPoint=0x000F2137134 IpmiSerialDxe.efi Device ID Response : CC 0, Device ID 0, Revision 0 BMC IP : 172.20.51.1 BMC IP Subnet Mask: 255.255.255.0 ``` Boot progress code dxe driver is included in the edk2-platforms to demonstrate the usage of IPMI in an edk2 driver. During the boot process, EDK2 transmits status codes to the BMC using the SBMR defined Boot Progress Code format. These boot progress codes are also exposed in the Redfish LastState property, as shown below. ``` SbmrSendBootProgressCode: CodeType=0x1 Value=0x3050007 Instance=0x0 CallerIdGuid=462CAA21-7614-4503-836E-8AB6F4662331 Data=F1747038 ``` ``` ubuntu-user:~$ curl -k -u root:0penBmc -X GET https://localhost:4223/redfish/v1/Systems/system { ... "BootProgress": { "LastState": "SetupEntered", "LastStateTime": "2024-12-20T01:38:29.906412+00:00" }, ... } ``` A sample commandline app named ipmiutil is ported from Ampere's implementation. It can be used to send any IPMI raw command to the BMC. It is executed from the UEFI shell. To enter into UEFI shell, Below is a sample IPMI command executed in the UEFI shell. ``` Shell> ipmiutil -r 0x6 0x46 0x1 72 6F 6F 74 00 00 00 00 00 00 00 00 00 00 00 00 C1 ``` #### Sample EDK2 RedfishPkg and Redfish Commandline Application During the boot sequence, the host issues IPMI commands to the BMC to construct the SMBIOS Type 42 record for the Redfish Host Interface, then outputs the following log: ``` ... BMC IP Discovery Type: 1 BMC IP format: 2 BMC IP : 192.168.122.10 BMC IP Subnet Mask: 255.255.255.0 Port 443 BMC Vlan ID: 0 BMC Mac Address: 72:45:77:AA:F7:87 ... ``` Use sample commandline applications, smbiosview and RedfishUtility.efi, to dump the SMBIOS Type 42 record and query the Redfish API. Execute the below command in the terminal_uart_ns_uart0 (AP) console to view SMBIOS Type 42 record from UEFI shell and access Redfish /redfish/v1 object. ``` Shell> smbiosview -t 42 ... Type=42, Handle=0x4 Dump Structure as: Index=11,Length=0x86,Addr=0xF25B0603 00000000: 2A 84 04 00 40 11 04 11-28 0D 04 02 00 72 45 77 **...@...(....rEw* 00000010: AA F7 87 01 00 FF FF 01-04 6A 18 36 89 C8 F8 80 *.........j.6....* 00000020: 4E 41 83 E1 67 E8 57 B2-12 C2 01 01 C0 A8 7A 09 *NA..g.W.......z.* 00000030: 00 00 00 00 00 00 00 00-00 00 00 00 FF FF FF 00 *................* 00000040: 00 00 00 00 00 00 00 00-00 00 00 00 01 01 C0 A8 *................* 00000050: 7A 0A 00 00 00 00 00 00-00 00 00 00 00 00 FF FF *z...............* 00000060: FF 00 00 00 00 00 00 00-00 00 00 00 00 00 BB 01 *................* 00000070: 00 00 00 00 0F 31 39 32-2E 31 36 38 2E 31 32 32 *.....192.168.122* 00000080: 2E 31 30 00 00 00 *.10...* Type: Management Controller Host Interface Length: 132 Handle: 4 MC Host Interface Type: Network Host Interface InterfaceTypeSpecificDataLength: 0x11 Dump InterfaceTypeSpecificData size=17: 00000000: 04 11 28 0D 04 02 00 72-45 77 AA F7 87 01 00 FF *..(....rEw......* 00000010: FF *.* DeviceType: USB v2 IdVendor: 0xD28 IdProduct: 0x204 MacAddress: 72:45:77:AA:F7:87 Characteristics: 0x1 CredentialBootstrappingHandle: 0xFFFF ProtocolType: Redfish over IP ServiceUUID: 00000000: 18 36 89 C8 F8 80 4E 41-83 E1 67 E8 57 B2 12 C2 *.6....NA..g.W...* HostIpAssignmentType: Static HostIpAddressFormat: IPv4 HostIpAddress: 192.168.122.9 HostIpMask: 255.255.255.0 RedfishServiceIpAssignmentType: Static RedfishServiceIpAddressFormat: IPv4 RedfishServiceIpAddress: 192.168.122.10 RedfishServiceIpMask: 255.255.255.0 RedfishServiceIpPort: 443 RedfishServiceVlanId: 0 RedfishServiceHostnameLength: 15 RedfishServiceHostname: 192.168.122.10 ``` ```bash Shell> FS0:RedfishUtility.efi /redfish/v1 ... HTTP Response : { "@odata.id": "/redfish/v1", "@odata.type": "#ServiceRoot.v1_15_0.ServiceRoot", "AccountService": { "@odata.id": "/redfish/v1/AccountService" }, "Cables": { "@odata.id": "/redfish/v1/Cables" }, "CertificateService": { "@odata.id": "/redfish/v1/CertificateService" }, "Chassis": { "@odata.id": "/redfish/v1/Chassis" }, "EventService": { "@odata.id": "/redfish/v1/EventService" }, "Id": "RootService", "JsonSchemas": { "@odata.id": "/redfish/v1/JsonSchemas" }, "Links": { "ManagerProvidingService": { "@odata.id": "/redfish/v1/Managers/bmc" }, "Sessions": { "@odata.id": "/redfish/v1/SessionService/Sessions" } }, "Managers": { "@odata.id": "/redfish/v1/Managers" ... ``` **Note:** - To enter the UEFI shell, select `Boot Manager` and then select `UEFI Shell` in the terminal_uart_ns_uart0 (AP) console - To exit the UEFI shell, type "exit" in UEFI Shell - To entry Linux distro, select `Boot Manager` and then select `UEFI Misc Device 2` in the terminal_uart_ns_uart0 (AP) console ### In-band Communication (Operating System) #### Ipmitool Application Use sample commandline applications, ipmitool, to send ipmi command to BMC. Log into Linux distro with username `root` and execute the below command in the terminal_uart_ns_uart0 (AP) console to get BMC network information. ```bash AP-Linux:~$ ipmitool -I serial-basic -D /dev/ttyAMA2 lan print 1 Set in Progress : Set Complete Auth Type Support : Auth Type Enable : Callback : : User : : Operator : : Admin : : OEM : IP Address Source : Static Address IP Address : 192.168.122.10 Subnet Mask : 255.255.255.0 MAC Address : 72:45:77:aa:f7:88 ``` #### Redfish-finder and Dmidecode Application Use sample commandline applications, dmidecode and redfish-finder, to dump the SMBIOS Type 42 record and enable Redfish Host Interface in the OS. Log into Linux distro with username `root` and execute the below command in the terminal_uart_ns_uart0 (AP) console to view the SMBIOS Type 42 record. ```bash AP-Linux:~$ dmidecode -t 42 # dmidecode 3.6 Getting SMBIOS data from sysfs. SMBIOS 3.4.0 present. Handle 0x0004, DMI type 42, 132 bytes Management Controller Host Interface Host Interface Type: Network Device Type: USB v2 idVendor: 0x0d28 idProduct: 0x0204 MAC Address: 72:45:77:aa:f7:87 Device Characteristics: Credential bootstrapping via IPMI is supported Credential Bootstrapping Handle: 0xffff Protocol ID: 04 (Redfish over IP) Service UUID: c8893618-80f8-414e-83e1-67e857b212c2 Host IP Assignment Type: Static Host IP Address Format: IPv4 IPv4 Address: 192.168.122.9 IPv4 Mask: 255.255.255.0 Redfish Service IP Discovery Type: Static Redfish Service IP Address Format: IPv4 IPv4 Redfish Service Address: 192.168.122.10 IPv4 Redfish Service Mask: 255.255.255.0 Redfish Service Port: 443 Redfish Service Vlan: 0 Redfish Service Hostname: 192.168.122.10 ``` Execute the below command in the terminal_uart_ns_uart0 (AP) console to enable Redfish Host Interface and access Redfish /redfish/v1 object. ```bash AP-Linux:~$ redfish-finder redfish-finder: Getting dmidecode info MAC = 72:45:77:aa:f7:87 redfish-finder: Building NetworkManager connection info Error: 'eth0' is not an active connection. Error: no active connection provided. Error: eth0 - no such connection profile. Connection 'eth0' (d221c829-01ac-4e49-a02b-c8ab8de7f1df) successfully added. redfish-finder: Obtaining OS config info redfish-finder: Converting SMBIOS Host Config to NetworkManager Connection info redfish-finder: Applying NetworkManager connection configuration changes Error: 'eth0' is not an active connection. Error: no active connection provided. Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3) redfish-finder: Adding redfish host info to OS config redfish-finder: Done, BMC is now reachable via hostname redfish-localhost ``` ```bash AP-Linux:~$ curl -k https://redfish-localhost/redfish/v1 { "@odata.id": "/redfish/v1", "@odata.type": "#ServiceRoot.v1_15_0.ServiceRoot", "AccountService": { "@odata.id": "/redfish/v1/AccountService" }, "Cables": { "@odata.id": "/redfish/v1/Cables" }, "CertificateService": { "@odata.id": "/redfish/v1/CertificateService" }, "Chassis": { "@odata.id": "/redfish/v1/Chassis" }, "EventService": { "@odata.id": "/redfish/v1/EventService" }, "Id": "RootService", "JsonSchemas": { "@odata.id": "/redfish/v1/JsonSchemas" }, "Links": { "ManagerProvidingService": { "@odata.id": "/redfish/v1/Managers/bmc" }, "Sessions": { "@odata.id": "/redfish/v1/SessionService/Sessions" } }, "Managers": { "@odata.id": "/redfish/v1/Managers" }, "Name": "Root Service", ... ``` ## Other PoC Showcases ### OpenBMC on AST2600 EVK and Arm FVP RD-V3-R1 This PoC demonstrates IPMI, PLDM/MCTP, and Redfish communication between OpenBMC running on AST2600 EVK hardware and Arm FVP RD-V3-R1 platform. The setup includes in-band IPMI communication over serial between Application Processors and BMC, and side-band PLDM/MCTP communication over serial between Management Control Processor and BMC. **Quick Start Commands:** ```bash ./build.sh setup ./build.sh build ast2600 ./run.sh -m /workspace/FVP_RD_V3_R1/FVP_RD_V3_R1 -ast -bmc_ip 192.168.1.10 -uart_to_ap ttyUSB1 -uart_to_mcp ttyUSB2 ``` For detailed setup instructions, hardware requirements, and testing procedures, please refer to [docs/ast2600evk_fvphost/README.md](docs/ast2600evk_fvphost/README.md). ### MCTP over PCC This PoC demonstrates a proof of concept utilising OpenBMC and Arm FVP platform to run MCTP over PCC communication. In PoC demostration, RD-N2 acts as our Reference Platform and Base-FVP as the BMC to verify the communication channel. For detailed setup instructions, please refer to [docs/mctp_pcc/README.md](docs/mctp_pcc/README.md). ## Reference * Neoverse Reference SW : https://neoverse-reference-design.docs.arm.com/en/latest/features/bmc.html * OpenBMC FVP Base : https://github.com/openbmc/openbmc/blob/master/meta-evb/meta-evb-arm/meta-evb-fvp-base/README.md