# Template Structure for SystemReady ES or SR Compliance Reports This repo structure is the template for collecting compliance evidence for a SystemReady ES or SR certification. (For the IR band, see the [systemready-template].) [systemready-template]: https://gitlab.arm.com/systemready/systemready-template ## Prerequisites: - Check basic info in https://developer.arm.com/documentation/102529/0100/Test-SystemReady-ES-compliance - Flash/update the firmware again to clear UEFI variable store/NVRAM right before running the tests, which will avoid fake failures that are caused by using up the UEFI variable store/NVRAM space. - Use a USB disk enclosure with a fast SSD or SATA drive for creating ACS USB drive, which would significantly shorten the testing time and will be more stable than using USB stick/flash drive. - Running SCT with USB stick may take more than 10 hours and run into some unexpected problem caused by USB HW issue. - Go to UEFI/BIOS setup menu to change your system’s "OS hardware description" option to "ACPI mode". You may find the option in Device Manage > O/S Hardware Description Selection. - Go to UEFI/BIOS setup menu to move the ACS media device (USB drive) to the top of the boot order. - Install network cable to all network ports and connect to a DHCP server for network related testing. ## General Instructions General instructions for collecting SystemReady ES or SR compliance logs: ### `./report.txt` Fill in with information about the system being certified ### `./acs-auto-results/` - Place the automation test result (entire copy of the `LUV-Results` or `RESULT` partitions in ACS USB drive/stick), as follows: - For Enterprise ACS, under the `LUV-Results` partition: ``` /luv-results- /sbsa_results /uefi /linux /SCT_RUN /sdei_results ``` - For SystemReady ES ACS, under the `RESULT` partition: ``` /acs_results /app_output /fwts /linux /linux_dump /sct_results /uefi /uefi_dump ``` ### `./manual-results/` - Place any results from manually performed tests in this directory tree. - Manual test results must include both a console log of the commands - executed and the result files (if any) generated by the test. ### `./manual-results/bsa-linux/` - If you're running the Enterprise ACS, please rename this folder from 'bsa-linux' to 'sbsa-linux' - Place any BSA or SBSA results collected manually from Linux (LUVOS or Linux BusyBox). - Note that if automation test works fine, you won't need to run manul test and put any file in this folder. - Examples for Enterprise-ACS: ``` Complete SBSA Linux results can be collected by running the following commands after booting LUVOS insmod /lib/modules/4.18.0-luv/extra/sbsa_acs.ko sbsa You can include verbose output (-v): sbsa -l 3 -v 2 To skip a specific problematic test, use -skip sbsa -l 3 -v 2 -skip 413 ``` - Examples for ES ACS ``` Complete BSA UEFI results can be collected by running the following command after booting Linux BusyBox: insmod /lib/modules/bsa_acs.ko bsa You can include verbose output (-v): bsa -v 1 To skip a specific problematic tests, use -skip bsa -skip 104 ``` ### `./manual-results/bsa-uefi/` - If you're running the Enterprise ACS, please rename this folder from 'bsa-uefi' to 'sbsa-uefi' - Place any BSA or SBSA results collected manually from UEFI Shell. - Note that if automation test works fine, you won't need to run manul test and put any file in this folder. - Examples for Enterprise-ACS: ``` Complete SBSA UEFI results can be collected by running the following commands: sbsa.efi You can include verbose output (-v): Shell> sbsa.efi -v 1 To skip a specific problematic tests, use -skip , Shell> sbsa.efi -skip 448,449 ``` - Examples for ES ACS: ``` Complete BSA UEFI results can be collected by running the following command: Shell> bsa.efi You can include verbose output (-v): Shell> bsa.efi -v 1 To skip a specific problematic tests, use -skip Shell> bsa.efi -skip 606 ``` ### `./manual-results/fwts/` Place any additional FWTS results collected manually from Linux (LUVOS or Linux BusyBox): Note that if automation test works fine, you won't need to run manul test and put any file in this folder. Examples: ``` Complete SBSA UEFI results can be collected by running the following commands: fwts -r stdout -q --sbbr esrt uefibootpath To skip a specific problematic tests, use --skip-test= fwts -r stdout -q --sbbr --skip-test=dmicheck fwts -r stdout -q --sbbr --skip-test=uefirtmisc,uefirtvariable,uefirttime ``` ### `./manual-results/sct/` - Place any additional SCT results collected manually from UEFI shell: - Note: If automation test works fine, you won't need to run manul test and put any file in this folder. - Note: Before manually running SCT, please run "connect -r" in UEFI shell first. Examples: ``` Enterprise-ACS Complete SCT results can be collected by running the following commands FS1:\EFI\BOOT\sbbr\AARCH64>SCT.efi -s SBBR.seq Specific SCT test's result can be collected by running the following commands to use the Menu-Driven Interface For details, please check the section 2.2 in https://github.com/tianocore/edk2-test/blob/master/uefi-sct/Doc/UserGuide/UEFI_SCT_II_UserGuide_2_6_A.pdf FS1:\EFI\BOOT\sbbr\AARCH64>SCT.efi -u ES ACS Complete SCT results can be collected by running the following commands FS1:\EFI\BOOT\bbr\SCT>SCT.efi -s SBBR.seq Specific SCT test's result can be collected by running the following commands to use the Menu-Driven Interface For details, please check the section 2.2 in https://github.com/tianocore/edk2-test/blob/master/uefi-sct/Doc/UserGuide/UEFI_SCT_II_UserGuide_2_6_A.pdf FS1:\EFI\BOOT\bbr\SCT>SCT.efi -u ``` ### `./docs/` Place any firmware or device documentation, manuals, user guides, build instructions, etc.. ### `./fw/` #### `./fw/screenshots/` Place some screenshots showing the FW menus (such as UEFI Setup, BMC web console, uboot shell) #### `./fw/uefi-sniff.log` Run the following commands in UEFI Shell and attach the logs (if not already done by ACS). The UEFI Shell can be run from the ACS image by pressing a key at the UEFI timeout prompt before tests begin to execute. ``` Shell> connect -r Shell> pci Shell> drivers Shell> devices Shell> devtree Shell> dmpstore Shell> dh -d -v Shell> memmap Shell> smbiosview Shell> acpiview -l Shell> acpiview Shell> acpiview -r 2 Shell> acpiview -s DSDT -d (if DSDT table is present) Shell> acpiview -s SSDT -d (if SSDT table is present) Shell> bcfg boot dump -v Shell> ifconfig -l Shell> ifconfig -s eth0 dhcp (If eth0 still hasn’t get IP, run "connect -r" again) Shell> ifconfig -l eth0 Shell> ping xxx.xxx.xxx.xxx (If don’t know any other system’s IP address, you can get DNS Server's IP address from ifconfig output and ping it) ``` ### `./os-logs/` #### `./os-logs/[distroname] [distroversion]/` - Please check ./os_logs/OS_image_download_links.txt to get the latest release for testing - Please rename the subfolders. For example, rename 'linux_distro_1' to 'Fedora Server 35' - Install the OS to a disk, and boot it. (Note that a network controller (PCIe or USB) is needed.) - The install log must begin when the platform is released from reset and must include: - All firmware output - Output from Linux installer - Output of reboot into installed OS. - Run the following commands and attach the logs: (Note that some commands may not work on some distros, and you may need to install the command or run an alternative command.) ``` sudo -s (Prevent from running into "Permission denied" error) dmesg lspci -vvv lscpu lsblk lsusb dmidecode uname -a cat /etc/os-release efibootmgr (Check the output message to know the ) efibootmgr -t 20 efibootmgr -t efibootmgr -c ifconfig (If ifconfig is not supported, use "ip addr show" and "ip link show" instead.) (If network port doesn't get IP address, run "dhclient" or "ip a add /255.255.255.0 dev eth" and "ip link set dev eth up") ping -c 5 www.arm.com (If the system is connected to intranet, please ping another system's IP address) Go to /sys/firmware and run "ls -lR" (Or use tree /sys/firmware/) ``` - Copy the entire content of /sys/firmware and attach zipped/archive file. You can run "cp -r sys/firmware " and then separately compress the files. - Note that directly running "tar -cvzf //sys_firmware.tgz /sys/firmware/" may result in corruption. #### `./os-logs/esxi/` - Please rename the subfolders. For example, rename 'esxi' to 'VMware ESXi 1.8' - Install VMWare ESXi to a disk, and boot it. (Note that a network controller (PCIe or USB) is needed.) - Collect the installation and OS boot logs and save in a text file. If no serial output log, attach screenshots or video instead. - Connect to vSphere web console over network and capture screenshots - Follow the instruction in https://kb.vmware.com/s/article/2004746 to enable ESXi Shell and SSH. - Connect ESXi shell through the SSH and Run the following commands and attach the logs: ``` dmesg lspci irqinfo localcli storage core adapter list localcli hardware pci list esxcli hardware cpu list esxcli hardware usb passthrough device list vim-cmd hostsvc/hosthardware vmware -v reboot -f ``` #### `./os-logs/winpe/` - Boot WinPE from USB key - Include screenshots and video of WinPE booting (from FW menu to OS startup, and keyboard working in command prompt) - Run SAC commands and/or WinPE commands below: ``` id d cmd ch Esc + Tab (Switch channel to command channel (like Cmd0001) for running WinPE commands) ver Systeminfo (This may not be supported) pnputil/enum-devices pnputil/enum-drivers Esc + Tab (Switch channel to SAC> for running SAC commands) shutdown ```