Add typing annotations to parser.py, which allows us to verify python
scripts with stricter mypy settings by default.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

We need a container with a recent version of Ubuntu to run mypy in strict
mode on validate.py.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Add typing annotations to validate.py and verify it with stricter mypy
settings.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

This is a first attempt at static checking our python scripts, to prepare
adding more type annotations in the future.

- Rework bits to make mypy happy.
- Add a mypy config file to deal with the junit.xml module.
- Check all scripts with mypy in `make check'.
- Mention mypy in the README and in `make help'.
- Add mypy install to the CI files.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

This is similar to what we do on gitlab.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Update the github workflow yaml file to better match what is done on
gitlab.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Mention that gitlab.arm.com is where the latest version is.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Add a CI job to generate the README.pdf.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Add a .gitlab-ci.yml file to run `make check' as CI.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Nhi Pham authored Mar 30, 2023 and Vincent Stehle committed May 24, 2023

Add summary table headings. Also, removing trailing white-space
on the markdown strings.

Signed-off-by: Nhi Pham <nhi@os.amperecomputing.com>

Also, add a unit test.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Describe the convention in comments and adjust existing naming accordingly.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

- Add a unit test with an empty .ekl file.
- Add identification tests.

While at it, refine the naming in the db.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Perform some sanity checks on the tests after loading.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Refine a bit the fields we create for the synthesized dropped tests.
This helps when the config file in use has rules referring those fields for
their criteria, and avoids exceptions.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

SIE.yaml: add known u-boot limitations

See merge request !18

The U-Boot QueryVariableInfo() implementation does not check properly for
all invalid inputs conditions instead of delegating to StMM.

Add a rule to override the result as "known u-boot limitation" in those
cases.

Link: https://lore.kernel.org/u-boot/20230126031512.17450-1-masahisa.kojima@linaro.org/


Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

The SIE errors, which we originally classified as "known rpmb limitations"
can happen not only with an eMMC but in the more general case with a small
storage for UEFI variables. This is typically the case with OP-TEE.
Let's re-classify those errors as "known size limitations". This is just
renaming at this stage.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

The ACS-IR 2.0 BET-1 SIE SCT is missing a few tests due to a patch not
applying properly during build. This results in dropped tests being
reported when parsing results.
Force the results of those tests as `KNOWN ACS LIMITATION'.

Also, update the schema.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Factorise tests `descr' and `sub set' enums definitions into the common
schema.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Constrain a bit more which fields a config file can update in the schema.
As of today, configs update either comments or results.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Add the EBBR sequence files from the ACS-IR v22.10_2.0.0_BETA-1 and update
the identification database accordingly.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

When running the new 32b ACS-IR, new HII Database failures are caught.
Triage them automatically after analysis.

While at it, update one comment in the config file to better match the
reality and update the schema.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Add the EBBR sequence files from the ACS-IR v22.10_IR_32b_0.7_BETA-0.
Update the identification database accordingly, mentioning those IR v1.x
config files as deprecated.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

This fixes the case when the path contains spaces.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Make the test for junit output depend on the availability of the junit_xml
python module.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Start factorizing our schemas, now that validate.py supports it.
We start with the obvious guid definition, as a proof of concept.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Enhance the validate.py script to allow referencing other schemas from a
schema, and load the local file with the script.

This will allow us to factorize our schemas.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

We need double quotes for proper multiline strings in YAML.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Reword our asserts to fix the errors reported by flake8, such as the
following:

  ./parser.py:375:11: E275 missing whitespace after keyword

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

This avoids an exception when junit is not installed.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Collect a few meta-data during execution, such as the date, the parser
command line, the sequence file identification and the git commit of the
parser when possible.

We add a new `--print-meta' command line option to display the collected
meta-data.

We output the collected meta-data to the generated files for the following
formats: md, yaml and template.

Also, we gracefully ignore the meta-data when reading back a result.md file
with meta-data.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

We add schemas to validate the generated YAML files contents and use them
in the unit test. The constraints are a bit loose but this is better than
nothing.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

JSON schema definitions are better put in the `$defs' object; fix our
schemas accordingly.

While at it, add `$id' identifiers and bump `$schema' version.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

We borrow the validate.py script from the SystemReady scripts
repository[1] to validate our YAML files against schemas.

We deprecate the `--validate-config' and the `--validate-seq-db'
command line options of the parser.

We adapt the `make check' commands.

[1]: https://gitlab.arm.com/systemready/systemready-scripts



Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Add a shell script to test functionalities of the parser and run it from
`make check'.

Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>

Sequence files

See merge request !16