codecheck.py is a script that finds all shell script files with given tree,
and runs shellcheck on all files that are not in excluded list.
It also runs pycodestyle check for all python files,
within the same directory tree.

This code check script is using venv to isolate required packages,
from user environment. After check is finished, created venv is deleted.
Logging can be changed with '--log' option,
three log levels are available: warning(default), info and debug.

Sample usage:
  $ meta-ewaol-config/tools/codecheck.py . --exclude tools

Issue-Id: SCM-2586
Signed-off-by: Kamil Dziezyk <kamil.dziezyk@arm.com>
Change-Id: I21289169b07b3d9ec590075b08fb32554a81ff36

This patch adds license disclaimers in the meta-ewaol files:
-Headers in all the source files.
-New license section in the README.

Issue-Id: SCM-2638
Signed-off-by: Hugo L'Hostis <hugo.lhostis@arm.com>
Change-Id: Ibeb28a797387e71092dd60a88bfe01f5e3000566

Issue-Id: SCM-2363
Signed-off-by: Richard Neill <richard.neill@arm.com>
Change-Id: Ie74a63861e3d71372772a0393bd77b70672d47b9

Richard Neill authored May 25, 2021 and Richard Neill committed May 25, 2021

Artifacts now unpack with an appropriate subdirectory: conf, images, logs.

Issue-Id: SCM-2701
Signed-off-by: Richard Neill <richard.neill@arm.com>
Change-Id: I8b93a94bca5bc4a763f5cfd9ee895e98596750c4

This patch provides three improvements:
* Allow standalone test runner to be executed from any directory.
* Adjust conditional check for BATS success to check return status rather than
  output.
* If the BATS variables $output and $status are null or unset, they now
  simply default to empty string, rather than emit an error.

Issue-Id: SCM-2363
Signed-off-by: Richard Neill <richard.neill@arm.com>
Change-Id: I71da12353e5ce019441f85728112e200b9ec1d27

Richard Neill authored May 20, 2021 and Diego Sueiro committed May 20, 2021

Provide a YAML file to define build targets (build machines, build modifiers,
CI builds) that can be queried via the kas build script. The list of CI build
targets will be built when "all" is passed to the build script.

Also provide a configuration file to configure a CI build in which the
workspace is subsequently cleaned to conserve disk usage.

Issue-Id: SCM-2619
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I4e5f70734caf72d7da6133e60cebec67034b76ef

Richard Neill authored May 20, 2021 and Diego Sueiro committed May 20, 2021

* Allow the container's network mode to be set via an input argument.
* Validate that the input kas config files for each target can be found.
* Allow the container engine, image, and image version to be set via input
  arguments.
* Ensure the executed container engine subprocess is stopped on a keyboard
  interrupt or terminate signal to avoid dangling containers.
* More clearly log where artifacts were deployed, and reduce log verbosity.
* Include images in the artifacts deployment.
* Include 'pseudo' log files in the log-artifacts deployment.
* Allow for a log file to be given as an input argument, which will log the
  output of the script (duplicating the terminal output).
* If a log-file is provided then reduce the output verbosity to print only an
  important subset of messages to the terminal.
* Allow for container engine volumes to be defined using relative paths in the
  input arguments.
* Update the script to be PEP8 compliant.

Issue-Id: SCM-2619
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: If6bf65270480b9562dc08c6c03bcf4621c4e5566

Richard Neill authored May 10, 2021 and Diego Sueiro committed May 20, 2021

This patch enables a cleaner working directory for the kas build by having each
each repository downloaded to the layers/ subdirectory, and adds this
subdirectory to .gitignore.

Issue-Id: SCM-2619
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I7542127583f2aca0839b50ced318a58611edf83d

Richard Neill authored May 06, 2021 and Diego Sueiro committed May 20, 2021

Enable bitbake disk monitoring for EWAOL image builds, and trigger task stops
or aborts on disk usage thresholds.

Issue-Id: SCM-2619
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: Ie5563853b936575e84fa83608130c25dabc655c5

Richard Neill authored May 05, 2021 and Diego Sueiro committed May 20, 2021

To avoid contaminating the STDOUT stream returned from a Bats run function, we
redirect the STDERR of each OCI container command. Rather than redirecting
STDERR to /dev/null, this patch stores it in a temporary log file within the
OCI_TESTS_LOG_DIR directory.

If there is a test failure, this file's contents are then included in the
debugging output so that the error can be more easily investigated.

This patch also simplifies the logging to avoid significant code duplication,
and generally improves the scripts following shellcheck static analysis.

Issue-Id: SCM-2619
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I8be08b44e58d8f718e4b915296bd55785d840bd4

Richard Neill authored May 05, 2021 and Diego Sueiro committed May 20, 2021

This patch includes the tests.yml build configuration for the targets built
when passing "all" to the kas CI build script.

Issue-Id: SCM-2619
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I483c62aedd80afe8633d7786ddea475dce46a1f5

Filipe Rinaldi authored Apr 29, 2021 and Diego Sueiro committed May 20, 2021

- Simplify the DockerEngine API by removing mnt_and_env_format() and
mnt_and_env().
- Normalise arguments API using add_ prefix
- Use docstrings instead of hash comments
- Remove the intermediary True/False return value in run()

Issue-Id: SCM-2619
Signed-off-by: Filipe Rinaldi <filipe.rinaldi@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: Ie751a8415efa30a3be81f0ef137ee28d3036f0c8

Richard Neill authored May 04, 2021 and Diego Sueiro committed May 20, 2021

This patch updates the kas build configuration in base.yml to use the hardknott
branch of each repository by default for EWAOL image builds.

Issue-Id: SCM-2634
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I27af27d3561a6b479d3ff94e01f017211cb7deeb

Richard Neill authored May 20, 2021 and Diego Sueiro committed May 20, 2021

Adds the meta-ewaol-tests layer to support the validation of images built for
the EWAOL project.

This patch adds Bats-based runtime integration testing for OCI containers. The
integration tests are provided via the oci-runtime-integration-tests.bb recipe.
This recipe deploys tests to validate the ability for a container to execute an
example persistent workload, which involves the pulling of an image from an
image hub on the network, the creation and start of a detached container, and
its subsequent stop and removal.

The tests are made available to run with a standalone shell script via the
run-oci-runtime-integration-tests binary, and as a suite integrated with the
Yocto Package Test (ptest) framework using ptest-runner.

The tests can be included in the build by appending 'ewaol-test' to the build
configuration's DISTRO_FEATURES. A build configuration file
meta-ewaol-config/kas/tests.yml is included to support this inclusion via the
kas build tool.

Issue-Id: SCM-2357
Signed-off-by: Richard Neill <richard.neill@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I2659e27fca12ae2cf9b9d0cd13041eafced7ebe1

Nathan Dunne authored Apr 27, 2021 and Diego Sueiro committed May 20, 2021

Added task containers_kernelcfg_check to validate that the kernel config
required for running containers is present in final kernel configuration.

Using docker.cfg from yocto-kernel-cache as required list.

Issue-Id: SCM-2356
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I3f50d9ef05483695817dded83e6f5c479db5ed41

Nathan Dunne authored May 20, 2021 and Diego Sueiro committed May 20, 2021

Introduced kas-ci-build.py wrapper script for building
ewaol images using kas, aiming to be used by the CI.

Building requires python3 and docker, and will pull the kas
docker container:
ghcr.io/siemens/kas/kas:latest

Issue-Id: SCM-2389
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I2bd3b266e125c997963acec2df1eac844099a335

Nathan Dunne authored Apr 12, 2021 and Diego Sueiro committed May 20, 2021

Created Repo README.md, as well as quickstart guide using Sphinx.

Issue-Id: SCM-2355
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I438718b9fe950c65f13f5aba3bd456a5ed78665b

Nathan Dunne authored Apr 12, 2021 and Diego Sueiro committed May 20, 2021

Created yaml files for kas builds targeting podman and docker based images
for both n1sdp and fvp-base machines. 'base.yml' allows a different machine
target by setting the KAS_MACHINE env variable

Issue-Id: SCM-2355
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I7390fdaa101709949dc3cc31d9fdf884b551cdfa

Nathan Dunne authored Apr 12, 2021 and Diego Sueiro committed May 20, 2021

Introduce targets ewaol-image-docker and ewaol-image-podman, as well as common
include file ewaol-image-core.inc

Issue-Id: SCM-2355
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: Ie53425a84a666dca3d75bb06422ec8c2ee7ad23c

Nathan Dunne authored Apr 12, 2021 and Diego Sueiro committed May 20, 2021

Created Distro 'ewaol', and structure for EWAOL specific distro features:
e.g. devel, tests...

EWAOL distro features are enabled through DISTRO_FEATURES, or a default
set via EWAOL_DISTRO_FEATURES_DEFAULT variable is selected if none is provided.

Issue-Id: SCM-2355
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Change-Id: I379cc643ebe2c3653ecc320640fcb2de657ef1f5

Signed-off-by: Filipe Rinaldi <filipe.rinaldi@arm.com>
Change-Id: Ibe7dbf53152bfceba8189aac00182050d429dc33