# Open Container Images > Builds container images as the first step in a pipeline to allow reproducible CI environments ## Getting Started Include the following in `.gitlab-ci.yml`: ```yaml include: - component: "gitlab.arm.com/ci/component/oci@" ``` ## Purpose The `.oci` job automatically builds container images in the `ci/docker/$NAME` directories, tags them with the current `git` SHA and pushes them to the GitLab container registry. This allows GitLab CI jobs to use a consistent, reproducible environment to run scripts in. ## Usage ### `Dockerfiles` The job expects a container to build in the `ci/docker/$NAME` folder. An easy way to get started is to copy the image definitions from this repository: ```sh cp -r ci/component your/project/root ``` Other images can easily be created by creating a new folder and `Dockerfile`: ```sh mkdir your/project/root/ci/docker/$NAME $EDITOR your/project/root/ci/docker/$NAME/Dockerfile ``` Refer to the [documentation][builder] for writing the `Dockerfile`. #### Examples This project dogfoods CI image to test the component. It is a safe starting point to copy the necessary image folder from the [ci/docker](/ci/docker) folder. ##### Re-use Images in `FROM` The container build passed in the `CI_REGISTRY_IMAGE`/`CI_COMMIT_SHA` and `CI_COMPONENT_REGISTRY_IMAGE`/`CI_COMPONENT_IMAGE_TAG` variables into the build. A `ci/component` image can be used as a basis: ``` ARG CI_COMPONENT_REGISTRY_IMAGE ARG CI_COMPONENT_IMAGE_TAG ARG VERSION FROM ${CI_COMPONENT_REGISTRY_IMAGE}/python:${VERSION}-${CI_COMPONENT_IMAGE_TAG} ``` A locally built OCI can be used as the basis for another image: ``` ARG CI_REGISTRY_IMAGE ARG CI_COMMIT_SHA FROM ${CI_REGISTRY_IMAGE}/custom:${CI_COMMIT_SHA} ``` This requires that the build of the images is ordered in the `.gitlab-ci.yml`: ```yaml oci-base: extends: .oci variables: NAME: base oci-derived: extends: .oci variables: NAME: derived needs: # Guarantees that the `base` image will be built and pushed - oci-base ``` ###### Installing Packages The base images in `ci/component` have removed the hooks that clean up downloaded package manager packages in favour of using cache mounts. When installing packages in derived images, use cache mounts to share downloads on the build instances. ####### `apt` ``` RUN \ --mount=type=cache,target=/var/cache/apt,sharing=locked \ --mount=type=cache,target=/var/lib/apt,sharing=locked \ apt update \ && apt --yes --no-install-recommends install \ ``` ####### `pip` ``` RUN --mount=type=cache,target=/home/ci/.cache/pip,uid=1337,gid=1337 \ python3 -m pip install --user \ ``` ####### `npm` ``` RUN --mount=type=cache,target=/home/ci/.npm,uid=1337,gid=1337 \ && npm install \ ``` ### Configuration #### Container Registry The registry defaults to the GitLab container registry. To customise the container registry set the following variables: - `REGISTRY`: URL to the registry endpoint - `USERNAME`: username for the registry login - `PASSWORD`: authentication for the registry user When pushing to another registry, the following variables will likely need to be customised: - `IMAGE`: the image name to be pushed to the registry, by default this includes the GitLab project name, which is often not necessary for external registries. ```yaml oci: extends: .oci ``` #### `NAME` The component exposes a hidden `.oci` job component. [extends][extends] can be used to use the `.oci` job and define the relevant image names to build. The image names can be provided easily with a [matrix][matrix] and will be built in parallel on the CI: ```yaml oci: extends: .oci parallel: matrix: - NAME: - pre-commit - semantic-release - icon ``` #### `CONTEXT` Defines the location of the Docker image build context. This defaults to `$ROOT/$NAME` but can be overridden to a specific folder. Useful when building a `Dockerfile` at the true root of a project: ```yaml oci: extends: .oci variables: CONTEXT: . ``` #### `TAGS` Overrides the tags that are created within the container build. Defaults to `$CI_COMMIT_SHA $CI_COMMIT_REF_SLUG`. #### `CACHE` The tags to use for the layer cache. Use the tags closest to the current build. Defaults to `$CI_COMMIT_REF_SLUG $CI_MERGE_REQUEST_DIFF_BASE_SHA` #### `PUSH` Parsed as a POSIX `test` expression to determine if the tags should be pushed. This can be used to only push branches on a certain branch: ```yaml oci: extends: .oci variables: # Only push the tags when on the default (usually `main`) branch PUSH: '"$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH"' ``` To avoid pushing entirely, set `PUSH: "false"` #### `VERSION` The `.oci` job also supports injecting a `VERSION` parameter into the build of the container. This can be used to select a base image efficiently: ```yaml oci: extends: .oci parallel: matrix: - NAME: python VERSION: - "3.9" - "3.10" - "3.11" ``` This `VERSION` parameter can be consumed in the `Dockerfile` with [ARG][arg]: ``` ARG VERSION # injected from container build FROM python:$VERSION-slim ``` The `VERSION` will be added to the image tag. The above configuration would build the `python:3.9-`, `python:3.10-` and `python:3.11-` images. #### `DOCKER NODES` The `oci` job supports building on multiple native nodes using the same builder instance. Using multiple native nodes provides better support for more complicated cases that are not handled by QEMU and generally have better performance. Native Docker Nodes can be supplied to the `DOCKER_NODES` variable and must be in the format `plaform:driver:protocol:endpoint`, e.g. setting a `DOCKER_NATIVE_NODES` variable in ci/cd vars to ``` linux/arm64:docker-container:ssh://ubuntu@10.0.0.2 linux/amd64:docker-container:ssh://ubuntu@10.0.0.3 ``` and setting `DOCKER_NODES` equal to it in `.gitlab-ci.yml` For Native docker nodes to work over ssh all of the following `DOCKER_NODES`, `SSH_KEYS`, `SSH_KEYSCAN` variables need to be set and credentials needs to be extended to read the SSH vars in. ```yaml include: - component: "gitlab.arm.com/ci/component/oci@" - component: "gitlab.arm.com/ci/component/credentials@" oci-native-docker-example: extends: - .oci - .credentials variables: DOCKER_NODES: |- ${DOCKER_NATIVE_NODES} ``` ##### `SSH_KEYS` ssh keys file that allows ssh onto the above docker builders, added as part of project CI/CD settings, see [ssh keys](credentials.md#ssh_keys) ##### `SSH_KEYSCAN` ssh keyscan file that allows ssh onto the above docker builders, added as part of project CI/CD settings, see [ssh keyscan](credentials.md#ssh_keyscan) #### `PLATFORMS` The `.oci` job supports building images for different platforms. Specify `PLATFORMS` variable as a space separated list of platforms to build for a container for: ```yaml oci: extends: .oci parallel: matrix: - NAME: pre-commit PLATFORMS: "linux/amd64" ``` | :boom: NOTICE :boom: | |:------------------------------------------------------------------------------------------------------------------------------- | | `linux/arm64` is not supported. We are moving ci/component to Gitlab CI components during which we will investigate enabling it. | #### `ROOT` Images are built from `ci/docker/$NAME` by default. To customise the root of the images, override `ROOT`: ```yaml oci: extends: .oci variables: ROOT: some/other/path parallel: matrix: - NAME: pre-commit PLATFORM: - linux/amd64 - linux/arm64 ``` #### `SECRETS` Provides a method to pass [secret][secret] information into the Docker build. This variable is space separated list of secrets to expose into the build. The secrets will not be available in the final image. Take care to not reveal the secrets in the build or store them in the image. The secrets can be used by mounting them into a `RUN` command: ``` # This assumes creation of a user at UID/GID 1000 RUN --mount=type=secret,id=CI_JOB_TOKEN,required=true,uid=1337,gid=1337 \ CI_JOB_TOKEN="$(cat /run/secrets/CI_JOB_TOKEN)" \ ``` ##### Environment Variables Define the environment variables to expose by name: ```yaml oci: extends: .oci variables: SECRETS: "CI_JOB_TOKEN NPM_TOKEN" # Can mount `/run/secrets/CI_JOB_TOKEN` and `/run/secrets/NPM_TOKEN` ``` By default the `CI_JOB_TOKEN` is exposed. Environment variables can be renamed from host to build: ```yaml oci: extends: .oci variables: SECRETS: "CI_JOB_TOKEN,env=NPM_TOKEN" # Can mount `/run/secrets/CI_JOB_TOKEN` # Will be the value of the host `NPM_TOKEN` ``` ##### Files It is possible to expose secret files: ```yaml oci: extends: .oci variables: SECRETS: "mysecret,src=mysecret.txt" # Where `mysecret.txt` is relative to the build root # Can mount `/run/secrets/mysecret` ``` #### `TRIVY` Set this to `true` to run security analysis on the built image. Each image built is ran through `trivy image ` A report will be added to a merge request with the found vulnerabilities. #### `TRIVY_SEVERITY` [Filters][trivy-filter] the severity of the security analysis to certain levels. ### Usage The images can be used in CI jobs from the GitLab container registry: ```yaml pre-commit: image: $CI_COMPONENT_REGISTRY_IMAGE/pre-commit:$CI_COMPONENT_IMAGE_TAG script: - pre-commit --version python: parallel: matrix: - VERSION: - "3.9" - "3.10" - "3.11" image: $CI_REGISTRY_IMAGE/python:${VERSION}-$CI_COMMIT_SHA script: - python -V ``` [builder]: https://docs.docker.com/engine/reference/builder [arg]: https://docs.docker.com/engine/reference/builder/#arg [matrix]: https://docs.gitlab.com/ee/ci/yaml/#parallelmatrix [extends]: https://docs.gitlab.com/ee/ci/yaml/#extends [secret]: https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information [image]: https://docs.gitlab.com/ee/ci/yaml/#image [trivy-filter]: https://aquasecurity.github.io/trivy/latest/docs/misconfiguration/options/filter/#by-severity