From 6fdb6b055c8bf02b2ba74598377101fa999dfe01 Mon Sep 17 00:00:00 2001 From: Matt Clarkson Date: Fri, 11 Oct 2024 13:30:41 +0100 Subject: [PATCH] fix(debian/launcher): patch the ELF interpreter We have an issue with QEMU relative paths. QEMU uses relative paths to the binary directory. The process is as so: - QEMU uses `/proc/self/exe` in `qemu_init_exec_dir` to get the current executable path - That usually returns the location of `qemu-system-{arm,aarch64,i386,x86_64}` - Relocatable paths, such as `QEMU_MODULE_DIR` are then found relative to the directory However, because we can invoking the binary with the ELF interpreter `/proc/self/exe` resolves the ELF interpreter path rather than the QEMU binary path. This throws off all of the relative paths that QEMU uses to find the data files. Concretely, this manifests as: ```console $ bazelisk run -- debian/amd64/qemu-system-arm:qemu-system-aarch64 -L help /home/matt-clarkson/.cache/bazel/_bazel_matt-clarkson/8a909ce9ddae247581c15927c03304e3/execroot/_main/bazel-out/k8-fastbuild/bin/debian/amd64/qemu-system-arm/unpack/lib/x86_64-linux-gnu/../share/qemu /home/matt-clarkson/.cache/bazel/_bazel_matt-clarkson/8a909ce9ddae247581c15927c03304e3/execroot/_main/bazel-out/k8-fastbuild/bin/debian/amd64/qemu-system-arm/unpack/lib/x86_64-linux-gnu/../share/seabios /home/matt-clarkson/.cache/bazel/_bazel_matt-clarkson/8a909ce9ddae247581c15927c03304e3/execroot/_main/bazel-out/k8-fastbuild/bin/debian/amd64/qemu-system-arm/unpack/lib/x86_64-linux-gnu/../lib/ipxe/qemu ``` Rather than: ```console $ bazelisk run -- debian/amd64/qemu-system-arm:qemu-system-aarch64 -L help /home/matt-clarkson/.cache/bazel/_bazel_matt-clarkson/8a909ce9ddae247581c15927c03304e3/execroot/_main/bazel-out/k8-fastbuild/bin/debian/amd64/qemu-system-arm/patched/usr/bin/../share/qemu /home/matt-clarkson/.cache/bazel/_bazel_matt-clarkson/8a909ce9ddae247581c15927c03304e3/execroot/_main/bazel-out/k8-fastbuild/bin/debian/amd64/qemu-system-arm/patched/usr/bin/../share/seabios /home/matt-clarkson/.cache/bazel/_bazel_matt-clarkson/8a909ce9ddae247581c15927c03304e3/execroot/_main/bazel-out/k8-fastbuild/bin/debian/amd64/qemu-system-arm/patched/usr/bin/../lib/ipxe/qemu ``` To resolve the paths correctly, we need to override the ELF interpreter path in the `PT_INTERP` ELF header to point at the downloaded Debian ELF interpreter. `patchelf` tool allows us to do that with `--set-interpreter`. The issue is that we cannot concretely set the path because the Bazel sandbox will always mount the Debian packages at a different absolute path. Linux does not support `${ORIGIN}` in `PT_INTERP`. Solaris does. This patch employs the following strategy: - Unpack Debian archives - Whilst unpacking the archives, yield any executable files - Use `patchelf` to set a fixed ELF interpreter path under `/tmp` - In the Debian launcher symlink the ELF interpreter under `/tmp` to the real Debian ELF interpreter This works but there is a race condition: Bazel does not necessarily mount a fresh `/tmp` into each sandbox so two actions running at the same time will thrash on the symlink. The patch atomically moves the symlink into place and each action running requiring the same symlink will always point at a concrete ELF interpreter. It is possible that the following could happen: - Action 1 starts and sets the ELF interpreter symlink to `execroot/1/ld-linux.so.2` - Action 2 starts and sets the ELF interpreter symlink to `execroot/2/ld-linux.so.2` - Commands from action 1 will still work because the second symlink points to the compatible ELF interpreter - Action 2 ends, unmounting the real ELF interpreter - Action 1 attempts to start a binary from within the already running binary - The ELF interpreter fails to resolve This is unlikely to happen because the launcher symlinks then launches the binary. In our usage, QEMU _likely_ won't start other host executables once it has launched. I will follow-up with a better strategy for the interpreter symlink but feel the risk is low enough to land this now. This patch also changes the Debian launcher to use Python rather than Bash to launch the executable. This allows us to use the robust Python runfiles library and not have to use the janky Bash runfiles library that requires unhermetic tools to find it. To do this it employs a trick that we can use in other modules to avoid using Bash in the future: - Write out baked executable arguments to a file - Use `ctx.runfiles#root_symlinks` to link the file to the root of the runfiles - Load the baked arguments using the runfiles library using the consistent symlink name - Add the file onto the front of the passed in arguments with an `@` symbol - Make sure `ArgumentParser#fromfile_prefix_chars` is set to `@` --- MODULE.bazel | 1 + MODULE.bazel.lock | 2 + debian/amd64/libc6/BUILD.bazel | 4 +- debian/amd64/qemu-system-arm/BUILD.bazel | 19 +-- debian/amd64/qemu-system-arm/srcs.bzl | 4 +- debian/amd64/qemu-system-x86/BUILD.bazel | 18 +-- debian/amd64/qemu-system-x86/srcs.bzl | 4 +- debian/amd64/tar/BUILD.bazel | 4 +- debian/arm64/libc6/BUILD.bazel | 4 +- debian/arm64/qemu-system-arm/BUILD.bazel | 18 +-- debian/arm64/qemu-system-arm/srcs.bzl | 4 +- debian/arm64/qemu-system-x86/BUILD.bazel | 18 +-- debian/arm64/qemu-system-x86/srcs.bzl | 4 +- debian/arm64/tar/BUILD.bazel | 4 +- debian/launcher/BUILD.bazel | 11 +- debian/launcher/launch.py | 135 +++++++++++++++++++++ debian/launcher/posix.tmpl.sh | 73 ------------ debian/launcher/rule.bzl | 64 +++++----- debian/patchelf/BUILD.bazel | 8 ++ debian/patchelf/InterpreterInfo.bzl | 26 +++++ debian/patchelf/defs.bzl | 7 ++ debian/patchelf/patch.py | 143 +++++++++++++++++++++++ debian/patchelf/rule.bzl | 53 +++++++++ e2e/MODULE.bazel.lock | 2 + 24 files changed, 451 insertions(+), 179 deletions(-) create mode 100644 debian/launcher/launch.py delete mode 100644 debian/launcher/posix.tmpl.sh create mode 100644 debian/patchelf/BUILD.bazel create mode 100644 debian/patchelf/InterpreterInfo.bzl create mode 100644 debian/patchelf/defs.bzl create mode 100644 debian/patchelf/patch.py create mode 100644 debian/patchelf/rule.bzl diff --git a/MODULE.bazel b/MODULE.bazel index 11389a88..7a8611a7 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -14,6 +14,7 @@ bazel_dep(name = "download_utils", version = "1.0.0-beta.2") bazel_dep(name = "rules_tar", version = "1.0.0-beta.3") bazel_dep(name = "rules_zstd", version = "1.0.0-beta.3") bazel_dep(name = "platforms", version = "0.0.10") +bazel_dep(name = "patchelf", version = "0.18.0") bazel_dep(name = "hermetic_cc_toolchain", version = "3.1.0", dev_dependency = True) diff --git a/MODULE.bazel.lock b/MODULE.bazel.lock index b4098ff3..c9b8a210 100644 --- a/MODULE.bazel.lock +++ b/MODULE.bazel.lock @@ -43,6 +43,8 @@ "https://bcr.bazel.build/modules/googletest/1.11.0/source.json": "c73d9ef4268c91bd0c1cd88f1f9dfa08e814b1dbe89b5f594a9f08ba0244d206", "https://bcr.bazel.build/modules/hermetic_cc_toolchain/3.1.0/MODULE.bazel": "ea4b3a25a9417a7db57a8a2f9ebdee91d679823c6274b482b817ed128d81c594", "https://bcr.bazel.build/modules/hermetic_cc_toolchain/3.1.0/source.json": "9d1df0459caefdf41052d360469922a73e219f67c8ce4da0628cc604469822b9", + "https://bcr.bazel.build/modules/patchelf/0.18.0/MODULE.bazel": "15a6beff7e828d585c5bd0f9f93589df117b5594e9d19e43096c77de58b9ae5f", + "https://bcr.bazel.build/modules/patchelf/0.18.0/source.json": "57caf6bcaa5ba515c6fb1c2eacee00735afbeb1ffacb34a57553fb139c8e4333", "https://bcr.bazel.build/modules/platforms/0.0.10/MODULE.bazel": "8cb8efaf200bdeb2150d93e162c40f388529a25852b332cec879373771e48ed5", "https://bcr.bazel.build/modules/platforms/0.0.10/source.json": "f22828ff4cf021a6b577f1bf6341cb9dcd7965092a439f64fc1bb3b7a5ae4bd5", "https://bcr.bazel.build/modules/platforms/0.0.4/MODULE.bazel": "9b328e31ee156f53f3c416a64f8491f7eb731742655a47c9eec4703a71644aee", diff --git a/debian/amd64/libc6/BUILD.bazel b/debian/amd64/libc6/BUILD.bazel index c627932c..74f1d725 100644 --- a/debian/amd64/libc6/BUILD.bazel +++ b/debian/amd64/libc6/BUILD.bazel @@ -2,9 +2,9 @@ load("@rules_tar//tar/filter:defs.bzl", "tar_filter") # FIXME: this should actually change the link to be relative instead tar_filter( - name = "data.tar.zst", + name = "data.tar.xz", src = "@amd64-libc6//:data.tar.xz", - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", + compress = "@rules_tar//tar/compress:xz", patterns = [ "!./lib64/ld-linux-x86-64.so.2", "**/*", diff --git a/debian/amd64/qemu-system-arm/BUILD.bazel b/debian/amd64/qemu-system-arm/BUILD.bazel index 59e9a88f..0a7d8329 100644 --- a/debian/amd64/qemu-system-arm/BUILD.bazel +++ b/debian/amd64/qemu-system-arm/BUILD.bazel @@ -1,29 +1,20 @@ load("//debian/launcher:defs.bzl", "debian_launcher") -load("@rules_tar//tar/concatenate:defs.bzl", "tar_concatenate") -load("@rules_tar//tar/unpack:defs.bzl", "tar_unpack") +load("//debian/patchelf:defs.bzl", "debian_patchelf") load(":srcs.bzl", "SRCS") -tar_concatenate( - name = "data.tar.zst", +debian_patchelf( + name = "patched", srcs = SRCS, - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", - duplicate = "skip", -) - -tar_unpack( - name = "unpack", - src = ":data.tar.zst", - visibility = ["//debian/qemu-system-aarch64/amd64:__pkg__"], ) debian_launcher( name = "qemu-system-arm", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) debian_launcher( name = "qemu-system-aarch64", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) diff --git a/debian/amd64/qemu-system-arm/srcs.bzl b/debian/amd64/qemu-system-arm/srcs.bzl index cf10ee25..c5abc160 100644 --- a/debian/amd64/qemu-system-arm/srcs.bzl +++ b/debian/amd64/qemu-system-arm/srcs.bzl @@ -1,8 +1,8 @@ visibility("//debian/amd64/qemu-system-arm/...") SRCS = ( - "//debian/amd64/libc6:data.tar.zst", - "//debian/amd64/tar:data.tar.zst", + "//debian/amd64/libc6:data.tar.xz", + "//debian/amd64/tar:data.tar.xz", "@all-adduser//:data.tar.xz", "@all-debconf//:data.tar.xz", "@all-iso-codes//:data.tar.xz", diff --git a/debian/amd64/qemu-system-x86/BUILD.bazel b/debian/amd64/qemu-system-x86/BUILD.bazel index 03e88f94..928b882e 100644 --- a/debian/amd64/qemu-system-x86/BUILD.bazel +++ b/debian/amd64/qemu-system-x86/BUILD.bazel @@ -1,28 +1,20 @@ load("//debian/launcher:defs.bzl", "debian_launcher") -load("@rules_tar//tar/concatenate:defs.bzl", "tar_concatenate") -load("@rules_tar//tar/unpack:defs.bzl", "tar_unpack") +load("//debian/patchelf:defs.bzl", "debian_patchelf") load(":srcs.bzl", "SRCS") -tar_concatenate( - name = "data.tar.zst", +debian_patchelf( + name = "patched", srcs = SRCS, - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", - duplicate = "skip", -) - -tar_unpack( - name = "unpack", - src = ":data.tar.zst", ) debian_launcher( name = "qemu-system-i386", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) debian_launcher( name = "qemu-system-x86_64", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) diff --git a/debian/amd64/qemu-system-x86/srcs.bzl b/debian/amd64/qemu-system-x86/srcs.bzl index 33db02f7..977b25a0 100644 --- a/debian/amd64/qemu-system-x86/srcs.bzl +++ b/debian/amd64/qemu-system-x86/srcs.bzl @@ -1,8 +1,8 @@ visibility("//debian/amd64/qemu-system-x86/...") SRCS = ( - "//debian/amd64/libc6:data.tar.zst", - "//debian/amd64/tar:data.tar.zst", + "//debian/amd64/libc6:data.tar.xz", + "//debian/amd64/tar:data.tar.xz", "@all-adduser//:data.tar.xz", "@all-debconf//:data.tar.xz", "@all-ipxe-qemu//:data.tar.xz", diff --git a/debian/amd64/tar/BUILD.bazel b/debian/amd64/tar/BUILD.bazel index b1927476..21385572 100644 --- a/debian/amd64/tar/BUILD.bazel +++ b/debian/amd64/tar/BUILD.bazel @@ -2,9 +2,9 @@ load("@rules_tar//tar/filter:defs.bzl", "tar_filter") # Remove dangling symlink tar_filter( - name = "data.tar.zst", + name = "data.tar.xz", src = "@amd64-tar//:data.tar.xz", - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", + compress = "@rules_tar//tar/compress:xz", patterns = [ "!./etc/rmt", "**/*", diff --git a/debian/arm64/libc6/BUILD.bazel b/debian/arm64/libc6/BUILD.bazel index f5bd5327..367f755a 100644 --- a/debian/arm64/libc6/BUILD.bazel +++ b/debian/arm64/libc6/BUILD.bazel @@ -2,9 +2,9 @@ load("@rules_tar//tar/filter:defs.bzl", "tar_filter") # FIXME: this should actually change the link to be relative instead tar_filter( - name = "data.tar.zst", + name = "data.tar.xz", src = "@arm64-libc6//:data.tar.xz", - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", + compress = "@rules_tar//tar/compress:xz", patterns = [ "!./lib64/ld-linux-x86-64.so.2", "**/*", diff --git a/debian/arm64/qemu-system-arm/BUILD.bazel b/debian/arm64/qemu-system-arm/BUILD.bazel index 7e03bba4..0a7d8329 100644 --- a/debian/arm64/qemu-system-arm/BUILD.bazel +++ b/debian/arm64/qemu-system-arm/BUILD.bazel @@ -1,28 +1,20 @@ load("//debian/launcher:defs.bzl", "debian_launcher") -load("@rules_tar//tar/concatenate:defs.bzl", "tar_concatenate") -load("@rules_tar//tar/unpack:defs.bzl", "tar_unpack") +load("//debian/patchelf:defs.bzl", "debian_patchelf") load(":srcs.bzl", "SRCS") -tar_concatenate( - name = "data.tar.zst", +debian_patchelf( + name = "patched", srcs = SRCS, - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", - duplicate = "skip", -) - -tar_unpack( - name = "unpack", - src = ":data.tar.zst", ) debian_launcher( name = "qemu-system-arm", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) debian_launcher( name = "qemu-system-aarch64", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) diff --git a/debian/arm64/qemu-system-arm/srcs.bzl b/debian/arm64/qemu-system-arm/srcs.bzl index 55e1acea..02eaabdb 100644 --- a/debian/arm64/qemu-system-arm/srcs.bzl +++ b/debian/arm64/qemu-system-arm/srcs.bzl @@ -1,8 +1,8 @@ visibility("//debian/arm64/qemu-system-arm/...") SRCS = ( - "//debian/arm64/libc6:data.tar.zst", - "//debian/arm64/tar:data.tar.zst", + "//debian/arm64/libc6:data.tar.xz", + "//debian/arm64/tar:data.tar.xz", "@all-adduser//:data.tar.xz", "@all-debconf//:data.tar.xz", "@all-iso-codes//:data.tar.xz", diff --git a/debian/arm64/qemu-system-x86/BUILD.bazel b/debian/arm64/qemu-system-x86/BUILD.bazel index 03e88f94..928b882e 100644 --- a/debian/arm64/qemu-system-x86/BUILD.bazel +++ b/debian/arm64/qemu-system-x86/BUILD.bazel @@ -1,28 +1,20 @@ load("//debian/launcher:defs.bzl", "debian_launcher") -load("@rules_tar//tar/concatenate:defs.bzl", "tar_concatenate") -load("@rules_tar//tar/unpack:defs.bzl", "tar_unpack") +load("//debian/patchelf:defs.bzl", "debian_patchelf") load(":srcs.bzl", "SRCS") -tar_concatenate( - name = "data.tar.zst", +debian_patchelf( + name = "patched", srcs = SRCS, - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", - duplicate = "skip", -) - -tar_unpack( - name = "unpack", - src = ":data.tar.zst", ) debian_launcher( name = "qemu-system-i386", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) debian_launcher( name = "qemu-system-x86_64", - src = ":unpack", + src = ":patched", visibility = ["//:__subpackages__"], ) diff --git a/debian/arm64/qemu-system-x86/srcs.bzl b/debian/arm64/qemu-system-x86/srcs.bzl index 8a85b381..40bcc82b 100644 --- a/debian/arm64/qemu-system-x86/srcs.bzl +++ b/debian/arm64/qemu-system-x86/srcs.bzl @@ -1,8 +1,8 @@ visibility("//debian/arm64/qemu-system-x86/...") SRCS = ( - "//debian/arm64/libc6:data.tar.zst", - "//debian/arm64/tar:data.tar.zst", + "//debian/arm64/libc6:data.tar.xz", + "//debian/arm64/tar:data.tar.xz", "@all-adduser//:data.tar.xz", "@all-debconf//:data.tar.xz", "@all-ipxe-qemu//:data.tar.xz", diff --git a/debian/arm64/tar/BUILD.bazel b/debian/arm64/tar/BUILD.bazel index 66ffe37f..480d5f08 100644 --- a/debian/arm64/tar/BUILD.bazel +++ b/debian/arm64/tar/BUILD.bazel @@ -2,9 +2,9 @@ load("@rules_tar//tar/filter:defs.bzl", "tar_filter") # Remove dangling symlink tar_filter( - name = "data.tar.zst", + name = "data.tar.xz", src = "@arm64-tar//:data.tar.xz", - compress = "@rules_zstd//zstd/toolchain/zstd:resolved", + compress = "@rules_tar//tar/compress:xz", patterns = [ "!./etc/rmt", "**/*", diff --git a/debian/launcher/BUILD.bazel b/debian/launcher/BUILD.bazel index dfd1c9fb..ce9c4a9d 100644 --- a/debian/launcher/BUILD.bazel +++ b/debian/launcher/BUILD.bazel @@ -1,6 +1,7 @@ -alias( - name = "template", - actual = select({ - "//conditions:default": ":posix.tmpl.sh", - }), +load("@rules_python//python:defs.bzl", "py_binary") + +py_binary( + name = "launch", + srcs = ["launch.py"], + deps = ["@rules_python//python/runfiles"], ) diff --git a/debian/launcher/launch.py b/debian/launcher/launch.py new file mode 100644 index 00000000..9011b781 --- /dev/null +++ b/debian/launcher/launch.py @@ -0,0 +1,135 @@ +from __future__ import annotations + +from argparse import REMAINDER, ArgumentParser +from logging import getLogger +from os import environ, getpid +from pathlib import Path, PurePath +from shutil import move +from subprocess import CalledProcessError, run +from sys import argv, exit +from typing import Final, Mapping + +from python.runfiles import Runfiles + +LOG: Final = getLogger(__file__) + + +class RunfileNotFoundError(FileNotFoundError): + pass + + +def runfile(path: Path) -> Path: + runfiles = Runfiles.Create() + resolved = Path(runfiles.Rlocation(path)) + if not resolved.exists(): + raise RunfileNotFoundError(path) + return resolved + + +def quoted(value: str) -> str: + return value.removeprefix("'").removesuffix("'") + + +def arguments(prsr: ArgumentParser) -> None: + prsr.add_argument( + "--ld-library-path", + required=True, + dest="llp", + metavar="PATHS", + help="`LD_LIBRARY_PATH` for the executable.", + type=quoted, + ) + prsr.add_argument( + "--interpreter", + required=True, + metavar="PATH", + help="The real ELF interpreter for the binary, relative to the `--root`.", + type=PurePath, + ) + prsr.add_argument( + "--symlink", + required=True, + metavar="PATH", + help="A symlink to create for the patch ELF interpreter path to the real ELF interpreter.", + type=PurePath, + ) + prsr.add_argument( + "--root", + required=True, + metavar="PATH", + help="The root of the unpacked Debian archives.", + type=runfile, + ) + prsr.add_argument( + "executable", + metavar="EXECUTABLE", + help="The ELF interpreter patched executabl to launch.", + type=Path, + ) + prsr.add_argument( + "arguments", + metavar="ARG", + nargs=REMAINDER, + help="Arguments to pass to the executable", + ) + + +def execute( + executable: Path, + *arguments: str, + interpreter: Path, + symlink: PurePath, + env: Mapping[str, str] = {}, +) -> int: + # FIXME: this is racey because Bazel does not provide a unique `/tmp` to each sandbox + symlink = Path(symlink) + tmp = symlink.with_suffix(f".{getpid()}") + tmp.parent.mkdir(parents=True, exist_ok=True) + tmp.symlink_to(interpreter) + move(tmp, symlink) + + assert executable.exists() + cmd = (f"{executable}", *arguments) + process = run(cmd, env=env) + return process.returncode + + +def main(exe: Path, *args: str) -> int: + prsr = ArgumentParser( + prog=str(exe), + description="Launches a Debian ELF interpreter patched executable.", + fromfile_prefix_chars="@", + ) + + arguments(prsr) + + try: + baked = runfile("launch.args") + except RunfileNotFoundError: + pass + else: + args = (f"@{baked}", *args) + + parsed = prsr.parse_args(args) + + try: + return execute( + parsed.root / parsed.executable, + *parsed.arguments, + interpreter=parsed.root / parsed.interpreter, + symlink=parsed.symlink, + env=environ + | {"LD_LIBRARY_PATH": parsed.llp.removeprefix("'").removesuffix("'")}, + ) + except CalledProcessError as e: + return e.returncode + except KeyboardInterrupt: + return 130 + + +def entry(): + exit(main(Path(argv[0]), *argv[1:])) + + +if __name__ == "__main__": + entry() diff --git a/debian/launcher/posix.tmpl.sh b/debian/launcher/posix.tmpl.sh deleted file mode 100644 index 815a1dfb..00000000 --- a/debian/launcher/posix.tmpl.sh +++ /dev/null @@ -1,73 +0,0 @@ -#! /usr/bin/env sh - -# Strict shell -set -o errexit -set -o nounset - -# Substitutions -ROOT="{{root}}" -BINARY="{{binary}}" -LLP="{{llp}}" -INTERPRETER="{{interpreter}}" -readonly ROOT BINARY LLP INTERPRETER - -# Check the root is available -if ! test -d "${ROOT}"; then - printf >&2 "Root is not a directory: %s\n" "${ROOT}" - exit 1 -fi - -# Check the binary is executable -if ! test -e "${BINARY}"; then - printf >&2 "Binary does not exist: %s\n" "${BINARY}" - exit 1 -elif ! test -f "${BINARY}"; then - printf >&2 "Binary not a file: %s\n" "${BINARY}" - exit 1 -elif ! test -x "${BINARY}"; then - printf >&2 "Binary is not executable: %s\n" "${BINARY}" - exit 1 -fi - -# Check the interpreter is executable -if ! test -e "${INTERPRETER}"; then - printf >&2 "ELF interpreter does not exist: %s\n" "${INTERPRETER}" - exit 1 -elif ! test -f "${INTERPRETER}"; then - printf >&2 "ELF interpreter is not a file: %s\n" "${INTERPRETER}" - exit 1 -elif ! test -x "${INTERPRETER}"; then - printf >&2 "ELF interpreter is not executable: %s\n" "${INTERPRETER}" - exit 1 -fi - -# FIXME: figure out a robust solution for launching the binary -# -# Launching with the ELF interpreter side-steps the fact that each -# Debian binary will have `/lib/ld-linux-${ARCH}.so.{1,2}` as the -# `PT_INTERP` ELF entry. -# -# Any binary that is launched will end up using the host ELF interpreter -# and crash with a `SIGABORT` (or worse). -# -# The `PT_INTERP` should be changed with `patchelf` to point at the correct -# ELF interpreter. However, that cannot be done ahead of time because the -# `${ROOT}` will be mounted at different sandbox execution roots. Linux -# does not support `${ORIGIN}` in the `PT_INTERP` as Solaris does. -# -# As the `${ROOT}` is mounted read-only, we cannot patch on the fly. -# -# Another option is to change into the `${ROOT}` with a fake-chroot but -# that would affect anything else running within the action sandbox. -# -# The _ideal_ solution would be a `fake-chroot` that runs _just_ the -# binary underneath: -# -# fake-chroot --root="${ROOT}" "${RELATIVE_BINARY_PATH}" "${@}" -# -# That would not affect anything else running in the sandbox and would -# solve the interpreter issue *and* solve any configuration file lookup -# issues where the absolute root path has been baked into the executable. - -# Run the binary -LD_LIBRARY_PATH="${LLP}" "${INTERPRETER}" "${BINARY}" "${@}" diff --git a/debian/launcher/rule.bzl b/debian/launcher/rule.bzl index fdc2a436..ad061241 100644 --- a/debian/launcher/rule.bzl +++ b/debian/launcher/rule.bzl @@ -1,3 +1,4 @@ +load("//debian/patchelf:defs.bzl", "DebianPatchelfInterpreterInfo") load("//debian/launcher/library/path:defs.bzl", "DebianLauncherLibraryPathInfo") load("//debian/launcher/elf/interpreter:defs.bzl", "DebianLauncherELFInterpreterInfo") @@ -8,6 +9,7 @@ DOC = "" ATTRS = { "src": attr.label( doc = "The unpacked Debian archives.", + providers = [DebianPatchelfInterpreterInfo], allow_single_file = True, mandatory = True, ), @@ -31,58 +33,56 @@ ATTRS = { providers = [DebianLauncherELFInterpreterInfo], default = "//debian/launcher/elf/interpreter", ), - "_template": attr.label( + "_launch": attr.label( doc = "The launcher script.", - allow_single_file = [".sh"], - default = ":template", + default = ":launch", + executable = True, + cfg = "exec", ), } +def _runfile(label, file): + path = file.short_path + if path.startswith("../"): + return path.removeprefix("../") + return "{}/{}".format(label.workspace_name or "_main", path) + def implementation(ctx): if not ctx.file.src.is_directory: fail("`src` must be a directory") - root = ctx.file.src.short_path relative = ctx.attr.executable or "usr/bin/{}".format(ctx.label.name) relative = relative.removeprefix("/") - def _escape(p): - split = p.split("\\$") - escaped = [s.replace("$", "\\$") for s in split] - joined = "\\$".join(escaped) - if not joined.startswith(("/", "\\$")): - return "{}/{}".format(root, joined) - return joined - - paths = ctx.attr._llp[DebianLauncherLibraryPathInfo].paths - paths = [_escape(p) for p in paths] - llp = ":".join(paths) - + llp = ":".join(ctx.attr._llp[DebianLauncherLibraryPathInfo].paths) interpreter = ctx.attr._interpreter[DebianLauncherELFInterpreterInfo].path + symlink = ctx.attr.src[DebianPatchelfInterpreterInfo].path + + arguments = ctx.actions.declare_file("{}.args".format(ctx.label.name)) + args = ctx.actions.args() + args.add("--ld-library-path", llp) + args.add("--interpreter", interpreter) + args.add("--symlink", symlink) + args.add("--root", _runfile(ctx.attr.src.label, ctx.file.src)) + args.add(relative) + ctx.actions.write(output = arguments, content = args) - rendered = ctx.actions.declare_file("{}.{}".format(ctx.label.name, ctx.file._template.extension)) - substitutions = ctx.actions.template_dict() - substitutions.add("{{root}}", root) - substitutions.add("{{binary}}", "{}/{}".format(root, relative)) - substitutions.add("{{interpreter}}", "{}/{}".format(root, interpreter)) - substitutions.add("{{llp}}", llp) - ctx.actions.expand_template( - output = rendered, - template = ctx.file._template, - computed_substitutions = substitutions, + executable = ctx.actions.declare_file(ctx.label.name) + ctx.actions.symlink( + output = executable, + target_file = ctx.executable._launch, is_executable = True, ) + files = depset([executable, arguments]) data = depset(transitive = [d.files for d in ctx.attr.data]) - runfiles = ctx.runfiles([ctx.file.src], transitive_files = data) + symlinks = {"launch.args": arguments} + runfiles = ctx.runfiles([ctx.file.src], transitive_files = data, root_symlinks = symlinks) + runfiles = runfiles.merge(ctx.attr._launch.default_runfiles) runfiles = runfiles.merge(ctx.attr.src.default_runfiles) runfiles = runfiles.merge_all([d.default_runfiles for d in ctx.attr.data]) - env = {k: ctx.expand_location(v, targets = ctx.attr.data) for k, v in ctx.attr.env.items()} - return [ - DefaultInfo(executable = rendered, runfiles = runfiles), - RunEnvironmentInfo(env), - ] + return DefaultInfo(executable = executable, files = files, runfiles = runfiles) debian_launcher = rule( doc = DOC, diff --git a/debian/patchelf/BUILD.bazel b/debian/patchelf/BUILD.bazel new file mode 100644 index 00000000..bbf159be --- /dev/null +++ b/debian/patchelf/BUILD.bazel @@ -0,0 +1,8 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +py_binary( + name = "patch", + srcs = ["patch.py"], + data = ["@patchelf"], + deps = ["@rules_python//python/runfiles"], +) diff --git a/debian/patchelf/InterpreterInfo.bzl b/debian/patchelf/InterpreterInfo.bzl new file mode 100644 index 00000000..2c86e517 --- /dev/null +++ b/debian/patchelf/InterpreterInfo.bzl @@ -0,0 +1,26 @@ +load("@bazel_skylib//lib:types.bzl", "types") + +visibility("//...") + +def init(path): + """ + Initializes a `DebianPatchelfInterpreterInfo` provider. + + Args: + path: The patched ELF interpreter path + """ + if not types.is_string(path): + fail("`DebianPatchelfInterpreterInfo.path` must be a string: {}".format(path)) + + return {"path": path} + +DebianPatchelfInterpreterInfo, debian_patchelf_interpreter_info = provider( + "Patched interpreter path.", + fields = ["path"], + init = init, +) + +InterpreterInfo = DebianPatchelfInterpreterInfo +interpreter_info = debian_patchelf_interpreter_info +Info = DebianPatchelfInterpreterInfo +info = debian_patchelf_interpreter_info diff --git a/debian/patchelf/defs.bzl b/debian/patchelf/defs.bzl new file mode 100644 index 00000000..376b3741 --- /dev/null +++ b/debian/patchelf/defs.bzl @@ -0,0 +1,7 @@ +load(":rule.bzl", _patchelf = "patchelf") +load(":InterpreterInfo.bzl", _InterpreterInfo = "InterpreterInfo") + +visibility("//debian/...") + +debian_patchelf = _patchelf +DebianPatchelfInterpreterInfo = _InterpreterInfo diff --git a/debian/patchelf/patch.py b/debian/patchelf/patch.py new file mode 100644 index 00000000..23e06718 --- /dev/null +++ b/debian/patchelf/patch.py @@ -0,0 +1,143 @@ +from __future__ import annotations + +from argparse import ArgumentParser +from logging import getLogger +from pathlib import Path, PurePosixPath +from stat import S_IXUSR +from subprocess import DEVNULL, PIPE, CalledProcessError, run +from sys import argv, exit, stderr +from tarfile import TarFile +from typing import Final, Iterable, Iterator + +from python.runfiles import Runfiles + +LOG: Final = getLogger(__file__) + + +def arguments(prsr: ArgumentParser) -> None: + runfiles = Runfiles.Create() + patchelf = Path(runfiles.Rlocation("patchelf/patchelf")) + assert patchelf.exists() + prsr.add_argument( + "--patchelf", + metavar="PATH", + help="The `patchelf` executable.", + type=Path, + default=patchelf, + ) + prsr.add_argument( + "--output", + metavar="PATH", + help="The output directory to unpack Debian packages into and patch executables.", + type=Path, + ) + prsr.add_argument( + "--set-interpreter", + dest="interpreter", + metavar="PATH", + help="The ELF interpreter path to set into the `PT_INTERP` ELF header.", + type=PurePosixPath, + ) + prsr.add_argument( + "archives", + metavar="ARCHIVE", + nargs="+", + type=Path, + help="Debian package archives to unpack.", + ) + + +def unpack(archive: Path, output: Path) -> Iterable[Path]: + """ + Unpacks an archive an yields executable paths extracted + + Args: + archive: archive to extract + output: output directory for archive files + + Returns: + Executable file interator + """ + with TarFile.open(archive, "r:*") as tar: + for member in tar: + tar.extract(member, output) + if member.isreg() and S_IXUSR & member.mode: + yield output / member.name + + +def current(patchelf: Path, executable: Path) -> str | None: + cmd = (patchelf, "--print-interpreter", f"{executable}") + process = run(cmd, stdout=PIPE, stderr=PIPE, encoding="utf-8") + if 0 == process.returncode: + return process.stdout.rstrip() + elif "cannot find section '.interp'" in process.stderr: + return None + elif "not an ELF executable" in process.stderr: + return None + else: + raise CalledProcessError( + returncode=process.returncode, cmd=cmd, stderr=process.stderr + ) + + +class PatchError(RuntimeError): + pass + + +def patch(patchelf: Path, executable: Path, interpreter: PurePosixPath) -> None: + if current(patchelf, executable) is None: + return + + cmd = (patchelf, "--set-interpreter", f"{interpreter}", f"{executable}") + process = run(cmd, stdout=DEVNULL, stderr=PIPE, encoding="utf-8") + if 0 != process.returncode: + raise PatchError( + f"Failed to patch `{executable}` interpreter to `{interpreter}`: {process.stderr}" + ) + + +def patched( + archives: Iterable[Path], + output: Path, + /, + patchelf: Path, + interpreter: PurePosixPath, +) -> Iterator[Path]: + for archive in archives: + for executable in unpack(archive, output): + patch(patchelf, executable, interpreter) + yield executable + + +def main(exe: Path, *args: str) -> int: + prsr = ArgumentParser( + prog=str(exe), + description="Unpacks Debian packages and patches executable interpreter.", + ) + + arguments(prsr) + + parsed = prsr.parse_args(args) + + try: + for executable in patched( + parsed.archives, + parsed.output, + patchelf=parsed.patchelf, + interpreter=parsed.interpreter, + ): + LOG.info("patched: %s", executable) + return 0 + except CalledProcessError as e: + print(f"fatal: {e.stderr.rstrip()}", file=stderr) + return e.returncode + except KeyboardInterrupt: + return 130 + + +def entry(): + exit(main(Path(argv[0]), *argv[1:])) + + +if __name__ == "__main__": + entry() diff --git a/debian/patchelf/rule.bzl b/debian/patchelf/rule.bzl new file mode 100644 index 00000000..f2a33c99 --- /dev/null +++ b/debian/patchelf/rule.bzl @@ -0,0 +1,53 @@ +load(":InterpreterInfo.bzl", "DebianPatchelfInterpreterInfo") + +visibility("//...") + +DOC = "" + +ATTRS = { + "srcs": attr.label_list( + doc = "The Debian packages unpack and patch.", + allow_files = [".tar.gz", ".tar.bz2", ".tar.xz"], + mandatory = True, + ), + "_patch": attr.label( + doc = "Script that will unpack the directory and patch each executable ELF", + executable = True, + cfg = "exec", + default = ":patch", + ), +} + +def implementation(ctx): + output = ctx.actions.declare_directory(ctx.label.name) + + srcs = depset(transitive = [s.files for s in ctx.attr.srcs]) + + interpreter = "/tmp/bazel-rules_labgrid-debian-patchelf-{}-{}-{}/ld-linux.so".format( + ctx.label.workspace_name or "_main", + ctx.label.package.replace("/", "-"), + ctx.label.name, + ) + + args = ctx.actions.args() + args.add("--set-interpreter", interpreter) + args.add("--output", output.path) + args.add_all(srcs) + + ctx.actions.run( + executable = ctx.executable._patch, + arguments = [args], + inputs = depset(transitive = [s.files for s in ctx.attr.srcs]), + outputs = [output], + mnemonic = "DebianPatchelf", + ) + + return DefaultInfo(files = depset([output])), DebianPatchelfInterpreterInfo(interpreter) + +debian_patchelf = rule( + doc = DOC, + attrs = ATTRS, + implementation = implementation, +) + +patchelf = debian_patchelf diff --git a/e2e/MODULE.bazel.lock b/e2e/MODULE.bazel.lock index e03015c2..a7962a36 100644 --- a/e2e/MODULE.bazel.lock +++ b/e2e/MODULE.bazel.lock @@ -43,6 +43,8 @@ "https://bcr.bazel.build/modules/googletest/1.11.0/source.json": "c73d9ef4268c91bd0c1cd88f1f9dfa08e814b1dbe89b5f594a9f08ba0244d206", "https://bcr.bazel.build/modules/hermetic_cc_toolchain/3.1.0/MODULE.bazel": "ea4b3a25a9417a7db57a8a2f9ebdee91d679823c6274b482b817ed128d81c594", "https://bcr.bazel.build/modules/hermetic_cc_toolchain/3.1.0/source.json": "9d1df0459caefdf41052d360469922a73e219f67c8ce4da0628cc604469822b9", + "https://bcr.bazel.build/modules/patchelf/0.18.0/MODULE.bazel": "15a6beff7e828d585c5bd0f9f93589df117b5594e9d19e43096c77de58b9ae5f", + "https://bcr.bazel.build/modules/patchelf/0.18.0/source.json": "57caf6bcaa5ba515c6fb1c2eacee00735afbeb1ffacb34a57553fb139c8e4333", "https://bcr.bazel.build/modules/platforms/0.0.10/MODULE.bazel": "8cb8efaf200bdeb2150d93e162c40f388529a25852b332cec879373771e48ed5", "https://bcr.bazel.build/modules/platforms/0.0.10/source.json": "f22828ff4cf021a6b577f1bf6341cb9dcd7965092a439f64fc1bb3b7a5ae4bd5", "https://bcr.bazel.build/modules/platforms/0.0.4/MODULE.bazel": "9b328e31ee156f53f3c416a64f8491f7eb731742655a47c9eec4703a71644aee", -- GitLab