Skip to content
ublue-osPublic

About

A caching layer for pre-built Fedora akmod RPMs

Topics

Resources

Code of conduct

Security policy

Stars

108 stars

Watchers

11 watching

Forks

Repository files navigation

ublue-os akmods

OpenSSF ScorecardBuild COREOS-STABLE akmodsBuild COREOS-TESTING akmodsBuild LONGTERM-6.18 akmodsBuild OGC akmodsBuild OGC-LTS akmodsBuild MAIN akmods

OCI images providing a set of cached kernel RPMs and extra kernel modules to Universal Blue images. Used for better hardware support and consistent build process.

How it's organized

The akmods images are built and published daily. However, there's not a single image but several, given various kernels we now support.

The akmods packages are divided up for building in a few different "groups":

  • common - any kmod installed by default in Aurora/Bazzite/Bluefin (or were originally in main images pre-Fedora 39)
  • extra - less common kmods used by only one Universal Blue project.
  • nvidia - only the nvidia proprietary kmod and addons
  • nvidia-open - only the nvidia-open kmod and addons
  • zfs - only the zfs kmod and utilities built for select kernels

Each of these images contains a cached copy of the respective kernel RPMs compatible with the respective kmods for the image.

Builds also run for different kernels:

  • main - Mainline Fedora Kernel
  • coreos-stable - Current Fedora CoreOS stable kernel version
  • coreos-testing - Current Fedora CoreOS testing kernel version
  • Longterm-6.18 - Fedora Kernel on Kernel 6.18 LTS
  • ogc - Open Gaming Collective kernel
  • ogc-lts - Open Gaming Collective kernel, LTS branch (:lts tag)

See images.yaml for which akmods packages are built for each Kernel

Features

Overview

The common images contain related kmod packages, plus:

  • ublue-os-akmods-addons - installs extra repos and our kmods signing key; install and import to allow SecureBoot systems to use these kmods

The nvidia and nvidia-open images contains

  • ublue-os-nvidia-addons - installs extra repos enabling our nvidia support
  • ublue-os-ucore-nvidia - a slightly lighter ublue-os-nvidia-addons for CoreOS/uCore systems

Kmod Packages

Group Package Description Source
common (ogc and ogc-lts only) nct6687d Linux kernel module for Nuvoton NCT6687D/NCT6687-R motherboard fan control Terra
common openrazer kernel module adding additional features to Razer hardware Terra
common v4l2loopback allows creating "virtual video devices" RPMFusion - free
common wl support for some legacy broadcom wifi devices RPMFusion - nonfree
common xone xbox one controller USB wired/RF driver modified to work along-side xpad (built from xonedo fork) badge
common xpadneo xbox one controller bluetooth driver negativo17 - fedora-multimedia
common (ogc and ogc-lts only) zenergy AMD Zen energy monitoring driver Terra
extra ayaneo-leds AYANEO joystick ring RGB LEDs (legacy EC models) Terra
extra ayaneo-platform AYANEO x86 handheld RGB LED control and bypass charging Terra
extra evdi DisplayLink USB graphics adapter support negativo17 - fedora-multimedia
extra gcadapter_oc Gamecube controller adapter overclocking badge
extra hid-fanatecff Fanatec wheel base force feedback Terra
extra hid-tmff2 Thrustmaster T300RS, T248, TX, T128, T598, T-GT II, TS-XW force feedback Terra
extra kvmfr KVM framebuffer relay kernel module for use with Looking Glass badge
extra logitech-trueforce Logitech RS50, G PRO, and G923 force feedback and TrueForce support Terra
extra new-lg4ff Logitech force feedback wheels Terra
extra ryzen-smu AMD Ryzen SMU (System Management Unit) driver badge
extra sc0710 Elgato 4K60 Pro MK.2 / 4K Pro capture card driver Terra
extra system76 System76 laptop hardware support badge
extra system76-io System76 Thelio I/O board driver badge
extra t150-driver Thrustmaster T150 steering wheel Terra
nvidia-open nvidia nvidia-open GPU drivers negativo17 - fedora-nvidia
zfs zfs OpenZFS advanced file system and volume manager zfs

Notes

NVIDIA Hardware Support

We build both the open and closed drivers from NVIDIA. The open driver is the only option for supporting the latest hardware. The closed driver is required for older hardware, but even it doesn't support "legacy" hardware.

nvidia-open - newest current and most open driver supports the following hardware:

nvidia - closed proprietary driver supports the following hardware:

  • GeForce RTX: 40 Series, 30 Series, 20 Series
  • GeForce: 16 Series, 10 Series, 900 Series, 700 Series
  • NVIDIA Turing: T4, T4G
  • NVIDIA Volta: V100
  • NVIDIA Pascal: Quadro: P2000, P4000, P5000, P6000, GP100; Tesla: P100, P40, P4
  • NVIDIA Maxwell: Quadro: K2200, M2000, M4000, M5000, M6000, M6000 24GB; - - Tesla: M60, M40, M6, M4

OpenZFS release signing keys

ZFS release signatures are verified with the pinned public keys in build_files/zfs/openzfs-release-keys.asc. Renovate tracks the ZFS version in images.yaml, but does not update the trusted keys. If a release uses a new or rotated key, independently verify its full fingerprint against the official OpenZFS signing keys and review the change before adding it. Obtain the public key by full fingerprint, inspect it with gpg --show-keys --with-fingerprint, and update both the armored keyring and the sorted fingerprint list in build_files/zfs/build-kmod-zfs.sh. Also refresh existing key material when appropriate to pick up revocations or updated signing subkeys; do not automatically trust a changed fingerprint.

Pinning the coreos-stable kernel

coreos_stable_kernel_pin in the Justfile is a temporary ceiling on the kernel used for every coreos-stable target (common, nvidia-lts, nvidia-open, zfs). It exists so publication can continue when Fedora CoreOS stable ships a kernel that OpenZFS cannot yet build.

Empty '' means follow upstream fedora-coreos:stable. A value like '6.17.12' is a major.minor.patch cap applied in get-kernel-version for coreos-stable only. Newer detected kernels are rewritten to that pin plus the detected release suffix, then verified in Koji. Kernels at or below the pin pass through.

Use the pin when all of these are true:

  1. coreos-stable ZFS builds are reliably failing, not flaking.
  2. The failure is kernel-version incompatibility: OpenZFS META Linux-Maximum is below the current CoreOS stable kernel, or configure rejects the kernel as unsupported.
  3. Waiting for an OpenZFS release would keep the entire coreos-stable check job red, blocking publication of the other targets (and kernel CVE / nvidia updates) to downstream stable images such as ucore:stable.

Do not use the pin for transient CI or network flakes, failures that are not “kernel newer than OpenZFS supports,” coreos-testing (set zfs.linux_experimental in images.yaml if you intentionally want to track newer kernels), or as a long-term fork of the CoreOS stable kernel. Prefer the pin over inventing local OpenZFS kernel-compat backports when the only goal is to restore publication. Carry an already-upstreamed ZFS fix only when the team has explicitly decided to do that.

To apply the pin:

  1. Read the last successful major.minor.patch from a published image (ostree.linux on ghcr.io/ublue-os/akmods-zfs:coreos-stable-<ver>) or from a prior build/*/KCWD/rpms/cache.json.
  2. Set coreos_stable_kernel_pin := 'X.Y.Z' in the Justfile, then commit and merge.
  3. Clear it back to '' as soon as ZFS builds on the current CoreOS stable kernel.

Downstream stable images keep the older kernel until the pin is cleared. That is intentional: operations continue, but new kernel CVEs in the unpinned CoreOS kernel wait.

A coreos-stable ZFS job that fails for this unsupported-kernel reason prints a pointer to this section.

Usage

To install one of these kmods, you'll need to install any of their specific dependencies (checkout the build-prep.sh and the specific build-FOO.sh script for details), and ensure you are on a compatible kernel.

Using common images as an example, add something like this to your Containerfile, replacing TAG with the appropriate tag for the image:

COPY --from=ghcr.io/ublue-os/akmods:TAG / /tmp/akmods-common
RUN find /tmp/akmods-common
## optionally install remove old and install new kernel
# dnf -y remove --no-autoremove kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra
## install ublue support package and desired kmod(s)
RUN dnf install /tmp/rpms/ublue-os/ublue-os-akmods*.rpm
RUN dnf install /tmp/rpms/kmods/kmod-v4l2loopback*.rpm

For NVIDIA images, add something like this to your Containerfile, replacing TAG with the appropriate tag for the image:

COPY --from=ghcr.io/ublue-os/akmods-nvidia:TAG / /tmp/akmods-nvidia
RUN find /tmp/akmods-nvidia
## optionally install remove old and install new kernel
# dnf -y remove --no-autoremove kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra
## install ublue support package and desired kmod(s)
RUN dnf install /tmp/rpms/ublue-os/ublue-os-nvidia*.rpm
RUN dnf install /tmp/rpms/kmods/kmod-nvidia*.rpm

Note There is an nvidia-install script that most universal-blue images use located with the main repo that works for desktop Fedora installs.

Verification

Image Signatures

All architecture images and manifest indexes are signed by Podman with the shared Universal Blue static key. Prefer verifying immutable, digest-pinned subjects with Cosign v3.1.2:

cosign verify \
  --key https://raw.githubusercontent.com/ublue-os/main/main/cosign.pub \
  ghcr.io/ublue-os/akmods@sha256:DIGEST

For historical manifest digests with registry-attached GitHub provenance, use legacy bundle discovery:

cosign verify \
  --new-bundle-format=false \
  --key https://raw.githubusercontent.com/ublue-os/main/main/cosign.pub \
  ghcr.io/ublue-os/akmods@sha256:DIGEST

The ublue-os/main/cosign.pub key is the Universal Blue image trust root. Mutable tags such as main-44 or coreos-stable-44 are convenient for discovery, but a digest is the preferred verification target.

Build Provenance

GitHub build provenance is a separate trust decision from image signatures: it uses a short-lived GitHub OIDC certificate rather than the Universal Blue static key. Provenance currently covers the release and kernel-version manifest indexes, not architecture-only image digests.

gh attestation verify \
  oci://ghcr.io/ublue-os/akmods@sha256:DIGEST \
  --repo ublue-os/akmods

Provenance is stored in GitHub's attestations API. It is no longer published as a registry referrer, so registry provenance and GitHub linked-artifact storage records are not available for newly published digests.

Local Building/Testing

You can build these akmods locally with our test keys using the included Justfile. We strongly recommend using the provided devcontainer which contains all dependencies for building this project.

Using the Devcontainer

The repo's .devcontainer.json points at ghcr.io/ublue-os/devcontainer:latest, a public prebuilt image with just, podman, yq, jq, and everything else the Justfile needs, so you don't have to install build dependencies on your host. Pulling it requires no authentication.

With VS Code: install the Dev Containers extension, open this repo, then run "Dev Containers: Reopen in Container" from the Command Palette. Once it's open, use the integrated terminal exactly as you would locally (e.g. just build).

From the command line (no VS Code): install the devcontainer CLI (npm install -g @devcontainers/cli), then start/exec into the container with Podman as the backend. A few aliases make this painless:

alias dcup='devcontainer up --docker-path=podman --workspace-folder .'
alias dcexec='devcontainer exec --docker-path=podman --workspace-folder .'
alias dcjust='devcontainer exec --docker-path=podman --workspace-folder . just'
alias dcpodman='devcontainer exec --docker-path=podman --workspace-folder . podman'
dcup                 # build and start the devcontainer (run once, or after changing .devcontainer.json)
dcjust build         # run `just build` inside the devcontainer
dcpodman images      # run an arbitrary podman command inside the devcontainer

How to Use the Justfile

To build an akmods package, run the following:

just build

Since nothing additional was set. The following will occur. The build scripts will determine the current fedora kernel version, download the RPMs, and sign the kernel with the test key. It will then build the common set of akmods. To modify what gets built, modify the following environment variables:

  • AKMODS_KERNEL - The kernel flavor you are building
  • AKMODS_VERSION - The release version
  • AKMODS_TARGET - The akmods package to build
  • ZFS_VERSION - Overrides zfs.*.version from images.yaml (a full X.Y.Z). ZFS_MINOR_VERSION is no longer read. Renovate manages the pin.
  • RPMFUSION_MIRROR - Optional base URL for an RPM Fusion mirror (for example https://mirrors.ocf.berkeley.edu/rpmfusion). When set, the build installs the rpmfusion release RPMs from it and points the rpmfusion repos at it instead of the metalink. When unset, the build uses https://download1.rpmfusion.org and the normal metalink. It can go in .env.
AKMODS_KERNEL=longterm-6.18 AKMODS_VERSION=45 AKMODS_TARGET=zfs just build

Will determine the current longterm kernel version, download the rpms and sign them, and will then build the zfs package.

You can also populate a .env file to store your current settings.

You can see your current settings with just --evaluate

Additionally you can pass values as key/value pairs.

just kernel_flavor=main version=42 akmods_target=extra build

Which will build the extra package for main.

Note, the Justfile will compare your inputs to the images.yaml file to ensure you have a valid combination.

How to Use images.yaml

All build targets are defined in the images.yaml file. This is where the top level targets are defined. You can view the targets using:

yq 'explode(.).images' images.yaml

Regenerating GitHub Workflows

The .github/workflows/build-akmods-*.yml files are generated from images.yaml and the templates in workflow-templates/ (workflow.yaml.in, cache_kernel.yaml.in, job.yaml.in, check-status.yaml.in) by running:

just generate-workflows

Run this and commit the resulting diff whenever you edit images.yaml (adding/removing a kernel, flavor, or target) or anything in workflow-templates/. Nothing in CI does this for you.

Never hand-edit files under .github/workflows/build-akmods-*.yml directly — they're regenerated from the templates above, and manual edits will be silently discarded the next time someone runs just generate-workflows.

Each flavor's daily build is scheduled at a staggered UTC minute assigned once and preserved across regenerations, so an unrelated images.yaml edit shouldn't produce cron diffs unless that flavor's generated file was missing beforehand.

Kernel and module signing keys

Kernels and kmods are signed for Secure Boot and MOK enrollment. Each production key has one ID, KSIGN_YYYYMMDD, taken from its certificate's notBefore date. The same ID names the 1Password item, GitHub secret and files below.

Path Contents
certs/production/signing-keys Active production key IDs, one per line
certs/production/<ID>.der Public certificate for each production key
certs/production/compat-key ID of the certificate copied to certs/public_key.der
certs/test/signing-keys Test key IDs
certs/test/<ID>.der, certs/test/<ID>.key Committed test certificates and private keys

Every active key signs every kernel and module. Verification requires all expected signers.

Production. Pushes, scheduled runs and manual runs on main use KSIGN_MODE=production. Private keys are secrets in the ksign environment, which only main can use. just write-signing-keys writes each listed key from the environment variable of the same name and checks it against its certificate. A missing or mismatched key fails the build; production never falls back to test keys. Workflows run just remove-signing-keys during cleanup, even after a failure, to delete local production .key files and key-writer temporary files. This does not remove copies in container storage.

Tests and local builds. Pull requests and local builds default to the keys in certs/test/. Their private keys are public, so never enroll these certificates as MOKs with mokutil. To load locally built modules under Secure Boot, sign them with your own trusted key instead.

Compatibility copy. Downstream images use certs/public_key.der to verify kernels and enroll our key. It must stay byte-identical to the certificate named in compat-key; just test-signing-key checks this. Adding an active key does not change the compatibility copy.

Adding a key.

  1. Run just generate-signing-key <validity-days>. This creates certs/production/<ID>.der and <ID>.key but does not activate the key.
  2. Store the private key in the project 1Password vault under its ID, verify the backup, then run just remove-signing-keys. This removes all local production .key files, not just the newly generated one.
  3. Add the secret to the ksign environment under the same ID.
  4. Map it into the Retrieve Signing Keys step in both reusable workflows, using <ID>: ${{ secrets.<ID> }} with the actual key ID substituted.
  5. Add the ID to certs/production/signing-keys, commit the certificate and merge. The certificate digest changes the kernel cache key, so the next production build re-signs kernels automatically.

Retiring a key. Users must enroll its replacement first, and downstream images must stop requiring the old signature. Update compat-key and certs/public_key.der if needed, then remove the retired key's ID, certificate, workflow mappings and environment secret after validating the new signing path.

Skipping Unchanged Builds

Each image has an io.github.ublue-os.akmods.fingerprint label with a sha256 of the build inputs and an io.github.ublue-os.akmods.fingerprint-inputs label with the inputs themselves as JSON. An image labeled fingerprint=unavailable was built without a fingerprint and never matches.

The inputs are the target, version, architecture, and kernel release. They also include the git hashes of build_files/, files/, Containerfile.in, Justfile, fetch-kernel.sh, the committed public and test keys, reusable-build.yml, and reusable-cache-kernel.yml. The target's images.yaml entry, the ZFS version, any RPMFUSION_MIRROR override, and for NVIDIA targets the akmod-nvidia EVR are in there too. Local uncommitted edits add a random nonce, so a dirty build never matches. The fingerprint logic lives in the Justfile, which is itself an input, so changing how the fingerprint is computed changes every hash. Keep it in a file that's an input.

Some inputs are left out on purpose: the builder image digest, akmod versions from rpmfusion, copr, and terra, and NVIDIA userspace package releases. A rebuild after 7 days picks those up. To pick them up sooner, run a forced build. Committed signing certificates and key lists are fingerprinted; kernel caches also include the selected signing certificates, signing mode, Fedora version, full kernel release and architecture.

Before testing, CI runs just fingerprint once per target and arch. That writes build/<flavor>-<version>/fingerprint-<target>-<arch>.json, and both just test and just build read that same file. For NVIDIA targets, the file pins the akmod-nvidia EVR, and the build installs exactly that version and the driver packages for that version. Without a valid file, the build runs unpinned and the image is labeled unavailable. For a labeled local build, run:

just fingerprint && just test && just build

just build-needed compares the local fingerprint with the published per-arch image. It skips test, build, and push only when the hashes match, the published image's signature verifies, and the image is no older than AKMODS_MAX_AGE_DAYS (default 7, settable as a repository variable). Any error or missing value means build. just manifest-needed skips regenerating the manifests when both indexes already point at the current per-arch digests and are signed. Attestations are created on every run either way.

Pull requests always build with test keys and never publish. Pushes, scheduled runs, and manual runs on main use the build guard and publish with production keys from the ksign environment. Manual runs on other branches use test keys and do not publish. Merge queues are no longer used. To force every target in a flavor, run the workflow manually with force checked.

Adding Kernels and KMODs

Generally speaking, Kernels are only added if they will be used internally to Universal Blue.

KMODs as well will likely only be included if there is a need/desire to include them within the Universal Blue Project. Generally, KMODs for hardware enablement will be considered for inclusion or ones that fix/resolve a known feature gap.

Don't hesitate to file an issue asking about inclusion.

Metrics

Alt

About

A caching layer for pre-built Fedora akmod RPMs

Topics

Resources

Code of conduct

Security policy

Stars

108 stars

Watchers

11 watching

Forks

Releases

Packages

Used by

Contributors

Languages