Skip to content

linux: make canister_build work against the current kernel - #1675

Open
dcasota wants to merge 2 commits into
vmware:5.0from
dcasota:fix/canister-build-against-current-kernel
Open

dcasota wants to merge 2 commits into
vmware:5.0from
dcasota:fix/canister-build-against-current-kernel

Conversation

@dcasota

@dcasota dcasota commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Two defects keep canister_build=1 from producing a usable canister for the 6.12 kernels (SPECS/92/linux 6.12.113, subrelease 92 and later; SPECS/91/linux 6.12.111, subrelease 91), and a third lets the FIPS certification modes run where they cannot work.

1. The canister patch series does not apply. In pkcs1pad_verify() upstream dropped the WARN_ON() around !digest_size; that line is context for canister-creation patches 1004 and 1010. At 6.12.111 upstream also inserted a gcc workaround (CFLAGS_ecc.o, under CONFIG_ARM, CONFIG_KASAN_STACK and GCC) between the curve25519 and ecdh_generic lines of crypto/Makefile, the context of the last hunk of patch 1000. At --fuzz=0:

1000-FIPS-canister-creation.patch: 1 out of 9 hunks FAILED -- saving rejects to file crypto/Makefile.rej
1004-Move-__bug_table-...patch:    1 out of 2 hunks FAILED -- saving rejects to file crypto/rsa-pkcs1pad.c.rej
1010-rsa-pkcs1pad-...patch:        1 out of 5 hunks FAILED -- saving rejects to file crypto/rsa-pkcs1pad.c.rej

6.12.113 also rewrote the overflow checks in drbg_healthcheck_sanity(): the two drbg_generate() results now go to ret and are checked with BUG_ON(ret == 0) instead of BUG_ON(0 < len), which is the context of the crypto/drbg.c hunk of 1004:

1004-Move-__bug_table-...patch:    1 out of 8 hunks FAILED -- saving rejects to file crypto/drbg.c.rej

so no canister_build=1 build gets through %prep.

2. A canister that does build is rejected at boot.

FIPS(fips_integrity_init): processing 8 sections, 687696 bytes
Kernel panic - not syncing: FIPS canister verification failed!

gen_canister_relocs gives each section an ondx, which fips_integrity_init() uses to index its si[] array when reversing a relocation. si[] is built from canister_sections[], which holds only sections carrying both begin and end markers. .bss carries only a begin marker (it is not measured; it is listed so relocations can resolve against it) but still consumes an ondx. Every section laid out after .bss is numbered one too high, its relocations are reversed against the wrong section, and the reconstructed image does not match the recorded HMAC. This stays invisible only while the compiler happens to emit .bss after every measured section.

3. acvp_build and kat_build use x86_64 inputs on other architectures. The build system injects both toggles for every architecture. On aarch64, the 6.12 linux.spec sets fips back to 1 after its architecture block and pulls in the prebuilt canister, which cannot work there: the canister is arch/x86 crypto and its tooling handles only R_X86_64_* relocations. In the 6.12 specs and SPECS/90/linux/linux.spec alike, acvp_build adds config_x86_64_acvp, the only ACVP config, to an arm64 build, which then fails late for a reason that names neither toggle.

Change

The same change is made in SPECS/92/linux and SPECS/91/linux; the generator and patches 1000 and 1010 are byte-identical in both trees, and 1004 differs only in the crypto/drbg.c hunk, which follows each tree's kernel:

  • Patches 1000, 1004, 1010 (canister_builder/patches) are refreshed to apply to the current source. In 1000 the lines added to crypto/Makefile (ecc.o, drbg.o, $(ecdh_generic-y)) are unchanged; only context and offsets move, and every other file in the patch is byte-identical. In 1004 the WARN_ON(req->dst) -> fcw_warn_on(req->dst) conversion is kept, because WARN_ON emits a __bug_table entry and keeping __bug_table out of the canister is the purpose of that patch; 1010's context follows. In SPECS/92/linux the crypto/drbg.c hunk of 1004 is refreshed for the 6.12.113 lines and converts the same two BUG_ON() calls to fcw_bug_on().
  • gen_canister_relocs.c: only a section with an end marker (a measured section) consumes an ondx; others get -1. .bss is NOBITS and holds no relocations, so the sentinel is never dereferenced; should a relocation ever target a section without a valid index (the sentinel, or one beyond the unsigned char the interpreter reads), the generator stops with the section name instead of emitting a wrapped index. The measured set, the generated linker script and the canister HMAC are unchanged; only the numbering handed to the interpreter changes.
  • linux.spec: the acvp_build -> fips 1 override is confined to x86_64, and acvp_build / kat_build on any other architecture stop the parse with %{error:acvp_build and kat_build are x86_64-only; refusing to build for %{_target_cpu}}. They are refused rather than undefined because Photon's SpecParser.py cannot undefine an injected macro and would still compute an .acvp Release that rpm does not build. canister_build stays allowed on aarch64, where it is ignored. This shares the %changelog entry linux, linux-esx: share the canister/.config logic via canister_config.inc #1673 adds (6.12.114-2 in SPECS/92/linux, 6.12.111-4 in SPECS/91/linux).
  • SPECS/90/linux/linux.spec (6.1.190, subrelease 90): the same refusal block; its aarch64 block already sets fips to 0 after the acvp_build override, so only the refusal is needed. 6.1.190-2 -> 6.1.190-3, one %changelog entry. The canister patches and the generator concern only the 6.12 trees.

Testing

  • %prep with canister_build=1 at subrelease 92 (x86_64) applies every patch at --fuzz=0 to the checksum-verified linux-6.12.113.tar.xz, all 16 canister patches (1000-1013, 1015, 1016) included; without this change 1000, 1004 and 1010 fail with the rejects shown above. At subrelease 90 the prepared trees of linux.spec (x86_64 default and canister_build, aarch64 default) are identical file for file with and without this change.
  • gen_canister_relocs.c passes gcc -fsyntax-only -Wall without diagnostics.
  • rpmspec -P of both specs for SPECS/90/linux (90), SPECS/91/linux (91) and SPECS/92/linux (92, 93), head against base, with no toggle, canister_build, acvp_build and kat_build: on x86_64 only the canister_build and kat_build cells differ, and only in the canister patches 1000, 1004 and 1010 and in gen_canister_relocs.c; on aarch64 acvp_build and kat_build stop with the error above (the base parses them) and every other cell is identical. No new warnings.
  • With canister_build=1 this stack builds a canister locally (linux-fips-canister 6.12.113-3), and linux and linux-esx are built against it. A kickstart install of the minimal ISO with fips=1 reports fips_enabled=1, the canister loaded and stamped as built from 6.12.113-3, and 0 failed units; a full-ISO install with the same kernels and without fips=1 boots with 0 failed units as well. It is a test build, not a CMVP-validated canister.
  • check_spec.py --mainline 93 (with spec-checker: check a spec as the subrelease being checked sees it #1678) exits 0 for linux.spec, linux-esx.spec and linux-rt.spec in SPECS/90/linux at 90, for linux.spec and linux-esx.spec in SPECS/91/linux at 91 and in SPECS/92/linux at 92 and 93.

Dependencies

Stacked on #1673 (canister_config.inc and the shared changelog entry), which merges first.

🤖 Generated with Claude Code

@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 2 times, most recently from 5167f6a to 2110644 Compare September 2, 2026 08:10
@aabusair aabusair closed this Sep 2, 2026
@aabusair aabusair reopened this Sep 2, 2026
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 6 times, most recently from 94ccc61 to c4a4e52 Compare September 9, 2026 11:19
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 5 times, most recently from 632de3c to 2994258 Compare September 13, 2026 05:18
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch from 2994258 to 8419a4e Compare September 14, 2026 09:28
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 7 times, most recently from c7f6ae1 to a62c1bb Compare September 19, 2026 11:44
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 3 times, most recently from aa8a578 to 66284df Compare September 30, 2026 11:49
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 2 times, most recently from 5729a16 to 2b139b8 Compare October 9, 2026 07:24
@dcasota dcasota changed the title linux, linux-esx: make canister_build work against the current kernel linux: make canister_build work against the current kernel Oct 9, 2026
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch 2 times, most recently from 754f4d0 to 92796a0 Compare October 24, 2026 16:16
@jaankit

jaankit commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Hi @dcasota, thanks for documenting the failures and preparing these fixes. Could you clarify why you want to rebuild the canister against the latest v6.12 tag? Is this for development and certification testing, or production use?
We designed the canister so that, once finalized and certified, subsequent kernel updates consume the existing approved binary rather than rebuild it against each new kernel tag. The spec reflects this through the separate canister_build / canister_usage paths and the pinned canister dependency. A newly built canister should not be assumed to inherit the existing certification.
If FIPS compliance is not required, building with the spec’s fips=0 setting provides the regular kernel crypto path without the canister. Alternatively, the existing FIPS-capable kernel supports booting with fips=0, which disables FIPS restrictions and integrity verification while retaining the linked canister implementations.
Are you encountering a compatibility issue with the existing prebuilt canister, or do the failures occur only when explicitly enabling canister_build=1? Understanding that would help clarify the need for the rebuild changes.

@dcasota

dcasota commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Hi @jaankit,
To answer directly: this is for development and certification testing.

The default canister_usage path still links the certified linux-fips-canister 6.12.60-18.2 without any issue, this PR leaves its rpmspec -P output unchanged, and I agree a rebuilt canister inherits no certification (the 6.12.113-3 canister in the Testing section is labelled a test build).
The failures occur only with canister_build=1, which kat_build (CMVP_DEMO) also forces: at 6.12.113 (btw. 6.12.114 is out), patches 1000, 1004 and 1010 are rejected in %prep because upstream moved their context in crypto/Makefile, crypto/rsa-pkcs1pad.c and drbg_healthcheck_sanity(), so the next certification would start with these rejects.
Independently, gen_canister_relocs numbers .bss as if it were measured, and fips_integrity_init() panics with "FIPS canister verification failed" as soon as a compiler places .bss before a measured section, a latent defect in every future canister build.

The main reason for canister builds in the open-source flavor of Photon OS is kernel CVEs inside the canister boundary:
CVE-2026-64306 (crypto/drbg.c: drbg_ctr_generate() can return success on failure, leaving the output uninitialized) and CVE-2026-64313 (crypto/ecc.c: carry overflow in vli multiplication) were fixed upstream in 6.12.96, but x86_64 kernels that link the 6.12.60 canister still run the unfixed code, even when booted with fips=0, which keeps the linked implementations.
A fips=0 build avoids the canister without exercising it, so only a canister built from the current tree can show that these fixes build, pass the self-tests and integrity check, and boot with fips=1 before a re-certification.

I hope this helps
Daniel

Btw. in my lab, I use a lab-tool called sharukhan-cli. sharukhan-cli is a Rust tool that rebases pending Photon OS 5.0 pull requests onto new upstream bases and builds isos from them, with either the certified prebuilt FIPS canister or an "equivalent" test-only canister rebuilt from the current kernel. It then boots each iso through a matrix of installs, then publishes the changes that pass after verifying them on a staged VM. I apologize for not testing the OVA formatting as well (sorry, Oliver, still a task todo!).

With six iso all kickstart tests (ext4/btrfs/+, generic/hypervisor-optimized, without/with stig-hardening, etc.) can be accomplished. The test isos generated this weekend were:

  • full-poi2.8-equivalent__photon-5.0-727909c4b8.x86_64-49b065fa0c26.iso
  • full-poi2.8-prebuilt__20261025T141231Z-photon-5.0-727909c4b8.x86_64.iso
  • full-poilatest-prebuilt__20261025T143713Z-photon-5.0-727909c4b8.x86_64.iso
  • minimal-poi2.8-equivalent__photon-minimal-5.0-727909c4b8.x86_64-4c7260da37ff.iso
  • minimal-poi2.8-prebuilt__20261025T123408Z-photon-minimal-5.0-727909c4b8.x86_64.iso
  • minimal-poilatest-prebuilt__20261025T131124Z-photon-minimal-5.0-727909c4b8.x86_64.iso

Sharukhan can also use a out-of-branch kernel release such as e.g. 7.3. It creates aarch64 isos, inject drivers for Azure.

  • 7.3_full-poilatest-equivalent__photon-5.0-cb2b916647.x86_64.iso
  • photon-5.0-c6db6a0d6.aarch64.azure.iso
  • photon-5.0-c6db6a0d6.aarch64.iso

dcasota and others added 2 commits October 30, 2026 13:23
…g.inc

linux.spec and linux-esx.spec implemented the same canister Kconfig handling
in two different idioms, and they disagreed on one constellation.

linux-esx.spec used %if 0%{?fips} / %else, and its %else branch cleaned the
GCC_PLUGIN_{MATCH,PAD}_CANISTER_STRUCTS comments out of .config. linux.spec
instead used two independent %if canister_build / %if canister_usage blocks
with no %else, so when fips=0 nothing ran: the shipped .config kept the
"is not set" comments, make olddefconfig dropped them, and the
check_for_config_applicability.inc diff guard failed %prep. That is an
x86_64 kernel built without the canister: fips is set by %global inside
%ifarch, so it is 1 on x86_64 unless that line is edited to 0, and
config_x86_64 carries both "is not set" comments. The aarch64 configs
never carried them.

Move the three conditional blocks into SPECS/linux/canister_config.inc,
pulled in by both specs as Source5 + %include, the same mechanism both
already use for check_for_config_applicability.inc. Divergence between the
two flavours is now impossible by construction.

SPECS/91/linux, which builds subrelease 91 from its own copy of the 6.12.111
specs, carries the same defect and gets the same change.

linux-esx.spec additionally gains the derived-flag block linux.spec already
had (canister_build=0, canister_usage=fips), nested inside the existing
%if fips so it does not shadow -D. esx never builds a canister - it untars a
prebuilt one - so fips=1 implies canister_usage=1, reproducing the old
branch exactly.

Tested across 168 cells: SPECS/91/linux at subrelease 91 and SPECS/linux at
92 and 93 x 2 specs x {x86_64, aarch64} x 14 flag combinations of fips /
canister_build / canister_usage / acvp_build / kat_build. All x86_64 cells are byte-identical apart from the release string.
The aarch64 fips=0 cells gain the two deletions, which are no-ops there: no
aarch64 config carries the "is not set" comments. The esx aarch64
reordering is likewise a no-op: the canister and jitterentropy seds touch
disjoint symbols.

Change-Id: If4107c14d8f15752dff0c0f25c6b4dee504e3041
Signed-off-by: Daniel Casota <dcasota@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two defects keep canister_build=1 from producing a usable canister, and a
third lets the FIPS build modes run where they cannot work.

1. The patch series no longer applies. Upstream dropped the WARN_ON()
wrapper around !digest_size in pkcs1pad_verify(), and that single line is
shared context for canister-creation patches 1004 and 1010, so %prep
fails at --fuzz=0 with

	1 out of 2 hunks FAILED -- crypto/rsa-pkcs1pad.c.rej

Rebase both onto the 6.12 series. In 1004 the WARN_ON -> fcw_warn_on
conversion is load-bearing rather than cosmetic: WARN_ON emits a
__bug_table entry, and keeping __bug_table out of the canister is what
that patch exists to do.

6.12.111 then inserted a gcc workaround (CFLAGS_ecc.o, under CONFIG_ARM,
CONFIG_KASAN_STACK and GCC) between the curve25519 and ecdh_generic lines
of crypto/Makefile, which is the context of the last hunk of 1000:

	1 out of 9 hunks FAILED -- saving rejects to file crypto/Makefile.rej

Rebase 1000 onto it as well. The lines it adds there are unchanged -
canister += ecc.o, drbg.o and $(ecdh_generic-y) - only context and
offsets move, and every other file in the patch is byte-identical.

6.12.113 rewrote the overflow checks in drbg_healthcheck_sanity(): the
two drbg_generate() results now go to ret and are checked with
BUG_ON(ret == 0) instead of BUG_ON(0 < len), so the crypto/drbg.c hunk
of 1004 fails:
	1 out of 8 hunks FAILED -- saving rejects to file crypto/drbg.c.rej
Rebase that hunk onto the new lines. It converts the same two BUG_ON()
calls to fcw_bug_on() as before.

2. A canister that does build is rejected at boot:

	FIPS(fips_integrity_init): processing 8 sections, 687696 bytes
	Kernel panic - not syncing: FIPS canister verification failed!

gen_canister_relocs gives each section an "ondx", the number
fips_integrity_init() uses to index its si[] array when reversing a
relocation. si[] is built from canister_sections[], which holds only the
sections carrying both markers. .bss carries a begin marker only - it is
not measured, it is listed so relocations can resolve against it - yet it
still consumed an ondx, so every section laid out after .bss was numbered
one too high and its relocations were reversed against the wrong section.
Let only a measured section consume an index. .bss is NOBITS and cannot
hold relocations, so the sentinel is never dereferenced; should a
relocation ever target a section without a valid index (the sentinel, or
one beyond the unsigned char the interpreter reads), the generator now
stops with the section name instead of emitting a wrapped index. The
measured set, the generated linker script and the canister HMAC are
unchanged.

3. acvp_build and kat_build build x86_64 inputs on other architectures.
The build system injects both toggles for every architecture. On aarch64,
SPECS/linux/linux.spec re-enabled fips after its architecture block and
pulled in canister machinery that cannot work there: the canister is
arch/x86 crypto and its tooling handles only R_X86_64_* relocations.
Independently, in SPECS/linux/linux.spec and SPECS/90/linux/linux.spec
alike, acvp_build selects config_x86_64_acvp, the only ACVP config, for an
arm64 kernel, and the build fails late for a reason that names neither
toggle.

Confine the fips override to x86_64, and refuse acvp_build and kat_build
on any other architecture with the same block in both specs, so the build
stops at once and says why. They are refused rather than undefined because
Photon's SpecParser.py cannot undefine an injected macro, and would still
compute a .acvp Release that rpm no longer builds. canister_build stays
allowed on aarch64, where it is already ignored.

SPECS/91/linux, which builds subrelease 91 from its own copy of the 6.12.111
specs, carries the same patch series, generator and fips override, and gets
the identical change, with 1004 as it applies to 6.12.111. With it, %prep
applies every patch at --fuzz=0 to 6.12.111 (subrelease 91) and 6.12.113
(92), for linux and linux-esx, by default and with canister_build and
kat_build.

Change-Id: Ib5878beeec1dd79e2bcdfefc33639d2b40d69d14
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dcasota
dcasota force-pushed the fix/canister-build-against-current-kernel branch from 92796a0 to 39fc4e0 Compare October 31, 2026 06:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants