Repository navigation
J700: add opt-in NV2 KVM virtualization and PSCI - #33
Open
RyanTheTide wants to merge 14 commits into
Open
RyanTheTide wants to merge 14 commits into
RyanTheTide wants to merge 14 commits into
Conversation
The monitor owns the secondary CPUs, so it can give the OS real PSCI instead of spin-table. Trap SMC to EL2 (HCR_EL2.TSC; Apple has no EL3) and service it in src/psci.c: VERSION 1.1, FEATURES, CPU_ON, CPU_OFF, CPU_SUSPEND, AFFINITY_INFO, MIGRATE_INFO_TYPE 2, SYSTEM_OFF/RESET, SMCCC_VERSION. CPU_ON releases a parked core into the OS through the existing pool. CPU_OFF re-parks the dying core on a fresh stack (smp_reenter_park); the park loop is factored out of smp_secondary_entry so both reach it. CPU_SUSPEND is a register-preserving WFI for a retention idle state. kboot adds a /psci node and sets enable-method = "psci" on each cpu when the monitor runs the OS at EL1; an EL2 boot keeps spin-table. Verified on the Neo: all six CPUs boot via CPU_ON (enable-method psci, PSCIv1.1), and on the proxy the CPU_OFF re-park and CPU_ON cycle runs 60 times across every secondary in both the IPI and SEV wake modes with no fault. Assisted-by: Claude Opus 5.5
Begin nested virtualization so KVM in the EL1 Linux can run VMs. New src/nv2.c runs the OS at virtual EL2 (HCR_EL2.NV|NV2|E2H, per-CPU VNCR/AVNCR pages) and dispatches the two guest-hypervisor traps the port rests on: - the ERET gate: when the guest erets into its own VHE-host userspace, do the eret here (the one constant NV2 cost); the team's SPTM "native SK owner" check is dropped since there is no secure kernel under this monitor; - the EL2-register emulator (EC 0x18): VHE-aliased regs applied to their real _EL12 copies, CNTHCTL/MDCR/CPTR/HACR/ICC_SRE held in per-CPU slots, anything else shadowed and logged so a vEL2 Linux boot reveals what it touches. monitor_enter_os is split so nv2_enter_os reuses the EL2 passthrough. nv2_active is off, so the shipping EL1+PSCI boot is unchanged (verified: still boots to login). Measured on the Neo (T8140, no SPTM) via the proxy: the ERET trap fires, and the emulator round-trips SCTLR_EL2->SCTLR_EL12, a HACR_EL2 slot and a shadowed ACTLR_EL2, with no fault. Next: wire the vEL2 boot and iterate on the trap set, then direct VTTBR/VTCR install for L2 and fault/TLBI/AT forwarding. Assisted-by: Claude Opus 5.5
With aura-nv2 (or an NV2_BOOT development build) every CPU enters the
OS at EL1 with HCR_EL2.NV|NV2 and per-CPU VNCR/AVNCR pages, so the
kernel finds itself at EL2 and KVM initialises in VHE mode. A plain EL1
boot is unchanged: the ERET gate in the vectors costs a compare and a
branch when NV2 is not armed.
- An assembly ERET gate in _v_sp0_sync completes the guest's returns to
its own userspace and to itself without building a frame.
- The EL2 registers that NV2 traps on T8140 are emulated: VHE aliases
go to the real EL1 register, the rest are held for the guest's guest.
- An ERET to EL1, or to EL0 with TGE clear, enters a child: the EL1
register file, SP_EL1 and CNTKCTL swap with the VNCR page, and with
HCR.VM the guest's own VTTBR/VTCR go into the hardware as they are
(no SPTM: the OS has no stage 2, so KVM's tables are already in PAs),
with its timers, list registers, FGT and trap controls. Exits are
handed to the OS's lower-EL vector as the hardware would.
- TLBI VMALLE1{,IS,OS} trap through HFGITR, not HCR.TTLB, so the OS's
own TLB maintenance stays native; AT traps through HCR.AT. Both run
under the guest's context when the OS's TGE is clear.
- The EL2 MMU stays on after the handoff, as NV2 redirection goes
through the EL2 regime, and secondaries enable theirs on entry.
Tested on J700: the Aurora kernel boots to login on all six CPUs with
"CPU: All CPU(s) started at EL2" and "kvm: VHE mode initialized
successfully", /dev/kvm present. nv2_probe_call runs bounded virtual-
EL2 stubs from the proxy; a stub exercised the ERET gate, a VM-disabled
child and a stage-2 guest's HVC, fault (HPFAR) and WFI exits.
Assisted-by: Claude Opus 5.5
A guest exit put back zeroed fine-grained traps instead of the ones the
OS runs with, so on SME parts the OS's TPIDR2_EL0 accesses trapped and
read as zero from the first VM exit on. Snapshot the OS's HFGxTR, HDFGxTR,
HCRX, CPTR and CNTHCTL when a CPU is armed and restore those.
NV2_TRACE development builds log entries, exits, emulated registers,
TLBI/AT and any trap the monitor could not handle to a per-CPU ring in a
no-map reserved-memory node ("aura,nv2-trace"), which the OS can read
through /dev/mem while it runs. The monitor no longer prints from these
paths: once the OS owns the DockChannel console a monitor printf can
stall a CPU for up to a second per byte.
Tested on J700 under the Aurora kernel with KVM: a raw KVM API guest
ran 1M in-kernel HVC exits (1.10 us each on a P core, 1.55 us on an E
core) and 1M userspace MMIO exits (2.7/4.1 us) on each of five CPUs,
and 50 VM create/destroy cycles, with no unhandled trap in the ring.
Assisted-by: Claude Opus 5.5
_exc_return skipped x19-x29 on the grounds that C preserves them, so an emulated register read into any of them returned stale contents to the guest. Under TID3 the kernel's mrs x19, ID_AA64PFR2_EL1 came back with junk, it took GICv5 as present, read ICC_IDR0_EL1 and died before its console was up. Reload them from the frame like the others. For L1, HCR.TID3 now traps the group-3 ID registers. They return the hardware's values (identical at EL2, EL1 and virtual EL2 on T8140) except ID_AA64MMFR4_EL1, which reports E2H0 = NI_NV1 and NV_frac = NV2_ONLY: the monitor exposes VHE only and never sets NV1, so a KVM in L1 must not offer nVHE to its guests. An ID read from L1's userspace is handed to L1 as if taken at its EL1. A guest hypervisor (vHCR.NV|NV2, never NV1) can now be entered: its VNCR page address is an L1 virtual address, translated with AT S1E1W while L1's stage 1 is live (L1 has no stage 2, so this is the PA); Apple EL2 registers of the guest go to a scratch AVNCR page, as Linux's KVM knows no AVNCR. NV2_TRACE_PRINT development builds also print the first trace records while the monitor still owns the console. Tested on J700: the Aurora kernel boots on six CPUs at virtual EL2 with TID3 and KVM initialises in VHE mode; proxy probe_nv2_tid3 checks the values and the EL0 reflection. Assisted-by: Claude Opus 5.5
EC 0x1a reaches EL2 only from EL1 with HCR_EL2.NV set, and the monitor sets NV only together with NV2, so reading HCR_EL2 and SPSR_EL2 in the gate decides nothing. Measured on J700 from the proxy: 168 ns per gated ERET round trip before and after; the cost is the trap itself. Assisted-by: Claude Opus 5.5
Two things break a guest hypervisor that L1's KVM runs under NV2, both from L1 itself running at virtual EL2: - KVM emulates a guest hypervisor's TLBIs with every EL1 TLBI form under that guest's VMID and its TGE clear, not only VMALLE1*. Untrapped, they hit VMID 0 and left the guest's own TLB stale (its first page-table updates read back old entries and hit a BUG in map_mem). - L1 writes its CPTR_EL2 through CPACR_EL1, whose hardware has no TCPAC. Without FEAT_NV2p1 KVM sets TCPAC to keep a guest hypervisor's CPACR accesses trapped; with the bit lost the guest's FP enable went stale and its kernel took its own FP trap on a SHA3 instruction. A CPU that has entered a guest hypervisor therefore switches to nested mode: all EL1 TLBIs from L1 trap (and run under the guest's VTTBR when L1's TGE is clear), and CPACR_EL1 writes trap into a shadow of L1's full CPTR_EL2 from which a guest's real CPTR_EL2 is built. The first entry's TCPAC follows the architectural rule above. Plain KVM guests pay none of this. L1's own TLBIs are no longer logged to the trace ring. Tested on J700 with kvm-arm.mode=nested in L1: a 2-vCPU Linux guest booted at its EL2 with KVM in VHE mode and ran to completion. Assisted-by: Claude Opus 5.5
Forward Apple PMU trap syndromes and switch the PMCR1 filter bank when entering and leaving nested guests. Document the T8140 fixed-counter contract and proxy evidence. Assisted-by: Codex Sol 6.1
Contributor
|
Some notes from implementing PSCI CPU_ON/CPU_OFF (over HVC, Linux at EL1 under the monitor) on M4 Max (T6041, J616C). They may save time for whoever takes this PR to M4. Two bugs that made bringing a core back kill the whole machine. Linux's offline/online looked fine until the core re-entered the kernel, and then every CPU died at once with nothing printed:
With both, offline/online works on every core, including the iBoot boot core. M4-specific things we hit:
SYSTEM_SUSPEND, in case anyone tries it: doing what XNU's system sleep does on the last core ( |
The J813 (T8142) in the kcbz-0 Mach-O mode confines its secondaries to the CTXR windows at EL2 exactly like T8140: a chainloaded Stage 2 that started them outside the boot object took an exception storm on every secondary as soon as it handed over. With the secondaries left in Stage 1 and the monitor forced, Linux reports "All CPU(s) started at EL1" and runs (network, NVMe adoption, GPU probe), so let the probe decide on T8142 too. Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
dt_set_cpus() switches the secondaries to enable-method "psci" when the monitor services PSCI, but skips the boot CPU, which keeps spin-table. cpuidle-psci initialises every present CPU and gives up when one of them does not use PSCI, so on J813 (boot CPU 0x10100) it failed with "Failed to create psci-cpuidle device". Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
J813: Linux at EL1 under the monitor, the boot CPU on PSCI (on #33)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
This branch adds the J700/T8140 Aura monitor support required to run Linux at virtual EL2 and use KVM, while preserving the normal physical-EL1 boot path.
aura-nv2or anNV2_BOOT=1development image arms the path; ordinary boots leave NV2 disabled.The Linux-side fixed-counter T8140 PMU registration and stage-2 invalidation optimization are in the companion Linux PR #731.
Validation
Proxy and J700 validation covered the monitor behavior before Linux boot, including PMU syndrome forwarding and PMCR1 restoration on both CPU clusters. The Aura release chainloading build passes, and the changed C/H files pass clang-format checks.
On J700, the NV2 image was validated with:
Measured performance
The trace-disabled NV2 runs measured 1.31–1.37 μs per in-kernel HVC exit on a P-core and 2.55–2.57 μs on an E-core. Userspace MMIO exits measured 2.68–2.70 μs on P-cores and 4.16–4.95 μs on E-cores. The ERET gate itself measured 168 ns per round trip in the proxy. These are measured NV2 costs; they are not a comparison against real-EL2 KVM or a claim of zero overhead.
Known limits
The emulated TPM reports Windows TPM/attestation errors. EFI runtime-service behavior during the diskless EDK2 shutdown path and suspend/resume at virtual EL2 remain unverified. High-rate PMU sampling in recursive guests can be throttled by Linux; the diagnostic unthrottled mode is disposable test configuration only. Unsupported T8140 programmable PMU events remain rejected rather than mapped from unverified event numbers.