Repository navigation
0.15.0 — PNM, PAM and PFM input, metadata for animations, four conversions brought to parity - #61
Merged
Merged
Conversation
Binary graymaps (P5), pixmaps (P6) and PAM (P7) are now an input format, sniffed by content like every other. - crates/skidbladnir-encode/src/pnm.rs ports libwebp 1.6.0's imageio/pnmdec.c rule for rule: the line-at-a-time header, whole-line comments, the PAM fields and TUPLTYPEs it knows, and the 8-bit rescale (v * 255 + maxval / 2) / maxval. What cwebp refuses (P1-P4, a one-line header, a truncated raster) is refused too. - jxl.rs: a PNM goes to libjxl as cjxl's lib/extras/dec/pnm.cc hands it over: the samples at log2(maxval + 1) bits, read with JXL_BIT_DEPTH_FROM_CODESTREAM, sRGB with the perceptual intent. A maximum cjxl refuses (not 2^n - 1) is encoded like any format cjxl does not read. - source.rs, metadata.rs: SourceFormat::Pnm; a PNM carries no metadata. - frontend/pages/index.vue: the picker lists .pnm, .pgm, .ppm and .pam. Gate: tests/pnm.rs — 11 fixtures (8/16-bit, maxval 15/1000/1023, gray, gray+alpha, RGBA, comments, no TUPLTYPE) x 13 WebP settings: 143 of 143 byte-identical to cwebp 1.6.0; 20 of 20 byte-identical to cjxl 0.12.0; cwebp's refusals checked against cwebp. Mutation: dropping the rescale's rounding fails 52 of 143; the relative intent and pixel-format bit depth (the first attempt) failed all 20 cjxl cases. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…-metadata A GIF's colour profile and XMP live in application extensions (ICCRGBG1012, XMP DataXMP). gif2webp -metadata copies them; Skidbladnir dropped them. - gif_input.rs: metadata() walks the GIF's blocks and reads the first extension of each kind the way gifdec.c's GIFReadMetadata does: a profile is its sub-blocks' data; an XMP packet keeps each sub-block's length byte (GIF's XMP encoding writes the packet raw) and loses its 257-byte magic trailer. - animation.rs: with_metadata() re-muxes the encoded animation with libwebp's mux, ICCP then XMP, as gif2webp does. - source.rs: AnimatedSource carries the metadata and encodes itself, so conversion and preview (src-tauri/src/preview.rs) agree. The window's WebP "Copy ICC profile" / "Copy XMP" toggles are gif2webp's -metadata icc / xmp for a GIF; Exif is not a gif2webp choice. An animated WebP source copies none, as img2webp. Gate: tests/gif.rs matches_reference_gif2webp_keeping_metadata — four GIFs (ICC and XMP before the frames, after the first frame, two of each, ICC only; a 700-byte profile over three sub-blocks) x five choices, whole encode_file conversions: 20 of 20 byte-identical to gif2webp 1.6.0 -metadata, which is checked to really write the chunks. Mutation: keeping the trailer fails 6, last-wins fails 2. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
cjxl reads PFM, Netpbm's 32-bit float form, and keeps the floats.
- pnm.rs: pfm() reads PF (colour) and Pf (gray) as libjxl 0.12's
lib/extras/dec/pnm.cc does: its single-whitespace header grammar, the
scale's sign as the byte order (its size ignored), rows stored bottom
to top. pfm_rgba() reduces it to 8- and 16-bit RGBA for the other
encoders, clamped to the nominal 0..1; no other reference tool reads
PFM, so there is nothing else to match.
- jxl.rs: a PFM goes to libjxl as 32-bit float samples (JXL_TYPE_FLOAT,
8 exponent bits), sRGB with the perceptual intent, as cjxl hands it
over. A cropped or resized PFM encodes from the 16-bit reduction.
- source.rs: SourceFormat::Pfm, sniffed by content. The picker lists .pfm.
Gate: tests/pnm.rs matches_cjxl_reading_pfm — big- and little-endian
colour, gray, a non-unit scale, samples slightly outside 0..1, x three
settings: 12 of 12 byte-identical to cjxl 0.12.0. Mutation: reading the
rows top-down fails 12 of 12. Found: libjxl 0.12.0 fails a lossless
float encode at effort 3 ("Residual overflow", enc_encoding.cc:314) —
in cjxl too; the test pins that both sides refuse it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…IJG allowance - cargo update: tauri 2.12.0 -> 2.12.1 (with tauri-build/-codegen/-macros 2.7.1, -runtime(-wry) 2.12.1, -utils 2.10.1), tauri-plugin-dialog 2.7.3 -> 2.8.1, tauri-plugin-fs 2.6.0, tauri-plugin-updater 2.13.1, cc 1.5.1 and smaller bumps. Covers Dependabot's #58 and #59. - npm update in frontend/: @tauri-apps/api and cli 2.12.1, @tauri-apps/plugin-dialog 2.8.1 (Dependabot's #56), @nuxt/ui 4.11.3, @iconify-json/lucide 1.2.138 (#57). - deny.toml: the IJG licence allowance named mozjpeg, which 0.14.0 replaced with the libjpeg-turbo submodule; cargo deny reported it as unmatched. libjpeg-turbo is built from source, not a crate, and its notice stays in THIRD-PARTY-NOTICES.md. Verified: npm run generate; cargo test --workspace 246 passed, 0 failed, with every parity suite required (SKIDBLADNIR_REQUIRE_PARITY=1) against references built by scripts/build-reference-tools.sh; clippy -D warnings; cargo deny check (advisories, bans, licences, sources ok); the release binary (cargo tauri build --no-bundle) passes scripts/smoke-test.sh. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
scripts/smoke-test.sh writes a P6 pixmap, a 16-bit RGBA PAM and a little-endian PFM, and checks in the running app that each is recognised by content (inspect_dropped_paths) and converts to WebP and to JPEG XL at its own size. Passes against the 2.12.1 release binary, with the app's own libheif beside it as the installers place it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…measurement; research ROADMAP.md: - Phase 4: AVIF ICC/XMP ticked (closed by 0.14.0's libavif port, left unticked); GIF ICC/XMP in animated WebP ticked; animated-WebP-source metadata split out as its own open item. - Phase 7: the Windows-CI stack overflow, measured on Linux (each test body needs under 50 KiB of stack, debug or optimised C), with the failing run's evidence: only lossless-decode tests, and LNK4098 mixed C runtimes. New item: one C runtime on Windows. Tauri 3 re-checked (3.0.0-alpha.4); libwebp re-checked (1.6.0). New: drop macos-private-api, which Tauri 2.12.1 no longer needs for transparency (tauri-apps/tauri#16166). New: update the README when Chrome 155 enables JPEG XL by default (jpegxl.com/news). The old 'prepare JPEG XL' item ticked as superseded. - Phase 8: PNM/PAM and PFM input ticked; libjxl 0.12.0's effort-3 lossless float failure recorded; the remaining input formats listed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ty cases The input list gains PNM/PAM and PFM (with how each tool reads them and that PFM to formats other than JPEG XL has no reference); the parity table counts this run's cases (WebP 2,397 with 143 PNM conversions; animated WebP 87 with 20 gif2webp -metadata conversions; JPEG XL 234 with 20 PNM and 12 PFM); the metadata gap now says what is kept for animations. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…p does A TIFF whose alpha is associated (premultiplied; ExtraSamples = [1]) came out darker than cwebp's conversion of it: the image crate hands back the stored, premultiplied samples and they were encoded as straight colour. 272 of 276 conversions differed from cwebp. cwebp gets the samples from libtiff as stored and un-multiplies them itself (imageio/tiffdec.c, MultARGBRow: 24-bit fixed point, black under alpha 0). source.rs now does exactly that for such a TIFF, and un-multiplies the 16-bit samples the other encoders take, exactly. Gate: tests/parity.rs matches_reference_cwebp_through_an_associated_alpha_tiff — TIFFs written byte by byte in the test (8- and 16-bit, every alpha value present) x the 138-setting surface: 276 of 276 byte-identical to cwebp 1.6.0. Mutation: plain rounded division fails 79; no un-multiply fails 272. The unassociated-alpha pin and the TIFF gate still pass. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Windows test binaries linked libjxl and native/heic_jpeg.cc against the static runtime (libcmt) and Rust plus the rest against the dynamic one, and the linker said so on every binary: "LNK4098: defaultlib 'libcmt.lib' conflicts with use of other libs" (run 36521408006, the run whose animation tests overflowed their stacks). Two runtimes in one process mean two heaps and two sets of CRT state. - .cargo/config.toml: +crt-static for x86_64-pc-windows-msvc, so Rust and the C cc compiles (libwebp, the AVIF and HEIC shims) use the static runtime too. No other target is affected. - crates/skidbladnir-encode/build.rs: libjpeg-turbo, libaom and libavif set CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded under it, since CMake's default (CMP0091) is the dynamic runtime whatever flags cmake-rs passes. - libheif stays a DLL with its own runtime; nothing crosses between them. Verified here: Linux is unchanged (cargo test --workspace 247 passed with every parity suite required; clippy; fmt). NOT verified here: Windows, which only CI can build — the rust (standard and GPL) and window-smoke (installed NSIS app) jobs on this branch are the gate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
cjxl reads GIF with its own reader (lib/extras/dec/gif.cc) and counts it a lossy input, like JPEG: with no -d or -q it encodes losslessly, and it labels the pixels sRGB with the perceptual intent. Skidbladnir sent a GIF down the generic path (relative intent, distance 1), so its JPEG XL differed from cjxl's in every case, and was lossy where cjxl's is lossless. jxl.rs: Input::gif — the generic input with cjxl's GIF colour and lossless default. The pixels are already what cjxl's reader makes of a still GIF (transparent index as 0,0,0,0; alpha only when present). Gate: tests/gif.rs matches_cjxl_reading_a_still_gif — four still GIFs x five settings, including no target at all: 20 of 20 byte-identical to cjxl 0.12.0. Mutation: the generic path fails 16; the distance-1 default fails 8. Workspace: 248 passed with every parity suite required. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
src-tauri/Cargo.toml is the product version's only copy (tauri.conf.json inherits it). Minor: new input formats (PNM/PAM, PFM), a GIF's ICC and XMP kept, and two conversions that now match their reference tool (premultiplied-alpha TIFF to WebP, still GIF to JPEG XL). tauri.conf.json's longDescription still listed the 0.6-era controls. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The parity job names its test files, and tests/pnm.rs (PNM/PAM against cwebp and cjxl, PFM against cjxl) was not among them, so CI would have skipped it rather than required it. The still-GIF-to-JPEG-XL gate is in tests/gif.rs, which the job already runs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Re-encoding an animated WebP dropped its metadata whatever the WebP panel's toggles said. img2webp, the reference for the re-encode, has no -metadata; the route a command-line user takes is re-encode, then webpmux -set icc|exif|xmp, so webpmux is the reference. - source.rs: an animated WebP's ICCP, EXIF and XMP chunks travel with it (metadata::webp_chunks, as cwebp reads a still WebP's) and are set back under the toggles. - animation.rs: with_metadata sets EXIF too. - scripts/build-reference-tools.sh builds webpmux beside cwebp from the same libwebp; CI's parity job gets SKIDBLADNIR_REFERENCE_WEBPMUX. Gate: tests/animation.rs keeps_an_animations_metadata_as_webpmux_sets_it — a source made with webpmux -set, five choices x lossless/lossy: 10 of 10 byte-identical to re-encoding then webpmux -set. Mutation: dropping Exif fails 4. Workspace: 249 passed with every parity suite required. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
heif-enc reads TIFF through libtiff (heifio/decoder_tiff.cc), and the HEIC gate only ever compared PNG and JPEG inputs. tests/heic_parity.rs matches_heif_enc_across_tiff_inputs: 8-bit RGB (plain and LZW) and straight-alpha RGBA x two settings, 6 of 6 byte-identical; nothing needed changing. ROADMAP records an owner decision this exposes: heif-enc takes premultiplied TIFF alpha as plain alpha, where Skidbladnir now un-multiplies it for every format. README counts 153 HEIC cases. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
heif-enc's WebP reader (heifio/decoder_webp.cc) decodes a lossy WebP straight into YCbCr 4:2:0 planes and alpha (WebPDecode into MODE_YUV(A)) and encodes those, never touching RGB. Skidbladnir decoded to RGB and let libheif convert back, so every lossy WebP converted to HEIC differed from heif-enc's (4 of 8 in the new gate; the lossless ones already matched). - encoder.rs: webp_yuv420_planes — the YUV decode the cwebp route already uses (yuv_picture), as tightly packed planes. - heic.rs: Layout::Yuv420 and Input::webp — a lossy, untouched WebP goes to the shim as its own planes with the file's ICCP/EXIF/XMP; a lossless, cropped or resized one, or 10-bit (GPL) output, goes through RGB. - native/heic_shim.c: input layout 2 builds the heif_image as decoder_webp.cc does (Y, Cb, Cr at 4:2:0, then alpha, not premultiplied). Gate: tests/heic_parity.rs matches_heif_enc_across_webp_inputs — lossless and lossy, with and without alpha, plus an odd-sized lossy one: 9 of 9 byte-identical to heif-enc -e kvazaar 1.23.5. Workspace: 251 passed with every parity suite required. The GPL edition shares the path and is built and tested by CI only. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…f and XMP heif-enc reads a HEIF input with libheif (heifio/decoder_heif.cc): the primary image decoded in its own colourspace and chroma, its NCLX passed through and its transformations applied, and its first Exif and XMP blocks kept. Skidbladnir decoded a HEIC to RGB and encoded that, and dropped the source's Exif and XMP on the way. - native/heic_shim.c: load_heif makes those libheif calls on the file's bytes, as the JPEG path hands a JPEG to heif-enc's JPEG reader; the window's Exif and XMP toggles still leave the blocks out. - heic.rs: an untouched HEIC (not cropped or resized, 8-bit output, ICC kept — the reader cannot drop a profile) goes that way; anything else through RGB as before. Gate: tests/heic_parity.rs matches_heif_enc_across_heic_inputs — sources written by heif-enc (RGB, RGBA, gray, an odd size, every kind of metadata) x two settings: 10 of 10 byte-identical, and the Exif/XMP toggles checked to keep and drop the blocks. Mutation: the old route fails it. Workspace: 252 passed with every parity suite required. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both go through heif-enc's own readers since this run (YCbCr planes for a lossy WebP, a native-colourspace decode for a HEIC). Passes against the 0.15.0 release binary with the app's libheif beside it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The previous commit's +crt-static never reached CI: a RUSTFLAGS environment variable replaces .cargo/config.toml's target rustflags, and CI sets RUSTFLAGS=-D warnings for every job — so the Windows jobs went green on the dynamic runtime, and the linker still warned LNK4098 three times. release.yml sets no RUSTFLAGS, so a release would have shipped the runtime switch untested. The rust job's Windows legs and the Windows window-smoke job now set RUSTFLAGS='-D warnings -C target-feature=+crt-static'. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…; no unverified CRT claim ROADMAP: the owner-decision item now records the run — heif-enc encodes a premultiplied (100,50,25)@128 TIFF pixel, true colour (200,100,50), as (100,50,25)@128. .cargo/config.toml no longer claims the Visual C++ redistributable is unneeded; that was never tested. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Routine run 2026-10-01 (started 09:21 CDT). Ten increments, version 0.15.0: new input formats (PNM/PAM, PFM), metadata kept for animations, and four conversions that differed from their reference tool now match it byte for byte.
How this run was verified
CARGO_TARGET_DIRon tmpfs. It was deleted at the end of the run.scripts/build-reference-tools.sh, built from the pinned sources this run:cwebp/img2webp/gif2webp/webpmux1.6.0,avifenc1.4.2,cjxl0.12.0 andheif-enc1.23.5.cargo test --workspacegave 252 passed, 0 failed, 0 skipped, withSKIDBLADNIR_REQUIRE_PARITY=1, so every parity suite had to run. Clippy-D warningsis clean, nightly rustfmt is clean,cargo denyis ok, andnpm run generateis green.cargo tauri build --no-bundle) passesscripts/smoke-test.sh, every check including this run's new ones. The binary has the app's own libheif beside it, as the installers place it; a bare binary otherwise loads the system's libheif, which has no Kvazaar. The axe audit finds nothing at or above serious.Increments
cwebpandcjxlread it (c54fc29). Phase 8.src/pnm.rsports libwebp'simageio/pnmdec.crule for rule:(v*255 + maxval/2)/maxval;cwebprefuses is refused too.jxl.rs, a PNM reaches libjxl the waycjxl'spnm.cchands it over: atlog2(maxval+1)bits, withJXL_BIT_DEPTH_FROM_CODESTREAM, as perceptual sRGB.tests/pnm.rs):cwebp, from 11 fixtures × 13 settings;cjxl;cwebp's refusals are checked againstcwebpitself.cjxlcases (relative intent, pixel-format bit depth) until it readpnm.cc.gif2webp -metadatakeeps them (fe72d0a). Phase 4.gif_input::metadatamirrorsgifdec.c'sGIFReadMetadata: the first extension of each kind wins; XMP keeps the sub-block length bytes and loses the 257-byte trailer.animation::with_metadatare-muxes with libwebp's mux.gif2webp -metadata. Mutation: keeping the trailer fails 6; last-wins fails 2.cjxlreads it (de1a799): 32-bit floats, bottom-up rows, the byte order taken from the sign of the scale.cjxl. Mutation: reading rows top-down fails 12/12.cjxlitself fails the same way. This is libjxl's open #4902. The gate pins that both sides refuse it.c6856f0):deny.toml's stale IJG allowance (mozjpeg is gone).a383b15,f3b0083): new checks for PNM, PAM and PFM input, and for lossy WebP → HEIC and HEIC → HEIC, in the running app.2176ec8). Phase 2.cwebp's. We encoded premultiplied samples as straight colour;cwebpun-multiplies them (MultARGBRow, 24-bit fixed point).cjxl(732cfca).cjxlcounts GIF as a lossy input, so its default is lossless, and it labels it perceptual sRGB. Every case differed before.3d5eedf). Phase 4.webpmux -set, becauseimg2webphas no-metadata.webpmuxjoins the reference tools, both in the build script and in CI.heif-enc(aa500d4,ea388b4,662f298):heif-encdecodes it straight to YCbCr 4:2:0 planes; we went through RGB, so 4 of 8 differed. Now 9/9, with a third input layout in the shim.heif-encdecodes in the file's native colourspace and keeps Exif/XMP; we went through RGB and dropped the source's Exif and XMP. The shim now loads the file asdecoder_heif.ccdoes: 10/10, and the toggles are checked directly.1b77778,c5dfd2e). Phase 7.LNK4098). Everything is now on the static runtime:.cargo/config.toml+crt-static, plusCMAKE_MSVC_RUNTIME_LIBRARYfor libjpeg-turbo, libaom and libavif.RUSTFLAGSsilently overrides the config's rustflags, so the first CI run of this was a no-op. The rust and window jobs now pass the flag themselves. Without that,release.yml, which sets noRUSTFLAGS, would have shipped the switch untested.LNK4098appears 0 times (it was 3);Also in this PR:
ci.yml's parity job now runs--test pnm. The PNM/PFM gates would otherwise not have been enforced in CI.longDescriptionis brought up to date.Parity cases: 2,861 → 3,387.
Done vs open
[ ]:macos-private-apiheif-encdoes not un-multiply; confirmed by running it)patchelf(owner-gated)Research folded into ROADMAP.md (sources read this run)
macos-private-apifor transparency or fullscreen (fix: enable macos private apis regardless of feature flags tauri-apps/tauri#16166). New item to drop the feature once macOS can be launched. https://github.com/tauri-apps/tauri/releases/tag/tauri-v2.12.1STOP REASON: no workable next item. The PR opened 3h08m in and the run closes past the 4h mark with the merge and the release. Every remaining item is one of:
patchelf, SVG;macos-private-api;avifenc/heif-encare path-based C, so even the first slice needs a design decision; raw YUV needs new UI; animation output was deferred by the owner;cjxlreads samples against the full 16-bit range.Recommended next step:
avifenc/heif-enc(the shims would need path-based readers, or a memory port ofy4m.c).🤖 Generated with Claude Code