Skip to content

perf(worker): Reduce peak memory in ImageProxyMiddleware proxy generation #165

Description

@bitbiter-dev

Context

Follow-up to #153 / #157. That change removed redundant decoding (3 decodes → 1 per file), which was the right call, but it traded decode time for peak memory in the same scenario #153 cited as motivation: high-resolution sources.

Problem

ImageProxyMiddleware.InvokeAsync now decodes once and hands each generator a Clone():

using var image = await Image.LoadAsync<Rgba32>(ms, ct);
...
using var clone = image.Clone();
var bytes = await generator.GenerateAsync(clone, ct);

Two compounding effects:

  1. The source buffer stays alive for the whole scope, so a full-res clone coexists with it. Each generator's Mutate(Resize(...)) downscales after the clone is already allocated at full resolution, so the saving comes too late.
  2. Rgba32 is now forced. Previously the thumbnail and preview paths used non-generic Image.LoadAsync, which decodes JPEG to Rgb24 (3 B/px). All three paths now use Rgba32 (4 B/px), because blurhash requires it.

On a 50 MP source that is ~200 MB per Rgba32 buffer, and Worker:MaxConcurrentFiles defaults to 4, so the ceiling scales 4×.

Also worth noting for anyone picking this up: Task.WhenAll over the generators buys almost no CPU parallelism, because Mutate() is synchronous and runs before the first await. The clone is required for mutation isolation — each generator resizes in place, so sharing one image would compound the resizes and generate the preview from an already-thumbnailed image. Any fix must preserve that isolation. (#157's description was corrected before merge to state this; earlier revisions of it framed the clone as being about race safety, which was wrong.)

Possible approach

Generate in a downscale chain instead of cloning the full-res source per generator: preview from the source, thumbnail from the preview, blurhash sample from the thumbnail. Only one full-res buffer would ever exist, and each subsequent step operates on progressively smaller data.

This changes proxy pixel output — a thumbnail resampled from an already-downscaled preview is not byte-identical to one resampled from the source, and quality needs checking before committing to it. That is why this is a separate issue rather than a change to #157.

Smaller cleanup to fold in

IImageProcessor.CreateThumbnailAsync / CreateFullPreviewAsync / ComputeBlurhashAsync and IImageProxyGenerator.GenerateAsync all mutate the Image<Rgba32> they are handed, and nothing in the signature communicates that. A caller that passes the shared image instead of a clone gets silent cross-proxy corruption. This warrants a doc comment on the interface under the "only when they add clear benefit" rule.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions