MitraFSnap registry

Part of MitraFSnap, Build Less, Pull Less, Restart Never: Fast Container Image Distribution and In-Place Updates. The paper is under review. Project page, videos and results

MitraFSnap updates the data mounted in a running container without replacing the container. It consists of an OCI compatible builder, this registry and a containerd snapshotter. The registry stores images whose models and assets are packaged as independent allotments, and it lets each client choose the allotments it needs first.

A client selects a partition of the image with a suffix on the tag, for example app:v2--0.0.0.1. The registry returns the complete image and marks the lazy layers of the selected allotments with a prefetch annotation. The snapshotter of the client prefetches these layers and fetches every other allotment on demand, so workers start with different working sets from the same image and can still read the rest of it.

Repository	Role in MitraFSnap
builder	builds independent, lazy loadable allotments and their chunk indexes
registry (this repository)	turns a partition selection into prefetch hints over the complete image
snapshotter	lazy pulling and refresh of running containers
demo	live demo that compares Docker, eStargz, 2DFS and MitraFSnap
evaluation	build, pull and update benchmarks

Run

The repository builds a container image that serves the registry on port 5000 with cmd/registry/config-dev.yml.

git clone https://github.com/mitrafsnap/registry.git
cd registry
docker build -t mitrafsnap-registry .
docker run -d -p 5000:5000 -v registry-data:/var/lib/registry --name registry mitrafsnap-registry

Images built with the MitraFSnap builder are pushed with tdfs image push, and clients pull them with ctr-remote images rpull from the MitraFSnap snapshotter.

Configuration

MitraFSnap adds a tdfs section to the registry configuration.

tdfs:
  stargzsupport: true    # annotate eStargz and zstd:chunked layers with their TOC digest
  prefetchsupport: true  # mark the layers of the selected partition for prefetch

With prefetchsupport disabled, the registry returns all lazy layers without prefetch hints. With stargzsupport disabled, it returns only the selected allotments, as the 2DFS registry does. The registry keeps the layer order of the image, so the chunk indexes and annotations of two versions line up.

Relation to the 2DFS registry

This repository is a fork of the 2DFS registry, which extends the CNCF distribution registry with partition selection for 2DFS images. MitraFSnap adds support for eStargz and zstd:chunked allotments, the TOC and prefetch annotations, and the tdfs configuration section. The upstream documentation is in UPSTREAM.md and docs/.

License

Apache License 2.0, as upstream. See LICENSE.