Improve README for community engagement by lastras · Pull Request #47 · generative-computing/granite-switch

lastras · 2026-05-20T02:46:55Z

Summary

Add animated SVG benchmark visualization showing adapter stacking and accuracy improvements across 3B/8B/30B models
Add aLoRA vs LoRA race screenshot with link to live race and Colab reproduction
Rewrite opening paragraph with software analogy tied to tagline
Add aLoRA value proposition (small models outperform larger generalists)
Replace inference example with guardian safety check (Mellea + vLLM)
Rewrite How It Works with technical detail (control tokens, KV cache isolation, per-position routing)
Add Ecosystem section (Granite Libraries, Mellea, Granite Switch)
Add tutorials table with direct Colab links and badges
Streamline install section (lead with single command)
Add Contributing section, live HuggingFace download badges
Soft hints that the architecture generalizes beyond Granite

Test plan

Verify SVG animation renders on GitHub
Verify race screenshot renders and links to live race page
Verify Colab badges link to correct notebooks
Verify HuggingFace badges show live download counts
Review copy for accuracy against adapter catalog data

Adds dynamic shields.io badges for corelib, raglib, and guardianlib showing monthly download counts pulled from the HuggingFace models API. Badges are placed in the top badge strip alongside License and Python version, use the HuggingFace logo, and follow the same flat/rounded style convention as the existing badges.

Parametric SVG generator (docs/generate_benchmark_svg.py) produces an animated visualization cycling through 18 slots across 3B/8B/30B model sizes. Each slot shows 4 adapters from corelib/raglib/guardianlib with real performance data extracted from the adapter catalog. LoRA/aLoRA type designations follow the catalog: prefer aLoRA when available, fall back to LoRA otherwise. Also fixes broken "Fast" link in README to point to the correct tutorial notebook path.

GitHub's SVG renderer doesn't support transform-box: fill-box, causing scaleX-based bar animations to scale from the viewport origin (x=0) instead of the element's left edge. Switch to animating the width property directly, which works universally. Also updates header to "CUSTOM GRANITE SWITCH MODEL".

- Rewrite opening to lead with action chain and aLoRA value prop - Add curated tutorials table with time estimates - Fix "high inference throughput" link to live race HTML - Update Browse Adapters link to interactive catalog - Rename "Models on HF" to "Pre-composed Models on HF" - Rephrase vLLM version note positively - Make Inference-ready bullet more concrete

Technical rewrite explains aLoRA routing mechanism: control tokens, KV cache isolation, and per-position routing. Removes Hello Adapter and aLoRA vs LoRA Race from tutorials table to focus on the three most actionable notebooks.

Remove HuggingFace example and swap query rewrite for a guardian social_bias check using Mellea — more compelling first impression of what the safety adapters can do.

Soft signals in the opening paragraph ("starting with the Granite family") and How It Works ("technique is architecture-general; Granite is the first supported family") without making commitments.

Links to the live race page. Shows aLoRA completing 20/32 queries while LoRA is still at 0/32 — visual proof of KV cache reuse.

fix table header, clarify ecosystem, add contributing section - Lead install with single pip command, collapse extras below - Bold "skip composition" callout for pre-composed models - Fix empty table header cell in tutorials - Reframe Granite Switch ecosystem bullet as "(this repo)" - Replace thin "IBM and Open Source AI" with Contributing section - Tighten license line

* Add live HuggingFace download badges for granitelib libraries Adds dynamic shields.io badges for corelib, raglib, and guardianlib showing monthly download counts pulled from the HuggingFace models API. Badges are placed in the top badge strip alongside License and Python version, use the HuggingFace logo, and follow the same flat/rounded style convention as the existing badges. * Add animated benchmark SVG to README Parametric SVG generator (docs/generate_benchmark_svg.py) produces an animated visualization cycling through 18 slots across 3B/8B/30B model sizes. Each slot shows 4 adapters from corelib/raglib/guardianlib with real performance data extracted from the adapter catalog. LoRA/aLoRA type designations follow the catalog: prefer aLoRA when available, fall back to LoRA otherwise. Also fixes broken "Fast" link in README to point to the correct tutorial notebook path. * Fix SVG rendering on GitHub: use width animation instead of scaleX GitHub's SVG renderer doesn't support transform-box: fill-box, causing scaleX-based bar animations to scale from the viewport origin (x=0) instead of the element's left edge. Switch to animating the width property directly, which works universally. Also updates header to "CUSTOM GRANITE SWITCH MODEL". * Improve README engagement: sharpen value prop, add tutorials table - Rewrite opening to lead with action chain and aLoRA value prop - Add curated tutorials table with time estimates - Fix "high inference throughput" link to live race HTML - Update Browse Adapters link to interactive catalog - Rename "Models on HF" to "Pre-composed Models on HF" - Rephrase vLLM version note positively - Make Inference-ready bullet more concrete * Strengthen opening paragraph: tie tagline to software analogy * Add Ecosystem section: Granite Libraries, Mellea, and Granite Switch * Rewrite How It Works section and trim tutorials table Technical rewrite explains aLoRA routing mechanism: control tokens, KV cache isolation, and per-position routing. Removes Hello Adapter and aLoRA vs LoRA Race from tutorials table to focus on the three most actionable notebooks. * Replace Run Inference example with guardian safety check Remove HuggingFace example and swap query rewrite for a guardian social_bias check using Mellea — more compelling first impression of what the safety adapters can do. * Link tutorials directly to Colab and add badge column * Hint that the architecture generalizes beyond Granite Soft signals in the opening paragraph ("starting with the Granite family") and How It Works ("technique is architecture-general; Granite is the first supported family") without making commitments. * Add aLoRA vs LoRA race screenshot below benchmark SVG Links to the live race page. Shows aLoRA completing 20/32 queries while LoRA is still at 0/32 — visual proof of KV cache reuse. * Move race screenshot below Key Features section * Add caption and Colab CTA below race screenshot * Clarify race caption: different adapter technology * Polish README: streamline install, promote pre-composed models, fix table header, clarify ecosystem, add contributing section - Lead install with single pip command, collapse extras below - Bold "skip composition" callout for pre-composed models - Fix empty table header cell in tutorials - Reframe Granite Switch ecosystem bullet as "(this repo)" - Replace thin "IBM and Open Source AI" with Contributing section - Tighten license line * Fix How It Works opener: clarify LoRA's KV cache limitation * Improve accuracy of aLoRA description in How It Works * Remove SVG generator script (kept in fork only) --------- Co-authored-by: Luis A Lastras <lastrasl@us.ibm.com>

…/x3) CI-runnable serving smoke closing the #47 e2e box. Boots a real composed, audio-enabled GraniteSwitch checkpoint under vLLM and drives text-only, adapter-control-token, and audio x1/x2/x3 requests through one live engine. Markers slow+requires_model+gpu (opt-in via -m); model built through the compose CLI with --enable-audio; synthetic audio keeps it asset-free. Verified green on an A100 with ibm-granite/granite-4.1-3b + granitelib-core-r1.0: 5 passed.

…/x3) CI-runnable serving smoke closing the #47 e2e box. Boots a real composed, audio-enabled GraniteSwitch checkpoint under vLLM and drives text-only, adapter-control-token, and audio x1/x2/x3 requests through one live engine. Markers slow+requires_model+gpu (opt-in via -m); model built through the compose CLI with --enable-audio; synthetic audio keeps it asset-free. Verified green on an A100 with ibm-granite/granite-4.1-3b + granitelib-core-r1.0: 5 passed. Signed-off-by: BAR HAIM <barha@il.ibm.com>

Adds the first correctness test for the "adapters + audio together" #47 box: the RAG answerability adapter judging a query against a document supplied as speech rather than text. The document is delivered as audio by placing a single <|audio|> marker as the document text (documents=[{"text": "<|audio|>"}]); the Granite template renders each document via `doc | tojson`, so the marker lands in the <documents> block exactly where vLLM's ASR processor splices the transcript — the document the adapter reasons over is the transcribed speech. The <|answerability|> control token is inserted by the same template (aLoRA fallback path), so the switch fires the adapter at the generation prompt. Ground truth comes from one real clip (hf-internal-testing/ librispeech_asr_dummy row 0), driving both classes without shipping a fixture: an answerable question whose answer is spoken, and an unanswerable one about content the clip never mentions. Both are distinctive enough to survive ASR word-errors, so the assertion tests the answerability decision, not transcription accuracy (WER is a separate box). FLAC is decoded with soundfile from the raw bytes to avoid requiring torchcodec (datasets>=4). Where test_audio_serving_smoke.py only proves an adapter control token doesn't crash the audio path, this proves the switch produces the correct verdict end-to-end through a live vLLM engine. Verified on an A100 with granite-4.1-3b + granitelib-rag-r1.0: 2 passed. Markers slow+requires_model+gpu; opt-in via pytest -m "slow and requires_model and gpu" Signed-off-by: BAR HAIM <barha@il.ibm.com>

Adds the 'all default adapters route correctly with audio enabled' coverage for issue #47. Composes the full default set (RAG + Core + Guardian) with --enable-audio and asserts the switch maps each adapter_token_ids[i] to index i+1 (pre-control positions stay on base 0), observed via the HF backend's _last_adapter_indices. For granite-4.1-3b, 12 adapters resolve (context_relevance has no 4.1-3b flavor); verified 12/12 route correctly on A100. Loads with ignore_mismatched_sizes=True: the <|audio|> token bumps vocab_size one past the switch's control_to_substitute_lut buffer, which SingleSwitch rebuilds from config, so the freshly-built buffer is kept. Signed-off-by: BAR HAIM <barha@il.ibm.com>

* tests: cover audio/adapter coexistence, chat-template marker, empty clips, clip ceiling Signed-off-by: BAR HAIM <barha@il.ibm.com> * tests: add e2e vLLM audio serving smoke (text + adapter + audio x1/x2/x3) CI-runnable serving smoke closing the #47 e2e box. Boots a real composed, audio-enabled GraniteSwitch checkpoint under vLLM and drives text-only, adapter-control-token, and audio x1/x2/x3 requests through one live engine. Markers slow+requires_model+gpu (opt-in via -m); model built through the compose CLI with --enable-audio; synthetic audio keeps it asset-free. Verified green on an A100 with ibm-granite/granite-4.1-3b + granitelib-core-r1.0: 5 passed. Signed-off-by: BAR HAIM <barha@il.ibm.com> * tests: e2e answerability adapter over an audio-delivered document (#47) Adds the first correctness test for the "adapters + audio together" #47 box: the RAG answerability adapter judging a query against a document supplied as speech rather than text. The document is delivered as audio by placing a single <|audio|> marker as the document text (documents=[{"text": "<|audio|>"}]); the Granite template renders each document via `doc | tojson`, so the marker lands in the <documents> block exactly where vLLM's ASR processor splices the transcript — the document the adapter reasons over is the transcribed speech. The <|answerability|> control token is inserted by the same template (aLoRA fallback path), so the switch fires the adapter at the generation prompt. Ground truth comes from one real clip (hf-internal-testing/ librispeech_asr_dummy row 0), driving both classes without shipping a fixture: an answerable question whose answer is spoken, and an unanswerable one about content the clip never mentions. Both are distinctive enough to survive ASR word-errors, so the assertion tests the answerability decision, not transcription accuracy (WER is a separate box). FLAC is decoded with soundfile from the raw bytes to avoid requiring torchcodec (datasets>=4). Where test_audio_serving_smoke.py only proves an adapter control token doesn't crash the audio path, this proves the switch produces the correct verdict end-to-end through a live vLLM engine. Verified on an A100 with granite-4.1-3b + granitelib-rag-r1.0: 2 passed. Markers slow+requires_model+gpu; opt-in via pytest -m "slow and requires_model and gpu" Signed-off-by: BAR HAIM <barha@il.ibm.com> * tests: use committed local speech clip for answerability-over-audio Replace the runtime hf-internal-testing/librispeech_asr_dummy download with a committed 3s 16 kHz WAV (SpeechT5, MIT-licensed) so the test runs offline and carries clean provenance for merge-to-main. The spoken sentence states a location that distil-whisper transcribes cleanly, so the questions key on the location rather than the proper noun. Verdicts verified: 2 passed on A100 (granite-4.1-3b + granitelib-rag-r1.0). Signed-off-by: BAR HAIM <barha@il.ibm.com> * tests: sweep all default adapters route correctly with audio enabled Adds the 'all default adapters route correctly with audio enabled' coverage for issue #47. Composes the full default set (RAG + Core + Guardian) with --enable-audio and asserts the switch maps each adapter_token_ids[i] to index i+1 (pre-control positions stay on base 0), observed via the HF backend's _last_adapter_indices. For granite-4.1-3b, 12 adapters resolve (context_relevance has no 4.1-3b flavor); verified 12/12 route correctly on A100. Loads with ignore_mismatched_sizes=True: the <|audio|> token bumps vocab_size one past the switch's control_to_substitute_lut buffer, which SingleSwitch rebuilds from config, so the freshly-built buffer is kept. Signed-off-by: BAR HAIM <barha@il.ibm.com> --------- Signed-off-by: BAR HAIM <barha@il.ibm.com>

Luis A Lastras added 18 commits May 19, 2026 22:45

Strengthen opening paragraph: tie tagline to software analogy

f053182

Add Ecosystem section: Granite Libraries, Mellea, and Granite Switch

5e55c42

Rewrite How It Works section and trim tutorials table

8db8057

Technical rewrite explains aLoRA routing mechanism: control tokens, KV cache isolation, and per-position routing. Removes Hello Adapter and aLoRA vs LoRA Race from tutorials table to focus on the three most actionable notebooks.

Replace Run Inference example with guardian safety check

cd48558

Remove HuggingFace example and swap query rewrite for a guardian social_bias check using Mellea — more compelling first impression of what the safety adapters can do.

Link tutorials directly to Colab and add badge column

8b8f2e5

Hint that the architecture generalizes beyond Granite

c5509da

Soft signals in the opening paragraph ("starting with the Granite family") and How It Works ("technique is architecture-general; Granite is the first supported family") without making commitments.

Add aLoRA vs LoRA race screenshot below benchmark SVG

e6aa158

Links to the live race page. Shows aLoRA completing 20/32 queries while LoRA is still at 0/32 — visual proof of KV cache reuse.

Move race screenshot below Key Features section

e71d05a

Add caption and Colab CTA below race screenshot

d67d549

Clarify race caption: different adapter technology

62d8bc0

Fix How It Works opener: clarify LoRA's KV cache limitation

263804f

Improve accuracy of aLoRA description in How It Works

f9dbf28

Remove SVG generator script (kept in fork only)

ea7c9b3

lastras requested review from antonpibm, freunda and yairallouche as code owners May 20, 2026 02:46

antonpibm approved these changes May 20, 2026

View reviewed changes

antonpibm merged commit 85f8bed into generative-computing:main May 20, 2026

Navigation Menu

Uh oh!

Improve README for community engagement - #47

Improve README for community engagement#47
antonpibm merged 18 commits into
generative-computing:mainfrom
lastras:pr/readme-improvements

lastras commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

lastras commented May 20, 2026

Summary

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants