[llm][4/4] llama_main CLI flags + chat_formatter wrapper + universal Jinja docs

[seyeong-han]

Summary

Part 4 of the chat-template support stack split out of #16987 per @kirklandsign's request. This is the user-facing surface that wires everything together — the CLI flags for llama_main and the universal-Jinja documentation.

Stack overview

PR	Subject
1/4	#19533 Library + tests
2/4	#19534 TextLLMRunner echo gating + EOS merge
3/4	#19535 Python bindings + LlamaRunner integration
4/4 (this PR)	llama_main CLI flags + chat_formatter wrapper + universal Jinja docs

What this PR adds

Example-local ChatFormatter (examples/models/llama/runner/chat_formatter.h)

ChatFormatter abstraction with NoChatFormatter and a JinjaChatFormatterAdapter wrapping executorch::extension::llm::JinjaChatFormatter.

• parse_chat_format is case-insensitive and trims whitespace, so "Llama3", " llama3 ", "LLAMA3" all map correctly.
• create_chat_formatter throws std::invalid_argument when chat_format=jinja is passed without --chat_template_file (no more silent no-op — addresses the Copilot review comment from [llama] Add chat format support for Llama 3 Instruct models #16987).

CLI flags (examples/models/llama/main.cpp)

Adds:

• --chat_format=llama3|gemma3|jinja|none (default: none)
• --chat_template_file=<path> (any HF / vLLM Jinja template)
• --system_prompt="<text>"
• --echo=true|false (wired to GenerationConfig.echo)

Wraps the prompt with the chat formatter, catches invalid_argument / std::exception from formatter creation with clear error messages.

Build wiring

• examples/models/llama/runner/CMakeLists.txt + targets.bzl: link llama_runner against jinja2cpp (transitive include in chat_formatter.h)
• examples/models/llama/CMakeLists.txt: a guarded FetchContent_Declare(jinja2cpp) so the example builds standalone (when the parent build hasn't already added jinja2cpp via extension/llm/chat_template), without redeclaring when it has

Universal Jinja docs

• examples/models/llama/README.md: documents the new flags AND the recommended workflow of passing any HuggingFace / vLLM Jinja template via --chat_template_file
• extension/llm/runner/README.md: documents universal Jinja support — points at vLLM examples and HF tokenizer_config.json files as supported template sources

Universal Jinja example

After this PR, users can drop in any chat template — no recompile required:

# Use any vLLM template:
curl -L -o llama3.2.jinja \
  https://raw.githubusercontent.com/vllm-project/vllm/main/examples/tool_chat_template_llama3.2_pythonic.jinja

cmake-out/examples/models/llama/llama_main \
  --model_path=<model.pte> \
  --tokenizer_path=<tokenizer.json> \
  --chat_template_file=llama3.2.jinja \
  --system_prompt="You are a helpful assistant." \
  --echo=false \
  --prompt="What is the capital of France?"
# Output: The capital of France is Paris.

Why this is split out

This is the user-facing CLI integration that depends on PRs A and C. It's the most reviewable in isolation since it's example code with lower blast radius — reviewers can focus on the CLI ergonomics and docs without re-reading library internals.

Sample vLLM templates are NOT checked in (per @metascroy's reviewer feedback); documentation here points users to vLLM's examples/ directory and HuggingFace tokenizer_config.json files, which the universal Jinja support handles directly.

Test Plan

• Build with cmake --workflow llm-release
• Build with make llama-cpu
• Test --chat_format=llama3 with Llama-3.2-1B-Instruct
• Verify generation stops at <|eot_id|> token
• Test --echo=false produces clean output without special tokens
• Test --echo=true produces raw output WITH special tokens
• Test --system_prompt affects model behavior
• Test --chat_template_file=<vLLM template> works for arbitrary HF/vLLM Jinja files
• Test --chat_format=jinja without --chat_template_file now produces a clear error (not silent no-op)
• Test --chat_format="Llama3" / " llama3 " (case-insensitive + trim) parses correctly
• Backward compatible with --chat_format=none (default)
• Standalone build of examples/models/llama works (FetchContent guard does not redeclare)

Depends on

• PR-A: #19533 — JinjaChatFormatter library
• PR-C: #19535 — Python LlamaRunner (independent runtime path; this PR's chat_formatter.h is C++-only)

Original PR

Splitting #16987 into 4 reviewable PRs. Once this stack lands, #16987 will be closed as superseded.

cc @kirklandsign @larryliu0820 @metascroy

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19536

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.