Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator

[qti-horodnic]

Summary

This PR adds the pipeline orchestrator (GenAIPipeline) that wires together all the components from PRs 1 and 2. It assembles stages from EngineProxy, builds InputConfig objects from OutputConfig results, executes stages sequentially with timing, and returns InferenceOutputConfig. No existing files are modified.

What's included

Pipeline orchestrator:

• GenAIPipeline: main orchestrator class with:

  • _STRATEGY_REGISTRY: maps (stage_name, engine_type) → (StageClass, StrategyClass) for extensibility
  • from_proxy(): factory method that resolves stages from EngineProxy
  • invoke(): executes model_preparation → quantization → compilation → inference
  • Private _run_* methods: each builds an InputConfig, calls the stage, returns the OutputConfig
  • Structured logging with [GenAIPipeline] StageName started/completed in X.Xs format per LLD Section 5.2

• executorch_model_preparation_strategy.py: ExecuTorch model preparation strategy stub (implementation in a subsequent PR)

Unit tests (13 tests):

• from_proxy(): creates all stages, skip stages, default engines
• from_proxy() error paths: unsupported engine for each stage
• invoke(): full pipeline with mock strategies, compile-only, no stages
• Data wiring: quantization receives soc_model, compilation receives backend_type, inference receives prompt

PR Review Checklist

• All new classes follow single responsibility (one class per file) - Yes.
• All dependencies are injected via constructor with sensible defaults - Yes.
• All external calls are behind injectable interfaces - Yes (via strategy pattern).
• Unit tests cover every public method - Yes.
• No existing files are modified (Phase 1 constraint) - Yes (only __init__.py updated to add GenAIPipeline export).
• Docstrings on all public classes and methods - Yes.
• Type annotations on all function signatures - Yes.
• Logging follows the strategy in the LLD - Yes ([GenAIPipeline] prefix, timing).

Related PRs

• PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
• PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795.
• PR 3: Pipeline orchestrator: this pr.
• PR 4: Adapter interfaces & default implementations: pending.
• PR 5: Model preparation & quantization strategies: pending
• PR 6: Compilation & inference strategy implementations: pending.
• PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/test_genai_pipeline.py \
  -v

All 13 unit-tests passed.

Test Coverage

Command to run:

coverage run --rcfile=backends/qualcomm/.coveragerc \
  -m pytest \
    backends/qualcomm/genai_pipeline/tests/test_genai_pipeline.py \
  -v

coverage report --rcfile=backends/qualcomm/.coveragerc \
  --include="backends/qualcomm/genai_pipeline/genai_pipeline.py"

Result:

Name                                                 Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/genai_pipeline.py      98      7     12      1    93%   184-195
------------------------------------------------------------------------------------------------
TOTAL                                                   98      7     12      1    93%

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21149

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 3 Unrelated Failures

As of commit 428f08e with merge base a439afd ():

NEW FAILURES - The following jobs have failed:

• pull / test-lora-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5af85147fab98632afeabc796e140b30f6190593dd049261ca8fd6c1daeefc51 /exec failed with exit code 1
• pull / test-lora-multimethod-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8ab9739bdca18d02095634c4e584d68c9045cd7cb13adc2f537c95a9169574ed /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / linux / linux-job (gh) (trunk failure)
  extension/llm/modules/test/test_turboquant_kv_cache.py::TestCodebook::test_rotation_varies_with_seed
• pull / unittest-editable / linux / linux-job (gh) (trunk failure)
  extension/llm/modules/test/test_turboquant_kv_cache.py::TestCodebook::test_rotation_varies_with_seed
• pull / unittest-nxp-neutron / linux-job (gh) (trunk failure)
  test_concat_cluster_quantization__conv_7_sigmoid

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[qti-horodnic]

@pytorchbot label "release notes: qualcomm"

[psiddh]

@claude review this PR

[claude]

Claude finished @psiddh's task in 2m 13s —— View job

---

PR Review: GenAI Pipeline Orchestrator (PR3)

• Read changed files and dependencies (configs, stages, proxy, context)
• Analyze orchestrator correctness & data wiring
• Review tests
• Post review

---

Solid, well-structured PR — clean strategy/registry pattern, good docstrings and type annotations, and the test suite covers the wiring intent nicely. A few issues worth addressing before merge.

🔴 Bug: GenAIPipeline added to __all__ but never imported

backends/qualcomm/genai_pipeline/__init__.py:39 adds "GenAIPipeline" to __all__, but the file has no corresponding import of the class. As a result the advertised public export doesn't actually work:

from executorch.backends.qualcomm.genai_pipeline import GenAIPipeline  # AttributeError
from executorch.backends.qualcomm.genai_pipeline import *              # raises: no attribute 'GenAIPipeline'

The unit tests don't catch this because they import from the submodule directly (...genai_pipeline.genai_pipeline import GenAIPipeline, test line 20), bypassing the package export.

Fix — add the import in __init__.py:

from executorch.backends.qualcomm.genai_pipeline.genai_pipeline import GenAIPipeline

(Note: "PipelineStage" is also listed in __all__ without an import — pre-existing from PR2, but worth fixing in the same pass since it has the same failure mode.)

Fix this →

🟡 Data-wiring gap: prepared model can't reach compilation when quantization is skipped

In invoke(), _run_compilation builds its CompilationInputConfig solely from quant_output.quantized_model (genai_pipeline.py:235). The model_prep_output.model_module is only consumed by quantization. So in an FP16 / skip-quantization flow (skip_stages={STAGE_QUANTIZATION}), quantization returns a default QuantizationOutputConfig() with quantized_model=None, and compilation receives model=None — the prepared module is silently dropped.

If the intent is "compile the prepared model when quant is skipped," compilation needs a fallback to model_prep_output.model_module. If that path is deferred to a later PR, a comment or TODO would make the current limitation explicit. Worth confirming this is intentional.

🟡 soc_model type inconsistency

PipelineContext.soc_model is a str (e.g. "SM8750", pipeline_context.py:24), but QuantizationInputConfig/CompilationInputConfig/InferenceInputConfig all type soc_model as QcomChipset. The orchestrator passes the raw string through with no conversion (e.g. genai_pipeline.py:213, :233, :259). Not exercised at runtime yet since strategies are stubs, but it's a latent mismatch — decide whether the string→QcomChipset resolution happens here in the orchestrator or inside each strategy, and align the type hints accordingly.

🟢 Minor

• Timing on failure: the ... completed in %.1fs logs only emit on success. If a stage raises, the elapsed time is lost. A try/finally around each _run_* body would make timing/telemetry robust to failures — optional given this is orchestration scaffolding.
• %.1fs precision: sub-second stages will log 0.0s. Consider %.2f or %.3f, or ms. Cosmetic.
• Coverage note: the reported missing lines (184–195) are the model-preparation branch of invoke(). Since the only model-prep strategy is a NotImplementedError stub, that path can't be driven end-to-end yet — expected, no action needed now.

Nits (non-blocking)

• _resolve_stage uses a bare set/untyped return; a -> Optional[PipelineStage] hint and Set[str] param would match the typing rigor of the rest of the file.

Overall the design is clean and extensible. The __init__.py export bug is the one item I'd consider blocking, since it breaks the documented public API surface this PR is meant to add.
· branch pr1c

[psiddh]

@qti-horodnic Is this CI job : pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (pull_request) failure unrelated ?

[qti-horodnic]

@qti-horodnic Is this CI job : pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (pull_request) failure unrelated ?

@psiddh
Yes, CI for this will only be enabled once the whole framework is in place. This code is not called in CI jobs currently

[qti-horodnic]

Regarding the Claude comments:

1: Fixed, added the imports.

2: Fixed. run_compilation now falls back to model_prep_output.model_module when quant_output.quantized_model is None, so the prepared module flows through correctly for FP16/skip-quantization flows.

3: This is intentional for now. PipelineContext is user-facing and carries str; the config type (QcomChipset) represents the contract the strategy expects. The str → QcomChipset resolution is deferred to the strategy/adapter layer, where DefaultQuantizerAdapter.make_quantizer already normalizes both forms. Added a comment at the pass-through point making this explicit. Resolution lands in PR5 where the strategy implementation decides.