Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations

[qti-horodnic]

Summary

This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.

What's included

Strategy implementations:

• ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter

  • Validates model, example_inputs, soc_model, and backend_type are present
  • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
  • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
  • Filters context.extra_options to a compilation-relevant allow-list
  • Returns artifact paths and optional ETRecord

• ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter

  • Validates artifact_paths and adapter are present (no default adapter — device config is required)
  • Push → execute → pull results flow
  • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
  • Returns inference results, performance metrics, and optional ETDump

DefaultCompilerAdapter:

• compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
  argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
  passes_job, constant_methods) are explicit parameters rather than extra_options keys
• Body deliberately raises NotImplementedError: the version this package needs is
  the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
  sharing), so it lands with the strategy-level fan-out that calls it rather than
  being written single-graph and then replaced. Inject a custom CompilerAdapter for now.

Config addition:

• CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs

Orchestrator wiring:

• genai_pipeline.py _run_compilation: passes example_inputs=model_prep_output.example_inputs

Unit tests:

• test_executorch_compilation_strategy.py
• test_executorch_inference_strategy.py
• test_compilation_input_config.py

PR Review Checklist

• All new classes follow single responsibility (one class per file) - Yes.
• All dependencies are injected via constructor with sensible defaults - Yes.
• All external calls are behind injectable interfaces - Yes (adapter pattern).
• Unit tests cover every public method - Yes (100% coverage on strategy impls).
• Docstrings on all public classes and methods - Yes.
• Type annotations on all function signatures - Yes.
• Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).

Related PRs

• PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
• PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
• PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
• PR 4: Adapter interfaces, default implementations & dataset providers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR4: Adapter interfaces, default implementations & dataset providers #21751
• PR 5: Model preparation & quantization strategies: Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations #21899
• PR 6: Compilation & inference strategy implementations: this pr.
• PR 7: Integration & E2E tests: pending.

Test plan

Run only tests added in this PR:

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
  backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
  backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
  -v

Run only this PR's tests with coverage:

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
  backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
  backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
  --cov=backends/qualcomm/genai_pipeline/strategies/compilation \
  --cov=backends/qualcomm/genai_pipeline/strategies/inference \
  --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
  --cov-report=term-missing

Result:

Name                                                                                         Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py                  7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py                     11      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py      45      0     10      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py                  14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py          43      0      6      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py                      7      0      0      0   100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                          127      0     16      0   100%

Run all genai_pipeline tests:

python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
  --cov=backends/qualcomm/genai_pipeline \
  --cov-config=backends/qualcomm/.coveragerc \
  --cov-report=term-missing

Result:

Name                                                                                                     Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py                                        12      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py                                        8      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py                                          12      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py                                          9      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py                                   7      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py                                 12      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py                                       13      0      0      0   100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py                                       5      0      0      0   100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py                                        5      0      0      0   100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py                               26      0      6      0   100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py                                  14      0      2      0   100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py                                           5      0      0      0   100%
backends/qualcomm/genai_pipeline/engine_proxy.py                                                            20      0      4      0   100%
backends/qualcomm/genai_pipeline/exceptions.py                                                              20      0      6      0   100%
backends/qualcomm/genai_pipeline/genai_pipeline.py                                                          99      7     12      1    93%   190-201
backends/qualcomm/genai_pipeline/pipeline_context.py                                                        52      0     14      0   100%
backends/qualcomm/genai_pipeline/pipeline_stage.py                                                           5      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py                                                14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py                                                  14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py                                          14      2      0      0    86%   30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py                                               14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py                              7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py                                 11      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py                  45      0     10      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py                              14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py                      43      0      6      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py                                  7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py      68      0     14      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py                        9      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py                  7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py                61      0     18      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py                            7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py                                9      0      0      0   100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                      668      9     92      1    99%

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 5 Pending

As of commit df39537 with merge base 7f4b8ad ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[qti-horodnic]

@pytorchbot label "release notes: qualcomm"

[psiddh]

@claude review this PR

[claude]

Claude finished @psiddh's task in 2m 5s —— View job

---

PR Review: Compilation & Inference Strategy Implementations

• Read changed files
• Review compilation strategy
• Review inference strategy
• Review adapters & config
• Post review feedback

Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.

---

1. Auto-wired pipeline can never run compilation or inference (design gap)

The orchestrator instantiates strategies with no arguments:

# genai_pipeline.py:163
return stage_cls(strategy_cls())

But both strategies added here depend on an injected adapter that has no working default:

• ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
• ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.

So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.

I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.

2. Input data is never pushed to the device

DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:

# executorch_inference_strategy.py:90
self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)

InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.

3. inference_results type vs. output_data contents

# executorch_inference_strategy.py:109
inference_results = result.output_data           # Optional[List[Any]]

InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.

4. example_inputs presence check vs. empty tuple

_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."

---

Minor / nits

• compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
• file_name unused downstream. context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
• The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.

---

Tests

Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).

Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.
· branch pr4c

[qti-horodnic]

Addressing Claude's comments:

1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.