Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers by qti-horodnic · Pull Request #20795 · pytorch/executorch

qti-horodnic · 2026-07-08T23:53:43Z

Summary

This PR adds the strategy interfaces, and stage wrappers for the GenAI pipeline. This is the second of 7 PRs establishing the framework skeleton. It builds on PR1 (data model + engine routing) and introduces the Strategy pattern ABCs, concrete stage delegation wrappers and Executorch strategy stubs. No existing files are modified (except __init__.py introduced in PR1 for imports).

What's included

Strategy ABCs (one class per file):

ModelPreparationStrategy — in strategies/model_preparation/
QuantizationStrategy — in strategies/quantization/
CompilationStrategy — in strategies/compilation/
InferenceStrategy — in strategies/inference/

Each ABC defines invoke(context, input_config) -> output_config.

Strategy stubs (raise `NotImplementedError`):

ExecuTorchModelPreparationStrategy
ExecuTorchQuantizationStrategy
ExecuTorchCompilationStrategy
ExecuTorchInferenceStrategy

Stage wrappers:

ModelPreparationStage, QuantizationStage, CompilationStage, InferenceStage.
Each delegates to its injected strategy via invoke().
name property returns stage constant from pipeline_types.py.

Pipeline stage ABC:

PipelineStage — abstract base with name property and invoke() method.

Unit tests (16 tests across 6 test files):

Strategy ABC enforcement (cannot instantiate, must implement invoke()).
Strategy stub tests (NotImplementedError raised).
Stage delegation tests (assert_called_once_with).
Stage name tests (use constants).

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - N/A, no external calls.
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only __init__.py updated to add PipelineStage export).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - N/A, no logging in this PR.

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: this PR.
PR 3: Pipeline orchestrator: pending.
PR 4: Adapter interfaces & default implementations: pending.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
  backends/qualcomm/genai_pipeline/tests/strategies/ \
  backends/qualcomm/genai_pipeline/tests/stages/ \
  -v

All 16 unit-tests passed.

Test Coverage

Command to run:

coverage run --rcfile=backends/qualcomm/.coveragerc \
  -m pytest \
    backends/qualcomm/genai_pipeline/tests/strategies/ \
    backends/qualcomm/genai_pipeline/tests/stages/ \
  -v

coverage report --rcfile=backends/qualcomm/.coveragerc \
  --include="backends/qualcomm/genai_pipeline/pipeline_stage.py,backends/qualcomm/genai_pipeline/strategies/*,backends/qualcomm/genai_pipeline/stages/*"

Result:

Name                                                                                           Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/stages/compilation_stage.py                                      14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py                                        14      0      0      0   100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py                                14      3      0      0    79%   24, 28, 35
backends/qualcomm/genai_pipeline/stages/quantization_stage.py                                     14      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py                    7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py         7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py             7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py                        7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py        7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py       7      0      0      0   100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py                  7      0      0      0   100%
------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                            110     3      0      0    97%

pytorch-bot · 2026-07-08T23:53:48Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20795

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

Unrelated MacOS MPS test jobs time out (30 min) due to stale cpp_extension lock file poisoning self-hosted runners

✅ You can merge normally! (1 Unrelated Failure)

As of commit b666b79 with merge base 477d278 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / android / build-android (gh) (trunk failure)
ImportError: /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/executorch/extension/training/pybindings/_training_lib.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c104impl3cow23materialize_cow_storageERNS_11StorageImplE

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-07-08T23:53:52Z

@pytorchbot label "release notes: qualcomm"

… & Stage Wrappers

psiddh · 2026-07-21T20:52:02Z

Great to see a clean framework. Is the plan for this new pipeline to eventually replace / wrap / coexist with the existing oss_scripts/llama stack (which already has QuantizationStrategy, ModelInference, etc.)? Also, the genai_pipeline tests don't seem to be wired into CI (not in pytest.ini or the QNN test jobs). is that intentional for now, or should they be added so the suite actually runs?

psiddh · 2026-07-21T20:53:52Z

self-contained skeleton with zero consumers atm, lgtm

qti-horodnic · 2026-07-21T22:26:23Z

Great to see a clean framework. Is the plan for this new pipeline to eventually replace / wrap / coexist with the existing oss_scripts/llama stack (which already has QuantizationStrategy, ModelInference, etc.)? Also, the genai_pipeline tests don't seem to be wired into CI (not in pytest.ini or the QNN test jobs). is that intentional for now, or should they be added so the suite actually runs?

@psiddh

Yes, that's the plan:

Phase 1 (PRs 1-7, what's listed in the decription): The framework skeleton coexists alongside the existing oss_scripts/llama stack, no modifications to existing code.

Phase 2 (3 PRs, ready locally): The framework wraps the existing code. The existing TextDecoder.quantize(), HybridTextDecoder.compile(), MultiModalManager, etc. are called from within the framework's adapters, not duplicated. Both entry points work: wherein users can still run python llama.py directly or use python -m backends.qualcomm.genai_pipeline --model llama3_2-1b.

Phase 3: Once the framework is validated to produce identical outputs, the old llama.py becomes a thin wrapper (or is deprecated), and new models are onboarded exclusively via the pipeline.

Regarding the CI integration, that's intentional for now. The tests require only pytest + unittest.mock, so they can run in any CI environment. They're not wired into pytest.ini or the QNN test jobs yet because:

The plan is to add a dedicated CI job for genai_pipeline unit tests once the PR stack is merged

qti-horodnic · 2026-07-22T17:11:26Z

Hey @psiddh, can we merge this pr if I've answered all your questions above? I'd like to raise the next PR in the series

…or (#21149) ## Summary This PR adds the pipeline orchestrator (`GenAIPipeline`) that wires together all the components from PRs 1 and 2. It assembles stages from `EngineProxy`, builds `InputConfig` objects from `OutputConfig` results, executes stages sequentially with timing, and returns `InferenceOutputConfig`. No existing files are modified. ### What's included #### Pipeline orchestrator: - `GenAIPipeline`: main orchestrator class with: - `_STRATEGY_REGISTRY`: maps `(stage_name, engine_type)` → `(StageClass, StrategyClass)` for extensibility - `from_proxy()`: factory method that resolves stages from `EngineProxy` - `invoke()`: executes model_preparation → quantization → compilation → inference - Private `_run_*` methods: each builds an `InputConfig`, calls the stage, returns the `OutputConfig` - Structured logging with `[GenAIPipeline] StageName started/completed in X.Xs` format per LLD Section 5.2 - `executorch_model_preparation_strategy.py`: ExecuTorch model preparation strategy stub (implementation in a subsequent PR) #### Unit tests (13 tests): - `from_proxy()`: creates all stages, skip stages, default engines - `from_proxy()` error paths: unsupported engine for each stage - `invoke()`: full pipeline with mock strategies, compile-only, no stages - Data wiring: quantization receives `soc_model`, compilation receives `backend_type`, inference receives `prompt` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (via strategy pattern). - Unit tests cover every public method - Yes. - No existing files are modified (Phase 1 constraint) - Yes (only `__init__.py` updated to add `GenAIPipeline` export). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (`[GenAIPipeline]` prefix, timing). - ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795. - PR 3: Pipeline orchestrator: this pr. - PR 4: Adapter interfaces & default implementations: pending. - PR 5: Model preparation & quantization strategies: pending - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/test_genai_pipeline.py \ -v ``` All 13 unit-tests passed. ### Test Coverage Command to run: ``` coverage run --rcfile=backends/qualcomm/.coveragerc \ -m pytest \ backends/qualcomm/genai_pipeline/tests/test_genai_pipeline.py \ -v coverage report --rcfile=backends/qualcomm/.coveragerc \ --include="backends/qualcomm/genai_pipeline/genai_pipeline.py" ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ------------------------------------------------------------------------------------------------ backends/qualcomm/genai_pipeline/genai_pipeline.py 98 7 12 1 93% 184-195 ------------------------------------------------------------------------------------------------ TOTAL 98 7 12 1 93% ```

… default implementations & dataset providers (#21751) ## Summary This PR adds the __adapter layer__ that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies. ### What's included #### Adapter Protocols (6 files): - `QuantizerAdapter`: Protocol wrapping `make_quantizer`, `prepare_pt2e`, `calibrate`, `convert_pt2e` - `CompilerAdapter`: Protocol wrapping `ExportSession` compilation flow + `CompilationResult` dataclass - `DeviceRunnerAdapter`: Protocol wrapping `SimpleADB` push/execute/pull + `InferenceResult` dataclass - `ModelLoaderAdapter`: Protocol wrapping HuggingFace model/tokenizer loading - `CalibrationDataAdapter`: Protocol for calibration dataset construction - `TrainingDataAdapter`: Protocol for QAT training data (yields (features, labels) pairs) #### Default Implementations (6 files): - `DefaultQuantizerAdapter`: Delegates to `export_utils.make_quantizer` + `torchao.quantization.pt2e` - `DefaultCompilerAdapter`: Placeholder for recipe-based compilation (depends on `ExportRecipe`/`ExportSession` APIs not yet available). Raises `NotImplementedError` with guidance to inject a custom `CompilerAdapter` using `to_edge_transform_and_lower_to_qnn`. - `DefaultDeviceRunnerAdapter`: Delegates to `SimpleADB` for on-device execution - `DefaultModelLoaderAdapter`: Delegates to HuggingFace `AutoModelForCausalLM` + `AutoTokenizer` - `DefaultCalibrationDataAdapter`: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"] - `DefaultTrainingDataAdapter`: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized) #### Configuration: - `.coveragerc` updated to omit `default_*_adapter.py` files (integration-test-only, require real SDK/hardware) - `__init__.py` files updated to export new adapter types #### New `datasets/` package Dataset providers are a __cross-stage__ concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization *and* on-device result evaluation during inference (including pre-built `.pte` flows where model preparation never runs). They therefore live in a top-level `datasets/` package rather than under `strategies/model_preparation/`. ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (Protocol pattern). - Unit tests cover every public method - Yes. - No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: this pr. - PR 5: Model preparation & quantization strategies: pending - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/ \ -v ``` All existing tests continue to pass (no regressions). ### Test Coverage Command to run: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40 backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 475 10 44 1 98% 417 10 30 1 98% ```

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic requested review from abhinaykukkadapu and psiddh as code owners July 8, 2026 23:53

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 8, 2026

pytorch-bot Bot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Jul 8, 2026

Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces…

b666b79

… & Stage Wrappers

qti-horodnic force-pushed the pr1b branch from be96145 to b666b79 Compare July 8, 2026 23:55

psiddh approved these changes Jul 21, 2026

View reviewed changes

psiddh merged commit 53551c1 into pytorch:main Jul 22, 2026
181 of 182 checks passed

qti-horodnic mentioned this pull request Jul 22, 2026

Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149

Merged

qti-horodnic mentioned this pull request Aug 11, 2026

Qualcomm AI Engine Direct - [GenAI Pipeline] PR4: Adapter interfaces, default implementations & dataset providers #21751

Merged

qti-horodnic mentioned this pull request Aug 17, 2026

Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations #21899

Merged

qti-horodnic mentioned this pull request Aug 28, 2026

Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations #22284

Merged

Navigation Menu

Uh oh!

Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers - #20795

Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers#20795
psiddh merged 1 commit into
pytorch:mainfrom
CodeLinaro:pr1b

qti-horodnic commented Jul 8, 2026

Uh oh!

pytorch-bot Bot commented Jul 8, 2026 •

edited

Loading

Uh oh!

qti-horodnic commented Jul 8, 2026

Uh oh!

psiddh commented Jul 21, 2026

Uh oh!

psiddh commented Jul 21, 2026

Uh oh!

qti-horodnic commented Jul 21, 2026

Uh oh!

qti-horodnic commented Jul 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

qti-horodnic commented Jul 8, 2026

Summary

What's included

Strategy ABCs (one class per file):

Strategy stubs (raise NotImplementedError):

Stage wrappers:

Pipeline stage ABC:

Unit tests (16 tests across 6 test files):

PR Review Checklist

Related PRs

Test plan

Test Coverage

Uh oh!

pytorch-bot Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20795

❗ 1 Active SEVs

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

qti-horodnic commented Jul 8, 2026

Uh oh!

psiddh commented Jul 21, 2026

Uh oh!

psiddh commented Jul 21, 2026

Uh oh!

qti-horodnic commented Jul 21, 2026

Uh oh!

qti-horodnic commented Jul 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Strategy stubs (raise `NotImplementedError`):

pytorch-bot Bot commented Jul 8, 2026 •

edited

Loading