Add embedding kwargs to common AI hooks and operators

[jeff3071]

Why

Embedding model options such as custom dimensions cannot currently be configured in common AI hooks and operators.

Following the OpenAI provider design, these options are passed explicitly instead of through connection extra.

What

• Add embedding_kwargs to LangChain and LlamaIndex hooks.
• Forward it through LlamaIndex embedding and retrieval operators.

Test

Use testing dag below and self-host embedding model.

import math

from airflow.providers.common.ai.operators.llamaindex_embedding import LlamaIndexEmbeddingOperator
from airflow.providers.common.compat.sdk import dag, task


@task
def verify_embeddings(result: dict) -> None:
    """Verify that vLLM produced one finite 128-dimensional vector per document."""
    if result["document_count"] != 2 or result["chunk_count"] != 2:
        raise ValueError(
            f"Expected two documents and two chunks, got {result['document_count']} documents "
            f"and {result['chunk_count']} chunks"
        )

    vectors = [chunk["vector"] for chunk in result["chunks"]]
    dimensions = {len(vector) for vector in vectors}
    if dimensions != {128}:
        raise ValueError(f"Expected 128-dimensional vectors, got {sorted(dimensions)}")
    if not all(math.isfinite(value) for vector in vectors for value in vector):
        raise ValueError("Embedding response contains non-finite values")


@dag(schedule=None, tags=["example", "llamaindex", "vllm"])
def example_llamaindex_vllm_embedding():
    embed = LlamaIndexEmbeddingOperator(
        task_id="generate_embeddings",
        documents=[
            {
                "text": "Apache Airflow orchestrates data workflows.",
                "metadata": {"source": "airflow"},
            },
            {
                "text": "Qwen3-Embedding produces vector representations of text.",
                "metadata": {"source": "qwen"},
            },
        ],
        embed_model="text-embedding-3-small",
        embed_conn_id="llamaindex_vllm",
        embedding_kwargs={"dimensions": 128},
        chunk_size=128,
        chunk_overlap=16,
    )

    verify_embeddings(embed.output)


example_llamaindex_vllm_embedding()

vllm command

vllm serve Qwen/Qwen3-Embedding-0.6B     --runner pooling     --host 0.0.0.0     --port 8000     --api-key EMPTY --max-model-len 1024 --gpu-memory-utilization 0.5  --hf-overrides '{"is_matryoshka":true}'  --served-model-name text-embedding-3-small

Use text-embedding-3-small as the served model name because LlamaIndexHook validates the model name.

---

Was generative AI tooling used to co-author this PR?

• Yes (Codex:GPT-5.6-sol)
  Generated-by: (Codex:GPT-5.6-sol) following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[Lee-W]

mostly good. one remaining nitpick