Repository navigation
Cannot use TypedDict object when defining params #29358
Description
Apache Airflow version
2.5.1
What happened
Context: I am attempting to use TypedDict objects to maintain the keys used in DAG params in a single place, and check for key names across multiple DAGs that use the params.
This raises an error with mypy as params expects an Optional[Dict]. Due to the invariance of Dict, this does not accept TypedDict objects.
What happened: I passed a TypedDict to the params arg of DAG and got a TypeError.
What you think should happen instead
TypedDict objects should be accepted by DAG, which should accept Optional[Mapping[str, Any]].
Unless I'm mistaken, params are converted to a ParamsDict class and therefore the appropriate type hint is a generic Mapping type.
How to reproduce
Steps to reproduce
from typing import TypedDict
from airflow import DAG
from airflow.models import Param
class ParamsTypedDict(TypedDict):
str_param: Param
params: ParamsTypedDict = {
"str_param": Param("", type="str")
}
with DAG(
dag_id="mypy-error-dag",
# The line below raises a mypy error
# Argument "params" to "DAG" has incompatible type "ParamsTypedDict"; expected "Optional[Dict[Any, Any]]" [arg-type]
params=params,
) as dag:
passOperating System
Amazon Linux
Versions of Apache Airflow Providers
No response
Deployment
Docker-Compose
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
boring-cyborg commentedon Feb 3, 2023
Thanks for opening your first issue here! Be sure to follow the issue template!
jcobb-healx commentedon Feb 3, 2023
Relevant mypy discussion on passing
TypedDictas args annotated withDict: python/mypy#4976potiuk commentedon Feb 3, 2023
Feel free to work on it.
pierrejeambrun commentedon Feb 25, 2023
@jcobb-healx Are you still working on this ? Don't hesitate to reach out if you need some help :)
jcobb-healx commentedon Feb 27, 2023
@pierrejeambrun Sorry, I hadn't gotten around to looking at this. Spent a bit of time on this today, and it looks like converting the type hint to
Mappingisn't right, as theparamsare merged with the default params in anupdatecall, which isn't consistent withTypedDictbehaviour.Given the earlier
mypydiscussion, it probably makes sense to useMutableMappingto allow generic inputs, and let users perform casts, i.e. not acceptTypedDictobjects.jcobb-healx commentedon Feb 27, 2023
@pierrejeambrun I've made the changes to use
MutableMappingso that passing in aParamsDictobject works as expected. I've not changed anything that would allow my original case to work, so I've made no mention ofTypedDictin the PR