Repository navigation
Upgrade Helm chart dependency to bitnami/postgresql 13.x.x and PostgreSQL 16 #34817
Description
Official Helm Chart version
1.11.0 (latest released)
Apache Airflow version
2.7.1
Kubernetes Version
1.27
Helm Chart configuration
No response
Docker Image customizations
No response
What happened
Airflow Helm chart 1.11.0 depends on bitnami/postgresql:12.10.0 subchart and deploys PostgreSQL 11 by default.
Apart from good practice of keeping dependencies up to date, there is an end-of-life for PostgreSQL 11 coming in November 9, 2023 as described in PostgreSQL Versioning Policy.
What you think should happen instead
Next Airflow Helm chart release should use latest bitnami/postgresql:13.x.x and PostgreSQL 16 by default.
How to reproduce
Deploy Airflow Helm chart 1.11.0 with default configuration.
Anything else
This issue is created to highlight the problem and discuss how it should be fixed with minimal impact on end users.
PostgreSQL was intentionally downgraded from 15 to 11 after bitnami/postgresql upgrade from 10.5.3 to 12.1.9.
Related PRs:
- Upgrade bitnami/postgresql helm depenency from 10.5.3 to 12.1.9 #29071
- Fix PgBouncer after postgres subchart upgrade #29207
The PR #29207 also contains the discussion of future version upgrade.
Also related PRs with minor changes and fixes:
- Fix bitnami/postgresql subchart upgrade #29210
- Fix bitnami/postgresql default username and password #29478
- Chart: Update postgres subchart #33747
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
boring-cyborg commentedon Oct 7, 2023
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
Taragolis commentedon Oct 7, 2023
This would unlikely happen, Airflow CI do not have regular runs on Postgres 16 yet (#34543)
After EOL Postgres 11 I guess default would be Postgres 12, at least this happen for other components: default python is 3.8, default db backend the oldest one.
dnskr commentedon Oct 8, 2023
Airflow Helm chart has been released recently, so taking it into considiration, there is a good chance that Airflow can support Postgres 16 officially before next Helm chart release.
I'm fine with upgrading PostgreSQL to the newest possible version, either it's 15, 14, 13 or even 12. However, I'm not sure that upgrading to PostgreSQL 12 is a pragmatic solution, because end-of-life for PostgreSQL 12 coming next year (November 14, 2024).
Main challenge with the upgrade is that there is no auto-upgrade options for major versions of PostgreSQL as far as I know, i.e. major version upgrade flow requires users to manage it manually with some database downtime. See https://www.postgresql.org/docs/current/pgupgrade.html for more details.
eladkal commentedon Oct 17, 2023
We added Postgres 16 support to main branch #34543
dnskr commentedon Dec 6, 2023
Looks like there is no automated solution for the problem at the moment.
Related issue bitnami/charts#14926
potiuk commentedon Dec 7, 2023
We can continue upgrading it manually as we do now. And it's fine to upgrade to postgres 16 immediately. Postgres in our chart is not intended for production use and you SHOULD use external postgres. So we do not have to maintain any compatibility or upgrade path.