[kadubhumika]

Summary

PostgreSQL JSONB can change large exponent-form floating-point values in DagRun.conf into large integers when Airflow reads them back.

For example, 1.7E308 can be returned as a Python int instead of a float.

This PR:

• Adds a custom JSON type for DagRun.conf to preserve exponent-form floats.
• Restores these values as Python float when reading them back.
• Keeps normal floats, integers, strings, booleans, and nested values unchanged.
• Adds regression tests for large floats and other JSON value types.

Testing

• Full airflow-core/tests/unit/models/test_dagrun.py: 248 passed, 1 expected xfailed
• Regression test for exponent-form float handling: passed
• Ruff: passed
• git diff --check: passed
• PostgreSQL 18.3 JSONB behavior reproduced independently:
  • 1.7E308 is stored by PostgreSQL JSONB as a large integer-like numeric representation.
  • jsonb_typeof(...) reports the value as number.
• The previous airflow_shared import issue was resolved by installing the repository's shared/secrets_masker package in the development environment.

The SQLAlchemy → PostgreSQL end-to-end round-trip test is currently limited by the local PostgreSQL pg_hba.conf configuration for WSL connections.

Reproduced the PostgreSQL JSONB behavior independently with PostgreSQL 18.3: 1.7E308 is stored as a large integer-like numeric value.
Added regression coverage for DagRun.conf and verified the exponent-form value is restored as a Python float.
Ran the complete airflow-core/tests/unit/models/test_dagrun.py: 248 passed, 1 expected xfailed.
Ruff and git diff --check both pass.
Resolved the earlier airflow_shared local environment issue and reran the affected tests successfully.

I also verified the PostgreSQL JSONB behavior independently. The remaining SQLAlchemy → PostgreSQL round-trip test is currently blocked by my local PostgreSQL pg_hba.conf configuration for WSL connections, so I have not marked that integration test as passing.

Closes #74023

[kadubhumika]

@eladkal Hi! Quick update on the PR I’ve addressed the two CI issues that came up:

Fixed the MySQL Core test failure by making the regression test compatible with MySQL’s handling of very large integers.
Fixed the static-check/formatting issues reported by CI.
The affected regression test now passes locally, and Ruff + git diff --check are clean.
Whenever you have time, could you please review the PR again? If you notice any other issues or bugs during review, I’ll be happy to address them. Thanks for your time! 🙏

[ashb]

Do we need to this? Is there not an option to instead make anything larger than 64bit it force to float before we send to msgspec.

In other words, this isn't a storage or JSON vs JSONB issue, it's an encoding issue.

[kadubhumika]

Hi @ashb , I checked the downstream behavior.
1.7e308 can become a 309-digit Python int after the PostgreSQL JSONB round-trip, and msgspec.msgpack.encode() raises OverflowError for that value.
I also found that #73783 addresses the MessagePack overflow for large integers, but it does not restore the original float type after the JSONB round-trip.
I’ll now trace the Airflow encoding path to see whether converting oversized integers to floats before MessagePack is sufficient and safe, rather than changing the JSONB storage layer.

[kadubhumika]

I need one day i will solve fix inspect this part thoroughly.

[kadubhumika]

Thanks @ashb Sir, I traced the full path through _FrameMixin.as_bytes()
and also inspected #73783.
I see that #73783 already addresses the MessagePack OverflowError
by preserving oversized integers through a MessagePack extension.
The remaining difference is the separate float → int conversion caused
by the PostgreSQL JSONB round trip. For example, 1.7E308 is a float
before persistence but becomes a 309-digit int after reading it back.
So I understand now that the encoding crash and the numeric-type
preservation are separate issues. Before changing the implementation,
I'd like to confirm whether Airflow 3 is expected to preserve the
original float type here, or whether #73783's exact-int behavior is
the intended semantics. Sorry Next time before working on any issues or prs i will first t inspect all existing's prs on it.. Thank you .. I learned a lott..

[kadubhumika]

@ashb how would you like to proceed? Are you closing this PR for now, or is there a specific way you'd like me to update it ?
I'm completely happy to follow your lead.