Repository navigation
test_dttm_sql_literal() fails in non-UTC environment #39781
Description
Bug description
Two testcases fail depending on the local timezone.
It happens for those parameter settings where a timezone-naive timestamp is compared to a UNIX timestamp, which is in UTC and thus timezone-aware.
The bug does not show up if the TZ environment variable is set to UTC.
There is no mention of timezone-sensitive tests in the documentation.
I am mildly surprised that this issue has not been raised earlier.
Screenshots/recordings
No response
Superset version
master / latest-dev
Python version
3.9
Node version
16
Browser
Chrome
Additional context
Example:
((.venv-14) ) ➜ superset git:(master) ✗ env TZ='US/Eastern' python3 -m pytest tests/unit_tests/models/core_test.py -k test_dttm_sql_literal
================================================================================================ test session starts =================================================================================================
platform darwin -- Python 3.12.13, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/yuhan/Scratch/superset
configfile: pytest.ini
plugins: asyncio-0.23.8, cov-6.0.0, anyio-4.11.0, mock-3.10.0, pyfakefs-5.3.5
asyncio: mode=Mode.AUTO
collected 55 items / 49 deselected / 6 selected
tests/unit_tests/models/core_test.py FF.... [100%]
====================================================================================================== FAILURES ======================================================================================================
_______________________________________________________________________________ test_dttm_sql_literal[dttm0-col0-database0-1672536225] _______________________________________________________________________________
dttm = datetime.datetime(2023, 1, 1, 1, 23, 45, 600000), col = None, database = <[TypeError('__repr__ returned non-string (type NoneType)') raised in repr()] Database object at 0x16dcddf40>, result = '1672536225'
@pytest.mark.parametrize(
"dttm,col,database,result",
[
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(python_date_format="epoch_s"),
Database(),
"1672536225",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(python_date_format="epoch_ms"),
Database(),
"1672536225000",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(python_date_format="%Y-%m-%d"),
Database(),
"'2023-01-01'",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(column_name="ds"),
Database(
extra=json.dumps(
{
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
},
},
),
sqlalchemy_uri="foo://",
),
"'2023-01-01'",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(),
Database(sqlalchemy_uri="foo://"),
"'2023-01-01 01:23:45.600000'",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(type="TimeStamp"),
Database(sqlalchemy_uri="trino://"),
"TIMESTAMP '2023-01-01 01:23:45.600000'",
),
],
)
def test_dttm_sql_literal(
dttm: datetime,
col: TableColumn,
database: Database,
result: str,
) -> None:
> assert SqlaTable(database=database).dttm_sql_literal(dttm, col) == result
E AssertionError: assert '1672554225' == '1672536225'
E - 1672536225
E ? ^^
E + 1672554225
E ? ^^
tests/unit_tests/models/core_test.py:256: AssertionError
----------------------------------------------------------------------------------------------- Captured stderr setup ------------------------------------------------------------------------------------------------
2026-04-30 09:01:37,360:WARNING:superset.initialization:Debug mode identified with default secret key
2026-04-30 09:01:37,360:WARNING:superset.initialization:--------------------------------------------------------------------------------
WARNING
--------------------------------------------------------------------------------
2026-04-30 09:01:37,360:WARNING:superset.initialization:A Default SECRET_KEY was detected, please use superset_config.py to override it.
Use a strong complex alphanumeric string and use a tool to help you generate
a sufficiently random sequence, ex: openssl rand -base64 42
For more info, see: https://superset.apache.org/docs/configuration/configuring-superset#specifying-a-secret_key
2026-04-30 09:01:37,360:WARNING:superset.initialization:--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
2026-04-30 09:01:37,361:WARNING:py.warnings:/Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py:302: DeprecationWarning: '_app_ctx_stack' is deprecated and will be removed in Flask 2.4. Use 'g' to store data, or 'app_ctx' to access the current context.
queries = _app_ctx_stack.top.sqlalchemy_queries
2026-04-30 09:01:37,361:WARNING:py.warnings:/Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py:304: DeprecationWarning: '_app_ctx_stack' is deprecated and will be removed in Flask 2.4. Use 'g' to store data, or 'app_ctx' to access the current context.
queries = _app_ctx_stack.top.sqlalchemy_queries = []
2026-04-30 09:01:37,361:WARNING:py.warnings:/Users/yuhan/Scratch/superset/superset/utils/cache_manager.py:225: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
cache.init_app(app, cache_config)
2026-04-30 09:01:37,361:WARNING:py.warnings:/Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_caching/__init__.py:145: DeprecationWarning: Using the initialization functions in flask_caching.backend is deprecated. Use the a full path to backend classes directly.
self._set_cache(app, config)
2026-04-30 09:01:37,374:WARNING:py.warnings:/Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_appbuilder/security/sqla/manager.py:136: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
timestamp=datetime.utcnow(),
2026-04-30 09:01:37,374:WARNING:py.warnings:/Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_appbuilder/security/sqla/manager.py:170: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
timestamp=datetime.utcnow(),
2026-04-30 09:01:37,457:INFO:alembic.runtime.migration:Context impl SQLiteImpl.
2026-04-30 09:01:37,457:INFO:alembic.runtime.migration:Will assume non-transactional DDL.
2026-04-30 09:01:37,510:WARNING:py.warnings:/Users/yuhan/Scratch/superset/superset/utils/encrypt.py:57: DeprecationWarning: The 'EncryptedType' class will change implementation from 'LargeBinary' to 'String' in a future version. Use 'StringEncryptedType' to use the 'String' implementation.
return EncryptedType(*args, lambda: app_config["SECRET_KEY"], **kwargs)
2026-04-30 09:01:37,533:INFO:superset.app:Pending database migrations: run 'superset db upgrade'
2026-04-30 09:01:37,623:WARNING:flask_appbuilder.base:appbuilder.app is deprecated and will be removed in a future version. Use current_app instead
2026-04-30 09:01:37,639:WARNING:flask_appbuilder.base:View already exists LogRestApi ignoring
2026-04-30 09:01:37,644:INFO:superset.core.mcp.core_mcp_injection:MCP dependency injection initialized successfully
2026-04-30 09:01:37,644:INFO:superset.core.mcp.core_mcp_injection:MCP service app imported - host tools registered
------------------------------------------------------------------------------------------------- Captured log setup -------------------------------------------------------------------------------------------------
WARNING superset.initialization:__init__.py:650 Debug mode identified with default secret key
WARNING superset.initialization:__init__.py:632 --------------------------------------------------------------------------------
WARNING
--------------------------------------------------------------------------------
WARNING superset.initialization:__init__.py:633 A Default SECRET_KEY was detected, please use superset_config.py to override it.
Use a strong complex alphanumeric string and use a tool to help you generate
a sufficiently random sequence, ex: openssl rand -base64 42
For more info, see: https://superset.apache.org/docs/configuration/configuring-superset#specifying-a-secret_key
WARNING superset.initialization:__init__.py:642 --------------------------------------------------------------------------------
--------------------------------------------------------------------------------
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py:302: DeprecationWarning: '_app_ctx_stack' is deprecated and will be removed in Flask 2.4. Use 'g' to store data, or 'app_ctx' to access the current context.
queries = _app_ctx_stack.top.sqlalchemy_queries
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py:304: DeprecationWarning: '_app_ctx_stack' is deprecated and will be removed in Flask 2.4. Use 'g' to store data, or 'app_ctx' to access the current context.
queries = _app_ctx_stack.top.sqlalchemy_queries = []
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/superset/utils/cache_manager.py:225: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
cache.init_app(app, cache_config)
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_caching/__init__.py:145: DeprecationWarning: Using the initialization functions in flask_caching.backend is deprecated. Use the a full path to backend classes directly.
self._set_cache(app, config)
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_appbuilder/security/sqla/manager.py:136: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
timestamp=datetime.utcnow(),
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/.venv-14/lib/python3.12/site-packages/flask_appbuilder/security/sqla/manager.py:170: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
timestamp=datetime.utcnow(),
INFO alembic.runtime.migration:migration.py:207 Context impl SQLiteImpl.
INFO alembic.runtime.migration:migration.py:210 Will assume non-transactional DDL.
WARNING py.warnings:warnings.py:112 /Users/yuhan/Scratch/superset/superset/utils/encrypt.py:57: DeprecationWarning: The 'EncryptedType' class will change implementation from 'LargeBinary' to 'String' in a future version. Use 'StringEncryptedType' to use the 'String' implementation.
return EncryptedType(*args, lambda: app_config["SECRET_KEY"], **kwargs)
INFO superset.app:app.py:177 Pending database migrations: run 'superset db upgrade'
WARNING flask_appbuilder.base:base.py:234 appbuilder.app is deprecated and will be removed in a future version. Use current_app instead
WARNING flask_appbuilder.base:base.py:549 View already exists LogRestApi ignoring
INFO superset.core.mcp.core_mcp_injection:core_mcp_injection.py:297 MCP dependency injection initialized successfully
INFO superset.core.mcp.core_mcp_injection:core_mcp_injection.py:304 MCP service app imported - host tools registered
_____________________________________________________________________________ test_dttm_sql_literal[dttm1-col1-database1-1672536225000] ______________________________________________________________________________
dttm = datetime.datetime(2023, 1, 1, 1, 23, 45, 600000), col = None, database = <[TypeError('__repr__ returned non-string (type NoneType)') raised in repr()] Database object at 0x16ddd73e0>
result = '1672536225000'
@pytest.mark.parametrize(
"dttm,col,database,result",
[
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(python_date_format="epoch_s"),
Database(),
"1672536225",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(python_date_format="epoch_ms"),
Database(),
"1672536225000",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(python_date_format="%Y-%m-%d"),
Database(),
"'2023-01-01'",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(column_name="ds"),
Database(
extra=json.dumps(
{
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
},
},
),
sqlalchemy_uri="foo://",
),
"'2023-01-01'",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(),
Database(sqlalchemy_uri="foo://"),
"'2023-01-01 01:23:45.600000'",
),
(
datetime(2023, 1, 1, 1, 23, 45, 600000),
TableColumn(type="TimeStamp"),
Database(sqlalchemy_uri="trino://"),
"TIMESTAMP '2023-01-01 01:23:45.600000'",
),
],
)
def test_dttm_sql_literal(
dttm: datetime,
col: TableColumn,
database: Database,
result: str,
) -> None:
> assert SqlaTable(database=database).dttm_sql_literal(dttm, col) == result
E AssertionError: assert '1672554225000' == '1672536225000'
E - 1672536225000
E ? ^^
E + 1672554225000
E ? ^^
tests/unit_tests/models/core_test.py:256: AssertionError
============================================================================================== short test summary info ===============================================================================================
FAILED tests/unit_tests/models/core_test.py::test_dttm_sql_literal[dttm0-col0-database0-1672536225] - AssertionError: assert '1672554225' == '1672536225'
FAILED tests/unit_tests/models/core_test.py::test_dttm_sql_literal[dttm1-col1-database1-1672536225000] - AssertionError: assert '1672554225000' == '1672536225000'
===================================================================================== 2 failed, 4 passed, 49 deselected in 0.58s =====================================================================================
Checklist
- I have searched Superset docs and Slack and didn't find a solution to my problem.
- I have searched the GitHub issue tracker and didn't find a similar bug report.
- I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
Activity
dosubot commentedon Apr 30, 2026
Hi @hy144328! I'm Dosu and I’m helping the superset team.
Good catch. The root cause is in
dttm_sql_literalinsuperset/models/helpers.py— whenpython_date_formatis"epoch_s"or"epoch_ms", it callsdttm.timestamp()on the naive datetime [1]. Python'sdatetime.timestamp()on a naive datetime assumes the local system timezone, so the resulting epoch value varies by environment.The test constructs
datetime(2023, 1, 1, 1, 23, 45, 600000)(naive) and expects1672536225, which is the correct epoch only in UTC [2]. InUS/Eastern(UTC-5),timestamp()adds a 5-hour offset, producing1672554225— exactly matching your output.There are a couple of ways this could be fixed:
datetime(2023, 1, 1, 1, 23, 45, 600000, tzinfo=timezone.utc)) so the expected epoch value is stable regardless of the runner's timezone.dttm_sql_literaltreat naive datetimes as UTC before calling.timestamp()(e.g.,dttm.replace(tzinfo=timezone.utc).timestamp()). This is arguably the more correct fix since Superset generally treats naive datetimes as UTC internally.A related timezone fix was made in PR #37979 which corrected
datetime_to_epochto useastimezone(pytz.utc)instead ofreplace(tzinfo=pytz.utc)for timezone-aware datetimes, but it didn't address this naive datetime case indttm_sql_literal.To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.
hy144328 commentedon Apr 30, 2026
@dosu I do not agree that the implementation fix makes more sense than the test fix here.
Timezone-naive datetime objects mean local timezone as exemplified by the
datetime.datetime.timestamp()method.IMHO, it is not the job of a helper function to perform timezone magic.
I suggest that this responsiblity is delegated to the caller of the helper function.
What do you think?
dosubot commentedon Apr 30, 2026
That's a fair point, and I think you're right to push back on that. Python's
datetime.timestamp()on a naive datetime does indeed assume local timezone per the language spec — that's well-defined behavior, not a bug in the implementation.That said, looking at the broader codebase, Superset consistently uses naive datetimes throughout its query/filter pipeline. The Presto engine spec even documents this explicitly: "Superset only defines time zone naive
datetimeobjects" [1]. All the upstream datetime creation utilities (parse_human_datetime,dttm_from_timetuple,get_since_until) produce naive datetimes without timezone info [2]. Sodttm_sql_literalis always receiving naive datetimes from its callers [3].Given that context, fixing the test is the cleaner approach — the test's expected epoch values implicitly assume UTC, which is the actual bug here. The simplest fix would be to either:
tzinfo=timezone.utcso the expected epoch is deterministic, orfreezegun/ mock the timezone in the test to ensure a consistent environmentThis keeps
dttm_sql_literalbehaving according to standard Python semantics, as you suggest, and puts the timezone responsibility where it belongs.To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.