Skip to content

fix: give HiveDialect its own colspecs dict - #490

Open
rusackas wants to merge 1 commit into
dropbox:masterfrom
rusackas:fix/hive-dialect-own-colspecs
Open

rusackas wants to merge 1 commit into
dropbox:masterfrom
rusackas:fix/hive-dialect-own-colspecs

Conversation

@rusackas

@rusackas rusackas commented Oct 2, 2026

Copy link
Copy Markdown

Summary

HiveDialect never defines its own colspecs, so it inherits sqlalchemy.engine.default.DefaultDialect.colspecs directly -- the exact same dict object, not a copy:

>>> from sqlalchemy.engine.default import DefaultDialect
>>> from pyhive.sqlalchemy_hive import HiveDialect
>>> HiveDialect.colspecs is DefaultDialect.colspecs
True

Every SQLAlchemy built-in dialect (postgresql, mysql, etc.) defines its own colspecs dict, even when there's nothing to override yet, specifically to avoid this. The reason: mutating a dialect's colspecs in place (HiveDialect.colspecs[SomeType] = ...) is a completely normal way to customize how a type renders for one dialect. Without its own dict, that mutation silently corrupts type handling for every other dialect in the process that also hasn't defined its own colspecs -- not just Hive.

This isn't hypothetical. Apache Superset's Databricks engine spec (which subclasses PyHive's HiveDialect for Databricks support) patches HiveDialect.colspecs[types.String] = ... to customize string-type handling. Because of the shared object, that one line broke type compilation for ~38 unrelated third-party dialects loaded in the same process, until Superset worked around it on its own side: apache/superset#44707.

Fix

Give HiveDialect its own (currently empty) colspecs dict, same as every SQLAlchemy built-in dialect does:

colspecs = {}

Verified this fully isolates it with no other behavior change -- colspecs was and remains empty for this dialect; it's just no longer the same object as DefaultDialect's:

>>> HiveDialect.colspecs is DefaultDialect.colspecs
False
>>> HiveDialect.colspecs[str] = 'test'
>>> DefaultDialect.colspecs
{}

Testing

pyhive/tests/test_sqlalchemy_hive.py requires a live Hive server on localhost:10000, which I didn't have available to run the full suite. This change touches only a class-level attribute with no effect on connection or query logic, so I'm confident it's safe, but flagging that I couldn't run the existing integration tests end to end.

馃 Generated with Claude Code

HiveDialect didn't define colspecs, so it inherited
sqlalchemy.engine.default.DefaultDialect.colspecs directly -- the exact
same dict object, not a copy. Every SQLAlchemy built-in dialect
(postgresql, mysql, etc.) defines its own colspecs for this reason: a
mutation like HiveDialect.colspecs[SomeType] = ... -- a normal way to
customize type rendering per dialect -- otherwise corrupts type
handling for every other dialect sharing the inherited default, for
the rest of the process.

This is a real downstream hit, not a hypothetical: apache/superset's
Databricks engine spec patches colspecs on HiveDialect (PyHive's
Databricks support subclasses it) and the shared-object aliasing broke
type compilation for ~38 unrelated dialects loaded in the same process
until Superset worked around it on its own side
(apache/superset#44707).

Verified directly:

    >>> HiveDialect.colspecs is DefaultDialect.colspecs
    True   # before this fix
    False  # after

Zero behavior change otherwise -- colspecs was and remains empty for
this dialect, it's just no longer the same object as the default's.

Co-Authored-By: Evan Rusackas <evan@preset.io>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Oct 2, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants