Skip to content

presto - timestamps and decimal(38,18) returned as strings? #290

Description

@tooptoop4

Why are presto timestamp/decimal(38,18) data types returned a string (enclosed in u'') instead of python datetime/numeric types?

#presto jdbc:
select typeof(col1),typeof(col2),typeof(col3),typeof(col4),typeof(col5),typeof(col6) from hive.x.y
#result is
timestamp timestamp bigint decimal(38,18) varchar varchar

desc hive.x.y
#result is
for_dt timestamp NO NO NO NO 1
for_d timestamp NO NO NO NO 2
for_h bigint NO NO NO NO 3
value decimal(38,18) NO NO NO NO 4
metric varchar(2147483647) NO NO NO NO 5
lat_lon varchar(2147483647) NO NO NO NO 6

attempt 1
#python
from sqlalchemy.engine import create_engine
engine = create_engine('presto://u:p@host:port',connect_args={'protocol': 'https', 'requests_kwargs': {'verify': 'mypem'}})
result = engine.execute('select * from hive.x.y limit 1')
print(result.fetchall())
#result is
[(u'2010-02-18 03:00:00.000', u'2010-02-18 00:00:00.000', 3, u'-0.191912651062011660', u'hey', u'there')]

attempt 2
#python
from pyhive import presto
import requests
from requests.auth import HTTPBasicAuth

req_kw = {
'verify': 'mypem',
'auth': HTTPBasicAuth('u', 'p')
}

cursor = presto.connect(
host='host',
port=port,
protocol='https',
username='u',
requests_kwargs=req_kw,
).cursor()

query = '''select * from x.y limit 1'''
cursor.execute(query)
print cursor.fetchall()
#result is
[(u'2010-02-18 03:00:00.000', u'2010-02-18 00:00:00.000', 3, u'-0.191912651062011660', u'hey', u'there')]

Activity

  1. bkyryliuk commented on Apr 16, 2020

    @bkyryliuk
    Contributor

    @tooptoop4 we are not actively working on the library at the moment, however contributions are welcome.

  2. RJ3 commented on May 26, 2020

    @RJ3

    I find this is a limitation to both pyhive and the official python-presto-client. I find the need to cast it afterwards.

  3. tooptoop4 commented on Oct 11, 2024

    @tooptoop4
    Author

    🐝

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions