Skip to content

UTC Timestamps constructed from datetime.datetime (with datetime.timezone.utc) cannot be compared to pandas Timestamps #22133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
skaae opened this issue Jul 30, 2018 · 2 comments
Labels
Duplicate Report Duplicate issue or pull request Enhancement Timezones Timezone data dtype

Comments

@skaae
Copy link

skaae commented Jul 30, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd
import datetime
ts_to_datetime = pd.to_datetime(datetime.datetime(2018, 3, 8, 16, 58, 41, 182221, tzinfo=datetime.timezone.utc))

ts_utcnow = pd.Timestamp.utcnow()

ts_timestamp = pd.Timestamp(datetime.datetime(2018, 3, 8, 16, 58, 41, 182221, tzinfo=datetime.timezone.utc))

ts_to_datetime - ts_utcnow  # Timestamp subtraction must have the same timezones or no timezones
ts_timestamp - ts_utcnow # same error
ts_to_datetime - ts_timestamp # ok 

Problem description

Timestamps constructed from datatime.datetime objects with UTC timezone have tz that is type datetime.timezone.utc but pd.Timestamp.utwnow() have tz that is pytz.UTC.

Expected Output

In [2]: ts_to_datetime - ts_utcnow
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-5564b4c82b42> in <module>()
----> 1 ts_to_datetime - ts_utcnow

pandas/_libs/tslibs/timestamps.pyx in pandas._libs.tslibs.timestamps._Timestamp.__sub__()

TypeError: Timestamp subtraction must have the same timezones or no timezones

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.0.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.3
pytest: 3.0.5
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.13.3
scipy: 0.19.1
pyarrow: 0.7.1
xarray: 0.9.6
IPython: 5.1.0
sphinx: 1.5.1
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: 0.4.0
matplotlib: 2.0.2
openpyxl: 2.4.1
xlrd: 1.1.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.2
bs4: 4.5.3
html5lib: 0.9999999
sqlalchemy: 1.1.5
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@mroeschke
Copy link
Member

xref #17173

The main issue is that pandas do not yet support datetime.timezone objects. Currently pandas only supports pytz or dateutil timezone objects.

I anticipate pandas supporting datetime.timezone objects soon enough since it's part of the standard library and since pandas will drop Python 2 support, but it will require quite some work. Pull requests are always welcome!

@mroeschke mroeschke added Enhancement Timezones Timezone data dtype labels Jul 30, 2018
@mroeschke mroeschke changed the title UTC Timestamps constructed from datetime.datetime cannot be compared to pandas Timestamps UTC Timestamps constructed from datetime.datetime (with datetime.timezone.utc) cannot be compared to pandas Timestamps Jul 30, 2018
@jreback jreback added the Duplicate Report Duplicate issue or pull request label Jul 31, 2018
@jreback jreback added this to the No action milestone Jul 31, 2018
@jreback
Copy link
Contributor

jreback commented Jul 31, 2018

as indicated this is need of implementation ; closing as a duplicate

@jreback jreback closed this as completed Jul 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Duplicate Report Duplicate issue or pull request Enhancement Timezones Timezone data dtype
Projects
None yet
Development

No branches or pull requests

3 participants