Importing data using HDFStore with pre-epoch dates; "ValueError: timestamp out of range for platform localtime()/gmtime() function"

[surbas]

I have data with a DataFrame that goes back to 1949. I imported it from a csv into a hdf5 using HDFStore. That went fine, but when reading from the HDFStore to get a DF back, I get the below stack trace. When looking at the data in the store I see that the index has negative values for preepoch times...

ValueError: timestamp out of range for platform localtime()/gmtime() function
File "C:\dev\MktDB\test_continuation.py", line 59, in
main()
File "C:\Python27\lib\site-packages\pandas-0.3.0-py2.7-win32.egg\pandas\io\pytables.py", line 157, in _read_group
File "C:\Python27\lib\site-packages\pandas-0.3.0-py2.7-win32.egg\pandas\io\pytables.py", line 173, in _read_frame
File "C:\Python27\lib\site-packages\pandas-0.3.0-py2.7-win32.egg\pandas\io\pytables.py", line 210, in _read_index
File "C:\Python27\lib\site-packages\pandas-0.3.0-py2.7-win32.egg\pandas\io\pytables.py", line 227, in _unconvert_index
File "C:\Users\Shon\AppData\Roaming\Python-Eggs\pandas-0.3.0-py2.7-win32.egg-tmp\pandas\lib\tseries.pyd", line 45, in tseries.array_to_datetime (pandas\lib\src\tseries.c:14378)
File "C:\Users\Shon\AppData\Roaming\Python-Eggs\pandas-0.3.0-py2.7-win32.egg-tmp\pandas\lib\tseries.pyd", line 20, in tseries.to_datetime (pandas\lib\src\tseries.c:13910)

Any guidance would be most appreciated.
Shon

[wesm]

I will look into this when I get a chance-- the most robust way to handle it would be to use the NumPy datetime64 dtype, but that is still undergoing change. There might be another workaround

[wesm]

This has been fixed in the HDFStore class (completely overhauled over the last day or two) ...legacy files will still have the problem but new files will be able to store pre-epoch dates (and there is a unit test to prove it!)