-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix handling of timestamps in BigQuery insert data method #1250
Conversation
I just realized Bigtable has me trained too much. Wrote Bigquery instead of BigQuery in the commit messages. |
FWIW I cherry-picked the second commit onto head and ran the system test with the error reported by @kamatama41 and got the following: ======================================================================
ERROR: test_load_table_then_dump_table (bigquery.TestBigQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".../gcloud-python/system_tests/bigquery.py", line 232, in test_load_table_then_dump_table
rows, _, _ = table.fetch_data()
File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/bigquery/table.py", line 611, in fetch_data
rows_data = _rows_from_json(response.get('rows', ()), self._schema)
File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/bigquery/_helpers.py", line 81, in _rows_from_json
row_data.append(converter(cell['v'], field))
File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/bigquery/_helpers.py", line 42, in _datetime_from_json
return _datetime_from_microseconds(1e6 * float(value))
File ".../gcloud-python/.tox/system-tests/local/lib/python2.7/site-packages/gcloud/_helpers.py", line 235, in _datetime_from_microseconds
return _EPOCH + datetime.timedelta(microseconds=value)
OverflowError: date value out of range
----------------------------------------------------------------------
Ran 11 tests in 69.796s |
LGTM. OK to merge w/ pylint failing on Travis, pending #1248. |
I'll just wait for #1248 |
As stated in the docs > BigQuery stores TIMESTAMP data internally as a > UNIX timestamp with microsecond precision > ...Specifies the number of seconds since the epoch.
@tseaver Assuming your LGTM still holds after rebasing. |
Fix handling of timestamps in BigQuery insert data method
Great job! Thank you for fixing quickly 😄 |
Sure thing. Thanks for reporting and finding the bug. |
Fixes #1247.