fix: 103 batch write example #112
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #112 +/- ##
==========================================
+ Coverage 55.12% 59.52% +4.39%
==========================================
Files 33 33
Lines 2048 2048
==========================================
+ Hits 1129 1219 +90
+ Misses 919 829 -90 ☔ View full report in Codecov by Sentry. |
bednar
approved these changes
Sep 18, 2024
jstirnaman
approved these changes
Sep 20, 2024
There was a problem hiding this comment.
LGTM
Unrelated (I think), but tests/test_write_file.py is failing:
> pytest
========================================================================================================================================= test session starts ==========================================================================================================================================
platform darwin -- Python 3.11.9, pytest-7.4.1, pluggy-1.3.0
rootdir: /Users/ja/Documents/github/influxdb3-python
configfile: pytest.ini
plugins: cov-2.12.1, pytest_codeblocks-0.16.1, anyio-4.3.0, dotenv-0.5.2
collected 62 items
tests/test_api_client.py ....... [ 11%]
tests/test_dataframe_serializer.py ........................ [ 50%]
tests/test_date_helper.py ... [ 54%]
tests/test_deep_merge.py ....... [ 66%]
tests/test_influxdb_client_3.py ... [ 70%]
tests/test_influxdb_client_3_integration.py ssssss [ 80%]
tests/test_merge_options.py .... [ 87%]
tests/test_point.py .. [ 90%]
tests/test_polars.py .. [ 93%]
tests/test_query.py ... [ 98%]
tests/test_write_file.py F [100%]
=============================================================================================================================================== FAILURES ===============================================================================================================================================
__________________________________________________________________________________________________________________________________ TestWriteFile.test_write_file_csv ___________________________________________________________________________________________________________________________________
self = <tests.test_write_file.TestWriteFile testMethod=test_write_file_csv>
def test_write_file_csv(self):
mock_write = Mock(spec=WriteApi)
self.client._write_api.write = mock_write.write
self.client.write_file(file='tests/data/iot.csv', timestamp_column='time', measurement_name="iot-devices",
tag_columns=["building"], write_precision='s')
expected_df = pd.DataFrame({
"name": ['iot-devices', 'iot-devices', 'iot-devices'],
"building": ['5a', '5a', '5a'],
"temperature": [72.3, 72.1, 72.2],
"time": pd.to_datetime(["2022-10-01T12:01:00Z", "2022-10-02T12:01:00Z", "2022-10-03T12:01:00Z"])
.astype('datetime64[s, UTC]'),
})
expected = {
'bucket': 'my_db',
'record': expected_df,
'data_frame_measurement_name': 'iot-devices',
'data_frame_tag_columns': ['building'],
'data_frame_timestamp_column': 'time',
'write_precision': 's'
}
_, actual = mock_write.write.call_args
assert mock_write.write.call_count == 1
> assert expected_df.equals(actual['record'])
E assert False
E + where False = <bound method NDFrame.equals of name building temperature time\n0 iot-devices 5a...devices 5a 72.1 2022-10-02 12:01:00+00:00\n2 iot-devices 5a 72.2 2022-10-03 12:01:00+00:00>( name building temperature time\n0 iot-devices 5a 72.3 2022-10-01 12:01:0...-devices 5a 72.1 2022-10-02 12:01:00+00:00\n2 iot-devices 5a 72.2 2022-10-03 12:01:00+00:00)
E + where <bound method NDFrame.equals of name building temperature time\n0 iot-devices 5a...devices 5a 72.1 2022-10-02 12:01:00+00:00\n2 iot-devices 5a 72.2 2022-10-03 12:01:00+00:00> = name building temperature time\n0 iot-devices 5a 72.3 2022-10-01 12:01:0...-devices 5a 72.1 2022-10-02 12:01:00+00:00\n2 iot-devices 5a 72.2 2022-10-03 12:01:00+00:00.equals
tests/test_write_file.py:52: AssertionError
======================================================================================================================================= short test summary info ========================================================================================================================================
FAILED tests/test_write_file.py::TestWriteFile::test_write_file_csv - assert False
================================================================================| batch_size = 10 | ||
|
|
||
| def success(conf, data): | ||
| nonlocal write_success, write_count |
There was a problem hiding this comment.
❤️
I didn't know about nonlocal
| print(f"Elapsed time ms: {int(callback.elapsed() / 1_000_000)}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #103
Proposed Changes
batching_example.pyto show making calls after the context manager for the write has closed.Checklist