Skip to content

Releases: redivis/redivis-python

0.15.0

29 May 01:21
1463e89
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.14.9...v0.15.0

0.14.9

21 May 04:34
Compare
Choose a tag to compare
  • fix(uploads): make sure files are always opened in binary format

0.14.8

20 May 21:13
85a1126
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.14.7...v0.14.8

0.14.7

06 Apr 18:25
6a4c717
Compare
Choose a tag to compare

What's Changed

Full Changelog: v0.14.0...v0.14.7

0.14.0

16 Jan 03:47
Compare
Choose a tag to compare
  • Implement table.add_files() method
  • Fix issue where threads wouldn't exit while bulk downloading files from an index table
  • General performance and reliability improvements when uploading data

0.13.0

11 Jan 22:34
49dff0b
Compare
Choose a tag to compare

What's Changed

  • refactor: general cleanup and bugfixes by @imathews in #36

Full Changelog: v0.12.6...v0.13.0

0.12.6

10 Jan 16:18
Compare
Choose a tag to compare
  • fix(datasets): fix bad reference in dataset returned from create_next_vesion (closes #32)
  • feat(projects): add project.get(), project.exists(), and user.list_projects() methods

0.12.5

08 Jan 23:34
Compare
Choose a tag to compare
  • Fix file path resolution on windows (#34, thanks @jbiggsets!)
  • Fix an issue with the dataset.update() method
  • Added support for REDIVIS_TMPDIR environment variable, which can be used to specify the temp directory used by this library (if not specified, the system default is used)

0.12.1

17 Oct 04:54
Compare
Choose a tag to compare
  • fix table.list_files() error

0.12.0

10 Oct 02:21
303e6b6
Compare
Choose a tag to compare

This is a major update to the redivis-python client library, complete with significant performance improvements, new mechanisms for working with larger tables, and new integrations with pyarrow, polars, and dask. It also includes updates that allow for the generation of output tables within a redivis notebook environment.

New features 🚀

  • New methods on Table, Query, and Upload classes:
    • to_dask_dataframe(): read a redivis table as a dask dataframe
    • to_polars_lazyframe(): read a redivis table as a polars lazyframe
    • to_arrow_table(): read a redivis table as a pyarrow table
    • to_arrow_dataset(): read a redivis table as a pyarrow dataset
    • to_arrow_batch_iterator(): iterate over a table via pyarrow RecordBatches
    • to_geopandas_dataframe(): read a redivis table with a geography variable as a geopandas dataframe
    • to_pandas_dataframe(): read a redivis table as a pandas dataframe for a table with a geography variable, using the new pyarrow dtypes by default (supercedes the old to_dataframe() method, which has been deprecated)
  • Added batch_preprocessor argument to all above functions to efficiently pre-filter data as it is loaded
  • Added dtype_backend argument to to_[geo]pandas_dataframe() methods. Specify either pyarrow, numpy, or numpy_nullable
  • Add date_as_object boolean argument to_[geo]pandas_dataframe()` methods. Configure how dates should be stored in when using the legacy numpy dtypes.
  • Add redivis.current_notebook(), returning an instance of the current notebook within a Redivis notebook environment (returns null in other environments)
    • Add Notebook.create_output_table(df) method to generate an output table from a redivis notebook

Performance ⚡

  • Multi-threaded data streaming, alongside performance improvements in the underlying transfer protocol, should lead to at least a 3x improvement in data transfer speeds
  • Reduced memory overhead significantly, with new mechanisms to load larger-than-memory data

Deprecations

  • The to_dataframe method has been deprecated, in favor of the more explicit to_pandas_dataframe() and to_geopandas_dataframe() methods. The new methods default to the "pyarrow" dtype_backend — to continue using numpy as the pandas backend, you must specify dtype_backend="numpy" for the new methods. However, pyarrow is significantly faster, and supports nullable data types. It is recommended if numpy isn't required, and will generally work with existing code.

Dependencies

Bumped the following dependencies:

  • pandas >= 2.0.0
  • pyarrow >= 13.0.0
  • pyarrow >= 0.14.0

Added the following dependencies:

  • dask >= 2023.9.3
  • polars >= 0.19
  • shapely >= 2.0.1