All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Resources and Destinations must now be imported using their full path.
- Tests can now be run with a single command. See README for details.
- Added pre-commit and pre-commit hooks for Black
- Added option to pass a manual schema to GCP Destination from a JSON Resource
- Bug fix for manual schema option
- Support manual schema option for CSVResource
- Support custom Hive paths for CSVResource
- Add py.typed marker so that dependent libraries can use type vestapol type annotations
- Removed functionality that creates a header metadata file for
CSVResource
s - Removed the
has_header
parameter forCSVResource
s
- Fixed a bug where the BigQuery external table would ignore the first row of all CSV data, even if the data did not contain a header row.
- Added an optional
skip_leading_rows
parameter toCSVResource
to specify andinteger indicating the number of header rows in the source data.
- Fixed issue where an invalid
None
value could be passed to theskip_leading_rows
parameter
- Added support for deeply nested manual schemans in
gcp_destination
.
- Fixed a bug where the
skip_leading_rows
parameter incsv_resource
was not being passed to the base class.
- Added support for specifying delimiter in csv files in
CSVResource
- Added support for allowing quoted newlines in csv files in
CSVResource
- Updated the
manual_schema
parameter inCSVResource
to use the Optional type
- Updated the parameter data type in write_list (json_resource class)
- Updated the parameter data type in write_list (json_resource class)