This repository has been archived by the owner on Sep 23, 2024. It is now read-only.
Pipeline fails if archive enabled with s3 and tap table name is not in format <<schema_name>>__<<table>> #235
Labels
bug
Something isn't working
When using s3 with archive like:
archive_load_files: true
With taps that generate streams with naming
<<table>>
instead of<<schema_name>>__<<table>>
it will break on archiving into s3.Example taps: tap-pend, tap-gitlab
This happens because in STREAM_UTILS_L97-L11, where the schema_name will be set to None if the name do not follow the pattern
<schema_name>_<table>.
In UPLOAD_CLIENTS_L107 if the metadata has {"schema_name":None} that will throw an error on trying to convert to ascii.
To Reproduce
To reproduce use use in meltano.yaml
Use a tap like tap-gitlab or ta-pendo
Error: AttributeError: 'NoneType' object has no attribute 'encode'
From:
Expected behavior
I want to be able to archive data into s3 when data is comming from taps that only send <<table_name>> on stream list instead of <<schema_name>>_<<table_name>>
The text was updated successfully, but these errors were encountered: