This repository has been archived by the owner on Sep 23, 2024. It is now read-only.
CSV file uses wrong file format (uses JSON instead of CSV) #131
Labels
bug
Something isn't working
Comments
cwegener
added a commit
to cwegener/pipelinewise-target-postgres
that referenced
this issue
Jan 3, 2024
writing in JSON format does not make sense. The records in Postgres should arrive exactly as they were scraped by a Tap at the source. E.g. if a string in the sources has control characters like LF (0x0a), the same string should also have LF in the postgres target and not '\n' like JSON format uses. Fixes transferwise#131
cwegener
added a commit
to cwegener/pipelinewise-target-postgres
that referenced
this issue
Jan 3, 2024
writing in JSON format does not make sense. The records in Postgres should arrive exactly as they were scraped by a Tap at the source. E.g. if a string in the sources has control characters like LF (0x0a), the same string should also have LF in the postgres target and not '\n' like JSON format uses. Fixes transferwise#131
cwegener
added a commit
to cwegener/pipelinewise-target-postgres
that referenced
this issue
Jan 3, 2024
writing in JSON format does not make sense. The records in Postgres should arrive exactly as they were scraped by a Tap at the source. E.g. if a string in the sources has control characters like LF (0x0a), the same string should also have LF in the postgres target and not '\n' like JSON format uses. Fixes transferwise#131
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
This is a most obvious bug, but somehow I'm the first person to report this?
pipelinewise-target-postgres/target_postgres/db_sync.py
Line 351 in bef5a27
The Postgresql
COPYcommand does NOT support reading JSON formatted files. It only supports CSV formatted files (and Binary and Text formats)https://www.postgresql.org/docs/current/sql-copy.html
The text was updated successfully, but these errors were encountered: