Glide destination updates #12
Conversation
| r = requests.post( | ||
| self.url(f"tables"), | ||
| self.url(f"tables?onSchemaError=dropColumns"), |
There was a problem hiding this comment.
This does not appear to be documented: https://apidocs.glideapps.com/api-reference/v2/tables/post-tables
There was a problem hiding this comment.
The docs update was awaiting PR review on the monolith for a few days, but it should hopefully go out to prod today.
| @@ -1,7 +1,7 @@ | |||
| { | |||
| "documentationUrl": "https://docs.airbyte.com/integrations/destinations/glide", | |||
| "supported_destination_sync_modes": ["overwrite"], | |||
| "supportsIncremental": true, | |||
| "supportsIncremental": false, | |||
There was a problem hiding this comment.
Good catch on the whole incremental & state record management. I think I followed a pattern from the first day of developing it and never really got back to thinking it through or testing it.
| """ | ||
|
|
||
| def init(self, api_key, table_name, api_host="https://api.glideapps.com", api_path_root=""): | ||
| def init(self, api_key, table_name, columns, api_host="https://api.glideapps.com", api_path_root="", batch_size = DEFAULT_BATCH_SIZE): |
There was a problem hiding this comment.
combining init & set_schema is a nice simplification 🙇♂️
|
@chkn I just decided to merge it so I can put some changes on top of this one. Hope that's okay!
|
What
onSchemaErrorflag to ignore columns that aren't in the schemalatesttag inpush-docker-imagescriptThese bits have been pushed to the
glide-connectorsGAR as tag 0.0.30 and tested locally with 32,992 rows from an Airtable source.