Zendesk Support connector - Missing data

[alexchouraki]

Is this your first time deploying Airbyte: No
OS Version / Instance: Ubuntu 20.04 on AWS EC2
Deployment: Docker
Airbyte Version: 0.35.31-alpha
Source name: Zendesk Support (0.1.12)
Destination: Snowflake (0.4.15)

Hello,
It appears that some data is missing from the Zendesk Support integration - I can find some tickets in the UI that are not in my tables (which is easy to check for anyone using the connection because it means there are “gaps” in ids instead of having everything completely sequential.).

This is related to this : https://support.zendesk.com/hc/en-us/articles/4408887617050-About-ticket-archiving
Basically, after 120 days since resolution, tickets are archived, and don’t show up in certain streams with the API.

This has several implications :

• Discrepancies between Airbyte data and Zendesk’s internal reporting system
• On incremental syncs, any full reset and re-sync will result in data loss (because some tickets that have been written incrementally will have been archived by then)

It’s not a big deal for me, but I think it might be problematic for some people using the connector. There are other API endpoints that contain the correct info - maybe it’s just a matter of using those ?

[bazarnov]

I found out that the current implementation of the Tickets doesn't return archived/deleted records, it simply skips them, as mentioned in the documentation. We would need to move to the incremental/tickets.json endpoint to return all records.
Thanks for the issue, we will fix this as soon as possible.

[alexchouraki]

Wonderful, thank you! Eager to try it :)