🐛 Source Intercom: company_segment raises 404 error while calling n+1 companies stream.

[alafanechere]

Environment

• Airbyte version: example is 0.22.0-alpha
• OS Version / Instance: example macOS, Windows 7/10, Ubuntu 18.04, GCP n2. , AWS EC2
• Deployment: example are Docker or Kubernetes deploy env
• Source Connector and version: Intercom 0.1.12
• Destination Connector and version: Postgres 0.3.13
• Severity:Medium
• Step where error happened: Sync job

Current Behavior

When syncinc company_segments the companies stream is re-consumed to build company_segments URL.
While performing these N+1 companies calls, a scroll_param returned by the API and used as the next page token returns 404 when called.
Could it be related to the specific behavior of scrolls in the intercom's API?

Expected Behavior

Can this error be handled to avoid this sync crashing?

Logs

logs-78-0.txt

Slack conversation

https://airbytehq.slack.com/archives/C01MFR03D5W/p1641437317000700?thread_ts=1641261197.116300&cid=C01MFR03D5W

Steps to Reproduce

The errors might be transient, other sync attempts do not have the same error.

[kirstenhipolito]

Some additional details: checking the tables on Postgres, there's data written in the _airbyte_raw_* tables, such as _airbyte_raw_contacts and _airbyte_raw_companies, and it looks like the number of rows is correct. But there are 0 rows in the normalized tables, e.g. Contacts and Companies, even though the logs say that normalization was completed.