2022-05-18 19:51:59 INFO i.a.w.w.WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.38.4-alpha 2022-05-18 19:51:59 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Cloud storage job log path: /workspace/402/2/logs.log 2022-05-18 19:52:03 destination > SLF4J: Class path contains multiple SLF4J bindings. 2022-05-18 19:52:41 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $.access_token: object found, string expected, $.credentials: does not have a value in the enumeration [oauth2.0], $.credentials: must be a constant value oauth2.0 2022-05-18 19:52:41 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $.access_token: object found, string expected 2022-05-18 19:52:41 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $.credential: is not defined in the schema and the schema does not allow additional properties, $.part_size_mb: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_name: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_path: is not defined in the schema and the schema does not allow additional properties, $.keep_files_in_gcs-bucket: is not defined in the schema and the schema does not allow additional properties, $.method: must be a constant value Standard 2022-05-18 19:52:41 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $.credential.hmac_key_access_id: object found, string expected, $.credential.hmac_key_secret: object found, string expected 2022-05-18 19:51:59 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.38.4-alpha 2022-05-18 19:51:59 INFO i.a.w.DefaultReplicationWorker(run):109 - start sync worker. job id: 402 attempt id: 2 2022-05-18 19:51:59 INFO i.a.w.DefaultReplicationWorker(run):121 - configured sync modes: {null.accounts=full_refresh - overwrite} 2022-05-18 19:51:59 INFO i.a.w.p.a.DefaultAirbyteDestination(start):69 - Running destination... 2022-05-18 19:51:59 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = destination-bigquery-write-402-2-qamyv for airbyte/destination-bigquery:1.0.1 2022-05-18 19:51:59 INFO i.a.w.p.KubeProcessFactory(create):103 - destination-bigquery-write-402-2-qamyv stdoutLocalPort = 9008 2022-05-18 19:51:59 INFO i.a.w.p.KubeProcessFactory(create):106 - destination-bigquery-write-402-2-qamyv stderrLocalPort = 9009 2022-05-18 19:51:59 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):583 - Creating stdout socket server... 2022-05-18 19:51:59 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):601 - Creating stderr socket server... 2022-05-18 19:51:59 INFO i.a.w.p.KubePodProcess():515 - Creating pod destination-bigquery-write-402-2-qamyv... 2022-05-18 19:52:00 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files... 2022-05-18 19:52:00 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Init container present.. 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):312 - Init container ready.. 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess():546 - Copying files... 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_config.json 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/65fee0af-8b4f-49a5-9795-ef5325f31855/destination_config.json prod/destination-bigquery-write-402-2-qamyv:/config/destination_config.json -c init 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_catalog.json 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/137a4c02-7a79-4511-b77d-0f9e6190c4d5/destination_catalog.json prod/destination-bigquery-write-402-2-qamyv:/config/destination_catalog.json -c init 2022-05-18 19:52:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/5a5ae4c2-346e-4cbe-98eb-5da0068d4e30/FINISHED_UPLOADING prod/destination-bigquery-write-402-2-qamyv:/config/FINISHED_UPLOADING -c init 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess():549 - Waiting until pod is ready... 2022-05-18 19:52:02 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):592 - Setting stdout... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):604 - Setting stderr... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess():564 - Reading pod IP... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess():566 - Pod IP: 172.19.6.12 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess():569 - Creating stdin socket... 2022-05-18 19:52:03 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = source-zendesk-chat-read-402-2-uvgmw for us-east1-docker.pkg.dev/bi-data-science/airbyte/source-zendesk-chat:0.0.1 2022-05-18 19:52:03 INFO i.a.w.p.KubeProcessFactory(create):103 - source-zendesk-chat-read-402-2-uvgmw stdoutLocalPort = 9010 2022-05-18 19:52:03 INFO i.a.w.p.KubeProcessFactory(create):106 - source-zendesk-chat-read-402-2-uvgmw stderrLocalPort = 9011 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):583 - Creating stdout socket server... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):601 - Creating stderr socket server... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess():515 - Creating pod source-zendesk-chat-read-402-2-uvgmw... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files... 2022-05-18 19:52:03 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Init container present.. 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):312 - Init container ready.. 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess():546 - Copying files... 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: source_config.json 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/b39a6f17-ab87-45f8-9480-53acd838f53f/source_config.json prod/source-zendesk-chat-read-402-2-uvgmw:/config/source_config.json -c init 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: source_catalog.json 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/6b56f1c8-46e2-4457-9672-4a405306ef61/source_catalog.json prod/source-zendesk-chat-read-402-2-uvgmw:/config/source_catalog.json -c init 2022-05-18 19:52:05 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:06 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:06 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING 2022-05-18 19:52:06 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/fc49e48c-f0e7-406d-96dd-6e3fa35ac179/FINISHED_UPLOADING prod/source-zendesk-chat-read-402-2-uvgmw:/config/FINISHED_UPLOADING -c init 2022-05-18 19:52:06 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:06 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:06 INFO i.a.w.p.KubePodProcess():549 - Waiting until pod is ready... 2022-05-18 19:52:07 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):592 - Setting stdout... 2022-05-18 19:52:07 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):604 - Setting stderr... 2022-05-18 19:52:08 INFO i.a.w.p.KubePodProcess():564 - Reading pod IP... 2022-05-18 19:52:08 INFO i.a.w.p.KubePodProcess():566 - Pod IP: 172.19.3.57 2022-05-18 19:52:08 INFO i.a.w.p.KubePodProcess():573 - Using null stdin output stream... 2022-05-18 19:52:08 INFO i.a.w.DefaultReplicationWorker(run):163 - Waiting for source and destination threads to complete. 2022-05-18 19:52:08 INFO i.a.w.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):382 - Destination output thread started. 2022-05-18 19:52:08 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$6):292 - Replication thread started. 2022-05-18 19:52:08 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$6):334 - Source has no more messages, closing connection. 2022-05-18 19:52:09 INFO i.a.w.p.KubePodProcess(close):713 - (pod: prod / source-zendesk-chat-read-402-2-uvgmw) - Closed all resources for pod 2022-05-18 19:52:18 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. errors: $: null found, object expected 2022-05-18 19:52:18 ERROR i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null 2022-05-18 19:52:19 INFO i.a.w.p.KubePodProcess(close):713 - (pod: prod / destination-bigquery-write-402-2-qamyv) - Closed all resources for pod 2022-05-18 19:52:19 ERROR i.a.w.DefaultReplicationWorker(run):174 - Sync worker failed. java.util.concurrent.ExecutionException: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source cannot be stopped! at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:167) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:59) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] Suppressed: io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:131) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:59) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] Caused by: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source cannot be stopped! at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:338) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] ... 1 more Caused by: io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:336) ~[io.airbyte-airbyte-workers-0.38.4-alpha.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] ... 1 more 2022-05-18 19:52:19 INFO i.a.w.DefaultReplicationWorker(run):233 - sync summary: io.airbyte.config.ReplicationAttemptSummary@42c287e[status=failed,recordsSynced=0,bytesSynced=0,startTime=1652903519947,endTime=1652903539662,totalStats=io.airbyte.config.SyncStats@16d77ef2[recordsEmitted=0,bytesEmitted=0,stateMessagesEmitted=0,recordsCommitted=0],streamStats=[]] 2022-05-18 19:52:19 INFO i.a.w.DefaultReplicationWorker(run):262 - Source did not output any state messages 2022-05-18 19:52:19 WARN i.a.w.DefaultReplicationWorker(run):273 - State capture: No state retained. 2022-05-18 19:52:19 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-05-18 19:52:19 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):157 - sync summary: io.airbyte.config.StandardSyncOutput@9f65d3a[standardSyncSummary=io.airbyte.config.StandardSyncSummary@352fb173[status=failed,recordsSynced=0,bytesSynced=0,startTime=1652903519947,endTime=1652903539662,totalStats=io.airbyte.config.SyncStats@16d77ef2[recordsEmitted=0,bytesEmitted=0,stateMessagesEmitted=0,recordsCommitted=0],streamStats=[]],normalizationSummary=,state=,outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@59598734[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@a99d153[stream=io.airbyte.protocol.models.AirbyteStream@313d42b7[name=accounts,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"plan":{"type":["null","object"],"properties":{"sla":{"type":["null","boolean"]},"name":{"type":["null","string"]},"goals":{"type":["null","integer"]},"price":{"type":["null","number"],"maximum":1000000000000000000000000000000000,"minimum":-1000000000000000000000000000000000},"support":{"type":["null","boolean"]},"rest_api":{"type":["null","boolean"]},"analytics":{"type":["null","boolean"]},"high_load":{"type":["null","boolean"]},"long_desc":{"type":["null","string"]},"max_agents":{"type":["null","integer"]},"monitoring":{"type":["null","boolean"]},"short_desc":{"type":["null","string"]},"unbranding":{"type":["null","boolean"]},"file_upload":{"type":["null","boolean"]},"chat_reports":{"type":["null","boolean"]},"integrations":{"type":["null","boolean"]},"agent_reports":{"type":["null","boolean"]},"daily_reports":{"type":["null","boolean"]},"email_reports":{"type":["null","boolean"]},"ip_restriction":{"type":["null","boolean"]},"max_departments":{"type":["null","string"]},"operating_hours":{"type":["null","boolean"]},"agent_leaderboard":{"type":["null","boolean"]},"max_basic_triggers":{"type":["null","string"]},"max_concurrent_chats":{"type":["null","string"]},"widget_customization":{"type":["null","string"]},"max_advanced_triggers":{"type":["null","string"]},"max_history_search_days":{"type":["null","string"]}}},"status":{"type":["null","string"]},"billing":{"type":["null","object"],"properties":{"city":{"type":["null","string"]},"cycle":{"type":["null","integer"]},"email":{"type":["null","string"]},"phone":{"type":["null","string"]},"state":{"type":["null","string"]},"company":{"type":["null","string"]},"address1":{"type":["null","string"]},"address2":{"type":["null","string"]},"last_name":{"type":["null","string"]},"first_name":{"type":["null","string"]},"postal_code":{"type":["null","string"]},"country_code":{"type":["null","string"]},"additional_info":{"type":["null","string"]}}},"account_key":{"type":["null","string"]},"create_date":{"type":["null","string"],"format":"date-time"}}},supportedSyncModes=[full_refresh],sourceDefinedCursor=,defaultCursorField=[],sourceDefinedPrimaryKey=[[account_key]],namespace=,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=overwrite,primaryKey=[[account_key]],additionalProperties={}]],additionalProperties={}],failures=[io.airbyte.config.FailureReason@7951d9ca[failureOrigin=source,failureType=system_error,internalMessage=module 'pendulum' has no attribute 'DateTime',externalMessage=Something went wrong in the connector. See the logs for more details.,metadata=io.airbyte.config.Metadata@6c2491f3[additionalProperties={attemptNumber=2, jobId=402, from_trace_message=true}],stacktrace=Traceback (most recent call last): File "/airbyte/integration_code/main_dev.py", line 9, in from source_zendesk_chat import SourceZendeskChat File "/airbyte/integration_code/source_zendesk_chat/__init__.py", line 23, in from .source import SourceZendeskChat File "/airbyte/integration_code/source_zendesk_chat/source.py", line 11, in from airbyte_cdk.sources.streams.http.requests_native_auth import TokenAuthenticator File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/__init__.py", line 5, in from .oauth import Oauth2Authenticator File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py", line 13, in class Oauth2Authenticator(AuthBase): File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py", line 26, in Oauth2Authenticator token_expiry_date: pendulum.DateTime = None, AttributeError: module 'pendulum' has no attribute 'DateTime' ,retryable=,timestamp=1652903528444], io.airbyte.config.FailureReason@28f3c66d[failureOrigin=source,failureType=,internalMessage=io.airbyte.workers.DefaultReplicationWorker$SourceException: Source cannot be stopped!,externalMessage=Something went wrong within the source connector,metadata=io.airbyte.config.Metadata@7380013e[additionalProperties={attemptNumber=2, jobId=402}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source cannot be stopped! at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source cannot be stopped! at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:338) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more Caused by: io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:336) ... 4 more ,retryable=,timestamp=1652903529531]]] 2022-05-18 19:52:19 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):236 - Stopping temporal heartbeating... 2022-05-18 19:52:19 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Cloud storage job log path: /workspace/402/2/logs.log 2022-05-18 19:52:19 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.38.4-alpha 2022-05-18 19:52:19 INFO i.a.w.DefaultNormalizationWorker(run):47 - Running normalization. 2022-05-18 19:52:19 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):122 - Running with normalization version: airbyte/normalization:0.2.0 2022-05-18 19:52:19 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = normalization-normalize-402-2-vnzlv for airbyte/normalization:0.2.0 2022-05-18 19:52:19 INFO i.a.w.p.KubeProcessFactory(create):103 - normalization-normalize-402-2-vnzlv stdoutLocalPort = 9012 2022-05-18 19:52:19 INFO i.a.w.p.KubeProcessFactory(create):106 - normalization-normalize-402-2-vnzlv stderrLocalPort = 9013 2022-05-18 19:52:19 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):583 - Creating stdout socket server... 2022-05-18 19:52:19 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):601 - Creating stderr socket server... 2022-05-18 19:52:19 INFO i.a.w.p.KubePodProcess():515 - Creating pod normalization-normalize-402-2-vnzlv... 2022-05-18 19:52:19 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files... 2022-05-18 19:52:19 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Init container present.. 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):312 - Init container ready.. 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess():546 - Copying files... 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_config.json 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/9e444379-344a-494e-a740-e4c41dc49acb/destination_config.json prod/normalization-normalize-402-2-vnzlv:/config/destination_config.json -c init 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_catalog.json 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/58425240-a480-4433-b4ff-405523e7ecca/destination_catalog.json prod/normalization-normalize-402-2-vnzlv:/config/destination_catalog.json -c init 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/90dd184b-df06-4c65-bda2-7a14232f75df/FINISHED_UPLOADING prod/normalization-normalize-402-2-vnzlv:/config/FINISHED_UPLOADING -c init 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-05-18 19:52:21 INFO i.a.w.p.KubePodProcess():549 - Waiting until pod is ready... 2022-05-18 19:52:22 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):592 - Setting stdout... 2022-05-18 19:52:22 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):604 - Setting stderr... 2022-05-18 19:52:23 INFO i.a.w.p.KubePodProcess():564 - Reading pod IP... 2022-05-18 19:52:23 INFO i.a.w.p.KubePodProcess():566 - Pod IP: 172.19.2.201 2022-05-18 19:52:23 INFO i.a.w.p.KubePodProcess():573 - Using null stdin output stream... 2022-05-18 19:52:41 INFO i.a.w.p.KubePodProcess(close):713 - (pod: prod / normalization-normalize-402-2-vnzlv) - Closed all resources for pod 2022-05-18 19:52:41 INFO i.a.w.DefaultNormalizationWorker(run):71 - Normalization executed in 21 seconds. 2022-05-18 19:52:41 INFO i.a.w.DefaultNormalizationWorker(run):77 - Normalization summary: io.airbyte.config.NormalizationSummary@3a50f0d0[startTime=1652903539841,endTime=1652903561339] 2022-05-18 19:52:41 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-05-18 19:52:41 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):236 - Stopping temporal heartbeating... 2022-05-18 19:52:03 destination > SLF4J: Found binding in [jar:file:/airbyte/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2022-05-18 19:52:03 destination > SLF4J: Found binding in [jar:file:/airbyte/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2022-05-18 19:52:03 destination > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2022-05-18 19:52:03 destination > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 INFO i.a.i.b.IntegrationRunner(createSentryTransaction):339 - Sentry transaction event: 06914f22cacf4517b83de44a8b4e3b08 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 INFO i.a.i.b.IntegrationRunner(runInternal):121 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 INFO i.a.i.b.IntegrationRunner(runInternal):122 - Command: WRITE 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 INFO i.a.i.b.IntegrationRunner(runInternal):123 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-05-18 19:52:08 destination > 2022-05-18 19:52:04 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-05-18 19:52:08 destination > 2022-05-18 19:52:05 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):281 - Selected loading method is set to: GCS 2022-05-18 19:52:08 source > module 'pendulum' has no attribute 'DateTime' Traceback (most recent call last): File "/airbyte/integration_code/main_dev.py", line 9, in from source_zendesk_chat import SourceZendeskChat File "/airbyte/integration_code/source_zendesk_chat/__init__.py", line 23, in from .source import SourceZendeskChat File "/airbyte/integration_code/source_zendesk_chat/source.py", line 11, in from airbyte_cdk.sources.streams.http.requests_native_auth import TokenAuthenticator File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/__init__.py", line 5, in from .oauth import Oauth2Authenticator File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py", line 13, in class Oauth2Authenticator(AuthBase): File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py", line 26, in Oauth2Authenticator token_expiry_date: pendulum.DateTime = None, AttributeError: module 'pendulum' has no attribute 'DateTime' 2022-05-18 19:52:08 destination > 2022-05-18 19:52:06 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):124 - Partitioned Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, tableId=_airbyte_tmp_vrr_accounts}} created successfully 2022-05-18 19:52:08 destination > 2022-05-18 19:52:06 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"AVRO","flattening":"No flattening","part_size_mb":"5"} 2022-05-18 19:52:08 destination > 2022-05-18 19:52:06 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):23 - Default schema. 2022-05-18 19:52:08 destination > 2022-05-18 19:52:06 INFO i.a.i.d.g.a.GcsAvroWriter():67 - Avro schema for stream accounts: {"type":"record","name":"accounts","fields":[{"name":"_airbyte_ab_id","type":{"type":"string","logicalType":"uuid"}},{"name":"_airbyte_emitted_at","type":{"type":"long","logicalType":"timestamp-millis"}},{"name":"_airbyte_data","type":"string"}]} 2022-05-18 19:52:08 destination > 2022-05-18 19:52:06 INFO i.a.i.d.g.a.GcsAvroWriter():73 - Full GCS path for stream 'accounts': hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro 2022-05-18 19:52:08 destination > 2022-05-18 19:52:06 INFO i.a.i.d.s.u.S3StreamTransferManagerHelper(getDefault):55 - PartSize arg is set to 5 MB 2022-05-18 19:52:08 destination > 2022-05-18 19:52:07 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro with full ID ABPnzm6HliYdw96qVm5kOVF0Yb3de5UkaVdv0aEw8bzuN-p8NviCyvQgmm9TR3yDkbvInl4 2022-05-18 19:52:08 destination > 2022-05-18 19:52:07 INFO i.a.i.d.g.w.BaseGcsWriter(initialize):73 - Overwrite mode 2022-05-18 19:52:08 destination > 2022-05-18 19:52:07 INFO i.a.i.d.g.w.BaseGcsWriter(initialize):90 - Overwrite is finished 2022-05-18 19:52:08 destination > 2022-05-18 19:52:07 INFO i.a.i.d.b.BigQueryUtils(isKeepFilesInGcs):297 - All tmp files will be removed from GCS when replication is finished 2022-05-18 19:52:09 destination > 2022-05-18 19:52:09 ERROR i.a.i.b.IntegrationRunner(consumeWriteStream):184 - Received invalid message: {"type":"TRACE","trace":{"type":"ERROR","emitted_at":1.652903528444904E12,"error":{"message":"Something went wrong in the connector. See the logs for more details.","internal_message":"module 'pendulum' has no attribute 'DateTime'","stack_trace":"Traceback (most recent call last):\n File \"/airbyte/integration_code/main_dev.py\", line 9, in \n from source_zendesk_chat import SourceZendeskChat\n File \"/airbyte/integration_code/source_zendesk_chat/__init__.py\", line 23, in \n from .source import SourceZendeskChat\n File \"/airbyte/integration_code/source_zendesk_chat/source.py\", line 11, in \n from airbyte_cdk.sources.streams.http.requests_native_auth import TokenAuthenticator\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/__init__.py\", line 5, in \n from .oauth import Oauth2Authenticator\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py\", line 13, in \n class Oauth2Authenticator(AuthBase):\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py\", line 26, in Oauth2Authenticator\n token_expiry_date: pendulum.DateTime = None,\nAttributeError: module 'pendulum' has no attribute 'DateTime'\n","failure_type":"system_error"}}} 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 WARN i.a.i.b.IntegrationRunner(watchForOrphanThreads):229 - The main thread is exiting while children non-daemon threads from a connector are still active. 2022-05-18 19:52:10 destination > Ideally, this situation should not happen... 2022-05-18 19:52:10 destination > Please check with maintainers if the connector or library code should safely clean up its threads before quitting instead. 2022-05-18 19:52:10 destination > The main thread is: main (RUNNABLE) 2022-05-18 19:52:10 destination > Thread stacktrace: java.base/java.lang.Thread.getStackTrace(Thread.java:1610) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.IntegrationRunner.dumpThread(IntegrationRunner.java:264) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.IntegrationRunner.watchForOrphanThreads(IntegrationRunner.java:233) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.IntegrationRunner.runConsumer(IntegrationRunner.java:190) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.IntegrationRunner.lambda$runInternal$1(IntegrationRunner.java:163) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:38) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:163) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:105) 2022-05-18 19:52:10 destination > at io.airbyte.integrations.destination.bigquery.BigQueryDestination.main(BigQueryDestination.java:262) 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 WARN i.a.i.b.IntegrationRunner(watchForOrphanThreads):243 - Active non-daemon thread: pool-4-thread-1 (WAITING) 2022-05-18 19:52:10 destination > Thread stacktrace: java.base@17.0.1/jdk.internal.misc.Unsafe.park(Native Method) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:341) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:506) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3463) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3434) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1623) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:420) 2022-05-18 19:52:10 destination > at app//alex.mojaki.s3upload.StreamTransferManager$UploadTask.call(StreamTransferManager.java:484) 2022-05-18 19:52:10 destination > at app//alex.mojaki.s3upload.StreamTransferManager$UploadTask.call(StreamTransferManager.java:474) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.FutureTask.run(FutureTask.java:264) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.FutureTask.run(FutureTask.java:264) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.lang.Thread.run(Thread.java:833) 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 WARN i.a.i.b.IntegrationRunner(watchForOrphanThreads):243 - Active non-daemon thread: pool-4-thread-2 (BLOCKED) 2022-05-18 19:52:10 destination > Thread stacktrace: app//alex.mojaki.s3upload.StreamTransferManager$UploadTask.call(StreamTransferManager.java:483) 2022-05-18 19:52:10 destination > at app//alex.mojaki.s3upload.StreamTransferManager$UploadTask.call(StreamTransferManager.java:474) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.FutureTask.run(FutureTask.java:264) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.FutureTask.run(FutureTask.java:264) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) 2022-05-18 19:52:10 destination > at java.base@17.0.1/java.lang.Thread.run(Thread.java:833) 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):65 - Airbyte message consumer: succeeded. 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO i.a.i.d.b.BigQueryRecordConsumer(close):58 - Started closing all connections 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO i.a.i.d.b.u.AbstractBigQueryUploader(close):76 - Field fails during format : 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO i.a.i.d.b.f.BigQueryRecordFormatter(printAndCleanFieldFails):70 - No field fails during record format. 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO i.a.i.d.b.u.AbstractBigQueryUploader(close):79 - Closing connector:AbstractBigQueryUploader{table=_airbyte_raw_accounts, tmpTable=_airbyte_tmp_vrr_accounts, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.gcs.avro.GcsAvroWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.GcsAvroBigQueryRecordFormatter} 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO i.a.i.d.g.w.BaseGcsWriter(close):120 - Uploading remaining data for stream 'accounts'. 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro with id ABPnzm6Hl...yDkbvInl4]: Uploading leftover stream [Part number 1 containing 0.00 MB] 2022-05-18 19:52:10 destination > 2022-05-18 19:52:10 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro with id ABPnzm6Hl...yDkbvInl4]: Finished uploading [Part number 1 containing 0.00 MB] 2022-05-18 19:52:11 destination > 2022-05-18 19:52:11 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro with id ABPnzm6Hl...yDkbvInl4]: Completed 2022-05-18 19:52:11 destination > 2022-05-18 19:52:11 INFO i.a.i.d.g.w.BaseGcsWriter(close):122 - Upload completed for stream 'accounts'. 2022-05-18 19:52:11 destination > 2022-05-18 19:52:11 INFO i.a.i.d.b.u.AbstractGscBigQueryUploader(uploadData):57 - Uploading data to the tmp table _airbyte_tmp_vrr_accounts. 2022-05-18 19:52:11 destination > 2022-05-18 19:52:11 INFO i.a.i.d.b.u.AbstractGscBigQueryUploader(uploadDataFromFileToTmpTable):68 - Started copying data from gs://hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro GCS avro file to GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, tableId=_airbyte_tmp_vrr_accounts}} tmp BigQuery table with schema: 2022-05-18 19:52:11 destination > Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]} 2022-05-18 19:52:11 destination > 2022-05-18 19:52:11 INFO i.a.i.d.b.u.AbstractGscBigQueryUploader(uploadDataFromFileToTmpTable):77 - Created a new job GCS avro file to tmp BigQuery table: Job{job=JobId{project=bi-data-science, job=c4101229-f61b-4149-bf51-babb2bd1aed7, location=US}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1652903531235, endTime=null, startTime=1652903531376, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=airbyte@bi-data-science.iam.gserviceaccount.com, etag=MWBitAiHTzAFyBy2MKBNFw==, generatedId=bi-data-science:US.c4101229-f61b-4149-bf51-babb2bd1aed7, selfLink=https://www.googleapis.com/bigquery/v2/projects/bi-data-science/jobs/c4101229-f61b-4149-bf51-babb2bd1aed7?location=US, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, projectId=bi-data-science, tableId=_airbyte_tmp_vrr_accounts}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=null, writeDisposition=WRITE_TRUNCATE, formatOptions=FormatOptions{format=AVRO}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=[gs://hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro], schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=true, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} 2022-05-18 19:52:11 destination > 2022-05-18 19:52:11 INFO i.a.i.d.b.BigQueryUtils(waitForJobFinish):305 - Waiting for job finish Job{job=JobId{project=bi-data-science, job=c4101229-f61b-4149-bf51-babb2bd1aed7, location=US}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1652903531235, endTime=null, startTime=1652903531376, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=airbyte@bi-data-science.iam.gserviceaccount.com, etag=MWBitAiHTzAFyBy2MKBNFw==, generatedId=bi-data-science:US.c4101229-f61b-4149-bf51-babb2bd1aed7, selfLink=https://www.googleapis.com/bigquery/v2/projects/bi-data-science/jobs/c4101229-f61b-4149-bf51-babb2bd1aed7?location=US, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, projectId=bi-data-science, tableId=_airbyte_tmp_vrr_accounts}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=null, writeDisposition=WRITE_TRUNCATE, formatOptions=FormatOptions{format=AVRO}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=[gs://hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro], schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=true, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}}. Status: JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-05-18 19:52:14 destination > 2022-05-18 19:52:14 INFO i.a.i.d.b.BigQueryUtils(waitForJobFinish):307 - Job finish Job{job=JobId{project=bi-data-science, job=c4101229-f61b-4149-bf51-babb2bd1aed7, location=US}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1652903531235, endTime=null, startTime=1652903531376, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=airbyte@bi-data-science.iam.gserviceaccount.com, etag=MWBitAiHTzAFyBy2MKBNFw==, generatedId=bi-data-science:US.c4101229-f61b-4149-bf51-babb2bd1aed7, selfLink=https://www.googleapis.com/bigquery/v2/projects/bi-data-science/jobs/c4101229-f61b-4149-bf51-babb2bd1aed7?location=US, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, projectId=bi-data-science, tableId=_airbyte_tmp_vrr_accounts}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=null, writeDisposition=WRITE_TRUNCATE, formatOptions=FormatOptions{format=AVRO}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=[gs://hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro], schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=true, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} with status JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-05-18 19:52:14 destination > 2022-05-18 19:52:14 INFO i.a.i.d.b.u.AbstractGscBigQueryUploader(uploadDataFromFileToTmpTable):83 - Table is successfully overwritten by avro file loaded from GCS 2022-05-18 19:52:14 destination > 2022-05-18 19:52:14 INFO i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):96 - Uploading data from the tmp table _airbyte_tmp_vrr_accounts to the source table _airbyte_raw_accounts. 2022-05-18 19:52:14 destination > 2022-05-18 19:52:14 INFO i.a.i.d.b.u.AbstractBigQueryUploader(uploadDataToTableFromTmpTable):121 - Replication finished with no explicit errors. Copying data from tmp tables to permanent 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractBigQueryUploader(copyTable):187 - successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, tableId=_airbyte_tmp_vrr_accounts}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=zendesk_chat, tableId=_airbyte_raw_accounts}} 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):98 - Data is successfully loaded to the source table _airbyte_raw_accounts! 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):100 - Final state message is accepted. 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):112 - Removing tmp tables... 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):114 - Finishing destination process...completed 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractGscBigQueryUploader(deleteGcsFiles):97 - Deleting file gs://hurb_airbyte/zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractGscBigQueryUploader(lambda$deleteGcsFiles$1):110 - File is deleted : zendesk_chat_bigquery/accounts/2022_05_18_1652903526467_0.avro 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.d.b.u.AbstractBigQueryUploader(close):86 - Closed connector:AbstractBigQueryUploader{table=_airbyte_raw_accounts, tmpTable=_airbyte_tmp_vrr_accounts, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.gcs.avro.GcsAvroWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.GcsAvroBigQueryRecordFormatter} 2022-05-18 19:52:18 destination > 2022-05-18 19:52:18 INFO i.a.i.b.IntegrationRunner(runInternal):169 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-05-18 19:52:23 normalization > Running: transform-config --config destination_config.json --integration-type bigquery --out /config 2022-05-18 19:52:23 normalization > Namespace(config='destination_config.json', integration_type=, out='/config') 2022-05-18 19:52:23 normalization > transform_bigquery 2022-05-18 19:52:23 normalization > Running: transform-catalog --integration-type bigquery --profile-config-dir /config --catalog destination_catalog.json --out /config/models/generated/ --json-column _airbyte_data 2022-05-18 19:52:23 normalization > Processing destination_catalog.json... 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_ab1.sql from accounts 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_ab2.sql from accounts 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_ab3.sql from accounts 2022-05-18 19:52:23 normalization > Generating airbyte_tables/zendesk_chat/accounts.sql from accounts 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_plan_ab1.sql from accounts/plan 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_plan_ab2.sql from accounts/plan 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_plan_ab3.sql from accounts/plan 2022-05-18 19:52:23 normalization > Generating airbyte_tables/zendesk_chat/accounts_plan.sql from accounts/plan 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_billing_ab1.sql from accounts/billing 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_billing_ab2.sql from accounts/billing 2022-05-18 19:52:23 normalization > Generating airbyte_ctes/zendesk_chat/accounts_billing_ab3.sql from accounts/billing 2022-05-18 19:52:23 normalization > Generating airbyte_tables/zendesk_chat/accounts_billing.sql from accounts/billing 2022-05-18 19:52:23 normalization > detected no config file for ssh, assuming ssh is off. 2022-05-18 19:52:29 normalization > [--event-buffer-size EVENT_BUFFER_SIZE] 2022-05-18 19:52:29 normalization > --event-buffer-size EVENT_BUFFER_SIZE 2022-05-18 19:52:29 normalization > 2022-05-18 19:52:29 normalization > DBT >=1.0.0 detected; using 10K event buffer size 2022-05-18 19:52:29 normalization > 2022-05-18 19:52:32 normalization > 19:52:32 Running with dbt=1.0.0 2022-05-18 19:52:32 normalization > 19:52:32 Partial parse save file not found. Starting full parse. 2022-05-18 19:52:33 normalization > 19:52:33 [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources. 2022-05-18 19:52:33 normalization > There are 2 unused configuration paths: 2022-05-18 19:52:33 normalization > - models.airbyte_utils.generated.airbyte_views 2022-05-18 19:52:33 normalization > - models.airbyte_utils.generated.airbyte_incremental 2022-05-18 19:52:33 normalization > 2022-05-18 19:52:33 normalization > 19:52:33 Found 12 models, 0 tests, 0 snapshots, 0 analyses, 544 macros, 0 operations, 0 seed files, 1 source, 0 exposures, 0 metrics 2022-05-18 19:52:33 normalization > 19:52:33 2022-05-18 19:52:34 normalization > 19:52:34 Concurrency: 8 threads (target='prod') 2022-05-18 19:52:34 normalization > 19:52:34 2022-05-18 19:52:34 normalization > 19:52:34 1 of 3 START table model zendesk_chat.accounts.......................................................................... [RUN] 2022-05-18 19:52:37 normalization > 19:52:37 1 of 3 OK created table model zendesk_chat.accounts..................................................................... [CREATE TABLE (0.0 rows, 0 processed) in 2.76s] 2022-05-18 19:52:37 normalization > 19:52:37 2 of 3 START table model zendesk_chat.accounts_billing.................................................................. [RUN] 2022-05-18 19:52:37 normalization > 19:52:37 3 of 3 START table model zendesk_chat.accounts_plan..................................................................... [RUN] 2022-05-18 19:52:40 normalization > 19:52:40 2 of 3 OK created table model zendesk_chat.accounts_billing............................................................. [CREATE TABLE (0.0 rows, 0 processed) in 2.94s] 2022-05-18 19:52:40 normalization > 19:52:40 3 of 3 OK created table model zendesk_chat.accounts_plan................................................................ [CREATE TABLE (0.0 rows, 0 processed) in 2.87s] 2022-05-18 19:52:40 normalization > 19:52:40 2022-05-18 19:52:40 normalization > 19:52:40 Finished running 3 table models in 6.62s. 2022-05-18 19:52:40 normalization > 19:52:40 2022-05-18 19:52:40 normalization > 19:52:40 Completed successfully 2022-05-18 19:52:40 normalization > 19:52:40 2022-05-18 19:52:40 normalization > 19:52:40 Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3