🎉 Make credentials optional in BigQuery connector (issue #3657)

[sabifranjo]

What

Issue #3657

How

Solved by making the field optional and using default application credentials in code when the field is empty.

Pre-merge Checklist

Expand the checklist which is relevant for this PR.

Connector checklist

• Issue acceptance criteria met

• Unit & integration tests added as appropriate (and are passing)
  

• /test command documented here is passing.

  • Community members can skip this, Airbyters will run this for you.

• Code reviews completed

• Credentials added to Github CI if needed and not already present. instructions for injecting secrets into CI.

• Documentation updated

  • README
  • CHANGELOG.md
  • Reference docs in the docs/integrations/ directory.

• Build is successful

• Connector version bumped like described here

• New Connector version released on Dockerhub by running the /publish command described here

• No major blockers

• PR merged into master branch

• Follow up tickets have been created

• Associated tickets have been closed & stakeholders notified

[sherifnada]

@sabifranjo I am concerned that this is a breaking change/backwards incompatible. While I agree that this is the right way to solve this issue, we don't have a way to automatically gracefully upgrade connectors when a breaking change has been made to the config. Can we instead leave the schema the same and just remove credentials_json as a required field? We should also update its description to clarify that not including it would use the "default" authentication.

In the future once we have a graceful migration process for breaking changes on connectors this should be solved more elegantly.

[sabifranjo]

@sabifranjo I am concerned that this is a breaking change/backwards incompatible. While I agree that this is the right way to solve this issue, we don't have a way to automatically gracefully upgrade connectors when a breaking change has been made to the config. Can we instead leave the schema the same and just remove credentials_json as a required field? We should also update its description to clarify that not including it would use the "default" authentication.

In the future once we have a graceful migration process for breaking changes on connectors this should be solved more elegantly.

@sherifnada I did the changes and updated the description accordingly. So now only the destination-bigquery:0.3.5 image needs to be published.

[marcosmarxm]

@sabifranjo sorry took so long! I'm going to review this max later today! thanks for the contribution