Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GAds oauth demo #5996

Merged

Conversation

sherifnada
Copy link
Contributor

@sherifnada sherifnada commented Sep 11, 2021

What

This branch contains working code for demoing oauth with Google Ads. It builds on the branch chris/google-analytics-oauth (#5911).

How

  1. It uses Google Ads instead of Google Analytics because I realized the analytics connector doesn't support webflow oauth yet. Google Analytics source: support oauth  #4850
  2. It makes changes to allow using the GoogleOauthFlow as a superclass to perform oauth for all google connectors. The main changes are: injecting scope from the child class, and allowing the child class to specify how to parse the instancewide config to get client ID and client secret.
  3. Changes OauthConfigSupplier.injectConfig to handle merging nested configs. Previously it overwrote any keys. Now it merges any objects with their equivalent objects if they exist, prioritizing the fromConfig. This was needed to work with google ads as the connector nests its credentials inside the config.

Recommended reading order

For points 1 & 2:

  1. GoogleAnalyticsOauthFlow.java
  2. GoogleAdsOauthFlow.java
  3. GoogleOauthFlow.java

For point 3:

  1. OAuthConfigSupplier.java, OAuthConfigSupplierTest.java

Important Learnings from this PR

It uncovered a few points for improvement:

  1. Instancewide secrets should probably be tied to connector version https://github.com/airbytehq/airbyte-internal-issues/issues/237
  2. Instancewide secrets masks should applied with the correct type https://github.com/airbytehq/airbyte-internal-issues/issues/236
  3. The current assumption that instancewide secrets should always be injected is not necessarily correct. We should clarify the story around this OAuth flow: Instance-wide params should only be injected if a connector will be using oauth #5989.
  4. We consciously made this tradeoff to reduce delivery risk, but right now the fact that oauth logic for particular connectors is encoded in the server is pretty fragile. A single change in a connector's spec could cause a breaking change.
  5. We might want to have a spec that describes the instancewide variables schema expected for a particular connector/version. I don't love that the schema is not described anywhere.

@github-actions github-actions bot added area/api Related to the api area/platform issues related to the platform labels Sep 11, 2021
@@ -1184,6 +1184,29 @@ paths:
$ref: "#/components/responses/NotFoundResponse"
"422":
$ref: "#/components/responses/InvalidInputResponse"
/v1/source_oauths/oauth_params/create:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these were just moved for grouping with other oauth endpoints. no actual changes happened here.

@@ -69,15 +74,32 @@ public JsonNode injectDestinationOAuthParameters(UUID destinationDefinitionId, U
}
}

private void injectJsonNode(ObjectNode config, ObjectNode fromConfig) {
@VisibleForTesting
void injectJsonNode(ObjectNode mainConfig, ObjectNode fromConfig) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was cumbersome to test this via the calling methods so I exposed it here. Open to changing this if we think it's a bad idea.

public class GoogleAdsOauthFlow extends GoogleOAuthFlow {

public GoogleAdsOauthFlow(ConfigRepository configRepository) {
super(configRepository, "https://www.googleapis.com/auth/adwords");
Copy link
Contributor

@ChristopheDuong ChristopheDuong Sep 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw the GoogleAnalytics scope URL is encoded as https%3A//. We should probably handle URL encoding of query params automatically in the abstract class...

Copy link
Contributor

@ChristopheDuong ChristopheDuong Sep 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduce some additional code to handle these in #6018

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea. done

Comment on lines 58 to 59
private final String consentUrl = "https://accounts.google.com/o/oauth2/v2/auth";
private final String accessTokenUrl = "https://oauth2.googleapis.com/token";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious to know why these were changed from constant strings to attributes to the class?

Is it because you expect in the future to override these values when extending the base class maybe? In that case, is it more clear to initialize these in the constructor (as it is done with googleQueryParameters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially had them that way but then realized it's probably not going to be injected separately by each connector. Moved it back to static

public class GoogleAnalyticsOauthFlow extends GoogleOAuthFlow {

public GoogleAnalyticsOauthFlow(ConfigRepository configRepository) {
super(configRepository, "https%3A//www.googleapis.com/auth/analytics.readonly");
Copy link
Contributor

@ChristopheDuong ChristopheDuong Sep 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
super(configRepository, "https%3A//www.googleapis.com/auth/analytics.readonly");
super(configRepository, "https://www.googleapis.com/auth/analytics.readonly");

Comment on lines +103 to +104
// TODO state should be randomly generated, and the 2nd step of oauth should verify its value
// matches the initially generated state value
Copy link
Contributor

@ChristopheDuong ChristopheDuong Sep 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, the state value should be returned by the getConsentUrl and then, forwarded by the UI back to the completeOAuthFlow API endpoint?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe just passing the full "consentUrl" in the call from the UI? but that requires storing some state on that side.

@sherifnada sherifnada merged commit a50073e into chris/oauth-google-analytics Sep 13, 2021
@sherifnada sherifnada deleted the sherif/oauth-google-analytics-v2 branch September 13, 2021 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api Related to the api area/platform issues related to the platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants