Oauth for google redirect_uri error expecting HTTPS but sending HTTP

[gvillafanetapia]

Apache Airflow version:
Version: v2.1.0 (The problem still exists in v2.1.2)
Git Version: .release:2.1.0+304e174674ff6921cb7ed79c0158949b50eff8fe

Apache Airflow Provider versions : apache-airflow-providers-google==4.0.0

Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.13-eks-8df270", GitCommit:"8df2700a72a2598fa3a67c05126fa158fd839620", GitTreeState:"clean", BuildDate:"2021-07-31T01:36:57Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Cloud provider or hardware configuration: AWS
• OS: Debian GNU/Linux 10 (buster)
• Kernel: Linux 5.4.105-48.177.amzn2.x86_64
• Install tools: Helm version.BuildInfo{Version:"v3.4.2", GitCommit:"23dd3af5e19a02d4f4baa5b2f242645a1a3af629", GitTreeState:"clean", GoVersion:"go1.14.13"}

What happened:

On configuring Google Oauth login, when selecting an account to continue with we get a message from Google saying:

The redirect URI in the request, http://****/oauth-authorized/google, does not match the ones authorized for the OAuth client. To update the authorized redirect URIs, visit: https://console.developers.google.com/apis/credentials/oauthclient/****.apps.googleusercontent.com?project=*******

When visiting google cloud console there is no way of configuring a redirect_uri with HTTP (HTTPS is enforced now), and there is no way in Airflow to change the redirect_uri sent in the request.

What you expected to happen:

I expect Airflow to send in the oauth request my configured BASE_URL with HTTPS as the redirect_uri, or to use the redirect_uri configured in the OAUTH_PROVIDERS object variable set in webserver_config.py

How to reproduce it:

Get oauth credentials from google cloud console. Check that it won't let you set a redirect_uri with HTTP.

Install Airflow v2.1.* with the following webserver_config.py:

import os
      from airflow import configuration as conf
      from flask_appbuilder.security.manager import AUTH_OAUTH
      basedir = os.path.abspath(os.path.dirname(__file__))
 
      SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')
      CSRF_ENABLED = True
      AUTH_TYPE = AUTH_OAUTH
 
      # registration configs
      AUTH_USER_REGISTRATION = True  # allow users who are not already in the FAB DB
      AUTH_USER_REGISTRATION_ROLE = "Viewer"  # this role will be given in addition to any AUTH_ROLES_MAPPING
 
      GOOGLE_KEY = os.getenv('AIRFLOW_GOOGLE_CLIENT_ID', 'GOOGLE_KEY_NOT_SET')
      GOOGLE_SECRET = os.getenv('AIRFLOW_GOOGLE_CLIENT_SECRET', 'GOOGLE_SECRET_NOT_SET')
 
      # the list of providers which the user can choose from
      OAUTH_PROVIDERS = [{
          'name': 'google',
          'whitelist': ['@*****'],
          'token_key':'access_token',
          'icon':'fa-google',
          'remote_app': {
              'client_id': '*******.apps.googleusercontent.com',
              'client_secret': '*******',
              'api_base_url': 'https://www.googleapis.com/oauth2/v2/',
              'client_kwargs': {
                  'scope': 'email profile'
              },
              'request_token_url': None,
              'access_token_url': 'https://accounts.google.com/o/oauth2/token',
              'authorize_url': 'https://accounts.google.com/o/oauth2/auth'
          }
      }]

and these settings;

AIRFLOW__WEBSERVER__BASE_URL: "https://******" # SOME URI with HTTPS
AIRFLOW__WEBSERVER__AUTHENTICATE: "True"
AIRFLOW__WEBSERVER__AUTH_BACKEND: "airflow.contrib.auth.backends.google_auth"
AIRFLOW__WEBSERVER__RBAC: "True"

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[feluelle]

I can confirm this. Spent already a couple of hours or days trying to figure this out.

But tbh I think the bug is somewhere upstream. My guess is FAB.

I already tried a dozen things. In my case we have Airflow running behind a proxy, but the proxy fix did not solve this specific issue.
To better debug it, I used ksniff as we have Airflow running in K8s and I wanted to investigate the network traffic.
I used the following command to retrieve all http(s) get and post requests:

kubectl sniff <pod> -n airflow -p -f 'tcp dst port 80 or tcp dst port 443 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420 or tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504F5354 or tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x48545450 or tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x3C21444F' -o -

then I also piped this to tshark for better visualisation and I could only find http calls not a single https call being made.

I don't know what is going on there. Appreciate help as well! :)

[juanmanuelcarreira]

Hello!!!
We have exactly the same issue!!!

[jedcunningham]

I believe @feluelle is on the right track with this being a FAB issue.

Found this, which looks like it's the same issue: dpgaspar/Flask-AppBuilder#1666

You all should add any helpful context there. This is most likely a "wont-fix" from Airflow's side.

[potiuk]

I am going to collaborate with @dpgaspar soon (maybe even this week) on fixing some other - related - FAB issues, so I will add that to the list.

[gvillafanetapia]

Thank you for taking notice of this!

In the meantime, I found a workaround that got us up and running: setting AIRFLOW__WEBSERVER__ENABLE_PROXY_FIX to True.

[sensei100]

We solved this by adding @gvillafanetapia's workaround in the airflow.cfg file under the webserver section. This solution also works for Airflow 1.10.15.

[webserver]
enable_proxy_fix = True

[zachliu]

i hereby confirm the workaround works on 2.2.0 🎉

also, evil google's documentation is not clear on this part (probably a 🐛 on google's end):

In the GCP console, under OAuth consent screen, when the Publishing status is In production, we can still put http://localhost:8080/oauth-authorized/google under the Authorized redirect URIs without triggering the red error message Invalid Redirect. However, it doesn't work unless the app is in Testing status.

[potiuk]

Cool. BTW. Maybe you can submit a change to Google about the docs ? They have a great system (developed by the team I worked at in Cracow years ago) that you can visually mark errors on the page with docs and suggest a change. It will not open PR with the change, but i think there are some people (or bots now) looking at the feedback. I think you can find 'provide feedback for this page' or similar link on each page at the bottom :)

[zachliu]

yeah, i knew that system but Google seems to adopt the Facebook way "move fast and break things" so their documentation is always behind 😿 anyway, i already submitted the feedback to Google 🤞

[vparmeland]

I encountered the same issue for my Gitlab OAUTH :/ (My Airflow service is behind an AWS ALB)
Does anyone know a workaround ?

[potiuk]

@nk74 - ask in Flask App Builder forums. Here are docs about authentication in Flask App Builder https://flask-appbuilder.readthedocs.io/en/latest/security.html

[potiuk]

Converting it into discussion - as this is FAB issue not an Airflow one.