FIX to -> Username or password is incorrect (LoginException('BadAuthentication'))

[djsudduth]

AS OF JAN 2024 THIS METHOD BELOW DOES NOT WORK - THE ONLY METHOD THAT WORKS RIGHT NOW IS IN THE NEW VERSION 0.6.0 (RELEASED 1/28/2024) INSTRUCTIONS. ADDITIONAL INFORMATION IN THE MESSAGE BELOW:

#72 (comment)

The easiest fix is to install a fresh version Python 3.9.1, run KIM and then upgrade to >= 3.10.x once the Google token is saved in your keyring.

To do this execute a fresh install of 3.9.1, run pip install urllib3==1.25.1 and pip install gpsoauth==1.0.2, and then run KIM. Once that is done, then upgrade to a higher version of Python.

But, if you've already installed 3.10.x or 3.11.x do this:

1) This assumes you already had default system install of Python 3.10.x or 3.11.x
2) For Windows, install pyenv-win and then install Python 3.9.1 using the command pyenv install 3.9.1 (if you have a Mac or Linux - see this page for installation instructions -> https://github.com/pyenv/pyenv )
3) Set the Python version with pyenv global 3.9.1
4) Run the kim install -> pip install -r requirements.txt
5) Run pip install urllib3==1.25.1 and pip install gpsoauth==1.0.2 (or, pip3 if that is your Python 3 standard)
6) Generate a new Google app password
7) Run python kim.py and login so you can save the Google Keep token to the keyring
8) Uninstall pyenv by deleting the .pyenv directory in your Users/User/folder (see online instructions on how to delete)
9) Run the kim install -> pip install -r requirements.txt for your default Python >3.10.x install from step 1 if you haven't already

Now kim.py will work in Python 3.10.x and 3.11.x when it loads and gets the Google token from keyring

[djsudduth]

How to install pyenv for Windows:
https://github.com/pyenv-win/pyenv-win

[chrisbrasington]

Sorry unfortunately this wasn't successful for me. Debian 12. After installing from source python 3.9.1, setting my virtual python environment to python3.9, and install the pip packages you recommended - LoginException persisted. Bummer.

Also this url doesn't exist anymore:

Login attempt failed
 Technical Error Message: LoginException('BadAuthentication', None)
Please start your browser and copy-paste this URL in the address bar: https://accounts.google.com/DisplayUnlockCaptcha - then, try logging in again.

which python
~/keep-it-markdown/dev3.9/bin/python
python --version
Python 3.9.1

pip list
Package Version

---

certifi 2023.7.22
cffi 1.16.0
charset-normalizer 3.3.1
click 8.1.7
cryptography 41.0.5
future 0.18.3
gkeepapi 0.14.2
gpsoauth 1.0.2
idna 3.4
importlib-metadata 6.8.0
jaraco.classes 3.3.0
jeepney 0.8.0
keyring 24.2.0
more-itertools 10.1.0
pip 20.2.3
pycparser 2.21
pycryptodomex 3.19.0
requests 2.31.0
SecretStorage 3.3.3
setuptools 49.2.1
urllib3 1.25.1
zipp 3.17.0

[djsudduth]

Hmmmmm...darn. Did you happen to try and run it under sudo?

I'll try a fresh Debian 12 install.

[chrisbrasington]

under sudo, NoKeyringError - no backend available.

If you can think of anything to try just let me know. Thanks for the attention, this is a cool idea of a project.

[djsudduth]

@chrisbrasington - Debian 12 setup. Will test KIM tonight

[djsudduth]

@chrisbrasington - no luck on Debian 12 (python 3.8, 3.9, 3.11) - I'm suspecting the updated linux OpenSSL 3.0 is causing the issue. I was able to get Debian 11 to work. If you can find an older image on Windows or Linux you can retrieve the API key once logged in and then save it directly in the keystore on Debian 12 as noted in the README:

"If those steps don't work, find a Linux or Mac system and run python keep-test.py -t which will display the token with the -t flag. Copy and save this master token in a safe and secure place!!. You can then use that token in KIM with python kim.py -t <token value> which will save it in your keystore."

[chrisbrasington]

@djsudduth kind of you to attempt, that's pretty cool of you. nice to know of the token flag, I'm sure I can work around it.

[djsudduth]

@chrisbrasington I think the best answer will be creating a Docker image of Debian 11 with the right python version and KIM. The other solution that I'm working on is a way to pull the notes directly from the web page (TBD on that)

[djsudduth]

NOTE: AS OF JAN 2024 THIS NOTE WAS UPDATED TO USE UBUNTU VS DEBIAN TO FIX A NEW ISSUE

Ok @chrisbrasington - here is one fix for you. I was able to setup a Docker container and get the Keep token/key. Then transfer this key to Debian 12. Here is the Dockerfile (just a text file) needed:

FROM ubuntu:22.04
RUN apt update
RUN apt install -y python-is-python3 python3-pip
RUN apt-get install -y curl
RUN apt-get install ca-certificates
RUN curl -L -o tmp/keep-it-markdown-0.5.4.tar.gz https://github.com/djsudduth/keep-it-markdown/archive/refs/tags/0.5.4.tar.gz
RUN tar -zxvf tmp/keep-it-markdown-0.5.4.tar.gz
RUN pip install -r keep-it-markdown-0.5.4/requirements.txt
RUN pip install requests==2.23.0
RUN pip install gpsoauth==1.0.2
RUN pip install keyrings.alt

Steps:

1. Install Docker on any PC (find the online instructions)
2. Startup Docker (or it will autostart on reboot)
3. Save the Dockerfile steps above in file on your PC as "Dockerfile"
4. Go to the command line and run docker build -t kim . in the directory where you saved the Dockerfile (it will take about 5 min to create the image)
5. Run the image with docker run -it kim
6. Change the directory to Kim cd keep-it-markdown-0.5.4
7. Create a temp app password on Google
8. Run python keep-test.py -t to log in and display the Keep token (keep token will appear be very long - almost 2 lines)
9. Copy the token by highlighting the entire string and hitting enter
10. Paste and save the token somewhere safe
11. Exit the Docker image with exit
12. Now run KIM in your current OS with the -t switch to save it in the keystore (python kim.py -t <long token here> (you may need a new Google app password to do this)

Let me know if this works!
cc @ooker777

[chrisbrasington]

@djsudduth that's brilliant. I'll give it a shot and report back later, ty

[chrisbrasington]

@djsudduth Good news, that worked. No issues once that token is in keystore. Cool project, thank you.

If you're musing on future development, I run more than half my applications nowadays as flatpak. It's one reason why I chose the stability of debian 12 as my current OS but when I need apps to be more up to date than their stable branch, I go with flatpak. Well any sort of containerization, docker or flatpak of the entire app - it's a useful tool to avoid dependency hell.

Once again, thanks for going out of your way to provide a solution to me in my obsidian/note-taking journey.

[djsudduth]

That’s great!! Dependency hell is awful! I was eventually able to get the keychain to work using a Debian 11 image vs Ubuntu 20.04.  I’ll post that and try flatpak.

[TheGreatestAlan]

Hi! I'm also using your implementation with the master token. Looks like a couple weeks ago I started getting the username and password error as well. I've followed your instructions and run the docker container to get the token and am getting a failed login attempt and asking me to navigate to https://accounts.google.com/DisplayUnlockCaptcha

Navigating there then just shows a dead page that google no longer supports. Has google changed things? Or am I somehow following the instructions incorrectly?

Also, I just wanted to express my gratitude for your work, it's been working brilliantly and I very much appreciate it.

[djsudduth]

@TheGreatestAlan - hi! Be sure you download version 0.5.3 - that removed the captcha prompt. My guess is you need to setup a new one-time Google App pw and try that again. Then try your id and one-time pw (no spaces or dashes in the pw). That should work with the Docker image to get your latest token. Use the Docker image in the new version vs the one here in the comments and look at the detailed instructions in the README.

Hope that helps!

[TheGreatestAlan]

@djsudduth thanks for the response. After some deeper poking around, I figured out the issue. The issue wasn't with the actual login. Unfortunately when the gkeepapi logs in, it also loads the notes. Looks like google has changed something and begun adding a source that says something along the lines of "migrated by google assistant" or something like that. In the backend in gkeepapi it attempts to load the source as a weblink, and assumes that a title and description must exist. Looks like assistant does not include those things, so it throws an exception, which throws all the way out. So from this piece of code it looks like an error logging in, but it's actually a note parse exception.

I confirmed this by going into google keep and removing those "migrated by google assistant" sources in google keep, and lo and behold, my original implementation began working again. I may see if I can PR the gkeepapi code, but I don't have a lot of time, and this is a bigger issue with it's deserialization of the gkeep objects.

[Tinqerer]

I've set up the Docker container, and I'm still getting "BadAuthentication" in there. Is there something else I still need to tweak? Looks like it's using Ubuntu 22.04.3 & Python 3.10.12 yeah?

[djsudduth]

@Tinqerer you have to use the one-time Google App password to log in each time you run the Docker image and/or get your Keep token and save it so you can log in from your PC without Docker. See the README on the steps.

https://support.google.com/accounts/answer/185833?hl=en

[Tinqerer]

Yep I'm using a Google App password, not sure why it still isn't happy. No other Google security settings that need be modified are there?

[djsudduth]

@Tinqerer hmmmmm....I just ran it yesterday with Docker on windows. Be sure the app pw doesn't have spaces or dashes. What OS are you running?

[Tinqerer]

@djsudduth Yeah strange. I've been triple checking everything, can't see any mistakes I might be making. I'm on Windows 11, with a fresh install of Docker Desktop.

[djsudduth]

@Tinqerer - darn - yes! There's definitely a problem. Seems Google made an auth change and now we're caught in a circle. The auth used to work on python 3.9.x but then started failing a few weeks ago. The gkeepapi was updated for the fix but only works on 3.10+ and that won't auth.

I need to try more testing. I'm trying to pin other versions but no luck yet. Sorry for the trouble.

[Tinqerer]

@djsudduth No worries! Hopefully you can find a solution :)

[djsudduth]

@Tinqerer - it took some work and help from other owners of the underlying packages. Here's how you can get the token.
Here's the new Dockerfile (build it with the name 'kim'):

FROM ubuntu:22.04
RUN apt update
RUN apt install -y python-is-python3 python3-pip
RUN apt-get install -y git
RUN apt-get install -y curl
RUN apt-get install ca-certificates
RUN curl -L -o tmp/keep-it-markdown-0.5.4.tar.gz https://github.com/djsudduth/keep-it-markdown/archive/refs/tags/0.5.4.tar.gz
RUN tar -zxvf tmp/keep-it-markdown-0.5.4.tar.gz
RUN pip install keyrings.alt
RUN pip install git+https://github.com/simon-weber/gpsoauth.git@8a5212481f80312e06ba6e0a29fbcfca1f210fd1

After creating the docker image - log in with docker run --mount type=bind,source=<your keep path>/mdfiles,target=/keep-it-markdown-0.5.4/mdfiles -it kim

Then, run this python script by just copy and paste into the Docker image terminal:
python -c 'print(__import__("gpsoauth").exchange_token(input("Email: "), input("OAuth Token: "), input("Android ID: ")))'
you will be prompted for your Google email account name, OAuth token, and AndroidID

Here's the tricky part - to get the OAuth token - follow the "Second way" instructions here:
https://github.com/rukins/gpsoauth-java?tab=readme-ov-file

The AndroidID can just be a Mac address like: 00:00:5e:00:53:af

So, when you get the prompt when running the script:
Email: your google ID
OAuth Token: oauth2_4/......rest of token
Android ID: 00:00:5e:00:53:af

A lot of data will print out -> the Keep token is at the top - it should look like:
"aas_et/FKcp.............lots of characters.....................BjQ="

Copy that token and exit the Docker image. Then run Kim on your PC with: python kim.py -t <token here>

You should be good from that point on!

[Tinqerer]

That works! Thanks so much!

I did have to use the "Second way" on those linked instructions (couldn't find first way in my browser), but now I'm all set! Amazing.

[djsudduth]

How to get your Keep token - Jan 2024 (I believe you will need Python 3.10+ from now on to use the token). But, you only have to run this Docker image once.

Build a Docker image and call it 'kim' (if you don't know how to do this - see the README):
FROM ubuntu:22.04
RUN apt update
RUN apt install -y python-is-python3 python3-pip
RUN apt-get install -y git
RUN pip install git+https://github.com/simon-weber/gpsoauth.git@8a5212481f80312e06ba6e0a29fbcfca1f210fd1

After creating the docker image - log in with: docker run -it kim

Here's the tricky part - you need to get your OAuth token from a Google cookie. To get the OAuth token - follow the "Second way" instructions here:
https://github.com/rukins/gpsoauth-java?tab=readme-ov-file
Copy the oauth_token from the cookies in your local browser

Then, run this python script by just copy and paste into the Docker image terminal:
python -c 'print(__import__("gpsoauth").exchange_token(input("Email: "), input("OAuth Token: "), input("Android ID: ")))'
you will be prompted for your Google email account name, OAuth token, and Android ID

The AndroidID can just be a random Mac address like: 00:00:5e:00:53:af

So, when you get the prompt when running the script:
Email: your google ID
OAuth Token: oauth2_4/......rest of token
Android ID: 00:00:5e:00:53:af

A lot of data will print out -> the Keep token is at the top - it should look like:
"aas_et/FKcp.............lots of characters.....................BjQ="

Copy that token and exit the Docker image. Then run Kim on your PC with: python kim.py -t <token here>

You should be good from that point on! (Kim version 0.6.0 will have more - in progress 1/23/20234)

[djsudduth]

Version 0.6.0 released. Dockerfile updated and step by step instructions in INSTALL.md

[davidlang42]

NOTE: AS OF JAN 2024 THIS NOTE WAS UPDATED TO USE UBUNTU VS DEBIAN TO FIX A NEW ISSUE

Ok @chrisbrasington - here is one fix for you. I was able to setup a Docker container and get the Keep token/key. Then transfer this key to Debian 12. Here is the Dockerfile (just a text file) needed:

FROM ubuntu:22.04 RUN apt update RUN apt install -y python-is-python3 python3-pip RUN apt-get install -y curl RUN apt-get install ca-certificates RUN curl -L -o tmp/keep-it-markdown-0.5.4.tar.gz https://github.com/djsudduth/keep-it-markdown/archive/refs/tags/0.5.4.tar.gz RUN tar -zxvf tmp/keep-it-markdown-0.5.4.tar.gz RUN pip install -r keep-it-markdown-0.5.4/requirements.txt RUN pip install requests==2.23.0 RUN pip install gpsoauth==1.0.2 RUN pip install keyrings.alt

Steps:

1. Install Docker on any PC (find the online instructions)
2. Startup Docker (or it will autostart on reboot)
3. Save the Dockerfile steps above in file on your PC as "Dockerfile"
4. Go to the command line and run docker build -t kim . in the directory where you saved the Dockerfile (it will take about 5 min to create the image)
5. Run the image with docker run -it kim
6. Change the directory to Kim cd keep-it-markdown-0.5.4
7. Create a temp app password on Google
8. Run python keep-test.py -t to log in and display the Keep token (keep token will appear be very long - almost 2 lines)
9. Copy the token by highlighting the entire string and hitting enter
10. Paste and save the token somewhere safe
11. Exit the Docker image with exit
12. Now run KIM in your current OS with the -t switch to save it in the keystore (python kim.py -t <long token here> (you may need a new Google app password to do this)

Let me know if this works! cc @ooker777

I found this thread incredibly useful, and appreciate all the valuable comments that went into it.
To help anyone out in the future who needs to get a gkeep token in an ad-hoc manner, I've made this docker image that will let you do it via cli in 1 line.

Just thought I'd leave it here in case it helps anyone else in the future.

[djsudduth]

Thx @davidlang42! As newer versions appear, I'll add this.

[djsudduth]

Changes to gpsoauth module deprecated these issues. No longer valid.