Fix RefreshTokenException('Could not find the Guest token in HTML')

[LinqLover]

About the problem

This problem has recently begun to occur on some environments.
This doesn't happen every time, so if you are lucky, you don't get the error.

The cause is literally that twint could not find the Guest token in HTML.
Actually, sometimes token isn't included in HTML recently.

#!/usr/bin/env python3
# This program is WTFPL.
import requests
res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

twint require the result of running the above code is })();</script><script nonce="VALUE">document.cookie = decodeURIComponent("gt=VALUE; Max-Age=VALUE; Domain=.twitter.com; Path=/; Secure");</script>.
However, sometimes the result is only })();</script> and missing the
Guest token.

About the solution

In this patch, twint get the Guest token from https://api.twitter.com/1.1/guest/activate.json if could not find the one.
The author referred to the code of gallery-dl.

Author's note:

I don't understand session of requests, so the code may be not good.
I hope someone rewrite the patch better and create a pull request.

This commit was adopted from #1320 (comment) by @minamotorin.

Closes #1320.

[rangersmyth74]

Hey all..

I am confused a little.

Q1: Do I add this code to a file?
Q2: Which file please!

#!/usr/bin/env python3

This program is WTFPL.

import requests
res = requests.get('https://twitter.com')
print(res.text.split('\n')[-1])

OR...

Q3: Do I have to download or use this >>> https://api.twitter.com/1.1/guest/activate.json
Q4: The author referred to the code of gallery-dl. > Do I need this code also?

Thanks in advance!

[LinqLover]

All you need to do is to replace the token.py in your installation of the twint package. To find out where you have the package installed, simply open a python interpreter in your shell, import twint, and then print twint.__file__. Navigate to this directory and replace the contents of token.py with the contents from the file in this branch.

Alternatively, you can reinstall twint from this location:

pip3 install git+https://github.com/Museum-Barberini/twint.git@fix/RefreshTokenException#egg=twint

[minamotorin]

After reinstalling via pip3 install git+https://github.com/Museum-Barberini/twint.git@fix/RefreshTokenException#egg=twint, don't forget this patch.