[Patreon] "[warning] Cloudflare CAPTCHA"

[ZenythFactor]

Sooo I was doing a quick boot up for Gallery-DL, apparently this error came up!

C:\Users\...>gallery-dl https://www.patreon.com/... --cookies C:\Us
ers\...\gallery-dl\cookies.txt --write-metadata --password ... --user
name ...
[config][warning] Could not parse 'C:\Users\...\gallery-dl.conf': Extra dat
a: line 333 column 1 (char 7110)
[config][warning] Could not parse 'C:\Users\...\gallery-dl.conf': Extra dat
a: line 333 column 1 (char 7110)
[patreon][warning] Cloudflare CAPTCHA
[patreon][error] HttpError: '403 Forbidden' for 'https://www.patreon.com/.../posts'

Not sure what's the problem, but I hope there's a solution before next month come. I've already support then unsupported some of my favorite creators for financial reason atm.

and yes, I updated my cookies too in an attempt to retry. I use the "Export Cookies.txt" extension on Chrome for that.

[mikf]

Try it with -o browser=firefox or -o browser=chrome, although it already uses browser=firefox by default. You are using a gallery-dl version higher than v1.17.0, right?

You could also see if deleting all lines from your cookies.txt file except the one for session_id helps.

The error in your config file doesn't matter here, and --username and --password have no effect for Patreon either.

[Kickball]

I ran into this issue (Linux, version 1.18.3, auth via cookie) and using -o browser=chrome solved the issue for me.

Thanks for suggesting @mikf.

[Ogwalla]

I am having this issue as well, but I am using Windows 10 with version 1.18.3 and auth via cookies. I tried both browser options and deleting every line except for session_id.

[Ogwalla]

I think the problem might be with urllib3

[Kickball]

The other repo I used had this issue in the past and narrowed it down to Cloudflare requiring a captcha for any host that didn't use TLS 1.3 and HTTP/2.

This lead them to making the minimum versions required to run the software as Linux hosts with OpenSSL 1.1.1 or Windows 10 1903 or later.

It is worth checking that you are running the above minimum requirements but on the urllib3 side, from a scan of the changelog it looks like they added TLS 1.3 support in version 1.25. They don't support HTTP/2.

More info:

• https://github.com/AlexCSDev/PatreonDownloader#system-requirements
• Linux version is being blocked by cloudflare protection AlexCSDev/PatreonDownloader#64
• Can't download profiles with new version AlexCSDev/PatreonDownloader#61

[Hrxn]

Just FYI, the v2.0 Roadmap of urllib3 mentions an expected release in "mid-to-late 2021"..

The new version comes with "Modern Security by Default", so, uh, yeah it seems it's optimized for TLS1.2 and higher.

FYI 2:
There is another good way to do HTTP with Python: https://github.com/encode/httpx
Might be a worthwhile alternative to requests, but I think this is up to @mikf to decide.

Noteworthy about httpx is that they explicitly mention HTTP/2 support, and they do not depend on urllib3.
They seem to use httpcore, which is written by the same authors.

[mikf]

I think the problem might be with urllib3

That seems to be the case. I was also getting a Cloudflare CAPTCHA with the latest v.1.26.6, but not with an older v1.25.11, so maybe rolling back to a previous version helps?
(pip install -U urllib3==1.25.11)

httpx

I'll most likely switch to httpx for gallery-dl v2.0. Another alternative would be https://github.com/aio-libs/aiohttp, but it has to be something with at least async/await support.

[Ogwalla]

That seems to be the case. I was also getting a Cloudflare CAPTCHA with the latest v.1.26.6, but not with an older v1.25.11, so maybe rolling back to a previous version helps?
(pip install -U urllib3==1.25.11)

That is what I ended up doing after I posted.

[left1000]

Is there anyway for me to fix this myself? or do I need to wait for a new release of gallery-dl?

I already foolishly commented this in a closed thread instead of this open one.

#748 (comment)

which says the below:

windows 10

[patreon][warning] Cloudflare CAPTCHA
[patreon][error] HttpError: '403 Forbidden' for '

python --version
Python 3.9.0

"patreon": {
"browser": "firefox",
"cookies": {
"session_id": "justredidmycopypastehere"
}
},

what am I doing wrong?

edit2: more complete notes on my version

D:\gallery-dl>gallery-dl -v

[gallery-dl][debug] Version 1.19.0
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.25.1 - urllib3 1.25.11

D:\gallery-dl>python --version
Python 3.9.0

edit3: sidenote the last time I used gallery-dl to rip a patreon I followed was july31 2021 (not that that is relevant.)

edit4: WHOOPS I was on 1.18.3 because I downloaded 1.19.0 to the wrong folder, but well, it was irrelevent because running on 1.19.0 now I still can't rip patreon :)

[ZenythFactor]

Try it with -o browser=firefox or -o browser=chrome, although it already uses browser=firefox by default. You are using a gallery-dl version higher than v1.17.0, right?

You could also see if deleting all lines from your cookies.txt file except the one for session_id helps.

The error in your config file doesn't matter here, and --username and --password have no effect for Patreon either.

Tried that just now actually, nothing changed, and yes.
and I'll look into that. I use the "Export Cookies.txt" plugin to do all my cookie exporting needs in case that needs to be mentioned.

[lionpanthera]

Having this same problem with gallery-dl built from Homebrew on an Intel Mac.

[gallery-dl][debug] Version 1.19.1
[gallery-dl][debug] Python 3.10.0 - macOS-11.6.1-x86_64-i386-64bit
[gallery-dl][debug] requests 2.26.0 - urllib3 1.26.7

-o browser=chrome doesn't help.

Browsing the site in Safari using the same session_id cookie works fine, no captchas.

Notably, my urllib3 is already >1.25

[left1000]

As far as I can tell it's just flat out broken, and no one knows how to fix it. See #1922 which is where I've been commenting on the issue.

[Hrxn]

Yeah, the underlying issue is that the requests / urllib3 combo is a dead end, and all projects are scrambling to transition away from it..

[ZenythFactor]

The thing actually briefly work today and it's gone back to the Cloudflare CAPTCHA error again.
This is getting pretty silly here.

[ZenythFactor]

Honestly, can you like, bring up a temporary screen to do the captcha manually?