FlareSolverr: Bypass Cloudflare protection in Jackett #9029

ngosang · 2020-06-21T19:38:08Z

TLDR; Since Jun 2020 Jackett can't resolve the Cloudflare challenge. In this issue I will try to explain the cause, affected trackers and a tentative solution.

There are some torrent sites which use Cloudflare protection to avoid DDoS attacks. The challenge used to be a small JavaScript code with mathematical operations. The browser does the maths for you and after a few seconds you can see the web site.
Jackett has been using CloudflareSolverRe library to resolve Cloudflare challenges for years. That library was really fast and lightweight, but it had to be updated each time Cloudflare changed the challenge. In Jun 2020 Cloudflare increased the complexity of the challenge and it can't be solved with that library anymore. More details: #8840 (comment)

Affected trackers: bteye, gktorrent, limetorrents, pornleech, torrent9clone, torrentz2, zetorrents (could be more because they enable/disable Cloudflare on demand)

To solve the new challenge we need a full web browser. This brings new problems: huge RAM usage, more disk space, OS compatibility... so we decide to not include it in Jackett. See #8840 (comment)
Since Cloudflare is present in 7 of 500 supported trackers, it doesn't make sense to make it mandatory for all users, so it's optional.

If you are not interested in the affected trackers, you don't need to do anything. Jackett will work as usual. If you try to use those trackers you will see a Cloudflare error.
If you want to bypass Cloudflare keep reading

FlareSolverr is a new project developed by me. It's a proxy server that includes the web browser. Info/instructions: https://github.com/FlareSolverr/FlareSolverr
FlareSolverrSharp is another new project developed by me. It's a .Net library to interact with FlareSolverr server. Info: https://github.com/FlareSolverr/FlareSolverrSharp
Changes in Jackett to integrate FlareSolverr. This is not included in Jackett releases yet. FlareSolverr integration. Bypass Cloudflare protection #9028

Instructions to test the POC:

Install FlareSolverr (it can't be installed in ARM yet) => https://github.com/ngosang/FlareSolverr
Download Jackett beta FlareSolverr integration. Bypass Cloudflare protection #9028 (download drop folder) => https://dev.azure.com/Jackett/Jackett/_build/results?buildId=762&view=artifacts&type=publishedArtifacts
Run Jackett and configure the FlareSolverr URL and Apply the changes. If you want to use this for long term, disable the Updates. Regular releases don't have this feature.
Configure and search using a tracker protected by Cloudflare like torrentz2. If everything is fine Jackett will work and you will be able to see the results.
In FlareSolverr you will see some traces like this, but you don't have to worry about that:

2020-06-21T18:41:54.582Z INFO REQ-5 Incoming request: POST /v1
2020-06-21T18:41:54.583Z INFO REQ-5 Params: {"url":"https://torrentz2.eu/","userAgent":"Mozilla/5.0 (X11; Linux i686; rv:77.0) Gecko/20100101 Firefox/67.0","maxTimeout":100000}
2020-06-21T18:42:08.594Z INFO REQ-5 Successful response in 14.012 s

This feature is experimental. It may never be a part of Jackett. Everything will depend on whether it is useful for users.
Please help us to test it and if you find any problems in any of the three projects open an issue.

The text was updated successfully, but these errors were encountered:

h0ttz · 2020-06-25T17:00:17Z

What ??????
https://twitter.com/eastdakota/status/1273277839102656515

Aadniz · 2020-06-28T09:41:32Z

I'm not sure how long the challange codes last until they require a new code, but you can add the cookie inside .config/Jackett/Indexers/TorrentWebsite.json

Go to the desired website → Inspect Element → Network → Copy the cookie

Go to .config/Jackett/Indexers/TorrentWebsite.json and add this (change out value to the cookie you copied):

  {
    "id": "cookie",
    "type": "inputstring",
    "name": "Cookie",
    "value": "The cookie you copied"
  },
  {
    "id": "cookieheader",
    "type": "hiddendata",
    "name": "CookieHeader",
    "value": "The cookie you copied"
  }

The cookie might look something like this: "__cfduid=d28e57bf8432a47859c223d06b39016461591236972; cf_clearance=0141459bcb7925275670b0d3bed06e9f6997eea2-1592401728-0-573dbd01-150; uid=1871531; pass=Lbetv3iI7p7xgg2eJk5gfnFcGs6yqCu3; hideCats=0; hideTop=1; togTem=11592478342"

RaphaelDucay · 2020-07-06T21:16:58Z

Very needed feature ! Yggtorrent (which is the main french torrent indexer) uses cloudflare Upvote for implementing a durable solution to handle those dynamic cloudflare challenges

RaphaelDucay · 2020-07-11T12:18:14Z

Any updates on this ? For m'y knowledge, why dont you merge this PR ?

bmn001 · 2020-08-14T19:19:13Z

Add tvchaosuk to the growing list of trackers using the new cloudflare.

I'm trying to download the Jackett beta linked above, but the link goes to an empty web page.

ethanopp · 2020-08-17T21:11:40Z

@ilike2burnthing Is it possible to pull the Jackett beta version if running in docker?

ilike2burnthing · 2020-08-17T21:33:06Z

Unless @ngosang or someone else has Dockerised it (or you want to give it a shot), then no. Sorry :/

dubl3a · 2020-08-29T14:01:49Z

I'm trying to download the Jackett beta linked above, but the link goes to an empty web page.

This is the issue I am having as well. Would LOVE to be able to use this...

EDIT: I was able to pull the source and build it. Verified working with SceneTime at least. Never containerized something before but may give it a shot.

JakeShirley · 2020-09-20T02:04:38Z

Sounds like this feature is mostly working. Any updates on if this could be considered a first-class feature? Even as a separate docker container it'd be amazing.

alpe12 · 2021-03-03T13:15:09Z

Exception (idope): The cookies provided by FlareSolverr are not valid as well.
I have Flaresolverr on docker, and Jackett outside docker.
All on the same server.

My thought is that maybe Jackett is using IPv6 and Flaresolverr IPv4? If so, there should be an option to force Jackett to only use IPv4.

manalive · 2021-03-03T15:08:00Z

Exception (idope): The cookies provided by FlareSolverr are not valid as well.
I have Flaresolverr on docker, and Jackett outside docker.
All on the same server.

My thought is that maybe Jackett is using IPv6 and Flaresolverr IPv4? If so, there should be an option to force Jackett to only use IPv4.

This is a great suggestion but neither the jail nor the vm are configured to use IPv6.

alpe12 · 2021-03-03T15:12:31Z

Exception (idope): The cookies provided by FlareSolverr are not valid as well.
I have Flaresolverr on docker, and Jackett outside docker.
All on the same server.
My thought is that maybe Jackett is using IPv6 and Flaresolverr IPv4? If so, there should be an option to force Jackett to only use IPv4.

This is a great suggestion but neither the jail nor the vm are configured to use IPv6.

Then idk in your case what could be the issue.
Here Jackett can be using IPv6. So this could be the cause of the issue here.

ilike2burnthing · 2021-03-03T21:19:45Z

Does your ISP actually provide IPv6 support?
https://test-ipv6.com/
https://ipv6-test.com/

alpe12 · 2021-03-03T21:29:26Z

Mine? Yes, it does.

xfouloux · 2021-03-04T01:50:03Z

Unless you configure Ipv6 fully with NAT v6 from your docker host, or allocate the right IPv6 Range into your docker daemon it's going to be an issue.

So either configure IPv6 fully or completely disable IPv6 on your docker host

in any case, you should call your flaresolverr container from jackett container via it's name

say you got 2 containers named : jackett & flaresolverr
in jackett you got to put http://flaresolverr:8191

you can try if this works by doing from your docker host "docker exec jackett ping flaresolverr"
That should respond, of course, both container needs to be on same container networks

alpe12 · 2021-03-04T01:56:04Z

Are you talking to me or manalive?
I don't use docker for Jackett. Only for FlareSolverr.
My problem is that FlareSolverr uses IPv4, and Jackett IPv6. They connect just fine, but the cookie is invalid. #11224

xfouloux · 2021-03-04T03:13:55Z

Well, indeed it seems to be problematic

you can pass to your container on creation options like --sysctl net.ipv6.conf.all.disable_ipv6=1
see : moby/moby#20569 (comment)
that should disable IPv6 inside your container

you can also add this to your /etc/docker/daemon.json file on your host

{
  "ipv6": false,
}

then reload docker daemon

Alternate, go full IPv6
https://medium.com/@skleeschulte/how-to-enable-ipv6-for-docker-containers-on-ubuntu-18-04-c68394a219a2
https://www.digitalocean.com/community/questions/how-configure-docker-with-ipv6

alpe12 · 2021-03-04T12:48:53Z

that should disable IPv6 inside your container

Thanks, but I don't use docker for Jackett. Only for FlareSolverr.
Jackett runs natively. So I have no idea how to disable IPv6 only to it.

Alternate, go full IPv6

Yeah, I'm considering doing this. Appears to be the only way to make it work while running Jackett outside docker.

xfouloux · 2021-03-05T06:08:47Z

If your docker host is a windows PC, just open the properties of your network card, untick IPV6 and you're good
If it's a linux, well pass the sysctl option into your /etc/sysctl.conf file do a sysctl -p and you're good.

Enohriel · 2021-03-25T09:54:16Z

Did you find an elegant solution ?
When I disable IPv6 on the whole machine, flaresolverr and jackett do work, but it breaks some other stuff (like radarr update because it's trying to connect to an IPv6 host).
When I disable IPv6 for docker only (flaresolverr), flaresolverr doesn't work anymore.

lborruto · 2021-05-19T07:12:42Z

Anyone has bypassed the captcha error?

An error occurred while testing this indexer
Exception (yggtorrent): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Cloudflare Error: Captcha detected but no automatic solver is configured.: FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Cloudflare Error: Captcha detected but no automatic solver is configured.

Tried to configure hcaptcha-solver but it doesn't work.

Lawaes · 2021-05-29T15:51:04Z

Anyone has bypassed the captcha error?

An error occurred while testing this indexer
Exception (yggtorrent): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Cloudflare Error: Captcha detected but no automatic solver is configured.: FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Cloudflare Error: Captcha detected but no automatic solver is configured.

Tried to configure hcaptcha-solver but it doesn't work.

Same here :/

ilike2burnthing · 2021-05-30T00:26:04Z

At this time none of the captcha solvers work.

https://github.com/FlareSolverr/FlareSolverr#captcha-solvers

dkfusion2021 · 2021-12-07T23:21:53Z

Is it possible to change the maxTimeout setting in a windows environment?

ilike2burnthing · 2021-12-08T00:07:58Z

@ngosang ^

ngosang · 2021-12-12T16:27:36Z

In Jackett is not possible to change FlareSolverr maxTimeout, it's fixed to 55 s. It could be implemented of course.
If you are using FlareSolverr directly in your software it's possible, read the readme.

ilike2burnthing · 2021-12-13T00:06:22Z

User configurable maxTimeout might be worthwhile for those with slower systems.

dkfusion2021 · 2021-12-13T12:32:27Z

That would be great if it could. I am getting maxTimeout errors every time for oMg[WtF]trackr and a handful of others. I am using this with Jackett and the PC is also behind a VPN, has Jackett, Radarr, Sonarr, Lidarr and qBit running on the same PC.

ilike2burnthing · 2021-12-13T12:39:32Z

Make sure you're running FlareSolverr v2.1.0, was released 21hrs ago, resolved a lot of the recent issues.

SmartBoy84 · 2021-12-29T10:53:55Z

For people continuing to have the issue of Jackett reporting invalid cookies, try using -p 0.0.0.0:[port]:[port] when running the container (likewise for when using docker compose)

pharpe · 2022-03-22T21:58:16Z

I'm not able to access kickasstorrents.to indexer and Jacket is referencing a FlareSolverr log. Here is the log,

2022-03-22T16:39:00-05:00 INFO FlareSolverr v2.2.2
2022-03-22T16:39:00-05:00 INFO Testing web browser installation...
2022-03-22T16:39:03-05:00 INFO FlareSolverr User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0
2022-03-22T16:39:04-05:00 INFO Test successful
2022-03-22T16:39:04-05:00 INFO Listening on http://0.0.0.0:8191
2022-03-22T16:39:13-05:00 INFO REQ-1 Incoming request => POST /v1 body: {"maxTimeout":55000,"cmd":"request.get","url":"https://kickasstorrents.to/new/?sortby=time&sort=desc"}
2022-03-22T16:39:16-05:00 INFO REQ-1 Cloudflare detected
2022-03-22T16:39:16-05:00 INFO REQ-1 Cloudflare Error: No challenge selectors found, unable to proceed.
2022-03-22T16:39:16-05:00 INFO REQ-1 Response in 2.982 s
2022-03-22T16:39:34-05:00 INFO REQ-1 Process interrupted
2022-03-22T16:40:01-05:00 INFO FlareSolverr v2.2.2
2022-03-22T16:40:01-05:00 INFO Testing web browser installation...
2022-03-22T16:40:06-05:00 INFO FlareSolverr User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0
2022-03-22T16:40:06-05:00 INFO Test successful
2022-03-22T16:40:06-05:00 INFO Listening on http://0.0.0.0:8191
2022-03-22T16:40:39-05:00 INFO REQ-1 Incoming request => POST /v1 body: {"maxTimeout":55000,"cmd":"request.get","url":"https://rutracker.org/forum/tracker.php?nm=&f=22%2C941%2C1666%2C376%2C106%2C7%2C187%2C2090%2C2221%2C2091%2C2092%2C2093%2C2200%2C1950%2C2540%2C934%2C505%2C212%2C2459%2C1235%2C166%2C124%2C1543%2C709%2C1577%2C93%2C905%2C101%2C100%2C877%2C1576%2C572%2C2220%2C1670%2C2198%2C1457%2C2199%2C313%2C312%2C1247%2C2201%2C2339%2C140%2C194%2C352%2C549%2C1213%2C2109%2C514%2C2097%2C4%2C84%2C2343%2C930%2C2365%2C1900%2C2258%2C521%2C208%2C539%2C209%2C484%2C822%2C181%2C847"}
2022-03-22T16:40:41-05:00 INFO REQ-1 Cloudflare detected
2022-03-22T16:40:44-05:00 INFO REQ-1 Challenge solved
2022-03-22T16:40:45-05:00 INFO REQ-1 Response in 6.691 s
2022-03-22T16:40:45-05:00 INFO REQ-2 Incoming request => POST /v1 body: {"maxTimeout":55000,"cmd":"request.get","url":"https://kickasstorrents.to/new/?sortby=time&sort=desc"}
2022-03-22T16:40:48-05:00 INFO REQ-2 Cloudflare detected
2022-03-22T16:40:48-05:00 INFO REQ-2 Cloudflare Error: No challenge selectors found, unable to proceed.
2022-03-22T16:40:48-05:00 INFO REQ-2 Response in 3.05 s
2022-03-22T16:40:55-05:00 INFO REQ-3 Incoming request => POST /v1 body: {"maxTimeout":55000,"cmd":"request.get","url":"https://kickasstorrents.to/new/?sortby=time&sort=desc"}
2022-03-22T16:40:58-05:00 INFO REQ-3 Cloudflare detected
2022-03-22T16:40:58-05:00 INFO REQ-3 Cloudflare Error: No challenge selectors found, unable to proceed.
2022-03-22T16:40:58-05:00 INFO REQ-3 Response in 2.888 s

ilike2burnthing · 2022-03-22T22:22:19Z

@pharpe open a new issue over on the FlareSolverr repo with debug FlareSolverr logs - https://github.com/FlareSolverr/FlareSolverr#environment-variables

ngosang added PR Welcome! We would welcome a volunteer to prepare a PR to solve this problem! Core Ongoing labels Jun 21, 2020

ngosang pinned this issue Jun 21, 2020

ilike2burnthing mentioned this issue Aug 16, 2020

[Scenetime] Can't open open settings window #9352

Closed

garfield69 mentioned this issue Aug 18, 2020

[pornleech] Exception (pornleech): Clearance failed after 10 attempt(s).: Clearance failed after 10 attempt(s). (Test) #9356

Closed

This was referenced Sep 4, 2020

[iptorrents] Your cookie did not work: The requested name is valid, but no data of the requested type was found. (Config) #9429

Closed

[demonoid] Clearance failed after 10 attempt(s). (Config) #9480

Closed

seeyabye mentioned this issue Sep 13, 2020

Cannot Download from Avistaz #9433

Closed

ilike2burnthing mentioned this issue Mar 5, 2021

Error - Unsupported platform: freebsd FlareSolverr/FlareSolverr#95

Closed

rguichard mentioned this issue Mar 8, 2021

[yggtorrent] The cookies provided by FlareSolverr are not valid (Config) #11268

Closed

garfield69 removed the Ongoing label Nov 11, 2021

garfield69 closed this as completed Nov 11, 2021

garfield69 unpinned this issue Nov 11, 2021

RoiArthurB mentioned this issue Jan 10, 2022

Purchase is not completed Revadike/epicgames-freebies-claimer#172

Open

eSoares mentioned this issue Mar 6, 2022

Torznab: Add timeout parameter Flexget/Flexget#2738

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FlareSolverr: Bypass Cloudflare protection in Jackett #9029

FlareSolverr: Bypass Cloudflare protection in Jackett #9029

ngosang commented Jun 21, 2020 •

edited

Loading

h0ttz commented Jun 25, 2020

Aadniz commented Jun 28, 2020 •

edited

Loading

RaphaelDucay commented Jul 6, 2020

RaphaelDucay commented Jul 11, 2020

bmn001 commented Aug 14, 2020

ethanopp commented Aug 17, 2020

ilike2burnthing commented Aug 17, 2020

dubl3a commented Aug 29, 2020 •

edited

Loading

JakeShirley commented Sep 20, 2020

alpe12 commented Mar 3, 2021 •

edited

Loading

manalive commented Mar 3, 2021

alpe12 commented Mar 3, 2021

ilike2burnthing commented Mar 3, 2021

alpe12 commented Mar 3, 2021

xfouloux commented Mar 4, 2021

alpe12 commented Mar 4, 2021 •

edited

Loading

xfouloux commented Mar 4, 2021 •

edited

Loading

alpe12 commented Mar 4, 2021

xfouloux commented Mar 5, 2021

Enohriel commented Mar 25, 2021

lborruto commented May 19, 2021

Lawaes commented May 29, 2021

ilike2burnthing commented May 30, 2021

dkfusion2021 commented Dec 7, 2021

ilike2burnthing commented Dec 8, 2021

ngosang commented Dec 12, 2021

ilike2burnthing commented Dec 13, 2021

dkfusion2021 commented Dec 13, 2021

ilike2burnthing commented Dec 13, 2021

SmartBoy84 commented Dec 29, 2021

pharpe commented Mar 22, 2022

ilike2burnthing commented Mar 22, 2022

FlareSolverr: Bypass Cloudflare protection in Jackett #9029

FlareSolverr: Bypass Cloudflare protection in Jackett #9029

Comments

ngosang commented Jun 21, 2020 • edited Loading

h0ttz commented Jun 25, 2020

Aadniz commented Jun 28, 2020 • edited Loading

RaphaelDucay commented Jul 6, 2020

RaphaelDucay commented Jul 11, 2020

bmn001 commented Aug 14, 2020

ethanopp commented Aug 17, 2020

ilike2burnthing commented Aug 17, 2020

dubl3a commented Aug 29, 2020 • edited Loading

JakeShirley commented Sep 20, 2020

alpe12 commented Mar 3, 2021 • edited Loading

manalive commented Mar 3, 2021

alpe12 commented Mar 3, 2021

ilike2burnthing commented Mar 3, 2021

alpe12 commented Mar 3, 2021

xfouloux commented Mar 4, 2021

alpe12 commented Mar 4, 2021 • edited Loading

xfouloux commented Mar 4, 2021 • edited Loading

alpe12 commented Mar 4, 2021

xfouloux commented Mar 5, 2021

Enohriel commented Mar 25, 2021

lborruto commented May 19, 2021

Lawaes commented May 29, 2021

ilike2burnthing commented May 30, 2021

dkfusion2021 commented Dec 7, 2021

ilike2burnthing commented Dec 8, 2021

ngosang commented Dec 12, 2021

ilike2burnthing commented Dec 13, 2021

dkfusion2021 commented Dec 13, 2021

ilike2burnthing commented Dec 13, 2021

SmartBoy84 commented Dec 29, 2021

pharpe commented Mar 22, 2022

ilike2burnthing commented Mar 22, 2022

ngosang commented Jun 21, 2020 •

edited

Loading

Aadniz commented Jun 28, 2020 •

edited

Loading

dubl3a commented Aug 29, 2020 •

edited

Loading

alpe12 commented Mar 3, 2021 •

edited

Loading

alpe12 commented Mar 4, 2021 •

edited

Loading

xfouloux commented Mar 4, 2021 •

edited

Loading