reCAPTCHA blocked in China

[JacksonWuxs]

A pypi ID is necessary while Iwould like to share and upload a package on pypi. Unfortunnatly, there was something wrong with the website of "https://pypi.org/account/register/". After writing the all information, an ERROR was arise named "Recaptcha error.".
I'm not sure if that is caused by China Internet?

[ewdurbin]

It looks like our Register page may be blocking the "additional screening" required by the recaptcha on occasion. Generally a modal that pops over and requests that a user click on objects that look like a "street sign" or "storefront"

@di @nlhkabu any ideas?

[dstufft]

Might be something related to the CSP policy.

[di]

@JacksonWuxs Are you seeing a reCAPTCHA like this on the register page?

[UlionTse]

@di @ewdurbin I also don't see CAPTCHA in China. See below,

[dstufft]

Oh interesting, I wonder if China is blocking recaptcha.

[dstufft]

Okay, so according to @reaperhulk Recaptcha doesn't work in China, that is... unfortunate.

[dstufft]

Some more information google/recaptcha#87

[UlionTse]

@dstufft I see, thanks.

[dstufft]

Not sure what (if any) milestone we should add this to, but getting this fixed is super important since currently all of China can't interact with anything on Warehouse that requires a captcha. /cc @brainwane

[JacksonWuxs]

I'm sorry about writing back so late. In fact I didn't see this register page. Here is what I got after clicking the "Create Account" button.

[JacksonWuxs]

I will try to find the answer from the blog what you gave me. In any case, I am moved by your attention to this issue. By the way, Is the user who upload the library to Pypi really important? If not, could you help me upload my lib to Pypi please? I can't wait to share my library. hahaha... Here is my lib address: https://github.com/JacksonWuxs/datapy .The address includes CODES, README and EXAMPLES. If you need more information about this issue, please let me know. I will try my best to collect information to help you.

[di]

@JacksonWuxs If you let me know the email address and username you'd like to use, I can register an account on your behalf, and then we can do a password reset so you can get access to it.

[JacksonWuxs]

I'm really happy to hear that. I hope you could help me register an account and my information as follow: E-Mail:wuxsmail@163.com User Name: Jackson Woo Real Name: Xuansheng Wu

[di]

@JacksonWuxs Done, you should have a password reset email in your inbox.

Leaving this issue open until we resolve the larger problem of using reCAPTCHA in China.

[JacksonWuxs]

Done, I have finished to reset my password. Thanks, so much!

[JacksonWuxs]

I'm sorry I need to bother you again. Unfortunately, when I was uploading my program with command as "twine upload dist/*", I raised another error. The error said: "HTTPError: 403 Client Error: The user 'JacksonWoo' is not allowed to upload to project 'datapy'." How can I do?

[di]

@JacksonWuxs No apology necessary. The datapy project already exists, you'll need to choose a new name.

[JacksonWuxs]

Oh it so sad... Could you tell me which web site will show the name list? I mean the token names.

[di]

@JacksonWuxs You can search here: https://pypi.org/search/, for example https://pypi.org/search/?q=datapy.

[JacksonWuxs]

I upload my package to Pypi successful. But I faced another bigger problem I thought. After I install successful, it still couldn't be import? I have screenshot of the wrong process. Thank you! On 3/12/2018 22:02，Dustin Ingram<notifications@github.com> wrote： @JacksonWuxs No apology necessary. The datapy project already exists, you'll need to choose a new name. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[brainwane]

I'd love advice on how we should address this. Right now I believe this is the biggest blocker stopping us from going to the beta stage and doing our publicity push.

[dstufft]

We're probably going to have to use a different captcha solution, at least in china if not everywhere. At least that's the only thing I can think of.

[ncoghlan]

Some basic notes at https://stackoverflow.com/questions/23780387/recaptcha-availability-in-china that sum up to "Don't use recaptcha if you have users in China".

[tylerdave]

It's been a while but I've used this with success when I needed a captcha solution that worked in China in the past: https://captcha.com/ (note that there's no Python backend though.)

[pradyunsg]

Someone claimed that recaptcha.net would work, because that domain is not blocked according to greatfire.org, and its SSL certificate is Google's misc services cert (so it looks legit). However, someone else reported that recaptcha.net still makes a request to www.google.com, which would fail in China.

That's already been mentioned in this thread @cpdyj.

[batou-mtcapthca]

reCaptcha v2 is not guaranteed to work in china due to its dependencies on google.com which is blocked. One can look at https://captcha.com/ to host their own, or https://www.mtcaptcha.com for a paid for recaptcha alternative. Both would work in China.

[Dorro101]

Hi

I am currently residing in Nanjing, China. I was temporarily unable to create an account because of the reCAPTCHA issue and the blanket blocking of google.com and its APIs. I was, however, able to get around this using the audio function by clicking on the speaker/volume (sic) icon and entering a code provided by audio playback.

[JacksonWuxs]

Well there was long time after I rose the issue. Finally, I solved this problem with a proxy software which let me go through the fire wall of China Internet.

Due to the reason that China government locked most of responses from abroad, including APIs from Google, the reCAPTCHA issue happened. If anyone face the same issue in the future, the best solution is buying a proxy. At that moment, you can not only create an account, you also search papers or more information with Google conveniencely.

[di]

Unfortunately, we needed to re-enable reCAPTCHA in #13232 due to the changes in #3339 eventually becoming insufficient to prevent spam signups. For any user that is affected by this, your best option at this time would be to use a proxy. Our sincere apologies for this inconveneince!

[ewdurbin]

New guidance: https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally

With an additional step not noted in googles docs regarding the need to add gstatic.cn to CSP: https://stackoverflow.com/a/57855838

[di]

Looking at the comments there, it seems like this doesn't work anymore, however it might be worth doing anyways and the "proxying" solution there might be something we could implement instead.

I'm going to unlock this issue to give our users in China the ability to tell us if they are still encountering this.

[demonguy]

@di I got the same issue here, cannot see the recaptcha on regsiter page.

BTW, my problem is slightly different here. I got proxy to access so websites outside of China. I can access Google Facebook and so on. And i can also see recaptcha on those websites.
when i open console of the browser, i didn't see any http status_code but the request failed because of csp

And i tried to disable CSP of chrome, and it works. I think that's some sort of security update?
my Chrome version: 111.0.5563.111

[miketheman]

@demonguy thanks for the report, can you please provide more details on the specific blocked request? I’m looking for the full url the browser is attempting to load?
Clicking on the blocked line in the inspector ought to display more details. Also clicking on the Console tab should show an error detail as well.

[Teddy-van-Jerry]

@demonguy Same situation and issue with you on Chrome. Recapture displays correctly on Safari. (When setting global proxy I can see Recapture on Chrome too, but with some latency.)

[SimFG]

Same situation, help!!!

[demonguy]

Here is the request jar saved from Chrome @miketheman

pypi.org.zip

[SimFG]

@demonguy has you registered a account

[demonguy]

@SimFG yeah. I download a Chrome plugin which disable csp on Chrome and i successfully registed an account

I think this is another problem? Maybe all latest Chrome users will have such issue?

[SimFG]

@demonguy which plugin, i need it. help help help

[demonguy]

https://chrome.google.com/webstore/detail/disable-content-security/ieelmcmcagommplceebfedjlakkhpden?hl=en#:~:text=Click%20the%20extension%20icon%20to,you%20from%20cross%2Dsite%20scripting.

[miketheman]

Here is the request jar saved from Chrome @miketheman

pypi.org.zip

Thanks, I'll investigate with these details - and think we have a path forward.

[miketheman]

Thanks for your details and patience!

@demonguy @Teddy-van-Jerry @SimFG I've shipped a change to the Content Security Policy we set as headers for the recaptcha - this directive helps by telling browsers what the authors of a website have decided which scripts to securely allow, and prevent unintended other scripts loading.

If you wouldn't mind deactivating the plugins that disable the CSP behavior and confirming that the recaptcha loads on registration page https://pypi.org/account/register/ so we can close this out?

[demonguy]

@miketheman it works now

[demonguy]

BTW, another url is blocked, but i think it doesn't matter

Request URL: https://media.ethicalads.io/media/client/v1.4.0/ethicalads.min.js
Referrer Policy: origin-when-cross-origin
Provisional headers are shown
Learn more
DNT: 1
Origin: https://pypi.org
Referer: https://pypi.org/
sec-ch-ua: "Google Chrome";v="111", "Not(A:Brand";v="8", "Chromium";v="111"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36

[miketheman]

@demonguy Thanks - that's not being blocked by CSP - that might be another browser plugin like an Ad Blocker. It shouldn't have any negative impact on the operation of the site.

[miketheman]

Now that this is confirmed as resolved again, I'll go ahead and lock this conversation to prevent unnecessary notifications to the 20ish folks on this issue.
If there are other captcha-related issues loading in China, please open a new issue (and reference this one for context).