Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IP address leak! (Tor) (reCAPTCHA) #3612

Open
ghost opened this issue May 18, 2020 · 17 comments
Open

IP address leak! (Tor) (reCAPTCHA) #3612

ghost opened this issue May 18, 2020 · 17 comments
Labels
bug Issue is related to a bug privacy & data protection Issues regarding either NewPipe, Team NewPipe services or external services NewPipe uses

Comments

@ghost
Copy link

ghost commented May 18, 2020

Version

  • 0.19.3

Steps to reproduce the bug

  1. Use NetGuard to forward everything through Tor. In this case NewPipe
  2. Open a video
  3. Get reCAPTCHA challenge
  4. Tor IP
  5. Complete the challenge
  6. (Realize the reCAPTCHA is broken :/)
  7. Real IP

Expected behavior

Pretty harsh, but my expectation would be that NewPipe wouldn't leak my real IP to Google at ANY cost. Ya, I made a video recording this \o/

Actual behaviour

Would like my IP hidden from Google. Please.

Screenshots/Screen recordings

Same video, different hosts
https://anonfile.com/B9sfZezdo0/oh_no_webm
https://streamable.com/p6h6we

Logs

@ghost ghost added the bug Issue is related to a bug label May 18, 2020
@ghost ghost changed the title IP address leak! (Tor) IP address leak! (Tor) (reCAPTCHA) May 18, 2020
@Stypox
Copy link
Member

Stypox commented May 18, 2020

I don't get how this could be possible, since NewPipe never sends any ip explicitly. Could you capture a logcat of this happening with a debug apk (e.g. this one from #3478)?

@ghost
Copy link
Author

ghost commented May 18, 2020

Of course :) I'll get right onto it.

@ghost
Copy link
Author

ghost commented May 18, 2020

adb logcat > llogcatnewpipe.txt
llogcatnewpipe.txt

Please tell me if I did something wrong or if I should add any command options

@Stypox
Copy link
Member

Stypox commented May 18, 2020

Here I extracted the recaptcha-related data:

05-18 11:57:16.976   747  1088 I ActivityManager:                            START u0 {cmp=org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity (has extras)} from uid 10145
05-18 11:57:31.663   747  1682 I ActivityManager:                            START u0 {cmp=org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity (has extras)} from uid 10145
05-18 12:00:47.660  3245  3245 D DefaultKioskFragment@a78ef85:               onError() called with: exception = [org.schabi.newpipe.extractor.exceptions.ReCaptchaException: reCaptcha Challenge requested]
05-18 12:00:47.661  3245  3245 D DefaultKioskFragment@a78ef85:               onReCaptchaException() called
05-18 12:00:47.678   747  1310 I ActivityManager:                            START u0 {cmp=org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity (has extras)} from uid 10145

05-18 12:00:49.234  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: shouldOverrideUrlLoading: request.url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGJC6ifYFIhkA8aeDSyJiiE001w2n9x67c5NPMC3rQ6vVMgFy
05-18 12:00:49.234  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGJC6ifYFIhkA8aeDSyJiiE001w2n9x67c5NPMC3rQ6vVMgFy
05-18 12:00:49.238  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=null
05-18 12:00:51.764  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGJC6ifYFIhkA8aeDSyJiiE001w2n9x67c5NPMC3rQ6vVMgFy
05-18 12:00:51.766  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=null
05-18 12:00:58.702  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index
05-18 12:00:58.706  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=NID=204=kRglBs369QjPd51u392P51RN71W2hprYkRnQrMYo300luUKC_aWKew1tye9_7VwThXL9B3AJBpYdQ1gC9nd8mg3s3iLbg1hf-NsEkgxPBW3dNWoErb9G7bmEFFiBUiqq5Kw9OWkCLyBiFwKsUEdhhypkgNLS9700xZBBtnjeiDI
05-18 12:01:23.415  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index
05-18 12:01:23.416  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=NID=204=kRglBs369QjPd51u392P51RN71W2hprYkRnQrMYo300luUKC_aWKew1tye9_7VwThXL9B3AJBpYdQ1gC9nd8mg3s3iLbg1hf-NsEkgxPBW3dNWoErb9G7bmEFFiBUiqq5Kw9OWkCLyBiFwKsUEdhhypkgNLS9700xZBBtnjeiDI
05-18 12:01:23.416  3245  3245 D class org.schabi.newpipe.ReCaptchaActivity: saveCookiesAndFinish:     foundCookies=

05-18 12:01:47.391  3380  3380 D DefaultKioskFragment@f2a3cfc:               onError() called with: exception = [org.schabi.newpipe.extractor.exceptions.ReCaptchaException: reCaptcha Challenge requested]
05-18 12:01:47.392  3380  3380 D DefaultKioskFragment@f2a3cfc:               onReCaptchaException() called
05-18 12:01:47.409   747  1683 I ActivityManager:                            START u0 {cmp=org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity (has extras)} from uid 10145

05-18 12:01:49.033  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: shouldOverrideUrlLoading: request.url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGMy6ifYFIhkA8aeDS9F75V0Stol0xr5A8qoJLtgI5LIJMgFy
05-18 12:01:49.033  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGMy6ifYFIhkA8aeDS9F75V0Stol0xr5A8qoJLtgI5LIJMgFy
05-18 12:01:49.037  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=null
05-18 12:01:51.378  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGMy6ifYFIhkA8aeDS9F75V0Stol0xr5A8qoJLtgI5LIJMgFy
05-18 12:01:51.381  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=null
05-18 12:01:58.363  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index
05-18 12:01:58.366  3380  3380 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=NID=204=fx4snLbUHmXBxA4_nukjuUMkEhcKqKekAiYY1i5p5IGE95JGb5j08Qw4Xzn98AluWlN5O0Z4fmrZsI1u2tBhWhCyc6Arl7XAy5WyjMlSvSNzaB1w1YxiWBeuv6RDNx-Cx1o2zm0MbmE_65HgG3ayo4usvEjkuEehC12g9XXLVs4

05-18 12:02:53.014   435  1187 W SurfaceFlinger:                             Attempting to set client state on removed layer: org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity#0
05-18 12:02:53.014   435  1187 W SurfaceFlinger:                             Attempting to destroy on removed layer: org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity#0
05-18 12:02:53.017   435   865 W SurfaceFlinger:                             Attempting to set client state on removed layer: org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity#0
05-18 12:02:53.018   435   865 W SurfaceFlinger:                             Attempting to destroy on removed layer: org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity#0
05-18 12:02:53.379   747  1682 I WindowManager:                              WIN DEATH: Window{a45adef u0 org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity}
05-18 12:02:53.391   435  1546 W SurfaceFlinger:                             Attempting to destroy on removed layer: AppWindowToken{9bc627a token=Token{85a7ea5 ActivityRecord{5aa149c u0 org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity t1616}}}#0
05-18 12:02:59.057  3492  3492 D DefaultKioskFragment@f2a3cfc:               onError() called with: exception = [org.schabi.newpipe.extractor.exceptions.ReCaptchaException: reCaptcha Challenge requested]
05-18 12:02:59.058  3492  3492 D DefaultKioskFragment@f2a3cfc:               onReCaptchaException() called
05-18 12:02:59.074   747   758 I ActivityManager:                            START u0 {cmp=org.schabi.newpipe.debug.release/org.schabi.newpipe.ReCaptchaActivity (has extras)} from uid 10145

05-18 12:03:00.453  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: shouldOverrideUrlLoading: request.url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGJS7ifYFIhkA8aeDS-l79vt5A-rNVGKn2SGL8NSJ6K14MgFy
05-18 12:03:00.454  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGJS7ifYFIhkA8aeDS-l79vt5A-rNVGKn2SGL8NSJ6K14MgFy
05-18 12:03:00.458  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=null
05-18 12:03:02.970  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index?continue=https://www.youtube.com/results%3Fsearch_query%3Dtest%26pbj%3D1&q=EgS53GWLGJS7ifYFIhkA8aeDS-l79vt5A-rNVGKn2SGL8NSJ6K14MgFy
05-18 12:03:02.972  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=null
05-18 12:03:08.104  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: handleCookiesFromUrl:     url=https://www.google.com/sorry/index
05-18 12:03:08.106  3492  3492 D class org.schabi.newpipe.ReCaptchaActivity: handleCookies:            cookies=NID=204=iMEqZ2hPp5ik2u8a8gylclkWq_ZqanZbe8HdTXNOG4rxzzpPxguj4pnyfBMXYCNGOVgMit7rSoOGhXdRTHRj59Lpqd0CxjQBcjz8p_Qp-gqWe8183SNUQ9ZavE1ptWODeh6gTK7po7-mmov7EzD8Cbo2Wrij8XN49KRq79rBZjk

@Stypox
Copy link
Member

Stypox commented May 18, 2020

This is really strange: there is no data at all showing that an ip has been saved in cookies or something similar, so that can't be the cause (you can see saveCookiesAndFinish: foundCookies= nothing). So this could have something to do with the recaptcha javascript code running in the webview. I currently have no idea how this problem could be fixed.

@ghost
Copy link
Author

ghost commented May 18, 2020

Oh that's weird.

As you said, it's probably the JavaScript causing this. And disabling JavaScript on reCAPTCHA page would make the page nonfunctional. I guess this is one of those issues we can't fix then huh

@ghost
Copy link
Author

ghost commented May 18, 2020

Unless WebView uses WebRTC which is known for leaking IP addresses. This Reddit link is having the same problem.

Any Android app that uses WebView should be affected by WebRTC IP leak

Some people may go to the extent to disable Android System WebView (may require root), this will cripple usability.

@TheAssassin
Copy link
Member

@ingingin do you have any actual proof that could help identify the issue, if there is any? How did you notice your IP was leaked? I don't see any such evidence in the original issue description.

Generally, we take privacy serious and try to avoid data leakage, but please don't expect full protection. There is always a risk for a leak. If in doubt, you need to use different tools on an actually hardened system, e.g., the Tails linux distro on a safe computer. Android devices, generally anything mobile, is not 100% safe anyway.

@TheAssassin TheAssassin added the privacy & data protection Issues regarding either NewPipe, Team NewPipe services or external services NewPipe uses label May 24, 2020
@ghost
Copy link
Author

ghost commented May 24, 2020

What do you mean, actual proof? I noticed my IP was leaked because as I mentioned in the video, before doing all that, I had the IP address which Tor gave me, then suddently I get my own IP printed. No I don't have evidence to backup my statement cuz I don't wanna show my IP to anyone.

But using NetGuard, blocking access to every system app. On a non-googled tablet, running LineageOS. And only allowing NewPipe Internet access while routing through Tor on 127.0.0.1 port 9050. Tested and working.

Should we then give the android VPN implementation shit for this? I know these problems about the risks on tablets and phones generally but this feels like something more complicated. While I don't understand any of androids interior I would trust you know much more, which is why I really wouldn't know why or HOW its happening :/

@ghost
Copy link
Author

ghost commented May 24, 2020

But I know that Google displays your IP when getting reCAPTCHAs on the webpage too. The same thing happens on NewPipe. And I would guess you guys use WebView? Something about that messes up the, something. Now instead of using YouTube, wouldn't it be better to add Invidious? As far I remember you don't get any reCAPTCHAs from that. Which would solve this ?_?

@ale5000-git
Copy link

ale5000-git commented May 27, 2020

I'm not sure but I think the problem is NetGuard, if the requests done in the WebView are forwarded through Tor it shouldn't see your IP.

Also have you tried to install a different WebView and switch the default WebView to the new one?

@ale5000-git
Copy link

On a real browser the uBlock Origin add-on can prevent the IP leak, is it possible to replicate the behaviour?

See here: https://github.com/gorhill/uBlock/wiki/Prevent-WebRTC-from-leaking-local-IP-address

@ghost
Copy link
Author

ghost commented May 27, 2020

@ale5000-git Yeah, you just got to thinking again, I believe I didn't route the system through Tor actually, but using WebRTC with Tor, should leak my IP anyhow, because of the VPN situation ?? I can't think of a way to replicate that thing uBlock Origin uses on Android. Haven't seen anything like that.

Right now I'm trying to see if using Bromite's WebView would make a difference. I believe it's because I didn't actually route Android's WebView through Tor which I actually thought I did. And only having NewPipe through Tor and then WebView not, would mess up things.

Coming back tomorrow with news

@ghost
Copy link
Author

ghost commented May 27, 2020

I'm an idiot...I think

@ghost
Copy link
Author

ghost commented May 27, 2020

Okay I'm back, this is not tomorrow. But yes, even with Android System WebView denied Internet access + the whole system. It still leaks my IP address.

I also tried the same with Bromite's WebView from my Android phone. And it didn't leak my IP. Or so I don't think. The real problem goes down to the reCAPTCHA being broken on my Tablet. Because on my phone it works, the ✔️ mark is displayed. And the page doesnt refresh. On my Tablet, the reCAPTCHA box disappears completely and refreshes the page kinda. But I'll try to install Bromite's WebView onto my Tablet and see if that makes the difference. But if that's the cause. This issue can't be resolved right?

@ale5000-git
Copy link

ale5000-git commented May 27, 2020

In my opinion the safest option would be to include a web rendering engine directly inside NewPipe code but that would certainly add a lot of complications.

@ghost
Copy link
Author

ghost commented May 27, 2020

Wouldn't Invidious implementation solve this? I would vouch for that instead of a real fix for this. As Invidious doesn't seem to require any reCAPTCHAs because you're not accessing the Google captcha page. Going directly to googlevideo is great. And having Tor with that 👍

Or, I don't understand this, so I wouldn't know.

@ghost ghost mentioned this issue Jun 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue is related to a bug privacy & data protection Issues regarding either NewPipe, Team NewPipe services or external services NewPipe uses
Projects
None yet
Development

No branches or pull requests

3 participants