Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jable.tv Unable to download with Error HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>) #6799

Closed
9 of 11 tasks
cocnetdom opened this issue Apr 13, 2023 · 11 comments · Fixed by #9911
Closed
9 of 11 tasks
Labels
NSFW site-bug Issue with a specific website

Comments

@cocnetdom
Copy link

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Singapore

Provide a description that is worded well enough to be understood

Unable to download video from jable.tv via yt-dlp. It used to be working until recently.

yt-dlp https://jable.tv/videos/jul-676/
[Jable] Extracting URL: https://jable.tv/videos/jul-676/
[Jable] jul-676: Downloading webpage
ERROR: [Jable] jul-676: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U

Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)

Sample Link: https://jable.tv/videos/jul-676/

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://jable.tv/videos/jul-676/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (pip)
[debug] Python 3.11.2 (CPython arm64 64bit) - macOS-13.1-arm64-arm-64bit (OpenSSL 1.1.1t  7 Feb 2023)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[Jable] Extracting URL: https://jable.tv/videos/jul-676/
[Jable] jul-676: Downloading webpage
ERROR: [Jable] jul-676: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/jable.py", line 45, in _real_extract
    webpage = self._download_webpage(url, video_id)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1097, in _download_webpage
    return self.__download_webpage(url_or_request, video_id, note, errnote, None, fatal, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1048, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 882, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 839, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 821, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/yt-dlp/2023.3.4/libexec/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3742, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
@cocnetdom cocnetdom added site-bug Issue with a specific website triage Untriaged issue labels Apr 13, 2023
@bashonly
Copy link
Member

cloudflare strikes again

$ curl -i "https://jable.tv/videos/jul-676/"
HTTP/2 403 
date: Thu, 13 Apr 2023 16:40:34 GMT
content-type: text/html; charset=UTF-8
cache-control: max-age=15
expires: Thu, 13 Apr 2023 16:40:49 GMT
x-frame-options: SAMEORIGIN
set-cookie: __cf_bm=qxprLX.ZchAqQ5QOEkbbgBbHXp5HpOvWQd/iEA7ZftoQ5QOEkbWbl98eIHDG48x=; path=/; expires=Thu, 13-Apr-23 17:10:34 GMT; domain=.jable.tv; HttpOnly; Secure; SameSite=None
server: cloudflare
cf-ray: 7aebc0984e65f8-ORD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>

...

      <div class="cf-wrapper cf-header cf-error-overview">
        <h1 data-translate="block_headline">Sorry, you have been blocked</h1>
        <h2 class="cf-subheadline"><span data-translate="unable_to_access">You are unable to access</span> jable.tv</h2>
      </div><!-- /.header -->

@bashonly bashonly added the NSFW label Apr 13, 2023
@observeroftime01
Copy link

Maybe this is helpful. Setting a custom UA for the curl command returns the appropriate page:
curl -i --referer "https://jable.tv/" --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/111.0" "https://jable.tv/videos/jul-676/" :

HTTP/1.1 200 OK
Date: Thu, 13 Apr 2023 20:25:07 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
vary: Accept-Encoding
set-cookie: PHPSESSID=ohrl801c3nmackrlm8cfeohqbf; path=/; domain=.jable.tv
expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
pragma: no-cache
strict-transport-security: max-age=15768000
CF-Cache-Status: DYNAMIC
Set-Cookie: kt_ips=IP.GOES.HERE.SNIP; expires=Fri, 14-Apr-2023 20:25:07 GMT; Max-Age=86400; path=/; domain=.jable.tv
Set-Cookie: __cf_bm=K.X6FuKYGANuFpUFOmLCWFPFfNI.KMByKDjr.x6y860-1681417507-0-AXusgTjbZR9CeWEtB109W0kUBFtYqCXtzvh/sqbDgLaMgMFNh6Zz2hLBHJ1g5DncyCutjqdQXSJROgN6vlUIFy8=; path=/; expires=Thu, 13-Apr-23 20:55:07 GMT; domain=.jable.tv; HttpOnly; Secure; SameSite=None
Server: cloudflare
CF-RAY: 7b766fbb5e94f4b2-YVR
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

<!doctype html>
<html lang="en">
<head>
<title>JUL-676 人妻與學生時期的教師偶然再會便成為他性欲發洩的寵物 向井藍 - Jable.TV | 免費高清AV在線看 | J片 AV看到飽</title>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<meta name="description" content="此作品曾在本站上傳,現已更新至中文字幕版。" />
<meta name="keywords" content="中文字幕, 角色劇情, 制服誘惑, 主奴調教, 漁網, 口爆, 年齡差, 老師, 風俗娘, 凌辱, 調教, 校 服, 出軌, 人妻, 顏射, 少女, 向井藍" />

Maybe all that's needed is a custom UA and proper referer? jable certainly doesn't like the default curl UA. I'm not sure how to pass a custom UA and referer to yt-dlp, otherwise I would have checked that out.

@bashonly
Copy link
Member

bashonly commented Apr 13, 2023

It works with curl yes, but not with yt-dlp. Curl has a different TLS fingerprint. My above comment was to illustrate what is happening beyond just "403"

Similar issue for different site: #6748

@pukkandan pukkandan removed the triage Untriaged issue label Apr 14, 2023
@bashonly
Copy link
Member

maybe try adding --legacy-server-connect to your command and see if that helps

@bashonly bashonly mentioned this issue Apr 15, 2023
11 tasks
@bennettjw

This comment was marked as spam.

@bennettjw
Copy link

One more time: Identical Issue resolved here: Nandaka/PixivUtil2#1245
Fixed for me by adding --user-agent "Mozilla/5.0" to yt-dlp command line.

@ozslice
Copy link

ozslice commented May 5, 2023

Maybe this is helpful. Setting a custom UA for the curl command returns the appropriate page: curl -i --referer "https://jable.tv/" --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/111.0" "https://jable.tv/videos/jul-676/" :

HTTP/1.1 200 OK
Date: Thu, 13 Apr 2023 20:25:07 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
vary: Accept-Encoding
set-cookie: PHPSESSID=ohrl801c3nmackrlm8cfeohqbf; path=/; domain=.jable.tv
expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
pragma: no-cache
strict-transport-security: max-age=15768000
CF-Cache-Status: DYNAMIC
Set-Cookie: kt_ips=IP.GOES.HERE.SNIP; expires=Fri, 14-Apr-2023 20:25:07 GMT; Max-Age=86400; path=/; domain=.jable.tv
Set-Cookie: __cf_bm=K.X6FuKYGANuFpUFOmLCWFPFfNI.KMByKDjr.x6y860-1681417507-0-AXusgTjbZR9CeWEtB109W0kUBFtYqCXtzvh/sqbDgLaMgMFNh6Zz2hLBHJ1g5DncyCutjqdQXSJROgN6vlUIFy8=; path=/; expires=Thu, 13-Apr-23 20:55:07 GMT; domain=.jable.tv; HttpOnly; Secure; SameSite=None
Server: cloudflare
CF-RAY: 7b766fbb5e94f4b2-YVR
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

<!doctype html>
<html lang="en">
<head>
<title>JUL-676 人妻與學生時期的教師偶然再會便成為他性欲發洩的寵物 向井藍 - Jable.TV | 免費高清AV在線看 | J片 AV看到飽</title>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<meta name="description" content="此作品曾在本站上傳,現已更新至中文字幕版。" />
<meta name="keywords" content="中文字幕, 角色劇情, 制服誘惑, 主奴調教, 漁網, 口爆, 年齡差, 老師, 風俗娘, 凌辱, 調教, 校 服, 出軌, 人妻, 顏射, 少女, 向井藍" />

Maybe all that's needed is a custom UA and proper referer? jable certainly doesn't like the default curl UA. I'm not sure how to pass a custom UA and referer to yt-dlp, otherwise I would have checked that out.

it seems not work right now :(

@huang72590
Copy link

Hi, is there a solution to this problem?

@gamer191
Copy link
Collaborator

gamer191 commented Jul 5, 2023

Hi, is there a solution to this problem?

  1. visit the website in a browser
  2. visit https://www.google.com/search?q=what+is+my+user+agent in the same browser, and note your user-agent
  3. run yt-dlp URL --cookies-from-browser BROWSER --user-agent "USERAGENT", where BROWSER is the name of your browser (eg chrome) and "USERAGENT" is your user agent, in quotation marks (eg: yt-dlp "https://jable.tv/" --cookies-from-browser chrome --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36")

Let me know if this helps!

@aknkaplanoglu
Copy link

Hi, is there a solution to this problem?

  1. visit the website in a browser
  2. visit https://www.google.com/search?q=what+is+my+user+agent in the same browser, and note your user-agent
  3. run yt-dlp URL --cookies-from-browser BROWSER --user-agent "USERAGENT", where BROWSER is the name of your browser (eg chrome) and "USERAGENT" is your user agent, in quotation marks (eg: yt-dlp "https://jable.tv/" --cookies-from-browser chrome --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36")

Let me know if this helps!

It was working but now it is not.

@gamer191
Copy link
Collaborator

gamer191 commented Jul 6, 2023

@aknkaplanoglu did you try repeating the steps from step 1? Note that you'll have to visit the website in your browser occassionally, and also that your user agent changes every time your browser updates.

EDIT: your user-agent actually only changes on some updates

@coletdjnz coletdjnz moved this to works with curl-impersonate in Cloudflare-related Jul 15, 2023
@github-project-automation github-project-automation bot moved this from works with curl-impersonate to Done in Cloudflare-related May 12, 2024
@pukkandan pukkandan closed this as not planned Won't fix, can't repro, duplicate, stale May 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NSFW site-bug Issue with a specific website
Projects
Development

Successfully merging a pull request may close this issue.

9 participants