Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert reddit links to a valid video link #27

Merged
merged 4 commits into from
Oct 24, 2021
Merged

Convert reddit links to a valid video link #27

merged 4 commits into from
Oct 24, 2021

Conversation

Zeboto
Copy link
Contributor

@Zeboto Zeboto commented Oct 2, 2021

No description provided.

@mralext20 mralext20 requested a review from 7bitlyrus October 2, 2021 22:22
alexBot/cogs/video_dl.py Outdated Show resolved Hide resolved
alexBot/cogs/video_dl.py Outdated Show resolved Hide resolved
alexBot/cogs/video_dl.py Outdated Show resolved Hide resolved
It is not possible to use aiohttp instead of httpx for some reason. I have tested extensively with requests, aiohttp, and httpx. requests and httpx will always work, while aiohttp will always return too many requests.
@Zeboto Zeboto requested a review from mralext20 October 22, 2021 21:08
@mralext20
Copy link
Owner

i've tested this, it works

@mralext20 mralext20 requested a review from 7bitlyrus October 23, 2021 18:26
@7bitlyrus
Copy link
Contributor

It is not possible to use aiohttp instead of httpx for some reason. I have tested extensively with requests, aiohttp, and httpx. requests and httpx will always work, while aiohttp will always return too many requests.
- @Zeboto

Is it not possible this is a the result of aiohttp's user-agent? Did you test that?

Copy link
Contributor

@7bitlyrus 7bitlyrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good. Just waiting for a response on whether aiohttp's UA is the cause of 429s before approval.

@7bitlyrus
Copy link
Contributor

Everything looks good. Just waiting for a response on whether aiohttp's UA is the cause of 429s before approval.

Yeah, I looked into this. No clue why 429s are resulting, but it keeps to be due to aiohttp capitalizing a header or some SSL difference.

@7bitlyrus
Copy link
Contributor

After looking into it a bit more, I found a similar issue with aiohttp and the Telsa API returning 403s: aio-libs/aiohttp#5643.

Drawing from the discussion on that issue, it looks like Reddit is returning a 429 for SSL issues. This code to override the SSL ciphers resolves the aiohttp 429 issue for me (where setting the UA alone did not):

import aiohttp, asyncio, ssl

async def test():
    async with aiohttp.ClientSession() as session:
        headers = {'user-agent': 'AlexBot:v1.0.0'}
        url = 'https://www.reddit.com/r/funny/comments/5gn8ru.json'
        
        FORCED_CIPHERS = (
            'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
            'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES'
        )
        sslcontext = ssl.create_default_context()
        sslcontext.set_ciphers(FORCED_CIPHERS)

        resp = await session.get(url=url, headers=headers, ssl=sslcontext)
        print(resp)

loop = asyncio.get_event_loop()
loop.run_until_complete(test())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants