Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

squid proxy server says requests.get is "Unsupported Request Method and Protocol" #881

Closed
shoibalc opened this issue Oct 4, 2012 · 8 comments
Closed

squid proxy server says requests.get is "Unsupported Request Method and Protocol" #881

shoibalc opened this issue Oct 4, 2012 · 8 comments

Comments

@shoibalc
Copy link

shoibalc commented Oct 4, 2012

I am behind a squid proxy server (2.7) with authentication. request.get (version 0.14) is rejected by squid.

     r = requests.get('https://www.google.com')
     r.headers
    {'connection': 'close',
     'content-length': '1168',
     'content-type': 'text/html',
     'date': 'Thu, 04 Oct 2012 23:49:49 GMT',
     'server': 'squid/2.7.STABLE7',
     'via': '1.0 proxy_server:proxy_port (squid/2.7.STABLE7)',
     'x-cache': 'MISS from proxy_server',
     'x-cache-lookup': 'NONE from proxy_server:proxy_port',
     'x-squid-error': 'ERR_UNSUP_REQ 0'}

    r.text
    u'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html><head> <meta http-equiv="Content-Type"
    content="text/html; charset=utf-8"> <title>ERROR: The requested URL could not be retrieved</title> <style type="text/css"><!--   %l  body :lang(
    fa) { direction: rtl; font-size: 100%; font-family: Tahoma, Roya, sans-serif; float: right; } :lang(he) { direction: rtl; float: right; }  --><
    /style> </head><body> <div id="titles"> <h1>ERROR</h1> <h2>The requested URL could not be retrieved</h2> </div> <hr>  <div id="content"> <p>The
    following error was encountered while trying to retrieve the URL: <a href="https://www.google.com/">https://www.google.com/</a></p>  <
    blockquote id="error"> <p><b>Unsupported Request Method and Protocol</b></p> </blockquote>  <p>Squid does not support all request methods for
    all access protocols. For example, you can not POST a Gopher request.</p>  <p>Your cache administrator is <a href="mailto:webmaster%W">
    webmaster</a>.</p> <br> </div>  <hr> <div id="footer"> <p>Generated 05/Oct/2012:05:19:49 +0530 by proxy_server  (squid/2.7.STABLE7)</p> <!--
        ERR_UNSUP_REQ --> </div> </body></html> \n'
@kennethreitz
Copy link
Contributor

That's.... bizzare.

I hate proxies :)

@shoibalc
Copy link
Author

shoibalc commented Oct 5, 2012

501 error? Any idea what the issue is? Has anybody else faced something like this.

@shoibalc
Copy link
Author

shoibalc commented Oct 5, 2012

My bad! Solved.

proxies = { "http": "proxy_server:proxy_port"}
r = requests.get('https://www.google.com',proxies=proxies)

@sigmavirus24
Copy link
Contributor

@shoibalc I thought I saw an email about you mentioning that 'http_proxy' was the more widely used environment variable... am I hallucinating?

@kennethreitz
Copy link
Contributor

excellent :)

@sigmavirus24
Copy link
Contributor

Just for future reference, this pops up on PythonAnywhere as well but for HTTPS proxies for which this workaround does not seem to work.

@sigmavirus24 sigmavirus24 mentioned this issue Oct 22, 2012
Closed
@chicagoben
Copy link

I still get this with requests 1.2... is there a special patch I need?

Here's a post I submitted to request/request#552

I know I've seen lots of stack overflow questions about this. I cannot seem to figure out which version of python and which version of requests are needed to get this simple test working:

import requests.packages.urllib3 as urllib3
import unittest

class TestProxyConnect(unittest.TestCase):
    def setUp(self):
        self.proxy_url='http://192.168.201.138:3128/'
        self.https_url='https://www.google.com/index.html'
    def test_connect(self):
        proxy=urllib3.proxy_from_url(self.proxy_url)
        res=proxy.urlopen('GET', self.https_url)
        self.assertEqual(res.status, 200)

if __name__ == '__main__':
    unittest.main()
F
======================================================================
FAIL: test_connect (__main__.TestProxyConnect)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/test.py", line 11, in test_connect
    self.assertEqual(res.status, 200)
AssertionError: 501 != 200

It's just trying to use squid. I know this is more urllib3, but I am using the urllib3 that is in the requests 'packages' folder.

(venv)[root@cb-server-test urllib3]# pip freeze
requests==1.2.0
wsgiref==0.1.2

Can anyone help me figure out how to get HTTPS requests over HTTP to use CONNECT?

Thanks,
Ben

@Lukasa
Copy link
Member

Lukasa commented May 15, 2013

HTTPS requests over HTTP currently cannot use CONNECT. For a fuller explanation, see #1359.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@chicagoben @kennethreitz @sigmavirus24 @Lukasa @shoibalc