Errors on HTTP/2 should be propagated across all streams.

[ttc9082]

I'm using httpx in my project for http/2 requests, and my program sometimes get stuck at this line after working for a while(usually in 1 or 2 hour this will happen):

response = session.post(url_base, files={'imgfile': open(tmp_path, 'rb')}, headers=headers, timeout=Timeout(5.0))

note that the timeout doesn't work and it just hang there for forever.
how can I debug this?

some demo code here:

while True:
    url_base = "https://s.taobao.com/image"
    tmp_path = 'path to my pic'
    headers = {
        'referer': 'https://www.taobao.com/',
        'origin': 'https://s.taobao.com',
        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36",
        "X-Requested-With": "XMLHttpRequest"
    }
    session = httpx.Client(http2=True, proxies={'all': 'http://my_proxy_here'}, timeout=Timeout(5.0))
    print('requesting', url_base)
    response = session.post(url_base, files={'imgfile': open(tmp_path, 'rb')}, headers=headers,
                                    timeout=Timeout(5.0))
    print(response.json())

basically my program is POSTing a pic to this url, and get response back via http2. and it works good at first and get stuck after a while with the console prints "requesting https://s.taobao.com/image" at the end, without raising any error.

[tomchristie]

Okay, so the next thing to do would be to try to narrow this down a bit...

• Does it still replicate if you don't use http2=True?
• Does it still replicate if you make the request without using proxies?

[ttc9082]

Thank you, @tomchristie

I modified my program and now each worker has 4 requests:

1.using httpx, http2=True, with proxy.
2.using httpx, http2=False, with proxy.
3.using requests, with proxy.
4.using httpx, http2=True, without proxy.

I start up 7 workers like such above. And later I will check if they hang at the same position.

[ttc9082]

After 24 hours of running, 2 workers hang at the first request(1.using httpx, http2=True, with proxy.). While others are still working.

btw, for each request I use a while loop for a retry mechanism, when the request fails it will try to request with the same session, is that a problem?

    while tmp <= 10:
        try:
            tmp += 1
            print('requesting', url_base)
            response = session.post(url_base, files={'imgfile': open(tmp_path, 'rb')}, headers=headers,
                                    timeout=Timeout(5.0))
            print('tmp0 = ', tmp)
            break
        except Exception as e:
            print(e)

and the log shows one of the hanging worker stuck at the 3rd try, and the other stuck at the 10th try.

requesting https://s.taobao.com/image
_ssl.c:1074: The handshake operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image

requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image
The read operation timed out
requesting https://s.taobao.com/image

[tomchristie]

Just to confirm - you're only ever seeing this error when using a proxy and using http2=True, right?

[ttc9082]

Just to confirm - you're only ever seeing this error when using a proxy and using http2=True, right?

sorry for the late reply, and yes.

[tomchristie]

Okay, although we're not currently able to reproduce this, I do have a good idea of a likely candidate for the cause here.

Currently where an exception occurs on the HTTP/2 connection we're only raising that against the stream on which it occurs. So for example, if a server disconnect occurs, then one stream will see that exception, but it won't be raised on other streams that are being concurrently handled on the same connection.

[ttc9082]

Thank you @tomchristie and for now there is nothing I can do ?

[tomchristie]

Resolved via encode/httpcore#440