bug: intermittent HTTP 500 errors occur when accessing domain via HTTPS. #9801
Comments
hansedong
changed the title
|
Is this an error related to mTLS between apisix and upstream? |
No, the upstream service of APISIX use the HTTP protocol, not HTTPS. |
|
So this error occurs when you try to access apisix via a browser? Or when you try to access the website frontend? Have you setup mTLS between client and APISIX? |
|
Same here, Upgraded for 3.2.1 to .3.4.0 and now we get the same error messages. Connections are HTTPS. I looked at the relevant PR: #9322 |
|
same issue |
|
same issue for me too |
yeah, I downgrade apisix to 3.2.1, and it works fine. |
|
Deleting lines between 331-346 solves the issue for us in init.lua file. https://github.com/apache/apisix/blob/3.4.0/apisix/init.lua |
I have reviewed the key code in your proposal, and I think it is a good idea. I am planning to try it out. |
@alptugay It's working now, thank you. @kingluo It seems that this PR #9322 may has some issues. |
|
@hansedong could you confirm again the openresty version is really 1.21.4.1? |
APISIX and OpenResty version information is as follows (including the retrieval command) @kingluo the route info {
"value": {
"uri": "/*",
"labels": {
"env-type": "dev",
"ops": "ops"
},
"name": "ops-dev-aos-ab",
"host": "aos.ab",
"create_time": 1688557490,
"id": "468131118424524075",
"plugins": {
"proxy-rewrite": {
"host": "aosab.inner.do"
},
"redirect": {
"http_to_https": true
}
},
"upstream": {
"type": "roundrobin",
"nodes": [
{
"host": "10.72.79.4",
"weight": 1,
"port": 80
}
],
"timeout": {
"send": 6,
"connect": 6,
"read": 6
},
"pass_host": "pass",
"scheme": "http",
"keepalive_pool": {
"requests": 1000,
"size": 320,
"idle_timeout": 60
}
},
"enable_websocket": true,
"status": 1,
"update_time": 1688622920
},
"key": "/apisix/routes/468131118424524075",
"createdIndex": 10069,
"modifiedIndex": 21124
}the ssl info I need to provide a few additional pieces of information:
|
|
@hansedong I'll try to reproduce it, currently not yet. |
The execution of the curl command is fine (ignoring the certificate), I have already tried it. From my own perspective, this issue only occurs in Chrome browser. Safari, Firefox, and the curl command line all work without any problem. It seems that Chrome might not be carrying the correct certificate information to APISIX? what about you @alptugay @wadefelix |
@hansedong I can neither deny nor confirm :) We haven't collected the data about user agents. But I can at least say that Gitlab runners (User-agent: GitLab/15.0.5-ee) have encountered this issue |
My Browser is Edge: 版本 114.0.1823.82 (正式版本) (64 位) Chromium is the major player On PC. If All the chromium browsers go wrong with the apisix, 3.4.0 should be recalled. after I restart apisix, Edge's first request always works fine, but the following request go wrong with 500 error. |
|
I cannot reproduce the issue with apisix 3.4.0. I had tried chrome, but it's ok (refresh several times. |
Really appreciate your effort in reproducing this issue, thank you. Please keep the current testing environment intact and do not destroy it. Please observe for a while longer. |
|
I have the same issue the error appears randomly, I am using Brave as a browser, and as soon as I hard refresh (catch clean, etc...) the error occurs. But using private navigation it works fine each time. Here is my Route config: Edit: I have no mTLS configuration for any of my routes/upstreams |
|
This issue occurs for me ever time I call a gateway endpoint from postman without a CA : Steps to reproduce :
Note : Admin API - GET /apisix/admin/ssls retutns only one SSL (the newly added SSL above)
If I call the gateway on the path /converter/number, without a client certificate and key , Postman returns the error - Error: socket hang up. With a client cert and key (as per the image) , I get the 500 error : APISIX Error in the logs : And I will continue to get the error until I add a CA in postman : After adding the CA, all the requests to the endpoint work again. |
|
We are also experiencing this issue with known good certs, we are rolling back to
|
|
@hansedong @mrmm @wadefelix @adam-huganir https://github.com/openresty/lua-nginx-module#ssl_certificate_by_lua_block
I'll try to fix it later. |
|
@Sn0rt please take a look and try to fix it |
got. I will take a look |
|
Hello, is there any update on this issue please? |
|
THis issue is tracked here #9610 |
|
Same issue, just upgraded, happens in all Firefox, Chrome and Safari intermittently. Though my services was unbroken until I did a thorough test. |
|
Fix created for this and reasons explained here - #10066 |
Current Behavior
When I access the service through HTTPS, I intermittently encounter HTTP 500 errors. The browser is Chrome. It's very strange that the issue resolves itself when I force quit the browser.
Through the APISIX error log, you can clearly see the following errors:
Here are some additional information about my environment:
I have encountered this issue not only on the same Mac device, but my colleagues have also experienced it multiple times.Expected Behavior
No response
Error Logs
Steps to Reproduce
The occurrence of this issue is sporadic, and it is unclear how to reproduce it.
Environment
apisix version):3.4.0uname -a):Linux knode10-72-73-177 5.15.29-200.el7.x86_64 #1 SMP Thu Mar 31 14:09:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linuxopenresty -Vornginx -V):curl http://127.0.0.1:9090/v1/server_info):3.5.93.0.1luarocks --version):The text was updated successfully, but these errors were encountered: