attempt to index local 'matched_ssl' 500 internal error HTTPS issue #9610

neosis41 · 2023-06-06T08:44:26Z

Current Behavior

hello, i get back to you because im facing an issue to configure my apisix cluster. Actually i already set 3 node with apisix-dashboard-etcd on each. everything works fine in http but when i try in https i get weird behaviour. i already set my ssl certificate. the first time i connect on https, it work fines, my apisix server has a tcp connection open with the client, as long as the tcp connection is opened i can make the requests in https. but once the tcp connection reach the connection timeout and pass to time_wait, i try the same https request on the client browser and i receive a 500 internal error from openresty. i can see on the serveur another tcp connection opened but i get this 500 error code. in the nginx error.log i get a message "lua entry thread aborted: runtime error: /opt/apisix/apisix/init.lua:332: attempt to index local 'matched_ssl' (a nil value)
stack traceback:
coroutine 0:
/opt/apisix/apisix/init.lua: in function 'verify_https_client'
/opt/apisix/apisix/init.lua:560: in function 'http_access_phase'
access_by_lua(nginx.conf:329):2: in main chunk, client: 10.138.16.8, server: _, request: "GET /test7212212415527117122562121275915345814351514.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"" . if i retry some times later like 1h it works again until it fails again. Same thing if i restart apisix it works again until it fails again.

If i wait the server time_wait connection finish then my https request is success.

for apisix install , i did the git clone then 'bash utils/install-dependencies.sh' then install luarock 'curl https://raw.githubusercontent.com/apache/apisix/master/utils/linux-install-luarocks.sh -sL | bash -' and '. /root/.cargo/env && LUAROCKS_SERVER=https://luarocks.cn/ make deps' and then 'make install'

for the dahboard : docker run -d --name dashboard --network apisix-network -p 9000:9000 -v /opt/apisix/dashboard-conf.yaml:/usr/local/apisix-dashboard/conf/conf.yaml apache/apisix-dashboard

for etcd : docker run -d --network apisix-network -p 2379:2379 -p 2380:2380 -p 4001:4001 --name etcd quay.io/coreos/etcd:v3.4.26 /usr/local/bin/etcd --name etcd{{ cluster_id }} -advertise-client-urls=http://{{ private_ip }}:2379 -listen-client-urls http://0.0.0.0:2379/,http://0.0.0.0:4001/ -initial-advertise-peer-urls http://{{ private_ip }}:2380 -listen-peer-urls http://0.0.0.0:2380/ -initial-cluster-token etcd-cluster-1 -initial-cluster {{ cluster_range }} -initial-cluster-state new

heres below my config.yaml :
apisix:
node_listen: # This style support multiple ports
- 9080
ssl:
enable: true
listen: # APISIX listening port in https.
- port: 443
enable_http2: true
ssl_protocols: TLSv1.2 TLSv1.3
ssl_ciphers: ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl_session_tickets: false # disable ssl_session_tickets by default for 'ssl_session_tickets' would make Perfect Forward Secrecy useless.
# ref: mozilla/server-side-tls#135
ssl_session_cache: shared:SSL:1m
key_encrypt_salt: # If not set, will save origin ssl key into etcd.
- edd1c9f0985e76a2
deployment:
role: traditional
role_traditional:
config_provider: etcd
admin:
allow_admin:
- 0.0.0.0/0 # Please set it to the subnet address you obtained.
admin_key:
- name: admin
key: edd1c9f034335f136f87ad84b625c8f1 # using fixed API token has security risk, please update it when you deploy to production environment
role: admin
etcd:
host:
- "http://127.0.0.1:2379"
prefix: "/apisix"
timeout: 30

Expected Behavior

i expect all my https request return me a code 200 and not 200 the first time and 500 internal error after.

Error Logs

first itry https request at 8:19 it works and the backend server give me a 404 :
10.212.135.241 - - [06/Jun/2023:08:19:57 +0200] slot_featuretestnico2.mrvol.com "GET /test2888854.html HTTP/2.0" 404 1251 0.002 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0" 10.10.0.30:80 404 0.002 "http://slot_featuretestnico2.mrvol.com/"
and then i retry at https request at 8:28 and then get my 500 error :
10.212.135.241 - - [06/Jun/2023:08:28:17 +0200] slot_featuretestnico2.mrvol.com "GET /test28588854.html HTTP/2.0" 500 174 0.000 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0" - - - "http://slot_featuretestnico2.mrvol.com/"

with the message below in error.log :
2023/06/06 08:28:17 [error] 3756#3756: *6713294 lua entry thread aborted: runtime error: /opt/apisix/apisix/init.lua:332: attempt to index local 'matched_ssl' (a nil value)
stack traceback:
coroutine 0:
/opt/apisix/apisix/init.lua: in function 'verify_https_client'
/opt/apisix/apisix/init.lua:560: in function 'http_access_phase'
access_by_lua(nginx.conf:329):2: in main chunk, client: 10.212.135.241, server: _, request: "GET /test28588854.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"

when it works (in debug mode) :
2023/06/06 14:57:27 [info] 7786#7786: 124362 [lua] radixtree_sni.lua:79: create_router(): route items: [{"paths":["moc.lovrm."],"handler":"function: 0x7f00ae708270"}], context: ssl_certificate_by_lua*, client: 10.212.135.241, server: 0.0.0.0:443
2023/06/06 14:57:27 [info] 7786#7786: 124362 [lua] radixtree.lua:355: pre_insert_route(): path: moc.lovrm. operator: <=, context: ssl_certificate_by_lua, client: 10.212.135.241, server: 0.0.0.0:443
2023/06/06 14:57:27 [info] 7786#7786: 124362 [lua] radixtree.lua:243: insert_route(): insert route path: moc.lovrm. dataprt: 1, context: ssl_certificate_by_lua, client: 10.212.135.241, server: 0.0.0.0:443
2023/06/06 14:57:27 [debug] 7786#7786: *124362 [lua] radixtree_sni.lua:169: match_and_set(): sni: slot_featuretestnico2.mrvol.com
2023/06/06 14:57:27 [info] 7786#7786: *124362 [lua] radixtree_sni.lua:209: match_and_set(): debug - matched: {"key":"/apisix/ssls/1","createdIndex":1193,"clean_handlers":{},"modifiedIndex":1193,"value":{"create_time":1685969984,"status":1,"key":"Y...
2023/06/06 14:57:27 [info] 7786#7786: 124362 [lua] secret.lua:233: fetch_secrets(): retrieve secrets refs, context: ssl_certificate_by_lua, client: 10.212.135.241, server: 0.0.0.0:443
2023/06/06 14:57:27 [debug] 7786#7786: *124362 [lua] ssl.lua:202: create_obj_fun(): parsing cert for sni: slot_featuretestnico2.mrvol.com
2023/06/06 14:57:27 [debug] 7786#7786: *124362 [lua] ssl.lua:220: create_obj_fun(): parsing priv key for sni: slot_featuretestnico2.mrvol.com
2023/06/06 14:57:27 [debug] 7786#7786: *124361 [lua] radixtree_sni.lua:169: match_and_set(): sni: slot_featuretestnico2.mrvol.com
2023/06/06 14:57:27 [info] 7786#7786: *124361 [lua] radixtree_sni.lua:209: match_and_set(): debug - matched: {"key":"/apisix/ssls/1","createdIndex":1193,"clean_handlers":{},"modifiedIndex":1193,"value":{"create_time":1685969984,"status":1,"key":"...
2023/06/06 14:57:27 [info] 7786#7786: *124361 [lua] radixtree.lua:355: pre_insert_route(): path: / operator: <=, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"
2023/06/06 14:57:27 [info] 7786#7786: *124361 [lua] radixtree.lua:243: insert_route(): insert route path: / dataprt: 1, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"
2023/06/06 14:57:27 [info] 7786#7786: *124361 [lua] radixtree.lua:355: pre_insert_route(): path: moc.lovrm.2ocintseterutaef_tols operator: =, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"
2023/06/06 14:57:27 [info] 7786#7786: 124361 [lua] radixtree_host_uri.lua:163: match(): route match mode: radixtree_host_uri, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"
2023/06/06 14:57:27 [debug] 7786#7786: 124361 [lua] radixtree.lua:497: compare_param(): pcre pat: /((.|\n))
2023/06/06 14:57:27 [info] 7786#7786: 124361 [lua] init.lua:607: http_access_phase(): matched route: {"value":{"uri":"/","status":1,"create_time":1685455621,"upstream":{"nodes":[{"port":80,"host":"10.10.0.30","weight":1}],"timeout":{"read":10,"send":10,"connect":15},"hash_on":"vars","pass_host":"pass","scheme":"http","type":"roundrobin","keepalive_pool":{"requests":1000,"idle_timeout":60,"size":320},"parent":{"value":"table: 0x7f00b03bb6a8","update_count":0,"key":"/apisix/routes/462927046608159422","createdIndex":431,"orig_modifiedIndex":1065,"clean_handlers":"table: 0x7f00ae664f78","modifiedIndex":1065,"has_domain":false}},"host":"slot_featuretestnico2.mrvol.com","id":"462927046608159422","name":"test_slot_featuretestnico.mrvol.com","plugins":{"authz-keycloak":{"keepalive_pool":5,"grant_type":"urn:ietf:params:oauth:grant-type:uma-ticket","client_id":"Client ID","timeout":3000,"access_token_expires_in":300,"lazy_load_paths":false,"access_token_expires_leeway":0,"ssl_verify":true,"_meta":{"disable":true},"refresh_token_expires_in":3600,"refresh_token_expires_leeway":0,"http_method_as_scope":false,"permissions":["resource name#scope name"],"keepalive_timeout":60000,"token_endpoint":"http://127.0.0.1:8090/auth/realms/${realm}/protocol/openid-connect/token","policy_enforcement_mode":"ENFORCING","cache_ttl_seconds":86400,"keepalive":true},"openid-connect":{"unauth_action":"auth","bearer_only":false,"use_pkce":false,"access_token_in_authorization_header":false,"introspection_endpoint_auth_method":"client_secret_basic","realm":"Digitrips","redirect_uri":"https://slot_featuretestnico2.mrvol.com/","set_id_token_header":true,"client_id":"apisix","timeout":3,"set_refresh_token_header":false,"ssl_verify":false,"_meta":{"disable":true},"discovery":"https://sso.digitrips.com/realms/Digitrips/.well-known/openid-configuration","session":{"secret":"abcdefghabcdefghabcdefgh"},"set_userinfo_header":true,"logout_path":"/logout","set_access_token_header":true,"client_secret":"iGRPFw5Y16NJHmAnbn7syWgMSeOGMRDQ","scope":"openid"}},"update_time":1685965267,"priority":0},"update_count":0,"key":"/apisix/routes/462927046608159422","createdIndex":431,"orig_modifiedIndex":1065,"clean_handlers":{},"modifiedIndex":1065,"has_domain":false}, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"
2023/06/06 14:57:27 [info] 7786#7786: 124361 [lua] balancer.lua:195: pick_server(): route: {"value":{"uri":"/","status":1,"create_time":1685455621,"upstream":{"timeout":{"read":10,"send":10,"connect":15},"hash_on":"vars","scheme":"http","nodes_ref":[{"port":80,"host":"10.10.0.30","weight":1,"priority":0}],"pass_host":"pass","nodes":"table: 0x7f00ae5a3cd0","original_nodes":[{"port":80,"host":"10.10.0.30","weight":1}],"type":"roundrobin","keepalive_pool":{"requests":1000,"idle_timeout":60,"size":320},"parent":{"value":"table: 0x7f00b03bb6a8","update_count":0,"key":"/apisix/routes/462927046608159422","createdIndex":431,"orig_modifiedIndex":1065,"clean_handlers":"table: 0x7f00ae664f78","modifiedIndex":1065,"has_domain":false}},"host":"slot_featuretestnico2.mrvol.com","id":"462927046608159422","name":"test_slot_featuretestnico.mrvol.com","plugins":{"authz-keycloak":{"keepalive_pool":5,"grant_type":"urn:ietf:params:oauth:grant-type:uma-ticket","client_id":"Client ID","timeout":3000,"access_token_expires_in":300,"lazy_load_paths":false,"access_token_expires_leeway":0,"ssl_verify":true,"_meta":{"disable":true},"refresh_token_expires_in":3600,"refresh_token_expires_leeway":0,"http_method_as_scope":false,"permissions":["resource name#scope name"],"keepalive_timeout":60000,"token_endpoint":"http://127.0.0.1:8090/auth/realms/${realm}/protocol/openid-connect/token","policy_enforcement_mode":"ENFORCING","cache_ttl_seconds":86400,"keepalive":true},"openid-connect":{"unauth_action":"auth","bearer_only":false,"use_pkce":false,"access_token_in_authorization_header":false,"introspection_endpoint_auth_method":"client_secret_basic","realm":"Digitrips","redirect_uri":"https://slot_featuretestnico2.mrvol.com/","set_id_token_header":true,"client_id":"apisix","timeout":3,"set_refresh_token_header":false,"ssl_verify":false,"_meta":{"disable":true},"discovery":"https://sso.digitrips.com/realms/Digitrips/.well-known/openid-configuration","session":{"secret":"abcdefghabcdefghabcdefgh"},"set_userinfo_header":true,"logout_path":"/logout","set_access_token_header":true,"client_secret":"iGRPFw5Y16NJHmAnbn7syWgMSeOGMRDQ","scope":"openid"}},"update_time":1685965267,"priority":0},"update_count":0,"key":"/apisix/routes/462927046608159422","createdIndex":431,"orig_modifiedIndex":1065,"clean_handlers":{},"modifiedIndex":1065,"has_domain":false}, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"
2023/06/06 14:57:27 [info] 7786#7786: 124361 [lua] balancer.lua:196: pick_server(): ctx: {"matched_sni":["moc.lovrm."],"upstream_conf":"table: 0x7f00b02d20e0","upstream_version":"1065#table: 0x7f00b02d20e0","upstream_key":"462927046608159422","matched_ssl":"table: 0x7f00b035aeb8","global_rules":" etcd key: /apisix/global_rules","curr_req_matched":"table: 0x7f00ae39a8e0","conf_type":"route","upstream_scheme":"http","var":{"_ctx":{"matched_sni":"table: 0x7f00ae708d40","upstream_conf":{"timeout":{"read":10,"send":10,"connect":15},"hash_on":"vars","scheme":"http","nodes_ref":[{"port":80,"host":"10.10.0.30","weight":1,"priority":0}],"pass_host":"pass","nodes":"table: 0x7f00ae5a3cd0","original_nodes":[{"port":80,"host":"10.10.0.30","weight":1}],"type":"roundrobin","keepalive_pool":{"requests":1000,"idle_timeout":60,"size":320},"parent":{"value":{"uri":"/","status":1,"create_time":1685455621,"upstream":"table: 0x7f00b02d20e0","host":"slot_featuretestnico2.mrvol.com","id":"462927046608159422","name":"test_slot_featuretestnico.mrvol.com","plugins":{"authz-keycloak":{"keepalive_pool":5,"grant_type":"urn:ietf:params:oauth:grant-type:uma-ticket","client_id":"Client ID","timeout":3000,"access_token_expires_in":300,"lazy_load_paths":false,"access_token_expires_leeway":0,"ssl_verify":true,"_meta":{"disable":true},"refresh_token_expires_in":3600,"refresh_token_expires_leeway":0,"http_method_as_scope":false,"permissions":["resource name#scope name"],"keepalive_timeout":60000,"token_endpoint":"http://127.0.0.1:8090/auth/realms/${realm}/protocol/openid-connect/token","policy_enforcement_mode":"ENFORCING","cache_ttl_seconds":86400,"keepalive":true},"openid-connect":{"unauth_action":"auth","bearer_only":false,"use_pkce":false,"access_token_in_authorization_header":false,"introspection_endpoint_auth_method":"client_secret_basic","realm":"Digitrips","redirect_uri":"https://slot_featuretestnico2.mrvol.com/","set_id_token_header":true,"client_id":"apisix","timeout":3,"set_refresh_token_header":false,"ssl_verify":false,"_meta":{"disable":true},"discovery":"https://sso.digitrips.com/realms/Digitrips/.well-known/openid-configuration","session":{"secret":"abcdefghabcdefghabcdefgh"},"set_userinfo_header":true,"logout_path":"/logout","set_access_token_header":true,"client_secret":"iGRPFw5Y16NJHmAnbn7syWgMSeOGMRDQ","scope":"openid"}},"update_time":1685965267,"priority":0},"update_count":0,"key":"/apisix/routes/462927046608159422","createdIndex":431,"orig_modifiedIndex":1065,"clean_handlers":{},"modifiedIndex":1065,"has_domain":false}},"upstream_version":"1065#table: 0x7f00b02d20e0","upstream_key":"462927046608159422","matched_ssl":{"key":"/apisix/ssls/1","createdIndex":1193,"clean_handlers":{},"modifiedIndex":1193,"value":{"create_time":1685969984,"status":1,"key":"YnwwDKc5vNzo0OU4StTRQbwgCnTZ3dmYiBFm8aGnvTwyAh+Rz4EWXpJif7colyUhItBIvm962vfX5Me9G9WPJ7DuLfMov4Fb4Vrc1tmSL5fr6N895OAiHGOTuoRt+rH7cNWiWXmgp53bOC+mGcArvKEkvS7YPicjbFhsG8zlyvJh9Tx6G1z/OYaKcHtYe0uFjQZFFNB3d1d7u2YZKhGTAO7MAbQKDSX8k7U1APLFo5zNkST96MTJbHDYm1v9lrsui7fRM35syM6XIld5HlZKQJG7BXpJQkKGZ2vvTaEvsq3HWOwj+MMiO2J+3V7YasxfgPr3WTthgBBqAL5q2QEZHMD78U8JBiOEllbySWky1uIGEWN563EcWMeMsqpcCAnSZRg5eU0sJnM08B5N337qsb2KrjwdZGicFOV9RAuprN4D8KUt0mVc8bjwHE8wgV0Q7YmjGrBZOPSBmjOhlhRxjdXdhKQv2FhOyGF3qzm3DiIpvAUgNrRMXP2rdUc5jy1V4vg95Bj5Ven7MmKDWE8zwEG3MirTu2A0xRwG0ZZYsBLGJnrpn7We7y8iY2tCQAg+KxWbBuWEPLDB//w9T/FNTPhbxglFsJz14PdpgZl31VSg9I0yIy5Q8xfi/E+UWHhP7tk1Ea9xSj3dXcdAex6c/Letco2jpX0wSPKSGkfQ/0fBfMlcK4iqZrZjnSU9jq8T3+p7mz116lqq+5EVrZOrxehO3SerOLT7AlJEVC+yDRcwqgzbB90cqG/v/ByZCnc1xk6ZBFqF2EEXij2nvnCE0MayC0WD6jqu6YMozuLjHKxA6GZ8i6cMrGwtdbMV57WDwt75f+6IxNcsVFj7JzTnj1o2PiEDz8xednXgTBCZjv9ZIKSSWoQ+jrVbdT8mZ3Gx+eWmn2/uA8Rd/bfNozJnYwpsea21C3qgK5WJvlH71yHO+U0gNSIYidvQ5mlLdHNEMyN2xOv5hHibf09ncrO4/vOlwS28I1TvkLXHEiS5cpR1fO5L1S5gU1/XsymmBCbSPW3Az4diuKODYhX/OioHeIwRfWvJQ9Ke47ec7i5kcQG6JZUtpb+DQMN/5e3MN2bLHDOyEwWPdilpZ2mOP3oi65C76ourrokFXQUOrtUH1TYsBj7Iub6niY10zLlU412W69+Qop+KdC/cgJKA1eRCurEOMSUO4G6HrRjo1pLZCYw2v1ZjsFghF+PIZhAbsthacsQmdowfteFOAghKEuajJau1diZL9X/ruZK8EfUk6jF7TB7Z4lS/ZR23EV0rK1T/pwbDRCF7bK1fJ7n8xee5ImrI5Ca+
2023/06/06 14:57:27 [info] 7786#7786: *124361 [lua] balancer.lua:384: run(): proxy request to 10.10.0.30:80 while connecting to upstream, client: 10.212.135.241, server: _, request: "GET /test2858526188365541.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"

when it fails :
2023/06/06 14:56:17 [error] 7790#7790: *112661 lua entry thread aborted: runtime error: /opt/apisix/apisix/init.lua:332: attempt to index local 'matched_ssl' (a nil value)
stack traceback:
coroutine 0:
/opt/apisix/apisix/init.lua: in function 'verify_https_client'
/opt/apisix/apisix/init.lua:560: in function 'http_access_phase'
access_by_lua(nginx.conf:331):2: in main chunk, client: 10.212.135.241, server: _, request: "GET /test7212212629523483713991313639659953285668397322511633661532592517216556232655625512485275915553245814351514.html HTTP/2.0", host: "slot_featuretestnico2.mrvol.com"

Steps to Reproduce

start etcd docker container
start apisix which is directly install on host
start dashboard docker container
request https://slot_featuretestnico2.mrvol.com/test.html (its an A record in our local dns server) on a browser in my computer
wait the connectiontimeout setttings if its configured on 60sec then wait 60s and do the same https request and now it fails.
I have still the same weird behaviour on chrome and mozilla firefox. i didnt test another one.

Environment

APISIX version (run apisix version): 3.3.0
Operating system (run uname -a): ubuntu 20
OpenResty / Nginx version (run openresty -V or nginx -V): openresty/1.21.4.1
etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): v3.4.26
APISIX Dashboard version, if relevant: latest
Plugin runner version, for issues related to plugin runners:
LuaRocks version, for installation issues (run luarocks --version): /usr/local/bin/luarocks 3.8.0

The text was updated successfully, but these errors were encountered:

Revolyssup · 2023-07-21T08:45:02Z

@lingsamuel Please assign it to me

Revolyssup · 2023-07-25T08:51:30Z

@neosis41 When you start getting 500, can you check on /apisix/ssls key in etcd whether the certificate is present?

neosis41 · 2023-07-25T12:51:03Z

i cant check anymore cause now im using apisix as docker containers and it works better. for sure the certificate was present in etcd. my first http request works. it s only when i waited several minute and i retry that i got 500 each time. Le mar. 25 juil. 2023 à 10:51, Ashish Tiwari ***@***.***> a écrit :

…

@neosis41 <https://github.com/neosis41> When you start getting 500, can you check on /apisix/ssls key in etcd whether the certificate is present? — Reply to this email directly, view it on GitHub <#9610 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BALJA7QRMR4YDNMULYALWADXR6CJ3ANCNFSM6AAAAAAY4CKD3E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Revolyssup · 2023-07-25T13:15:14Z

@neosis41 I can't help debug if you cannot reproduce at this point. Anyhow, if your use case is solved then you can go ahead and close the issue.

neosis41 · 2023-07-27T13:01:22Z

actually, i think i found when the probleme is coming. i have sometimes this 500 internal error when the openid-connect plugin is enable on the route im checking. i just get the error :
023/07/27 12:55:26 [error] 50#50: *22525794 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/init.lua:332: attempt to index local 'matched_ssl' (a nil value)
stack traceback:
coroutine 0:
/usr/local/apisix/apisix/init.lua: in function 'verify_https_client'
/usr/local/apisix/apisix/init.lua:560: in function 'http_access_phase'
access_by_lua(nginx.conf:324):2: in main chunk, client: 10.7.7.27, server: _, request: "GET /media/1104/iziwifi.png HTTP/1.1", host: "slot_featurejenkins.mrvol.com", referrer: "https://slot_featurejenkins.mrvol.com/"
2023/07/27 12:55:26 [error] 56#56: *22525796 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/init.lua:332: attempt to index local 'matched_ssl' (a nil value)
stack traceback:
coroutine 0:
/usr/local/apisix/apisix/init.lua: in function 'verify_https_client'
/usr/local/apisix/apisix/init.lua:560: in function 'http_access_phase'
access_by_lua(nginx.conf:324):2: in main chunk, client: 10.7.7.27, server: _, request: "GET /media/1096/flexy.png HTTP/1.1", host: "slot_featurejenkins.mrvol.com", referrer: "https://slot_featurejenkins.mrvol.com/"

neosis41 · 2023-07-27T13:02:45Z

i think its a cache issue maybe a misconfiguration in my openid settings. if i retry on private browser i can authenticate and it works. but in my other broser a get this 500.

neosis41 · 2023-07-27T13:03:13Z

when i disable this plugin i dont have this issue anymore

neosis41 · 2023-07-27T13:03:53Z

here is my openid congiguration :
"plugins": {
"openid-connect": {
"access_token_in_authorization_header": true,
"bearer_only": false,
"client_id": "apisix",
"client_secret": "something",
"discovery": "https://sso.digitrips.com/realms/Digitrips/.well-known/openid-configuration",
"introspection_endpoint": "https://sso.digitrips.com/realms/Digitrips/protocol/openid-connect/token/introspect",
"introspection_endpoint_auth_method": "client_secret_basic",
"realm": "Digitrips",
"redirect_uri": "/redirect_uri",
"session": {
"secret": "something"
},
"set_access_token_header": true,
"set_id_token_header": true,
"set_userinfo_header": true,
"ssl_verify": false
},

n84ck · 2023-07-28T05:28:51Z

i think its a cache issue maybe a misconfiguration in my openid settings. if i retry on private browser i can authenticate and it works. but in my other broser a get this 500.

I have the same error (open-id, docker). Rollback to Apisix 3.3.0 solves the problem.

stonegithup · 2023-08-07T09:01:12Z

lines 332 after "if " add " matched_ssl and " because matched_ssl perhaps nil.
The reason for the higher-level code is this paragraph in the nginx.conf configuration file.

ssl_certificate_by_lua_block {
apisix.http_ssl_phase()
}

This conf does not execute at the h2 protocol, and the browser can reproduce it 2-3 minutes after opening the page.

Hope these leads help

Revolyssup · 2023-08-07T16:35:22Z

@neosis41 On which browser do you get the error? These two issues might be related. #9962
Here also rollback to 3.3.0 solves issue #9962 (comment)

neosis41 · 2023-08-08T07:37:20Z

yes its exactly the same problem. i thought it was related to openid-connect but really not. iwas still the easy when openid-connect is disabled. i got those issue in 3.3.0. i rollback to the LTS 3.2.2 and now everyhting is fine.

Revolyssup · 2023-08-09T07:26:09Z

@neosis41 You can close the issue then

macdoor · 2023-08-09T08:59:53Z

I am facing the same issue with version 3.4.1. I rollback to the LTS 3.2.2. I seems okay now

2023/08/09 16:42:37 [error] 25967#25967: *142132 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/init.lua:332: attempt to index local 'matched_ssl' (a nil value)
stack traceback:
coroutine 0:
	/usr/local/apisix/apisix/init.lua: in function 'verify_https_client'
	/usr/local/apisix/apisix/init.lua:560: in function 'http_access_phase'
	access_by_lua(nginx.conf:345):2: in main chunk, client: 10.14.69.11, server: _, request: "POST /api/v2/sync/xxxxxx HTTP/1.0", host: "xxxxx:19443"

anilaws · 2023-08-18T15:56:12Z

I am getting the same error with apisix version 3.4.0
Can you please provide an update on this issue

nijave · 2023-08-19T00:02:15Z

If this is the same as #9801, I was hitting this consistently when running wrk to benchmark against a simple Python backend with a few instances (Kubernetes pods). In #9801, someone mentioned it being a race condition which would explain why it was much more common under load.

I'm not using openidc. My setup was just with the key-auth plugin

Revolyssup · 2023-08-22T06:06:34Z

Fix created for this and reasons explained here - #10066

kingluo · 2023-09-12T12:28:30Z

We can not reproduce the issue on the master branch anymore, because the commit #9903 after 3.4.1: adds ssl_client_hello_by_lua_block. This phase used by apisix always constructs ngx.ctx.matched_ssl:

apisix/apisix/init.lua

Lines 205 to 207 in f47c2d7

    
           local ok, err = router.router_ssl.match_and_set(api_ctx, true, sni) 
        
           ngx_ctx.matched_ssl = api_ctx.matched_ssl

neosis41 changed the title ~~bug:~~ 500 internal error HTTPS issue Jun 7, 2023

neosis41 changed the title ~~500 internal error HTTPS issue~~ attempt to index local 'matched_ssl' 500 internal error HTTPS issue Jun 7, 2023

lingsamuel added the bug Something isn't working label Jun 21, 2023

leslie-tsang assigned Revolyssup Jul 24, 2023

neosis41 closed this as completed Jul 25, 2023

neosis41 reopened this Jul 27, 2023

This was referenced Aug 10, 2023

bug: 500 internal server error after refreshing page . #10001

Closed

help request: accessing an HTTPS route I receive a LUA runtime error: "attempt to index local 'matched_ssl' (a nil value)" #10016

Closed

Revolyssup mentioned this issue Aug 18, 2023

bug: intermittent HTTP 500 errors occur when accessing domain via HTTPS. #9801

Closed

This was referenced Aug 21, 2023

help request: Erro 500 after update to version 3.4.2 #9931

Closed

fix: error on https requests due to nil matched_ssl #10066

Closed

Revolyssup assigned kingluo Aug 28, 2023

kingluo closed this as completed Sep 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

attempt to index local 'matched_ssl' 500 internal error HTTPS issue #9610

attempt to index local 'matched_ssl' 500 internal error HTTPS issue #9610

neosis41 commented Jun 6, 2023 •

edited

Loading

Revolyssup commented Jul 21, 2023

Revolyssup commented Jul 25, 2023

neosis41 commented Jul 25, 2023 via email

Revolyssup commented Jul 25, 2023

neosis41 commented Jul 27, 2023

neosis41 commented Jul 27, 2023

neosis41 commented Jul 27, 2023

neosis41 commented Jul 27, 2023

n84ck commented Jul 28, 2023 •

edited

Loading

stonegithup commented Aug 7, 2023 •

edited

Loading

Revolyssup commented Aug 7, 2023 •

edited

Loading

neosis41 commented Aug 8, 2023

Revolyssup commented Aug 9, 2023

macdoor commented Aug 9, 2023 •

edited

Loading

anilaws commented Aug 18, 2023

nijave commented Aug 19, 2023

Revolyssup commented Aug 22, 2023

kingluo commented Sep 12, 2023

attempt to index local 'matched_ssl' 500 internal error HTTPS issue #9610

attempt to index local 'matched_ssl' 500 internal error HTTPS issue #9610

Comments

neosis41 commented Jun 6, 2023 • edited Loading

Current Behavior

Expected Behavior

Error Logs

Steps to Reproduce

Environment

Revolyssup commented Jul 21, 2023

Revolyssup commented Jul 25, 2023

neosis41 commented Jul 25, 2023 via email

Revolyssup commented Jul 25, 2023

neosis41 commented Jul 27, 2023

neosis41 commented Jul 27, 2023

neosis41 commented Jul 27, 2023

neosis41 commented Jul 27, 2023

n84ck commented Jul 28, 2023 • edited Loading

stonegithup commented Aug 7, 2023 • edited Loading

Revolyssup commented Aug 7, 2023 • edited Loading

neosis41 commented Aug 8, 2023

Revolyssup commented Aug 9, 2023

macdoor commented Aug 9, 2023 • edited Loading

anilaws commented Aug 18, 2023

nijave commented Aug 19, 2023

Revolyssup commented Aug 22, 2023

kingluo commented Sep 12, 2023

neosis41 commented Jun 6, 2023 •

edited

Loading

n84ck commented Jul 28, 2023 •

edited

Loading

stonegithup commented Aug 7, 2023 •

edited

Loading

Revolyssup commented Aug 7, 2023 •

edited

Loading

macdoor commented Aug 9, 2023 •

edited

Loading