bug: apisix2.8 dns解析IP变化导致栈溢出 #4941
Comments
|
Could you provide the logs before the stack overflow? |
这条连接的日志 dns每变化一次，deepcopy 就会增加一层递归拷贝.. 持续一段时间后就会出现栈溢出 2021/08/30 09:52:15 [error] 10780#10780: *35662 [lua] debug.lua:128: http_access_phase(): call require("apisix").http_access_phase() args:{}, client: 192.168.33.133, server: _, request: "GET / HTTP/1.1", host: "192.168.33.133:9080" |
|
I try to reproduce it but fail. Here is my route configuration: I used coredns and a python script to change DNS configuration each second, and the IP rotates from 127.0.0.1 to 127.0.0.8: Then I loaded test the I am curious about why the input can bypass the loop check: Lines 102 to 103 in a062a9c Maybe it only happens with special route configuration? What's your route configuration? |
|
路由配置 {"count":1,"node":{"nodes":[{"key":"/apisix/routes/365638203417298469","value":{"id":"365638203417298469","uris":["/*"],"create_time":1627466952,"methods":["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],"status":1,"name":"test","upstream":{"timeout":{"send":6,"read":6,"connect":6},"type":"roundrobin","scheme":"http","pass_host":"pass","nodes":[{"host":"www.test.com","port":80,"weight":100}]},"update_time":1630318367,"plugins":{"xxxxxx":{"hosts":[{"fails":0,"health_check_time":0,"port":80,"host":"192.168.33.112"},{"fails":0,"health_check_time":0,"port":80,"host":"192.168.33.111"}],"disable":false,"rejected_code":502,"block":true}}},"createdIndex":404,"modifiedIndex":609}],"key":"/apisix/routes","dir":true},"action":"get"} |
|
deepcopy中加了一些调试 local function _deepcopy(orig, copied)
local core = require("apisix.core")
ngx.log(ngx.ERR, core.json.delay_encode(orig))
-- prevent infinite loop when a field refers its parent
copied[orig] = true
-- If the array-like table contains nil in the middle,
-- the len might be smaller than the expected.
-- But it doesn't affect the correctness.
local len = #orig
local copy = new_tab(len, nkeys(orig) - len)
for orig_key, orig_value in pairs(orig) do
if type(orig_value) == "table" and not copied[orig_value] then
copy[orig_key] = _deepcopy(orig_value, copied)
ngx.log(ngx.ERR, "next")
else
ngx.log(ngx.ERR, "key-value : ", orig_key, "-", tostring(orig_value))
copy[orig_key] = orig_value
end
end
return copy
end以下为变化一次 dns 解析IP后打印出的调试信息 . 每变化一次 调试信息就会增加 2021/08/31 09:09:04 [error] 5066#5066: 2343 [lua] debug.lua:128: http_access_phase(): call require("apisix").http_access_phase() args:{}, client: 127.0.0.1, server: _, request: "GET / HTTP/1.1", host: "127.0.0.1:9080" |
It happened when we are handling upstream with domain inside a route. Fix apache#4938 Fix apache#4941 Signed-off-by: spacewander <spacewanderlzx@gmail.com>
Issue description
apisix2.8 dns解析IP变化导致栈溢出复现过程
现象：
服务器500错误
错误日志堆栈 ：
问题出现状况：
生产环境 cname 对应IP会不定时发生变化
复现过程：
apisix 上游服务目标节点配置假cname www.test.com
本机使用 dnsmasq 循环修改 www.test.com 对应的解析IP 127.0.0.1 / 192.168.33.101 (2个即可复现) 并持续访问
等待一段时间会出现即可复现堆栈、并且内存即将打满
相关代码：
parse_domain_in_route.lua
Environment
apisix version):2.8uname -a):nginx -Voropenresty -V):curl http://127.0.0.1:9090/v1/server_infoto get the info from server-info API):luarocks --version):Steps to reproduce
复现过程：
apisix 上游服务目标节点配置假cname www.test.com
本机使用 dnsmasq 循环修改 www.test.com 对应的解析IP 127.0.0.1 / 192.168.33.101 (2个即可复现) 并持续访问
等待一段时间会出现即可复现堆栈、并且内存即将打满
Actual result
apisix 500错误 table拷贝递归栈溢出
Error log
2021/08/30 09:52:15 [error] 10780#10780: *35662 lua entry thread aborted: runtime error: stack overflow
stack traceback:
coroutine 0:
[C]: in function 'nkeys'
/usr/local/apisix/apisix/core/table.lua:108: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
/usr/local/apisix/apisix/core/table.lua:111: in function '_deepcopy'
..., client: 192.168.33.133, server: _, request: "GET / HTTP/1.1", host: "192.168.33.133:9080"
The text was updated successfully, but these errors were encountered: