UDP downstream api

[alonbg]

Hi,

Would like to implement a udp proxy server (dns proxy) with stream and upstream modules.
Both send/receive messages are not longer than a single datagram.
I would also like to intercept response from upstream server (That is 'subrequest' in http module)

Would like to code/port missing parts to existing modules.
Any recommendations to where should I start ?

From what I have seen upstream-lua does not support udp and I'm not sure it's usable in the stream/server context and when considering porting then lua-nginx-module
does not support udp cosockets just yet

thanks

[agentzh]

@alonbg I think we can just extend the existing downstream cosocket API of ngx_stream_lua_module to support datagrams. ngx_stream_lua_module does support upstream datagram cosockets though. You can have a look at that for the downstream part (ideally, we should resuse the implementation if possible).

[alonbg]

looked at the code and bit confused. Would need some explicit guidelines or a starting push or can help in any other way ( so I guess "we" is good :-). What ever works.

Also when we say 'upstream' that is using the proxy_pass ? And would proxy_bind be in effect ?

[agentzh]

@alonbg When I say upstream, I mean anything talking to the backend. When I say downstream, I mean anything talking to the client side. I'm not referring to nginx's upstream facility in this context because I'm talking about cosockets.

Upstream cosocket API:

https://github.com/openresty/lua-nginx-module#ngxsockettcp

Downstream cosocket API:

https://github.com/openresty/lua-nginx-module#ngxreqsocket

[alonbg]

@agentzh I understand now. Thank you.

So as for extending existing downstream to support datagram. Code at most part seems agnostic to transport type as this is handled by nginx. Otherwise I do not see where the extensions should go.

As for the upstream I'd like to extend with capabilities similar to nginx 1.11 proxy_bind $remote_addr transparent. see ngx_event_connect_set_transparent. I guess that would require support for a socket operation bind and a working setoption.

[agentzh]

@alonbg I guess ngx.req.socket can just return a datagram-typed cosocket if the downstream is datagram-typed.

Regarding bind, yes, we can have a separate cosocket method. Just ensure it is compatible with LuaSocket's API wherever possible ;)

[agentzh]

@alonbg Thanks for looking into this!

[alonbg]

@agentzh downstream - I assume you mean ngx.req.socket should already be supporting datagram.
I'll check it out. Maybe a good way to start is to add udp tests ? What do you think ?
One location though (not sure) that might require modifications.

upstream - I will look into bind and setoptions (that is if you are not already .. ;)

[agentzh]

@alonbg No, ngx.req.socket only supports stream-typed socket operations atm. The object methods should be different for datagrams.

[daxiong380380]

@alonbg how do you implement a udp proxy server finally?

[bjne]

I am interested in basic udp server functionality as well, and progress/timeframe on when this might arrive?

[agentzh]

Hopefully in the next few months :)

[agentzh]

Patches and volunteers are always welcome :)

[daxiong380380]

@agentzh
some directives,such as set,access_by_lua_block,are being developped,aren't they?
if I set the proxy_pass value in the access phase,I can dispatch my stream to my will, is that right?
Thanks

[agentzh]

@daxiong380380 You'll have to rely on the balancer_by_lua* feature proposed in #7 for dynamic dispatching on ngx_stream_proxy_module. The access_by_lua* thing won't help much here since the nginx variables are not supported in nginx's stream subsystem (thus no proxy_pass $target usage).

[alonbg]

@agentzh
What do you suggest:

1. ngx.udp_req.socket (req would be nil)
2. ngx.req.udp_socket (req.socket would be nil)
   or
3. ngx.req.socket where the returned object can be either udp socket of tcp socket ?

[agentzh]

@alonbg I think I like ngx.req.udp_socketthe most :)

BTW, I'm also going to build a simple DNS name server atop this module for our openresty.org site :) Thank you for working on this!

[alonbg]

forked here

nginx.conf with both tcp and udp test servers

testing udp: not working ( just yet ... )
client: socat -T10 -t10 -d -d -d - udp:127.0.0.1:8070
echo server: socat -d -d -d udp-l:8071,reuseaddr,fork,crlf system:"cat"

testing tcp: working just fine
client: socat -d -d -d - tcp:127.0.0.1:5354
echo server: socat -d -d -d tcp-l:5355,reuseaddr,fork,crlf system:"cat"

@agentzh could you please have a quick look
thanks

[agentzh]

@alonbg Will you create a pull request on GitHub? This way is easier for us to review your changes. Many thanks!

[alonbg]

done

[webandbusiness]

Hello, so far today in November 2018, can we implement UDP server with nginx-lua-stream module
Kind regards

[agentzh]

It is already in the master.

[rbreslow]

@agentzh You mentioned this is in master, but I can't find anything that makes it seem like support exists to read UDP packets coming into OpenResty.

Can you share a link or connection to a PR?

[rbreslow]

Ah. It doesn't seem documented. https://openresty-reference.readthedocs.io/en/latest/Lua_Nginx_API/#ngxreqsocket says that ngx.req.socket returns tcpsock, but evidently it also returns a socket capable of inspecting UDP packets.

However, it also seems that :peek() is nil for UDP request sockets, which makes it useless to do any sort of filtering in preread_by_lua_block. Any suggestions?