Get native handle from net.Socket

[ghost]

It doesn't seem possible to get the native handle (file descriptor / SOCKET / HANDLE) from a net.Socket via addons. The Node SDK does not include the tcp_wrap.h or any of the derived from headers to allow one to get the uv handle and from there use uv_fileno.

Projects like node-ancillary (https://github.com/VanCoding/node-ancillary) actually end up with a private copy of the internal Node.js headers to allow fetching the uv_tcp_t pointer. This is of course not the right way to solve the problem as this solution will break on internal implementation changes.

Getting the native handle of a socket can be used to transfer the connection from Node.js into the addon or into any other server and this is what I currently do on Unix systems. I cannot do this on Windows since I cannot get the SOCKET handle.

[mscdex]

You should be able to just unwrap the object like node::Unwrap() does. Unwrapping gets you access to the HandleWrap instance, which has a GetHandle() method that returns the underlying uv_handle_t*.

[bnoordhuis]

Yes, but the installer doesn't install handle_wrap.h and friends. With good reason: the implementation is API nor ABI stable.

You would also be subverting node's internal bookkeeping, something that could blow up at any time for any reason. I don't think this is a use case we should support, even unofficially.

[ghost]

I don't think this is a use case we should support, even unofficially.

How do you mean? Are you against the whole idea? I do this on Unix and it works just fine. You could just add a property _external to the TCPWrap or any parent and return a pointer to the uv structure.

[bnoordhuis]

Are you against the whole idea?

Yes. What you are requesting is a way to manipulate a libuv handle that node.js owns without node.js knowing about it. That can go wrong in so many ways I won't even start enumerating them.

What might be acceptable is a way to transfer ownership from node.js to an add-on. There are practical objections, though. The libuv handle is embedded in the HandleWrap object so it's implicitly tied to the lifetime of the HandleWrap.

Dynamically allocating the libuv handle is an option but that's asking everyone to take a hit for a feature only a small percentage of users have a need for. There is a non-zero probability such a change would be rejected. (It's also not strictly necessary, you could uv_write2 the handle to clone it.)

If you just want a handle and also expose it as a net.Socket, a better solution is to create the handle yourself and the socket object with new net.Socket({ handle: handle }). The JS handle object should follow the contract of regular HandleWrap instances. That takes some upfront work but is otherwise transparent to node.js.

[ghost]

Yes you can shoot yourself in the foot pretty badly with this feature. If you want to restrict the use case to something easier to support I think an interface like this would be very useful:

net.Socket.peel(callback that takes a v8::External)

I call it peel because essentially you take a net.Socket and peel away all the overhead of JavaScript and end up with a native socket/file descriptor. This function could be implemented in various ways, but the main idea is to destroy/delete all of the net.Socket object and only get a native handle that is now completely owned by the caller.

Edit: I edited my text.

Edit2: You can call it export and maybe it could also be called with a pointer to the SSL structure.

[ghost]

I just managed to (hack-) port my addon to Windows by getting the native handle by sliding one byte at a time from the HandleWrap pointer until it recognized a uv_tcp_t. This worked, I dup the native handle and then call net.Socket.destroy and use the native handle just like on Unix. It really seems duping a socket is completely safe, Node.js doesn't care about the underlying internals of the descriptor, it just uses it like any other reference counted resource. It is hacky, yes, but it works without messing with any "bookkeeping" since the bookkeeping lies on the OS.

[ghost]

My point is, it can work and it would be a really nice feature to allow addons or any other process to take ownership of the native socket.

[ghost]

@bnoordhuis @indutny Just as a point of reference: you already expose the OpenSSL (internal) SSL pointer via net.Socket._handle.ssl._external (thanks to Fedor for showing me this!). Using that pointer you can of course fuck up the internals of Node.js, but it is also very useful since the structure is reference counted. That way you can transfer a complete connection with SSL and everything into the addon (I have done this now for about 6 months with success). It would be very easy to do the same for the uv_handle_t, it could be exposed as net.Socket._handle._external. The file / socket descriptor is also reference counted once duplicated so it too can be used to transfer the connection (same story).

Like I said, I already have this working on Unix and Windows, I just want to move from my haxxy solution described above to using something better supported. Of course Node.js does not need to care about what addons do with the pointer - if they want to crash the process that's up to them (delete (void *) 1; will do that just fine for you).

[clshortfuse]

I'd like some way to access the native windows socket, just so I can call setsockopt against it. Linux is fine because because of fd. I'm unsure what is meant my unwrapping the handle. Can I do this from nodeJS?

What the difficulty with adding net.Socket._handle.getWindowsHandle() ?

[ghost]

This is how I do it (yes, majorly haxxy but works):

uv_handle_t *getTcpHandle(void *handleWrap)
{
    volatile char *memory = (volatile char *) handleWrap;
    for (volatile uv_handle_t *tcpHandle = (volatile uv_handle_t *) memory; tcpHandle->type != UV_TCP
         || tcpHandle->data != handleWrap || tcpHandle->loop != uv_default_loop(); tcpHandle = (volatile uv_handle_t *) memory) {
        memory++;
    }
    return (uv_handle_t *) memory;
}

where void *handleWrap is args[0]->ToObject()->GetAlignedPointerFromInternalField(0) and args[0] is net.socket._handle.

Then you use uv_fileno on the uv_handle_t to get your SOCKET handle.

[clshortfuse]

I was hoping for something in pure JavaScript. The issue I have is enabling OOB packets on sockets. I've already solved this in Linux because of its use of file descriptor which I listed here: #8282

I'm not sure where to use your code on an existing socket. I would have to create a custom module and then pass the socket._handle object to the module's getTcpHandle function?

Official implementation would just need to add

env->SetProtoMethod(t, "getHandle", HandleWrap::GetHandle);

at

node/src/tcp_wrap.cc

Line 69 in b896057

env->SetProtoMethod(t, "ref", HandleWrap::Ref);

Which would call

node/src/handle_wrap.h

Line 50 in b896057

inline uv_handle_t* GetHandle() const { return handle__; }

I can build a PR, if that's all that's needed, unless I'm oversimplifying a bigger issue.

[ghost]

The issue is not implementation complexity, this could be solved very easily. The issue is that "they" do not want this feature, so if you feel lucky you need to haxx around it yourself. You cannot implement this in JS code alone, you need a native addon somewhere in the picture.

[clshortfuse]

@alexhultman I used your method and build a package for this

https://github.com/clshortfuse/node-getsockethandleaddress

and I got it to work. I'm unclear about why you loop through until you find UV_TCP though, but I was able to then use that socket number to set the needed socket option through ffi and ref.

[Trott]

This issue has been inactive for sufficiently long that it seems like perhaps it should be closed. Feel free to re-open (or leave a comment requesting that it be re-opened) if you disagree. I'm just tidying up and not acting on a super-strong opinion or anything like that.

[clshortfuse]

I'm just posting this for posterity. I upgraded my module to 1.1.0 and it should work on any OS now, not just Windows. If you try to run the function on another OS, it'll just return handle.fd for you.

https://github.com/clshortfuse/node-getsockethandleaddress

const address = require('getsockethandleaddress').getAddress(socket._handle);

So now it's safe to use on any operating system without having to do platform checks or using optional dependencies.

[megawattz]

I think the idea here is Nodejs is ideally cross platform. The file objects need to be usable across platforms. There's no guarantee that every platform implements files or sockets with a handle. Another example is, there's no REAL fork() function in Nodejs, at least not in the way Unix does it. I strongly suspect this is because Windows doesn't have real fork() so they can't put that into Nodejs.

I would like to have had access to the underlying handle for readonly operations. Specifically I want to know the message backlog in a UDP socket and NodeJS doesn't have a function to do that in the dgram class.

[dong-king]

If you just want a handle and also expose it as a net.Socket, a better solution is to create the handle yourself and the socket object with new net.Socket({ handle: handle }). The JS handle object should follow the contract of regular HandleWrap instances. That takes some upfront work but is otherwise transparent to node.js.

@bnoordhuis
My code is as follows：

const TCPWrap = process.binding('tcp_wrap');
const { TCP } = TCPWrap;
const handle  = new TCP(TCPWrap.constants.SOCKET);
console.log(handle)                 // handle is not null here
var conn = new net.Socket({handle});
console.log(conn._handle)	// print null

Why is conn._handle still null after executing new net.Socket({ handle: handle })？

[ClosetGeek-Git]

You should be able to just unwrap the object like node::Unwrap() does. Unwrapping gets you access to the HandleWrap instance, which has a GetHandle() method that returns the underlying uv_handle_t*.

Thank you, this is likely what I was looking for. As to other comments, this is typical of how to access native resources created in a scripts user space. Same basic thing in PHP and python as well when reaching into native resources. How would you even handle such a thing once you pass ownership? There is no “surrenderownsership” event/method within the nodejs socket object… you grab the resource, use it understanding possible issues, then leave it in a state that javascript will still be able to handle it, either for further use within the script or by freeing/destroying the resources as it normally would.

[ImBoop]

Any chance we could get this reopened? Per abi stability, any module that needs a UV handle from a node owned socket/pipe/file/process/etc will need to be rebuilt every major version, but I think that tradeoff is ok. We're leaving a lot of new features on the table. As long as the developer isn't doing anything to circumvent the state node knows about (which shouldn't ever be the case because Node would already support what they want), it'll always be safe.

For instance, I needed to be able to get the pipe handle today because Node doesn't support getting any information about who is on the client side of the named pipe (SO_PEERCRED on Linux, ImpersonateNamedPipeClient and some change on Windows).

But there are many other possible use cases, such as adding more ways to interact with processes, etc, without having to create a wrapper for each possible handle type you need.

[ClosetGeek-Git]

@ImBoop GetHandle()

[ImBoop]

@ImBoop GetHandle()

I use that, but it's not well documented, it requires GetAlignedPointerFromInternalField (only accessible with nan) and the index it lives at could change (it's at node::BaseObject::kSlot which is 1, not the 0 it used to be where the comment above hardcodes it to 0) - not to mention requiring separately imported node headers which complicates builds.