Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(node): ref/unref for udp #21521

Closed
wants to merge 18 commits into from
Closed

Conversation

CosminPerRam
Copy link

I attempted on adding ref/unref for UDP.
I do not expect this to be merged (as I do not think what I've done is exactly what is needed for it), but suggestions and directions would be gladly appreciated.

Related:

@CLAassistant
Copy link

CLAassistant commented Dec 9, 2023

CLA assistant check
All committers have signed the CLA.

@CosminPerRam CosminPerRam changed the title feat(ext/net): attempt ref/unref for udp feat(node): ref/unref for udp Dec 9, 2023
@CosminPerRam CosminPerRam marked this pull request as draft December 9, 2023 00:59
@CosminPerRam
Copy link
Author

Bump, any thoughts?

@bartlomieju
Copy link
Member

@littledivy please review

@CosminPerRam
Copy link
Author

Sorry for the linting issues, had some troubles running it at first, will fix asap when I'll get on my computer.

@littledivy littledivy self-requested a review December 14, 2023 03:28
ext/node/polyfills/internal_binding/udp_wrap.ts Outdated Show resolved Hide resolved
Comment on lines 320 to 322
if (this.#listener) {
this.#listener!.unref();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (this.#listener) {
this.#listener!.unref();
}
this.#listener?.unref();

Comment on lines 426 to 432
ref() {

}

unref() {

}
Copy link
Member

@littledivy littledivy Dec 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of exposing public APIs for ref and unref in Deno.Datagram it should be implemented internally. Something like this:

Add a new private fields this.#recvPromise = null and this.#unrefed = false in UdpWrap.

In #recieve:

[buf, remoteAddr] = (await this.#listener!.receive(p)) as [

Change this line to store the current recieve promise in the field.

this.#recvPromise = this.#listener!.receive(p);
if (this.#unrefed) core.unrefOpPromise(this.#recvPromise);

[buf, remoteAddr] = (await this.#recvPromise) as [
// ...
// UdpWrap
ref() {
  if (this.#recvPromise) core.refOpPromise(this.#recvPromise);
  this.#unrefed = false;
}

unref() {
  if (this.#recvPromise) core.refOpPromise(this.#recvPromise);
  this.#unrefed = true;
}

@CosminPerRam CosminPerRam marked this pull request as ready for review December 14, 2023 19:06
Comment on lines 987 to 991

Deno.test(
{ permissions: { read: true, run: true, net: true } },
async function netUdpListenUnrefAndRef() {
const p = execCode2(`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please test the Node.js API instead of Deno.listenDatagram. It can be written in cli/tests/unit_node/dgram_test.ts

@CosminPerRam
Copy link
Author

CosminPerRam commented Dec 16, 2023

Hope I don't miss anything, but there is no cli/tests/unit_node/dgram_test.ts file, so I've created it.

udp ref and unref fails with

[dgram_test 009.80] [node/dgram] udp ref and unref => ./cli/tests/unit_node/dgram_test.ts:9:6
[dgram_test 009.80] error: Leaking async ops:
[dgram_test 009.80]   - 1 async operation to receive a datagram message via UDP was started in this test, but never completed. This is often caused by not awaiting the result of `Deno.DatagramConn#receive` call, or not breaking out of a for await loop looping over a `Deno.DatagramConn`. 

udp unref hangs indefinetly (test integration::node_unit_tests::dgram_test has been running for over 60 seconds)

Copy link
Member

@littledivy littledivy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@bartlomieju bartlomieju enabled auto-merge (squash) December 29, 2023 23:42
@bartlomieju
Copy link
Member

@littledivy @CosminPerRam it appears there's still a failing test that leaks ops. Can you please take a look at it?

@CosminPerRam
Copy link
Author

CosminPerRam commented Jan 3, 2024

Unfortunately this beats me, @littledivy could you please take care of it (some afterwards explanations would be appreciated)?

Comment on lines +322 to +323
if (this.#recvPromise) core.refOpPromise(this.#recvPromise);
this.#unrefed = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (this.#recvPromise) core.refOpPromise(this.#recvPromise);
this.#unrefed = true;
if (this.#recvPromise) core.unrefOpPromise(this.#recvPromise);
this.#unrefed = true;

Comment on lines +21 to +23
udpSocket.on("message", (buffer, _rinfo) => {
resolve(Uint8Array.from(buffer));
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
udpSocket.on("message", (buffer, _rinfo) => {
resolve(Uint8Array.from(buffer));
});
let data;
udpSocket.on("message", (buffer, _rinfo) => {
data = Uint8Array.from(buffer);
udpSocket.close();
});
udpSocket.on("close", () => {
resolve();
});

Socket needs to be closed.

@littledivy
Copy link
Member

@CosminPerRam I opened #21777 with a different implementation that fixes bugs in this approach. The major one being that the promises returned by this.#listener are not ref/unrefable because they wrap the actual async op call.

@CosminPerRam
Copy link
Author

Thanks, I will close this PR as that one superseeds this one.

auto-merge was automatically disabled January 3, 2024 15:46

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants