Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX unhandled promises on transport.send() #289

Merged
merged 1 commit into from
May 4, 2022
Merged

FIX unhandled promises on transport.send() #289

merged 1 commit into from
May 4, 2022

Conversation

aricart
Copy link
Member

@aricart aricart commented May 4, 2022

These could bubble up and be trapped by the runtime, resulting in the client crashing.

FIXES #271

@aricart aricart requested a review from kozlovic May 4, 2022 20:41
Copy link
Member

@kozlovic kozlovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aricart aricart merged commit b6b2950 into main May 4, 2022
@aricart aricart deleted the fix-271 branch May 4, 2022 20:49
@intech
Copy link

intech commented May 4, 2022

Errors suppression was never considered a good pattern - wasn't it?
img

@intech
Copy link

intech commented May 5, 2022

@aricart @kozlovic ping

@aricart
Copy link
Member Author

aricart commented May 5, 2022

In this particular case it slipped through - the correct handling of the error was elsewhere but those promises were forgotten :(

@aricart
Copy link
Member Author

aricart commented May 5, 2022

In case it is not clear, these promises fail when the socket already failed, which is handled by the transport implementation. The rejections of the write are typically subsequent to this and can be safely ignored, as the client will enter its reconnect logic. As usual, this is core NATS so it is at most once delivery.

For deno the detection happens on a read error
https://github.com/nats-io/nats.deno/blob/main/src/deno_transport.ts#L187

For node/websocket the event handlers on the socket deal with it
https://github.com/nats-io/nats.js/blob/main/src/node_transport.ts#L278
https://github.com/nats-io/nats.ws/blob/main/src/ws_transport.ts#L144

@intech
Copy link

intech commented May 6, 2022

@aricart A more attractive solution would be to create an error event and emit such errors so that they can always be logged and processed if necessary.

@aricart
Copy link
Member Author

aricart commented May 6, 2022

The disconnect is reported via the status mechanism. If your app requires to know if your messages arrived, you use request reply.

@arjenvdhave
Copy link

Hi @aricart

I have this error:

Error: write EPIPE
at afterWriteDispatched (node:internal/stream_base_commons:160:15)
at writeGeneric (node:internal/stream_base_commons:151:3)
at Socket._writeGeneric (node:net:817:11)
at Socket._write (node:net:829:8)
at writeOrBuffer (node:internal/streams/writable:389:12)
at _write (node:internal/streams/writable:330:10)
at Socket.Writable.write (node:internal/streams/writable:334:10)
at NodeTransport.send (/usr/src/app/node_modules/nats/lib/src/node_transport.js:327:21)
at ProtocolHandler.processInfo (/usr/src/app/node_modules/nats/lib/nats-base-client/protocol.js:425:32)
at ProtocolHandler.push (/usr/src/app/node_modules/nats/lib/nats-base-client/protocol.js:461:22) {
errno: -32,
code: 'EPIPE',
syscall: 'write'

Could it be that the .catch should also be added to this call:
https://github.com/nats-io/nats.deno/blob/main/nats-base-client/protocol.ts#L509

@aricart
Copy link
Member Author

aricart commented May 25, 2022

@arjenvdhave that one is in catch but not awaited, may need to modify the transport interface. - really want to know when that fails right there.

@arjenvdhave
Copy link

Yep me to :)
For some reason I only get this in my app when it is running in a docker container on our k8s cluster.
I still haven't got a clue what the cause is.

Other services using the same setup don't have this issue.

@aricart
Copy link
Member Author

aricart commented May 25, 2022

@arjenvdhave the network overlay is possibly the problem, right now trying to remove the async nature of send - as I am interested in a possible exception that happens as part of the write operation, but not the actual write succeeding...
Do you have a repeatable case, if I publish a bundle for you to try?

@arjenvdhave
Copy link

I hacked the catch call in my docker image.
I now get this error:

error: NatsError: DISCONNECT
- error.js:100 Function.errorForCode
[app]/[nats]/lib/nats-base-client/error.js:100:16
- protocol.js:175
[app]/[nats]/lib/nats-base-client/protocol.js:175:40
- Array.forEach
- protocol.js:174 ProtocolHandler.resetOutbound
[app]/[nats]/lib/nats-base-client/protocol.js:174:15
- protocol.js:195 ProtocolHandler.prepare
[app]/[nats]/lib/nats-base-client/protocol.js:195:14
- protocol.js:244 ProtocolHandler.<anonymous>
[app]/[nats]/lib/nats-base-client/protocol.js:244:31
- Generator.next
- protocol.js:34
[app]/[nats]/lib/nats-base-client/protocol.js:34:75
- new Promise
- protocol.js:13 __awaiter
[app]/[nats]/lib/nats-base-client/protocol.js:13:16

And after that it retries and connects without issues

@arjenvdhave
Copy link

@aricart i can reproduce the issues on my cluster.
If you have "fix" i can try it.

I however do use the nats-js lib, not this deno directly

@aricart
Copy link
Member Author

aricart commented May 25, 2022

@arjenvdhave give me a few, posting the branch for the deno side, now need to change the usage on nats.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The network exception is not handled normally
4 participants