Skip to content

Race Condition Issue in PostgreSQL SSL Connection Process #3401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
edwardyoon2 opened this issue Mar 13, 2025 · 3 comments
Open

Race Condition Issue in PostgreSQL SSL Connection Process #3401

edwardyoon2 opened this issue Mar 13, 2025 · 3 comments

Comments

@edwardyoon2
Copy link

edwardyoon2 commented Mar 13, 2025

I have been encountering "Error: Client network socket disconnected before secure TLS connection was established" errors recently without any specific cause when connecting to PostgreSQL through RDS Proxy. These errors occur intermittently, making them difficult to reproduce consistently.
After examining the code in the pg module's lib/connection.js file, I suspect there is a race condition issue between the TCP connection and TLS connection process. The architecture of the connection flow appears problematic:

  • Client initiates TCP connection (stream.connect(port, host))
  • When TCP connection succeeds, a 'connect' event is emitted
  • However, the code for SSL/TLS connection is not directly linked to the 'connect' event
  • Instead, logic to wait for the server's SSL support response is set up separately (stream.once('data', function (buffer) {...})

This design creates potential timing issues because:

  • There's no explicit synchronization between the TCP connection establishment and when requestSsl() is called
  • The SSL handshake process depends on event listeners that operate asynchronously
  • When working with RDS Proxy as an intermediary layer, these timing sensitivities may become more problematic

I've verified this by testing direct connections to PostgreSQL without RDS Proxy, which work consistently. The issue only manifests when connecting through the proxy layer.
Please check if my understanding of this potential race condition is correct, and if there are any recommended approaches to mitigate this issue.

@saper
Copy link

saper commented Mar 13, 2025

Note to self: #3346 might require the change of the logic anyway

@brianc
Copy link
Owner

brianc commented Mar 15, 2025

I am not sure what you mean exactly by a race condition here. All network messaging happens asynchronously for hand-shake style things. The same socket is re-used when transitioning to SSL, it's just upgraded so you getting an issue about being disconnected before the TLS connection was established probably isn't related to a race condition...since there is no explicit disconnection being done by the library between establishing the first, non-TLS connection to the backend and then upgrading the same connection once the SSL support response is received.

@edwardyoon2
Copy link
Author

I suspect there might be an issue with RDS Proxy when using multiple database connections simultaneously through TypeOrmModule.forRootAsync() with the @InjectRepository decorator.

The error logs simply show intermittent occurrences of:

  • 'Client network socket disconnected before secure TLS connection was established'
  • 'Connection terminated unexpectedly'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants