Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set flags in JAVA_OPTS to speed up java connector startup latency in tests #31816

Closed
postamar opened this issue Oct 25, 2023 · 1 comment · Fixed by #32318
Closed

set flags in JAVA_OPTS to speed up java connector startup latency in tests #31816

postamar opened this issue Oct 25, 2023 · 1 comment · Fixed by #32318
Assignees
Labels
team/db-dw-sources Backlog for Database and Data Warehouse Sources team

Comments

@postamar
Copy link
Contributor

Topic

No response

Relevant information

The performance investigation of the tests in source-postgres revealed that java connector containers could require the better part of a second to do something as basic as respond to a spec API call. See #31298 (comment)

Part of what causes the latency to be high is that the JVM default settings are tuned for long-running operation. This is fine for us in a production setting but in tests these JVMs are typically short-lived. I've found that making the JIT less aggressive with -XX:TieredStopAtLevel=1 would speed things up noticeably. Perhaps there's other flags and settings worth using, I don't know yet.

Furthermore another cause of startup latency involves the logger. I don't think tests care much about the logs and overriding the log4j configuration with something like -Dlog4j.configurationFile=<some nonexistent path> would speed things up as well. Alternatively, we can add a very simple but valid log4j-for-integration-testing.propertiesresource file in the CDK somewhere and refer to it with-Dlog4j.configuration=`.

@postamar postamar added needs-triage team/db-dw-sources Backlog for Database and Data Warehouse Sources team labels Oct 25, 2023
@postamar postamar self-assigned this Oct 25, 2023
@postamar
Copy link
Contributor Author

postamar commented Nov 8, 2023

I'm wrong about the tests not caring about the connector logs. I found them to be quite useful the other day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team/db-dw-sources Backlog for Database and Data Warehouse Sources team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant