Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose StreamingDirectExchangeBuffer buffer utilization distribution as DataSource metrics #23107

Open
sopel39 opened this issue Aug 22, 2024 · 1 comment

Comments

@sopel39
Copy link
Member

sopel39 commented Aug 22, 2024

Similar as for output buffer utilization (#13463), we can track input buffer utilization (after first page is placed in the buffer). This would allow us to identify network bottlenecks

@sopel39
Copy link
Member Author

sopel39 commented Sep 3, 2024

The idea here is to track io.trino.operator.DirectExchangeBuffer#getRetainedSizeInBytes (distribution over time after first page is received, same as in https://github.com/trinodb/trino/blob/master/core/trino-main/src/main/java/io/trino/execution/buffer/OutputBufferMemoryManager.java#L66). Then such distribution can be exposed via io.trino.exchange.DirectExchangeDataSource#getMetrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant