Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Instrumentation to Runner #530

Closed
Tracked by #213 ...
darunrs opened this issue Jan 29, 2024 · 1 comment · Fixed by #602
Closed
Tracked by #213 ...

Add Instrumentation to Runner #530

darunrs opened this issue Jan 29, 2024 · 1 comment · Fixed by #602
Assignees

Comments

@darunrs
Copy link
Collaborator

darunrs commented Jan 29, 2024

While Grafana/Promethus provides general metrics to assess performance and faults, the granularity of data provided is not sufficient to understand performance characteristics of Runner. We will want to know in greater detail what is contributing to the latency of a runFunction execution, so that we can both detect and prioritize improvements to the workflow, aiming for the fastest backfill performance we can achieve.

The instrumentation can be gradually improved as we tackle various latency contributors.

Ideally, we will also integrate the instrumentation library into Grafana (and/or Prometheus) to reuse existing metrics infrastructure.

@darunrs darunrs self-assigned this Jan 29, 2024
@pkudinov pkudinov changed the title Add Instrumentation to Runner Deploy Zipkin with Runner instrumentation to Dev Feb 6, 2024
@pkudinov pkudinov changed the title Deploy Zipkin with Runner instrumentation to Dev Add Instrumentation to Runner Feb 6, 2024
@darunrs
Copy link
Collaborator Author

darunrs commented Feb 15, 2024

We don't have any intentions to release this into prod. We may release it to dev briefly to understand latency impacts in the future, but local tracing provided the insights we needed for now, to improve latency.

@darunrs darunrs closed this as completed Feb 15, 2024
@darunrs darunrs reopened this Mar 14, 2024
@darunrs darunrs linked a pull request Mar 14, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant