Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Come up with an automated way to measure the hot path latency of the scie-pants execution flow. #24

Open
jsirois opened this issue Dec 18, 2022 · 0 comments
Labels
performance Denotes work towards performance optimization and maintenance.

Comments

@jsirois
Copy link
Contributor

jsirois commented Dec 18, 2022

There are currently 3 executions here, 2 of which take place opaquely in the scie-jump launcher binary.

  1. scie-jump launches default ("") scie-pants scie command (scie-pants.bin)
  2. scie-pants.bin discovers pants.toml, then pants_version from that, then derives the Python version needed; finally it re-executes the scie-pants scie with SCIE_BOOT=pants.
  3. scie-jump launches the "pants" command, which is the Pants "pants" console script installed in the Pants venv.
  4. The pants console script executes.

Each of these 1st three steps takes ~300us totaling generally at just under 1ms. Unfortunately, the 4th step runtime drowns out the 1st 3 in the noise since even pants -V currently takes 500ms.

In order to get a measurement of the scie-pants overhead, we need to find some way to measure just the sum total of the 1st three steps. After we can do that, performance checks can be added to CI to, at the very least, have a handy eyeball on drift there and, at best, fail CI for deviations of greater than some margin above the established baseline.

@jsirois jsirois added the performance Denotes work towards performance optimization and maintenance. label Dec 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Denotes work towards performance optimization and maintenance.
Projects
None yet
Development

No branches or pull requests

1 participant