Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Optimize the CLI by completely avoiding Pydantic imports from everyvoice -h. #429

Merged
merged 2 commits into from
May 17, 2024

Conversation

joanise
Copy link
Member

@joanise joanise commented May 14, 2024

PR Goal?

Get the CLI even faster, by avoiding all pydantic imports. Prompted once again by typing everyvoice new<tab> and being annoyed at the delay getting results.

According to CI this PR brings everyvoice -h down from .28s to .21s, but on GPSCC I'm seeing .65 down to .50 so I care more, and where I also really wish I could optimize further...

Fixes?

My impatience. Well, no, it doesn't fix my impatience, it only satisfies a little bit of it. 😄

Feedback sought?

I'm not sure about the name fs2/types_expensive.py. First I called it fs2/shared_types.py to parallel what we do in ev/config, but it's not shared so that makes no sense. fs2/type_definitions.py was nicely self-documenting, and I feel I'm partially undoing that, so suggestions for a better name are very welcome.

Priority?

Normal

Tests added?

Should already be covered by CI.

How to test?

time everyvoice -h before and after the PR.

Confidence?

High for the code changes, low for the file name.

Version change?

no

@joanise
Copy link
Member Author

joanise commented May 14, 2024

Copy link
Contributor

github-actions bot commented May 14, 2024

CLI load time: 0:00.23
Pull Request HEAD: 8898f3bdf42311909362c68e01d69a2a0b5b8649
Imports that take more than 0.1 s:
import time: self [us] | cumulative | imported package

Copy link

codecov bot commented May 14, 2024

Codecov Report

Attention: Patch coverage is 75.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 75.98%. Comparing base (57df25a) to head (75a65db).

Current head 75a65db differs from pull request most recent head 8898f3b

Please upload reports for the commit 8898f3b to get more accurate results.

Files Patch % Lines
everyvoice/base_cli/checkpoint.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #429      +/-   ##
==========================================
- Coverage   76.02%   75.98%   -0.04%     
==========================================
  Files          42       42              
  Lines        2757     2757              
  Branches      455      455              
==========================================
- Hits         2096     2095       -1     
- Misses        576      577       +1     
  Partials       85       85              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@roedoejet
Copy link
Member

What about following the convention in utils with type_definitions_heavy.py or something like that? Ideally we should use the same naming convention as the division in utils in any case.

@joanise
Copy link
Member Author

joanise commented May 15, 2024

What about following the convention in utils with type_definitions_heavy.py or something like that?

That sounds better, I'll take a look.

Ideally we should use the same naming convention as the division in utils in any case.

✔️💯

@joanise
Copy link
Member Author

joanise commented May 15, 2024

Alright, investigation done, heavy.py is for some types defs but mostly utility functions requiring expensive imports. The parallel is not perfect so I won't try to match exactly, but I'll use the word "heavy" for reasonable consistency, which means I'm going to take your first suggestion: type_definitions_heavy.py.

@joanise joanise force-pushed the dev.ej/cli-perf-no-pydantic branch from 527a3a7 to 75a65db Compare May 15, 2024 14:13
@joanise
Copy link
Member Author

joanise commented May 15, 2024

@roedoejet OK, file renamed, and rebased onto main, so this is ready to re-review and possibly merge.

Copy link
Collaborator

@SamuelLarkin SamuelLarkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@joanise joanise force-pushed the dev.ej/cli-perf-no-pydantic branch from 75a65db to 8898f3b Compare May 17, 2024 15:30
@joanise joanise merged commit 8898f3b into main May 17, 2024
2 checks passed
@joanise joanise deleted the dev.ej/cli-perf-no-pydantic branch May 17, 2024 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants