Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Python glob.glob when scanning for main.nix files #1101

Closed
dsalaza4 opened this issue Jun 1, 2023 · 1 comment
Closed

Use Python glob.glob when scanning for main.nix files #1101

dsalaza4 opened this issue Jun 1, 2023 · 1 comment

Comments

@dsalaza4
Copy link
Contributor

dsalaza4 commented Jun 1, 2023

Current scanning function is implemented in Nix. It is very slow:

makes/src/evaluator/modules/default.nix

Line 94 in 1311f28

attrsFromPath = path: position:

Let's use Python glob.glob and pass all found attrs to Nix.
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 2, 2023
@dsalaza4
feat(back): fluidattacks#1101 globbed attrs
0a6f245 
- Collect attr paths using python globs
for initialization performance
- Pass them as attrPaths to evaluator
- Redefine evaluator logic for building outputs from attrs
- Initialization for universe takes less than 3 seconds now

Signed-off-by: Daniel Salazar <podany270895@gmail.com>
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 2, 2023
@dsalaza4
feat(back): fluidattacks#1101 globbed attrs
20a3958 
- Collect attr paths using python globs
for initialization performance
- Pass them as attrPaths to evaluator
- Redefine evaluator logic for building outputs from attrs
- Initialization for universe takes less than 3 seconds now

Signed-off-by: Daniel Salazar <podany270895@gmail.com>
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 5, 2023
@dsalaza4
feat(back): fluidattacks#1101 globbed attrs
b5d7d34 
- Collect attr paths using python globs
for initialization performance
- Pass them as attrPaths to evaluator
- Redefine evaluator logic for building outputs from attrs
- Initialization for universe takes 1 second now

Signed-off-by: Daniel Salazar <podany270895@gmail.com>
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 5, 2023
@dsalaza4
feat(back): fluidattacks#1101 globbed attrs
f9670ca 
- Collect attr paths using python globs
for initialization performance
- Pass them as attrPaths to evaluator
- Redefine evaluator logic for building outputs from attrs
- Initialization for universe takes 1 second now

Signed-off-by: Daniel Salazar <podany270895@gmail.com>
jpverde added a commit that referenced this issue Jun 5, 2023
@jpverde
Merge pull request #1102 from dsalaza4/main
fc31801 
feat(back): #1101 globbed attrs
@dsalaza4
Copy link
Contributor Author

dsalaza4 commented Jun 5, 2023

I will also release a new version of makes for pinning this CLI change.

dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 5, 2023
@dsalaza4
feat(cross): fluidattacks#1101 new makes version
30c85b1 
- Deploy new makes version that includes new CLI optimization
- Update doc
- Update pipelines
- Update containers

Signed-off-by: Daniel Salazar <podany270895@gmail.com>
dsalaza4 added a commit that referenced this issue Jun 5, 2023
@dsalaza4
Merge pull request #1103 from dsalaza4/main
d15f787 
feat(cross): #1101 new makes version
@dsalaza4 dsalaza4 closed this as completed Jun 5, 2023
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 22, 2023
@dsalaza4
perf(back): fluidattacks#1101 faster init
881f5ca 
- Stop checking if path exists as it was collected a few
milliseconds ago
- Only declare some trivial variables once
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 22, 2023
@dsalaza4
perf(back): fluidattacks#1101 faster init
80aafe7 
- Stop checking if path exists as it was collected a few
milliseconds ago
- Only declare some trivial variables once
dsalaza4 added a commit to dsalaza4/makes that referenced this issue Jun 22, 2023
@dsalaza4
perf(back): fluidattacks#1101 faster init
1760eca 
- Stop checking if path exists as it was collected a few
milliseconds ago
- Only declare some trivial variables once

Signed-off-by: Daniel Salazar <podany270895@gmail.com>
jpverde added a commit that referenced this issue Jun 22, 2023
@jpverde
Merge pull request #1113 from dsalaza4/main
3004ae7 
perf(back): #1101 faster init
jpverde added a commit to jpverde/makes that referenced this issue Jun 27, 2023
@jpverde
Merge pull request fluidattacks#1113 from dsalaza4/main
01b40ff 
perf(back): fluidattacks#1101 faster init
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dsalaza4