Skip to content

Conversation

@deitch
Copy link
Contributor

@deitch deitch commented Jul 7, 2023

We were generating the graph with each call to make, and we were calling make recursively with each package/<something> target. So make all would generate the graph every single time, costing 1.5-2s. That is fine for 3 or 4 packages, but for nearly 1000, it times out.

We have to use recursive make, because we need to pass the original target to the dependency. This below won't work:

package/%: packages/$(ARCH)/$(shell $(MELANGE) package-version $(yamlfile)))
packages/$(ARCH)/%:
    # do stuff

the "# do stuff" section needs to know the original package name, so it can mkdir -p ${sourcedir}, and call build melange build ${yamlfile}, both of which depend on the original package name, not the full version.

Make provides no way to do this. I also looked at target-specific variables, but they are evaluated in the context of the target to be run, e.g. in this case packages/aarch64/openssl-1.2.3-r1.apk, and not the original call package/openssl, leaving us no way to get the original package name of openssl, and hence the source dir and yaml file.

The only way to do that is to call recursive make and pass the information, which determines all vars again.

Therefore, we change it so that it only generates the graph when explicitly needed, i.e. for .buildpackages, list and list-yaml. This makes it very quick.

Signed-off-by: Avi Deitcher <avi@deitcher.net>
@deitch deitch requested a review from a team as a code owner July 7, 2023 13:16
@deitch deitch enabled auto-merge July 7, 2023 13:20
@deitch deitch added this pull request to the merge queue Jul 7, 2023
Merged via the queue into wolfi-dev:main with commit ae224a9 Jul 7, 2023
@deitch deitch deleted the avoid-regenerating-graph branch July 7, 2023 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants