Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ignore concurrency if not specified #7182

Merged
merged 5 commits into from
Mar 15, 2022

Conversation

gsquared94
Copy link
Contributor

@gsquared94 gsquared94 commented Mar 11, 2022

Fixes: #7181

Description

The build concurrency should be set to the minimum defined concurrency across all configs. Previously, imported configs that didn't define concurrency were defaulting to the value of 1, thereby lowering the overall concurrency to 1.
We remove setting this default value in the config, rather if the resolved build concurrency is defaulted to 1 at the end.

Testing instructions

  • In project examples/multi-config-microservices set the root skaffold.yaml config's build concurrency.
apiVersion: skaffold/v2beta28
kind: Config
build:
  local:
    concurrency: 2
requires:
- path: ./leeroy-app
- path: ./leeroy-web
  • Run skaffold build -v DEBUG. The output should have the following log lines showing that the build concurrency was updated to that value:
INFO[0000] build concurrency first set to 2 parsed from *local.Builder[3]  subtask=-1 task=DevLoop
INFO[0000] final build concurrency value is 2            subtask=-1 task=DevLoop

@gsquared94 gsquared94 requested a review from a team as a code owner March 11, 2022 07:24
@gsquared94 gsquared94 requested a review from briandealwis March 11, 2022 07:24
@gsquared94 gsquared94 requested a review from MarlonGamez March 11, 2022 07:29
@codecov
Copy link

codecov bot commented Mar 11, 2022

Codecov Report

Merging #7182 (b8639f2) into main (290280e) will decrease coverage by 2.00%.
The diff coverage is 56.96%.

@@            Coverage Diff             @@
##             main    #7182      +/-   ##
==========================================
- Coverage   70.48%   68.47%   -2.01%     
==========================================
  Files         515      560      +45     
  Lines       23150    26461    +3311     
==========================================
+ Hits        16317    18119    +1802     
- Misses       5776     7090    +1314     
- Partials     1057     1252     +195     
Impacted Files Coverage Δ
cmd/skaffold/app/cmd/deploy.go 52.00% <ø> (-1.85%) ⬇️
cmd/skaffold/app/cmd/dev.go 84.61% <0.00%> (ø)
cmd/skaffold/app/cmd/render.go 36.66% <0.00%> (-4.72%) ⬇️
cmd/skaffold/skaffold.go 0.00% <0.00%> (ø)
cmd/skaffold/app/cmd/inspect_tests.go 62.50% <14.28%> (-1.14%) ⬇️
cmd/skaffold/app/cmd/lsp.go 28.12% <28.12%> (ø)
cmd/skaffold/app/cmd/fix.go 68.85% <40.00%> (-7.62%) ⬇️
cmd/skaffold/app/cmd/lint.go 42.85% <42.85%> (ø)
cmd/skaffold/app/cmd/find_configs.go 48.88% <50.00%> (+0.24%) ⬆️
cmd/skaffold/app/skaffold.go 76.19% <70.00%> (-8.43%) ⬇️
... and 223 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

Copy link
Member

@briandealwis briandealwis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about extracting the minConcurrency logic into a separately testable function?

}
concurrency := *b.Concurrency()

// set mux concurrency to be the minimum of all builders' concurrency. (concurrency = 0 means unlimited)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add that 0 merits special treatment: if there are two configs A and B, and A has concurrency 0 and B has concurrency 1, then the "minimum" should be 1 since it's the most stringent.

Comment on lines 72 to 74
if b.Concurrency() == nil {
continue
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to be equivalent. Consider if I have two modules, A and B. Both have local build-envs. A does not specify a build concurrency. B specified concurrency 0. I believe that prior to this change, A would have had a default concurrency of 1 set which would have resulted in the minConcurrency==1 (serialized). Whereas now A would be skippedI think we will have minConcurrency 0.

Honestly, I think we should separate this out into a function whose results can be tested for different scenarios.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@briandealwis I verified via this test here that if B specified concurrency 0 and A does not set the concurrency, then it default to 1.

pkg/skaffold/build/builder_mux.go Outdated Show resolved Hide resolved
pkg/skaffold/build/builder_mux.go Outdated Show resolved Hide resolved
pkg/skaffold/build/builder_mux_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@MarlonGamez MarlonGamez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't have much to add that brian didn't already catch

pkg/skaffold/build/builder_mux_test.go Outdated Show resolved Hide resolved
@tejal29 tejal29 merged commit 59f488e into GoogleContainerTools:main Mar 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build concurrency from skaffold.yaml is not respected
4 participants