-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Cgroups] Extended cgroup support #21322
Closed
AlessandroPatti
wants to merge
17
commits into
bazelbuild:master
from
AlessandroPatti:apatti/cgroups
Closed
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
fd2d4eb
Remove suspicious cgroup deletetion
AlessandroPatti 642d3af
[Cgroups] Extended cgroup support
AlessandroPatti 38f8544
Log and continue gracefully when cgroups are not available
AlessandroPatti 4d74115
Cgroup dirs should not be writable
AlessandroPatti 38b6b29
Specialize v1 and v2 controllers
AlessandroPatti cebcc0d
Ignore IOException when creating root cgroup
AlessandroPatti e394d97
Fix tests
AlessandroPatti 25df3b8
Fix metrics
AlessandroPatti ccb6fea
Add tests
AlessandroPatti 01d1a86
Remove CgroupsInfo
AlessandroPatti 98bed77
Feedback: rename and document
AlessandroPatti b95f319
Feedback: override isLegacy
AlessandroPatti 427ce3d
Remove optional
AlessandroPatti c1f7fad
Keep old resource option around
AlessandroPatti d3a6c47
Restore CgroupInfo
AlessandroPatti 44cb649
Remove new execution requirement
AlessandroPatti c5f0158
Feedback
AlessandroPatti File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 changes: 6 additions & 0 deletions
6
src/main/java/com/google/devtools/build/lib/sandbox/Cgroup.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| package com.google.devtools.build.lib.sandbox; | ||
|
|
||
| public interface Cgroup { | ||
| int getMemoryUsageInKb(); | ||
| boolean exists(); | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
41 changes: 41 additions & 0 deletions
41
src/main/java/com/google/devtools/build/lib/sandbox/cgroups/BUILD
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| load("@rules_java//java:defs.bzl", "java_library") | ||
|
|
||
| package(default_applicable_licenses = ["//:license"]) | ||
|
|
||
| filegroup( | ||
| name = "srcs", | ||
| srcs = glob(["**"]) + [ | ||
| "//src/main/java/com/google/devtools/build/lib/sandbox/cgroups/controller:srcs", | ||
| ], | ||
| visibility = ["//src:__subpackages__"], | ||
| ) | ||
|
|
||
| java_library( | ||
| name = "cgroups", | ||
| srcs = [ | ||
| "Hierarchy.java", | ||
| "Mount.java", | ||
| "VirtualCGroup.java", | ||
| "VirtualCGroupFactory.java", | ||
| ], | ||
| deps = [ | ||
| "//src/main/java/com/google/devtools/build/lib/sandbox:cgroups_info", | ||
| "//src/main/java/com/google/devtools/build/lib/actions:exec_exception", | ||
| "//src/main/java/com/google/devtools/build/lib/events", | ||
| "//src/main/java/com/google/devtools/build/lib/sandbox/cgroups/controller", | ||
| "//src/main/java/com/google/devtools/build/lib/sandbox/cgroups/controller/v1", | ||
| "//src/main/java/com/google/devtools/build/lib/sandbox/cgroups/controller/v2", | ||
| "//src/main/protobuf:failure_details_java_proto", | ||
| "//third_party:auto_value", | ||
| "//third_party:flogger", | ||
| "//third_party:guava", | ||
| ], | ||
| visibility = [ | ||
| "//src/main/java/com/google/devtools/build/lib/metrics:__subpackages__", | ||
| "//src/main/java/com/google/devtools/build/lib/sandbox:__subpackages__", | ||
| "//src/main/java/com/google/devtools/build/lib/worker:__subpackages__", | ||
| "//src/test/java/com/google/devtools/build/lib/metrics:__subpackages__", | ||
| "//src/test/java/com/google/devtools/build/lib/sandbox/cgroups:__subpackages__", | ||
| "//src/test/java/com/google/devtools/build/lib/worker:__subpackages__", | ||
| ], | ||
| ) |
52 changes: 52 additions & 0 deletions
52
src/main/java/com/google/devtools/build/lib/sandbox/cgroups/Hierarchy.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| package com.google.devtools.build.lib.sandbox.cgroups; | ||
|
|
||
| import com.google.auto.value.AutoValue; | ||
| import com.google.common.collect.ImmutableList; | ||
| import com.google.common.io.Files; | ||
|
|
||
| import java.io.File; | ||
| import java.io.IOException; | ||
| import java.nio.charset.StandardCharsets; | ||
| import java.nio.file.Path; | ||
| import java.nio.file.Paths; | ||
| import java.util.List; | ||
| import java.util.regex.Matcher; | ||
| import java.util.regex.Pattern; | ||
|
|
||
| @AutoValue | ||
| public abstract class Hierarchy { | ||
| public abstract Integer id(); | ||
| public abstract List<String> controllers(); | ||
| public abstract Path path(); | ||
| public boolean isV2() { | ||
| return controllers().size() == 1 && controllers().contains("") && id() == 0; | ||
| } | ||
|
|
||
| /** | ||
| * A regexp that matches entries in {@code /proc/self/cgroup}. | ||
| * | ||
| * The format is documented in https://man7.org/linux/man-pages/man7/cgroups.7.html | ||
| */ | ||
| private static final Pattern PROC_CGROUPS_PATTERN = | ||
| Pattern.compile("^(?<id>\\d+):(?<controllers>[^:]*):(?<file>.+)"); | ||
|
|
||
| static Hierarchy create(Integer id, List<String> controllers, Path path) { | ||
| return new AutoValue_Hierarchy(id, controllers, path); | ||
| } | ||
|
|
||
| static List<Hierarchy> parse(File procCgroup) throws IOException { | ||
| ImmutableList.Builder<Hierarchy> hierarchies = ImmutableList.builder(); | ||
| for (String line : Files.readLines(procCgroup, StandardCharsets.UTF_8)) { | ||
| Matcher m = PROC_CGROUPS_PATTERN.matcher(line); | ||
| if (!m.matches()) { | ||
| continue; | ||
| } | ||
|
|
||
| Integer id = Integer.parseInt(m.group("id")); | ||
| String path = m.group("file"); | ||
| List<String> cs = ImmutableList.copyOf(m.group("controllers").split(",")); | ||
| hierarchies.add(Hierarchy.create(id, cs, Paths.get(path))); | ||
| } | ||
| return hierarchies.build(); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of introducing a separate mechanism for sandboxed spawns, I wonder if it's better to unify this with the specified
resource_setof the sandboxed action. Then just use--experimental_sandbox_enforce_resources_regexpto control for which actions to enforce this on.Is there a reason to have this as a separate mechanism?
cc @wilwell.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a couple of reasons why I find this flag very useful:
--experimental_sandbox_enforce_resources_regexp=DO_NOT_MATCH --experimental_sandbox_limits=cpu=<n> <target>to test<target>with a different limit other then the one specified at the target level. Without this flag, I'd have to go and change the BUILD file, which makes this harder to iterate on.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By this I assume you mean actions outside of your control - e.g. in build rules not owned by your project / default starlark resources.
While I don't think fast testing / iteration is a good reason to introduce this (whatever value specified here should be relatively stable without users having to tinker frequently), I can see a legitimate case for this in the event that resources are underestimated. Since the
ResourceManagermakes reservations purely off of estimated values (and kind-of makes a claim that the action will have some minimum amount of resources1), this serves as a mechanism to upper bound resources.1 Not entirely true since Bazel assumes it can use all the resources available to the machine and doesn't actively monitor actually available resources.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add this as a separate flag that overrides
--experimental_sandbox_memory_limit_mb(and document that behavior) so we don't make this a breaking change. Internally I don't believe we use this flag, but I'm not sure about external users.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done