Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't eagerly flatten a NestedSet in RepoMappingManifestAction #18349

Closed
wants to merge 1 commit into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented May 8, 2023

Delays the flattening from action creation to action execution in preparation for making this action accessible from Starlark.

Work towards #17941

@fmeum fmeum force-pushed the 17941-no-flatten branch 2 times, most recently from 96fda7a to 783a194 Compare May 8, 2023 20:10
@fmeum fmeum requested a review from Wyverald May 8, 2023 20:10
@fmeum fmeum marked this pull request as ready for review May 8, 2023 20:10
@fmeum fmeum requested a review from a team as a code owner May 8, 2023 20:10
@fmeum fmeum requested review from aiuto and removed request for a team May 8, 2023 20:10
@github-actions github-actions bot added awaiting-review PR is awaiting review from an assigned reviewer team-Configurability platforms, toolchains, cquery, select(), config transitions labels May 8, 2023
@fmeum fmeum removed the request for review from aiuto May 8, 2023 20:42
@fmeum fmeum added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. and removed team-Configurability platforms, toolchains, cquery, select(), config transitions labels May 8, 2023
@fmeum
Copy link
Collaborator Author

fmeum commented May 8, 2023

@rickylev This should unblock your planned fix for #17941.

Delays the flattening from action creation to action execution in
preparation for making this action accessible from Starlark.
Copy link
Member

@Wyverald Wyverald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beautiful!

@Wyverald Wyverald added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels May 10, 2023
@fmeum
Copy link
Collaborator Author

fmeum commented May 10, 2023

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label May 10, 2023
@keertk
Copy link
Member

keertk commented May 10, 2023

@bazel-io fork 6.3.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label May 10, 2023
@Pavank1992 Pavank1992 removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label May 11, 2023
@fmeum fmeum deleted the 17941-no-flatten branch May 11, 2023 14:04
ruleContext.getTransitivePackagesForRunfileRepoMappingManifest()),
runfiles),
ruleContext.getTransitivePackagesForRunfileRepoMappingManifest(),
runfiles.getAllArtifacts(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: getAllArtifacts() does a partial flattening -- symlinks and root symlinks are still flattened. But those those are usually small, so still a big win.

Copy link
Collaborator Author

@fmeum fmeum May 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be worth improving that further? Can Runfiles be serialized? Then that could make for a better action constructor argument.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so? SourceManifestAction, for example, accepts a Runfiles object directly.

I wouldn't expect a large gain in practice. The symlinks and root_symlinks are usually small. Runfiles.getAllArtifacts() is called in a few different places already.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your comment, it led me to discover that the repo mapping manifest didn't handle symlinks correctly: #18381

fmeum added a commit to fmeum/bazel that referenced this pull request May 16, 2023
Delays the flattening from action creation to action execution in preparation for making this action accessible from Starlark.

Work towards bazelbuild#17941

Closes bazelbuild#18349.

PiperOrigin-RevId: 531165675
Change-Id: Ia2c175834d409c30303dd3ecba0dd276ea2cd905
fmeum added a commit to fmeum/bazel that referenced this pull request May 16, 2023
Delays the flattening from action creation to action execution in preparation for making this action accessible from Starlark.

Work towards bazelbuild#17941

Closes bazelbuild#18349.

PiperOrigin-RevId: 531165675
Change-Id: Ia2c175834d409c30303dd3ecba0dd276ea2cd905
fmeum added a commit to fmeum/bazel that referenced this pull request May 19, 2023
Delays the flattening from action creation to action execution in preparation for making this action accessible from Starlark.

Work towards bazelbuild#17941

Closes bazelbuild#18349.

PiperOrigin-RevId: 531165675
Change-Id: Ia2c175834d409c30303dd3ecba0dd276ea2cd905
iancha1992 pushed a commit that referenced this pull request May 19, 2023
…18419)

Delays the flattening from action creation to action execution in preparation for making this action accessible from Starlark.

Work towards #17941

Closes #18349.

PiperOrigin-RevId: 531165675
Change-Id: Ia2c175834d409c30303dd3ecba0dd276ea2cd905
fweikert pushed a commit to fweikert/bazel that referenced this pull request May 25, 2023
Delays the flattening from action creation to action execution in preparation for making this action accessible from Starlark.

Work towards bazelbuild#17941

Closes bazelbuild#18349.

PiperOrigin-RevId: 531165675
Change-Id: Ia2c175834d409c30303dd3ecba0dd276ea2cd905
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants