Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally inject environment variables into EnvironmentAware #17013

Merged
merged 5 commits into from
Sep 27, 2022

Conversation

chrisjrn
Copy link
Contributor

This extends #17009 by making a generic mechanism to request environment variables when EnvironmentAware subsystems are constructed.

Generally speaking, this encapsulates both the requesting and consumption of environment variables into the subsystem itself, and this will save subsystem consumers from knowing which environment variables to request in order to call an option post-processing method. All option post-processing methods should safely be able to be converted to properties too. All very exciting.

Fixes #14612

@chrisjrn chrisjrn marked this pull request as ready for review September 26, 2022 21:26
@chrisjrn chrisjrn added the category:internal CI, fixes for not-yet-released features, etc. label Sep 26, 2022
Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elegant! Thanks!

src/python/pants/option/subsystem.py Show resolved Hide resolved
Christopher Neugebauer added 3 commits September 26, 2022 14:40
… subsystems at construct time.

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
…envvars

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@chrisjrn chrisjrn force-pushed the 14612-inject-environment-variables branch from 6324001 to 2c056b6 Compare September 26, 2022 21:40
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
def rules():
return [*collect_rules()]
@property
def environment_dict(self) -> dict[str, str]:
Copy link
Member

@stuhood stuhood Sep 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def environment_dict(self) -> dict[str, str]:
def environment_vars_dict(self) -> dict[str, str]:

Maybe?

This somewhat conflates environments with environment variables... while environment variables can be overridden by environments via options, they don't have to be.

Unless we expect all access to environment variables to move to being via Subsystems (maybe...?), this results in two APIs for consuming environment variables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, not really -- the specific use case for this is for the cases where the subsystem's options are private, and the public API post-processes based on environment variables. This happens all the time, and the EnvironmentVarsRequest API is brittle -- you have to request the specific env vars ahead of time, and then pass them into the method that consumes them. This makes those subsystems very difficult to consume correctly. Pre-fetching the envvars makes that entirely non-brittle.

That said, the intention here is that EnvironmentAware.env_vars is meant to be used solely for those post-processing methods. Would it help if env_vars were called _env_vars instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stuhood to be clear, environment_dict was a property that existed on PythonNativeCodeSubsystem before today.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be clear, environment_dict was a property that existed on PythonNativeCodeSubsystem before today.

Ack. Renaming it would I think make things more clear, even though it's existing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I don't feel strongly about it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Eric-Arellano I'll do that now. Too many things named "environment" 🤣

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@chrisjrn chrisjrn merged commit 81d07b3 into pantsbuild:main Sep 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[python-native-code] breaks Pantsd cache invalidation
3 participants