-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not get the environment variables twice for same property(Environment variables not identified through properties file) #14235
Conversation
reverse merge
reverse merge
reverse merge
reverse merge
reverse merge
reverse merge
reverse merge
Enforce removeSegment flow with _enableDeletedKeysCompactionConsisten…
reverse merge
Reverse merge from master
reverse merge
reverse merge
If the config is applied twice then the function |
Yeah, it is going through already lookedup env variables again which resulted in the env variables not applied at all as second time, it will be null for that variable. |
…etting picked up from env variables
merge master
@@ -170,8 +173,15 @@ public static List<Configuration> applyDynamicEnvConfig(List<Configuration> conf | |||
Map<String, String> environmentVariables) { | |||
return configurations.stream().peek(configuration -> { | |||
for (String dynamicEnvConfigVarName : configuration.getStringArray(ENV_DYNAMIC_CONFIG_KEY)) { | |||
configuration.setProperty(dynamicEnvConfigVarName, | |||
environmentVariables.get(configuration.getString(dynamicEnvConfigVarName))); | |||
//if the environment variable doesn't exist or the property is already checked do not add the property |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems we are not checking whether the property already exists. Is this expected? If so, let's modify the comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed the description
pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please take a look at the failed tests
pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java
Outdated
Show resolved
Hide resolved
pinot-spi/src/main/java/org/apache/pinot/spi/env/PinotConfiguration.java
Outdated
Show resolved
Hide resolved
Seems the failure is caused by #14237 and not related to this PR. Will take a look and get back |
This PR is causing |
Issue:
The environment variable is not setting right with the "dynamic.env.config" property through controller.conf file.
Reference PR: #12307
Root cause:
Ideally the dynamic variables should be added once, but BaseControllerStarter & PinotServiceManager(startRole) are both trying to apply dynamic variables and this caused an issue when setting the environment variables from the dynamic.env.config properties through config file.
Fix:
Test Result:
Changed the port to "CONTROLLER_PORT" for the controller which is "9001" from the existing config 9000:
Before Change: throws error when launching controller
After change, running controller locally on "9001" :