ARGUMENTS alternative for data came from files

[dustdfg]

I am almost sure it will be converted to issue but will ask firstly. There is ARGUMENTS and ARGLIST variables which allow to access raw user data from cli. But I need to get raw user data from files too. Is there a way?

There is the same workaround as for #4645 where user need to customly load files with exec... As the #4645 was solved it would be perfect to implement ARGMUNETS alternative for files and it will be possible to get rid of that workaround in one release

[mwichmann]

Not sure I'm completely getting the request here. Arguments are collected and made available as you note. If you want special handling, Variables give you a way to add extra features - saved settings files, predefined "types" of variables, help messages, etc. You want to add those features outside of the Variables concept?

[bdbaddog]

@dustdfg
Hmm. shouldn't be hard to add a list of set_by_file(s),
Curious what you're using this for?

[dustdfg]

Just added a Variables attribute (defaulted) that shows which were set in the env from their Variable defaults, so you will be able to determine from that the inverse - if it was set by the input sources

Where can I find and learn it?

It doesn't distinguish further between "set on command line" and "set in a saved-variables file"

Basically CLI needs to override values from files (at least in our case). We can need to have order.

Curious what you're using this for?

In Godot we have several use cases:

1. We have aliases. For example dev_mode option which automatically sets verbose="yes", warnings, werror, tests, strict_checks. But user can want to set dev_mode="yes" but without verbose="yes" which is automatically set by dev_mode so before setting verbose="yes" we need to know if user wanted or not verbose. And if user explicitly said I don't want it to be verbose we don't set verbose="yes". Yeah, here we need to get bool so we use SCons builtins for parsing bool but it doesn't matter much. ARGUMENTS allows to know if user wanted or not but if user sets it from file because always want to build with dev_mode="yes" verbose="no" it is not possible
2. We have different platform files. Each of them can specify default options for selected platform. For example for web platform it is disabling vulkan="no" because there is no support for vulkan and there is not much sense for user to desire it to be true. But for example the same web platform by default sets optimize="size" but if user explicitly wants it to be optimize="speed" there will be problems. Here just basically we need in pseudocode. But currently instead of this magic env.defined_by_user(opt) we check if option is inside ARGUMENTS

platform_defaults = platform.get_defaults()

for opt in platform_defaults:
    if env.defined_by_user(opt):
        continue # don't change value if it was set by user
    else:
        env[opt] = platform_defaults[opt] # change value if user not against 

3. I see one option which is deprecated in our code base. And I want to print a warning it is deprecated if user sets this option and not another... But I don't want to print it if it was set by default value from scons. I need to distinguish if it was a user. (Just formally I have a workaround for this one case but it looks a bit weird... And it is much more convenient to use that workaround of custom loading which is required in other cases...)

[dustdfg]

Hmm. shouldn't be hard to add a list of set_by_file(s),

It would be redundant as there are ARGUMENTS but at the same time for convenience wouldn't it be also good to have set_by_cli? But it seems to me that when Variables gets ARGUMENTS as parameter it can't distinguish it from any other dict so set_by_cki even couldn't be implemented?

[mwichmann]

Just added a Variables attribute (defaulted) that shows which were set in the env from their Variable defaults, so you will be able to determine from that the inverse - if it was set by the input sources

Where can I find and learn it?

Pretty much just in the code, in git.

scons/SCons/Variables/__init__.py

Line 239 in 01f77b1

def Update(self, env, args: dict | None = None) -> None:

There are a few manpage words ready to be pushed in a PR, once another PR clears the review (#4658).

[mwichmann]

@dustdfg Hmm. shouldn't be hard to add a list of set_by_file(s), Curious what you're using this for?

Indeed, this would be quite easy to add - basically in the same place in the files section where it now removes a name from defaulted if it was added from a file.

[bdbaddog]

Or you can use plain python to read files and do anything you want with that..
No ARGUMENTS is meant to unmodified what's on the command line only. No processing, nothing.

[dustdfg]

While I expect from user only direct values (without expecting there will be any expanding with refering to other values and etc) it is fine

[dustdfg]

It doesn't distinguish further between "set on command line" and "set in a saved-variables file", and of course we have no way to easily trap something that was just directly updated in the env by other code.

I assume here Variables.defaulted is just a set of names. So it can done when user does env["name"] = any_value you just pop that name from Variables.defaulted but it would mess for example with our approach of dev_mode and similar aliases