2164 update config from list of variables to class

[lucas-wilkins]

This PR is for a big housekeeping effort surrounding the configuration system.

It does the following things:

1. A new folder called system that replaces the custom_configs/local_config system which contains:
   a) config directory, which contains a class called config which controls configuration variables, config_meta describes the behaviour of this class, including loading and saving. More fully documented in the config class pramble.
   b) web a class containing all the urls and email addresses
   c) legal a class containing any legal stuff, currently just the copyright notice
   d) env a utility class for dealing with environment variables
   e) log a relocation of logger_config.py
   Config settings are now all defined though config.py and runtime changes to these variables are written to a local config.json file on closing (which is loaded on startup)
   There is now no writing python files, and variable usage is much more easily followed
2. Removal of config items that are no longer used, there were a lot of these.
3. Simplification of the way configuration data is utilised.

Things that probably need the most attention when reviewing:

1. Do the GPU settings still integrate properly with your hardware and with sasmodels? (@pkienzle ?)
2. Places issues might occur causing silent errors e.g. in fitting methods where config is used?
3. Have I changed the meaning of anything, or logic of anything that was handled by the local/custom config system?
4. Have I broken anything on other OSs than windows?

Note: this PR addresses multiple issues (linked) which will need to be manually closed.

[krzywon]

Looking through this, I have a few comments, but the direction you are taking looks good. My related PR (#2167) currently uses get_custom_config which you are deprecating, but that looks like the only clash we should have once you're finished.

[lucas-wilkins]

Status update:

I've finished making the config class, the next task is look the variables within it, throw out unused ones, replace bad uses of a config with more proper usages etc. Basically I've done steps 1, 2, 5 and 7 from my initial list.

Appropriate loading and saving on startup is yet to be implemented, basically because I'm not sure about where the user config file (which is now the only config file that should exist) should live.

There is pretty decent test coverage for these changes.

@krzywon

[pkienzle]

Mostly suggestions that you can ignore.

You should sort out the circular reference between setup logger and config.load, preferably by moving config.load after setup logger and out of the root sas namespace.

There are still some imports of sas.sasview that can be removed.

[lucas-wilkins]

@pkienzle @krzywon Will merge this tomorrow if there are no further changes.

[lucas-wilkins]

Will probably enable pretty printing when I have it open.

…

On Wed, Sep 28, 2022 at 10:16 PM Paul Kienzle ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/sas/system/config/config_meta.py <#2168 (comment)>: > + if "config_data" not in data: + raise MalformedFile("Malformed config file - no 'config_data' key") + + if not isinstance(data["config_data"], dict): + raise MalformedFile("Malformed config file - expected 'config_data' to be a dictionary") + + + # Check major version + file_version = data["sasview_version"] + file_major_version = file_version.split(".")[0] + sasview_major_version = sas.system.version.__version__.split(".")[0] + + if int(file_major_version) != int(sasview_major_version): + logger.warning(f"Attempting to used outdated config file (config is" + f" for {file_version}, this SasView version is {sas.system.version.__version__})") + self._write_disabled = True Oops… this doesn't preserve unused keys. Try: def load_from_file_object(self, file): data = json.load(file) self._stored = data ... def save_to_file_object(self): changed = {key: value for key, default in self._defaults.items() if default != (value := getattr(self, key))} data = {**self._stored, **changed} ... This is different from the behaviour of your algorithm when using a single version. With your code if you turn on OpenCL then it will save the "use opencl" flags, but if you turn it off again it will match the default and save nothing. This new algorithm will preserve the "don't use opencl" flags when you turn it off. This is only the case if you turn it off in a different session than you turn it on. If it is the same session then the "on" values won't be in stored, so turning it off again won't trigger stored test or the non-default test. A quick check shows that the key order in stored is preserved in the dict union constructor, though I'm not sure it matters since we are not round-tripping with preserved formatting and comments. And since we are not preserving formatting, we may want to sort keys as well as pretty-print on dump for debugging convenience. — Reply to this email directly, view it on GitHub <#2168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACKU4SRHKLQRDLNQRNWINTDWASYR3ANCNFSM57CLHS2Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Dr Lucas Wilkins +44 (0) 7505 915 726 Personal Website: http://www.lucaswilkins.com/ Alternate e-mail: ***@***.***

[pkienzle]

I moved the discussion on SAS_OPENCL to #2226

[lucas-wilkins]

@krzywon @krzywon Right, I think its ready!

[butlerpd]

@krzywon will take a final look.