Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible numerical underflow during quantization of log sampled UniformFloat #186

Closed
NeoChaos12 opened this issue Jul 13, 2021 · 0 comments · Fixed by #202
Closed

Possible numerical underflow during quantization of log sampled UniformFloat #186

NeoChaos12 opened this issue Jul 13, 2021 · 0 comments · Fixed by #202

Comments

@NeoChaos12
Copy link

NeoChaos12 commented Jul 13, 2021

I noticed that in the case of UniformFloatHyperparameter types that have log-sampling enabled, there is a possible numerical underflow issue with using a quantization factor. To illustrate this, consider the following minimum working example:

import ConfigSpace as CS
p = CS.UniformFloatHyperparameter("param", 0.0001, 1.0, log=True, q=0.0001)

:1: RuntimeWarning: invalid value encountered in log

space = CS.ConfigurationSpace("sp1")
space.add_hyperparameter(p)

Traceback (most recent call last):
File "", line 1, in
File "ConfigSpace/configuration_space.pyx", line 186, in ConfigSpace.configuration_space.ConfigurationSpace.add_hyperparameter
File "ConfigSpace/configuration_space.pyx", line 1062, in ConfigSpace.configuration_space.ConfigurationSpace._check_default_configuration
File "ConfigSpace/configuration_space.pyx", line 1427, in ConfigSpace.configuration_space.Configuration.init
File "ConfigSpace/configuration_space.pyx", line 1457, in ConfigSpace.configuration_space.Configuration.is_valid_configuration
File "ConfigSpace/c_util.pyx", line 38, in ConfigSpace.c_util.check_configuration
File "ConfigSpace/c_util.pyx", line 103, in ConfigSpace.c_util.check_configuration
ValueError: Active hyperparameter 'param' not specified!

More cases that fail:

p = CS.UniformFloatHyperparameter("param", 0.0001, 1.0, log=True, q=0.00001)
:1: RuntimeWarning: invalid value encountered in log

p = CS.UniformFloatHyperparameter("param", 0.00001, 1.0, log=True, q=0.00001)
:1: RuntimeWarning: invalid value encountered in log

This persists even when the upper bound is smaller, such as here:

p = CS.UniformFloatHyperparameter("param", 0.0001, 0.01, log=True, q=0.0001)
:1: RuntimeWarning: invalid value encountered in log

Reducing the lower bound, however, fixes this. That is, this works fine:

p = CS.UniformFloatHyperparameter("param", 0.001, 1.0, log=True, q=0.0001) # but lower=0.0001 does not!

and removing the quantization factor works fine as well, even for a very small lower bound:

p = CS.UniformFloatHyperparameter("param", 0.00001, 1.0, log=True)

I haven't tested other numerical hyperparameters for similar cases, but it is definitely possible that they will run into similar issues.

My environment information:

OS: Ubuntu 18.04
Python: 3.7
ConfigSpace version: 0.4.19
NumPy version: 1.21.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant