PiBO Pull Request 2 - Beta Hyperparameters (#222)

* Got structure and notes in place to implement PiBO - commenting and pdf
for NormalFloat

* Added class BetaFloatHyperparameter, and allowed for its support in
read_and_write

* Fixed bug where a NormalFLoatHyperparameter in log space could be
sampled outside its permitted range.

* Checked normalfloatparameter with inputs - now has a complete and working
pdf regardlessx of input

* Implemented BetaIntegerHyperparameter - onlb brief testing conducted

* Finished categorical hyperparameter and pdf for all hyperparameters.
Testing left.

* Added to_uniform for categorical (to uniform weights

* Added to_uniform in configspace, fixed some earlier mistakes in
hyperparameters.pyx where i changed Typing.

* Removed debugging method "get_probs"

* Fixed bug in beta neighborhood generation - still have not normalized
beta and normal distributions

* Changed neighbor generation to be consistent for Normal and Beta
parameters - now, they are generated the same way as the uniform (but
scaled, so that it makes sense even though they are not normalized)

* Changed back to standard local search procedure for the normal
hyperparamretersChanged back to standard local search procedure for the
normal hyperparamreters

* Forgot to change back neighbor generation for normal float

* Removed done TODO's and commentary

* Added return types for .pdf and removed a TODO

* Written user guide for placing priors on parameters and the pdf method

* Reformatted some faulty things with the user guide

* One more user guide fix

* Documented the pdf and _pdf methods. Moved them from the Hyperparameter
class down to the subclasses Float, Constant,  Integer, Categorical and Ordinal
Hyperparameters (as these almost certainly can have a well-defined pdf,
or that it would make sense to consider a pdf for these HP types).

* Fixed a few bugs in check_default, and added method get_max_density for
all the classes that implement a pdf

* Found a bug in normalfloat and removed a TODO

* Completing merge with master - not implemented get_size for beta
parameters

* Changed name of to_uniform and minor fixes

* Corrected for wrong placement in merge

* Changed minor bugs in the beta parameters and added the get_size methods
for both (identical to Uniform parameters

* Added unit tests for betafloathyperparameter - in line with the ones for
the other parameter types - follows as closely as possible

* Added additional bugfixes to beta parameters

* Added tests to BetaIntegerParameters

* Added skeleton of all remaining tests

* Changed some scaling issues and added a shape check in pdf method for
all parameters - _pdf does consequently not have a shape check

* Added unittests for constant and uniformfloat pdf and _pdf methods

* Added tests for normalfloat

* Added pdf and _pdf for betafloatHP

* Changed magnitude of pdf method for Uniform parameters in log space to
reflect the whole unlogged range (instead of the logged range)

* Removed some comments, changed a normalization and ensured Integer pdfs
work as intended

* Added tests for normalint pdf and _pdf

* Added betainteger pdf and _pdf tests

* Bugfixes in ordinal parameter pdf

* Completed last unit tests for pdf - ordinal parameter

* Further bugfixes regarding get_max_density

* Added tests for get_max_density

* Added test_remove_parameter_prior and the last bits and pieces of
testing

* Added attempt of bugfix to log Normal params - not done

* Some cleaning of unneccesary statements

* Added all suggested changes and fixed bugs in
NormalFloat/NormalIntegerHP parameters. Now, lognormal parameters give
proper default values, i.e. a lognormal with mu=0 has default value e^0= 1
unless otherwise specified. Tests are changed to accomodate.

This issue also caused normalized_default_value for lognormal parameters
to frequently be NaN, which is now fixed.

* Update ConfigSpace/hyperparameters.pyx

Co-authored-by: Matthias Feurer <lists@matthiasfeurer.de>

* Added numpydoc docstrings for pdf and fixed a change in user guide

* Changed remove_parameter_priors to check getattr(to_uniform)

* Fixed doctests for parameters, renamed remove_parameter_priors in
configurationspace (and tests renamed accordingly)

* Fixed doctest in User guide example

* One last fix in Categorical pdf

* Fixed documentation, indentation in doctests and copy in remove_priors

* Added method to be able to add conditions to the uniform config space

* Implemented solution to copy both conditions and forbiddens over to the
new configspace

* Added better test for remove_parameter_priors with new methods
implemented

* Added test for substitute_hps_in_conditions

* Added complete tests to the substitute methods and made them
staticmethods, since it seemed to be the most sensible design choice

* Reordered to make the static methods last in the configuration_space
class

* Added normalization (and lack thereof) for normalintegerparameters
without bounds

* Added links to pibo where appropriate

* Changed the to_integer methods to be consistent (and correct) across the
bboard

* Added all bugfixes - removed everything else

* Small fixes

* Added the BetaHyperparameters

* Added everything related to the beta parameters

* Changed a comment

* Fixed numpy array hashing error, and the changes in randcomness for
categoricalHPs

* All tests pass except for PCS

* All tests now work, rewrote PCS to account for the fact that
Categoricals cannot be saved

* Passed linting

* Cleanup

* Fix?

* Changed Categorical HParam to be based around weights, not probabilities

* Weights now a tuple

* Final fixes after merge with bugfixes - had to revert some previous
cleanup

* Changed BetaFloat to subclass UniformFloat - rewrote the tests to
accound for new RNG

* Finished converting BetaIntegerHP to subclass UniformFloatHP

* Update ConfigSpace/hyperparameters.pyx

Co-authored-by: Matthias Feurer <lists@matthiasfeurer.de>

* Minor fixes and edits - formatting etc.

* Removed __init__.py in tests

* Added metainfo passing to all the to_uniform methods, and fixed one test
error in BetaFloatHP

* Added all tests for betafloatHPs, including very thorough checks for
default values. Notably, quantization plus logging does not yield
correct default values (i.e., not the mode of the distribution) all the
time, as there is some undesired adjustment of the bounds ._lower and
._upper in UniformFloatHyperparameter which skews it.

* Fixed all the necessary tests for BetaHPs

* Formatted, pre-commit now passes.

* Update test/test_hyperparameters.py

Co-authored-by: Matthias Feurer <lists@matthiasfeurer.de>

Co-authored-by: Matthias Feurer <lists@matthiasfeurer.de>
Co-authored-by: Matthias Feurer <feurerm@informatik.uni-freiburg.de>

ConfigSpace/configuration_space.pyx

@@ -41,6 +41,9 @@ from ConfigSpace.hyperparameters import (
  Hyperparameter,
  Constant,
  FloatHyperparameter,
+ UniformFloatHyperparameter,
+ UniformIntegerHyperparameter,
+ OrdinalHyperparameter
 )
 from ConfigSpace.conditions import (
  ConditionComponent,