add random forest prior to Bayesian Optimization

[libbyandhelen]

This PR add random forest as prior to Bayesian Optimization algorithm. RF is more suitable for discrete and categorical parameters, while gaussian process if more suitable for categorical parameters.

[libbyandhelen]

/assign @gaocegege @YujiOshima

[YujiOshima]

@libbyandhelen Thank you!
For validation of logic, can you add any citation for this algorithm?
If there is any difference to the citation, please specify the difference.

[libbyandhelen]

@YujiOshima
The reference paper is

Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2011. Sequential modelbased
optimization for general algorithm configuration.
https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf

shall I put this into the code?

[gaocegege]

I do not think we should put the paper citation in the code. Maybe we could maintain a doc about the algorithms that we support and list the citations there. WDYT

[YujiOshima]

Yes, I agree with @gaocegege .
It should be put in docs.
You can add BO to README and cite here.

[libbyandhelen]

@YujiOshima @gaocegege
The citation is added to the doc, which is for the whole BO algorithm in general, and it mentions the use of random forest prior.

[gaocegege]

/ok-to-test

[libbyandhelen]

/retest

[gaocegege]

@libbyandhelen Please rebase the upstream/master and have a try 😄

[gaocegege]

@YujiOshima PTAL

[YujiOshima]

I'm going to take a look.

[YujiOshima]

/lgtm

[YujiOshima]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: YujiOshima

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [YujiOshima]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment