[WIP] Make Random Forest observable. #4655

geektoni · 2019-05-30T14:54:13Z

No description provided.

examples/meta/src/multiclass/random_forest.sg

src/shogun/base/base_types.h

src/shogun/machine/BaggingMachine.cpp

src/shogun/machine/BaggingMachine.h

karlnapf · 2019-05-31T15:17:01Z

tests/unit/labels/MockLabels.h

@@ -12,6 +12,14 @@ namespace shogun {
 			MOCK_METHOD0(get_values, SGVector<float64_t>());

 			virtual const char* get_name() const { return "MockCLabels"; }
+
+		protected:
+			virtual CSGObject *create_empty() const


i dont understand why you need this as there is never an empty instance created ... or am I wrong?

There were situations in which we were emitting MockLabels from a machine and therefore I needed to implement the create_empty() method to make clone() work.

tests/unit/multiclass/BaggingMachine_unittest.cc

* Round of clang format.

* Round of clang format; * Remove CombinationRule casting inside BagginMachine unit tests.

src/shogun/machine/BaggingMachine.cpp

* Round of clang format; * Remove CombinationRule casting inside BagginMachine unit tests.

stale · 2020-02-26T15:52:12Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

geektoni · 2020-04-08T17:34:51Z

@geektoni is this still needed?

I'm picking it up again these days. Let's see after I fix all these merge conflicts.

* Round of clang format; * Remove CombinationRule casting inside BagginMachine unit tests.

* Fix CBaggingMachine unit tests by using the put()/get() methods; * Fix RandomForest constructors (remove missing methods); * Fix RandomForest unit tests;

It can be called now without passing an explicit CEvaluation object. This will be generated automatically, given the features' type, and it will be used to compute the oob error.

It will emit oob error during training and each of the trained machines.

* Add CCombinationRule to base_types.h; * Add missing SG_REF()/SG_UNREF() pair.

These are added to enable put() semantic. * Fix random forest meta examples;

* Round of clang format; * Remove CombinationRule casting inside BagginMachine unit tests.

* Removed some observable code until the get_oob_error() gets fixed (or updated);

* observe() now accept a const reference to the ObservedValue; * BaggingMachine should be better observable;

* Fix RandomForest unit test (casting error); * Remove compute_oob observable; * Remove Random Forest meta example (they were probably left there when rebasing against develop); * Fix code style;

The OOB error is computed only if the combination rule and the evaluation metric are specified. Moreover, there must be observers attached to the machine.

karlnapf · 2020-05-07T13:50:50Z

src/shogun/machine/BaggingMachine.cpp

+			if (this->m_combination_rule && this->m_oob_evaluation_metric &&
+			    this->get_num_subscriptions() != 0)
+			{
+				auto oob_error = this->get_oob_error();


question: you now always compute the oob error whenever there is an observer ...Is that good? I forgot how the specifying to be observed variables works, but can't the user somehow pick what she wants to observe? In that case this should only be computed if the user requested it

So, the idea is that as long as the model has some observer attached, then it will output all the available observations. This means that the OOB error will be computed regardless of the user preferences.

It will be the observer job to filter the observations and keep just the ones that the user wants.

At the moment, there is no way to tell an observable model to emit only certain observations.

I don’t think that’s good and we should probably think about a way to avoid this. Passing (and ignoring) pointers to existing data is one thing, but expensive operations should only be executed if desired.
I could imagine that this could be done with a lambda function that is only executed if the observer actually wants (as in user instructed) to store the data. Maybe @gf712 has an idea he is the lambda master?

I see the problem. One small issue here is that the model does not have any idea about which things the observers want to observe. I guess the solution could be emitting functions (lambdas) instead which can then be executed by the observers if they want to store that kind of data. Is this what you meant @karlnapf?

yes, the model doesnt know. In general, it should offer the observer to observe something (which the observer then can or cannot do, depending on its settings). Passing a pointer is a good example for that. The observer can clone the underlying structure for example.
But for costly things, passing a lambda that would compute something expensive is a better choice imo, so yet.

Okay, that makes sense! I will do some experimenting to try adding this lambda feature. I'm leaving this one as it is (or maybe we could merge it in a feature branch) and then I will open a new PR with the lambda observers.

I say let's just keep this PR open, and then you could rebase it once the the lambda thing works. Alternatively, just remove the oob error stuff and we can merge everything else?

…ions. * Fix test style;

stale · 2020-11-05T22:06:20Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale · 2020-11-12T23:59:26Z

This issue is now being closed due to a lack of activity. Feel free to reopen it.

stale · 2021-06-09T00:20:44Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale · 2021-06-16T01:09:41Z

This issue is now being closed due to a lack of activity. Feel free to reopen it.

geektoni force-pushed the feature/observable_randomforest branch 5 times, most recently from fe8419a to 9068b55 Compare May 31, 2019 08:57