Linear decision trees improvements #60

bytesnake · 2020-11-20T15:30:57Z

This PR continues the work of #43

Rebase to master branch
Use new traits for linfa-trees
Add more tests to linfa-trees
Add function to generate tikz code describing the tree

…from decision trees

mossbanay · 2020-11-25T10:54:40Z

I have exams this week and next but I'm interested in helping out with this afterward. There are only a few changes left mentioned in the previous PR.

* use toy test from sklearn * use four perfectly separable uniform blobs

This hyper-parameter can be estimated from the input data and is therefore uneccessary in the API.

linfa-trees/src/decision_trees/algorithm.rs

mossbanay

Tests & lints want a review?

mossbanay

Test & lints want an approval?

mossbanay · 2020-12-06T04:42:46Z

For looking at random forests (which I think should be in a separate PR to these improvements) we can either:

Pass in a mask over the features
Generate datasets with the features masked out
It's not immediately clear to me which would be better in the long-term so I suggest we pursue (2) for the moment since it doesn't clutter the API for now.

bytesnake · 2020-12-06T09:32:01Z

they were moved to #66

codecov-io · 2020-12-06T09:47:53Z

Codecov Report

Merging #60 (5fd321a) into master (a3eede5) will decrease coverage by 0.56%.
The diff coverage is 2.39%.

@@            Coverage Diff            @@
##           master     #60      +/-   ##
=========================================
- Coverage    9.97%   9.40%   -0.57%     
=========================================
  Files          47      49       +2     
  Lines        2507    2657     +150     
=========================================
  Hits          250     250              
- Misses       2257    2407     +150

Impacted Files	Coverage Δ
linfa-logistic/src/lib.rs	`0.00% <0.00%> (ø)`
linfa-svm/src/classification.rs	`0.00% <ø> (ø)`
linfa-trees/src/decision_trees/algorithm.rs	`0.00% <0.00%> (ø)`
linfa-trees/src/decision_trees/hyperparameters.rs	`0.00% <0.00%> (ø)`
linfa-trees/src/decision_trees/iter.rs	`0.00% <0.00%> (ø)`
linfa-trees/src/decision_trees/tikz.rs	`0.00% <0.00%> (ø)`
src/dataset/mod.rs	`50.00% <ø> (ø)`
src/dataset/impl_dataset.rs	`9.37% <6.66%> (-7.30%)`	⬇️
src/metrics_classification.rs	`77.41% <66.66%> (ø)`
src/dataset/impl_targets.rs	`44.44% <100.00%> (ø)`
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3eede5...5fd321a. Read the comment docs.

* introduce node iterator * rewrite `max_depth`, `num_leaves`, `features` in iterator syntax

mossbanay

Happy to merge this then continue on with RFs in that other PR

bytesnake · 2020-12-06T10:26:29Z

awesome 👍 I will just write a quick function which can generates a tikz styled tree (for example here) and then merge

bytesnake · 2020-12-06T16:56:31Z

from the example, this is not as pretty as possible, but for now is sufficient

mossbanay · 2020-12-07T07:21:54Z

Nice work! That looks great. One of the more premium features in ML libraries for sure.

fgadaleta and others added 28 commits November 20, 2020 11:34

add debug trait to structs

05e1f02

initial commit for random forest implementation

7c3b1c0

WIP first iter of random forest

e5c77ca

add benches

7d2d512

setup bench and cleanup

da77696

cleanup

701182d

add max_n_rows for single decision tree fitting

75bebb4

implement random forest feature importance as collection of features …

d045e5e

…from decision trees

implement random forest feature importance as collection of features …

1b754e9

…from decision trees

remove unused var

7fbfa01

remove unused var

da30535

run clippy

b808415

assert test success for feature importance

f75d334

clippy and fmt

5461488

store references of nodes to queue

d84a8b5

WIP voting classifier and predictor trait

4cfd3d4

WIP voting classifier and predictor trait

2846ef1

implement and test VotingClassifier hard voting

b571894

implement predict_proba for random forest and tested

37c2bec

documentation, examples, cleanup

46d8491

cleanup

0efe3ab

implement LinfaError for Predictor trait

34f3c40

fixed tests and CI/CD pipeline

79abd05

renamed predict_classes to predict in logreg for consistency

139ba82

implement ProbabilisticPredictor whenever needed

7b4103e

votingclassifier implements predictor trait

dbf9184

PR-43 Moss comments addressed

92f20ab

Switch linfa-tree to new infrastructure

90a5a2a

bytesnake mentioned this pull request Nov 20, 2020

[WIP] Feature/linfa ensemble random forest #43

Closed

bytesnake added 2 commits December 4, 2020 16:55

Add more tests to linear decision trees

5864e76

* use toy test from sklearn * use four perfectly separable uniform blobs

Remove number of classes hyper-parameter

6d0cc1d

This hyper-parameter can be estimated from the input data and is therefore uneccessary in the API.

mossbanay reviewed Dec 4, 2020

View reviewed changes

linfa-trees/src/decision_trees/algorithm.rs Outdated Show resolved Hide resolved

mossbanay reviewed Dec 4, 2020

View reviewed changes

linfa-trees/src/decision_trees/algorithm.rs Outdated Show resolved Hide resolved

mossbanay reviewed Dec 4, 2020

View reviewed changes

mossbanay approved these changes Dec 4, 2020

View reviewed changes

Remove ensemble algorithm

6f5f78f

bytesnake changed the title ~~[WIP] Trees and ensemble algorithms~~ Linear decision trees improvements Dec 5, 2020

Address issue with toy test

b2b41a0

Merge branch 'master' of github.com:rust-ml/linfa into trees

82b62c4

Simplify tree inspection methods

70402ab

* introduce node iterator * rewrite `max_depth`, `num_leaves`, `features` in iterator syntax

mossbanay approved these changes Dec 6, 2020

View reviewed changes

bytesnake added 6 commits December 6, 2020 11:53

Add max depth testing and hyperparameter validation

1f1849f

Fix parameter syntax in benchmarks

09348a0

Run cargo fmt

7df68bc

Add tikz export builder

f0bdc1c

Improve decision tree formatting

311fc94

Add pruning

fa410f7

bytesnake added 4 commits December 6, 2020 17:56

Adjust syntax of tikz snippet

635f98c

Merge branch 'master' of github.com:rust-ml/linfa into trees

c658bd9

Run cargo fmt

b268de2

Run cargo fmt

5fd321a

bytesnake merged commit bfa5aeb into rust-ml:master Dec 6, 2020

bytesnake mentioned this pull request Dec 10, 2020

Rust implementation of DecisionTree::fit does not match results from sklearn #40

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Linear decision trees improvements #60

Linear decision trees improvements #60

bytesnake commented Nov 20, 2020 •

edited

Loading

mossbanay commented Nov 25, 2020

mossbanay left a comment

mossbanay left a comment

mossbanay commented Dec 6, 2020

bytesnake commented Dec 6, 2020

codecov-io commented Dec 6, 2020 •

edited

Loading

mossbanay left a comment

bytesnake commented Dec 6, 2020

bytesnake commented Dec 6, 2020

mossbanay commented Dec 7, 2020

Linear decision trees improvements #60

Linear decision trees improvements #60

Conversation

bytesnake commented Nov 20, 2020 • edited Loading

mossbanay commented Nov 25, 2020

mossbanay left a comment

Choose a reason for hiding this comment

mossbanay left a comment

Choose a reason for hiding this comment

mossbanay commented Dec 6, 2020

bytesnake commented Dec 6, 2020

codecov-io commented Dec 6, 2020 • edited Loading

Codecov Report

mossbanay left a comment

Choose a reason for hiding this comment

bytesnake commented Dec 6, 2020

bytesnake commented Dec 6, 2020

mossbanay commented Dec 7, 2020

bytesnake commented Nov 20, 2020 •

edited

Loading

codecov-io commented Dec 6, 2020 •

edited

Loading