FEA implement InstanceHardnessCV cross-validation splitter

[fritshermans]

As discussed on LinkedIn with @glemaitre, I implemented the InstanceHardnessCV splitter as described in my blog post.

'Instance hardness' refers to the difficulty to classify an instance. The way hard-to-classify instances are distributed over train and test sets has significant effect on the test set performance metrics.

Todo:

• add example to examples/ folder
• write documentation

[fritshermans]

I made a small change in the way samples with large instance hardness are distributed over the folds. I first sort by label and then by proba before assigning the folds. Documentation is now also created. Let me know if there are questions or if there's something more I can do!

[glemaitre]

Since I was late to review and I'm planning a release, I took the liberty to push a couple of changes directly. @fritshermans It looks really good already. I just change the module name to align on the scikit-learn naming (cross_validation was unfortunate because it is an old name of a deprecated module). I change slightly the API to not depend on a random state and always request an estimator and handle the positive label properly.

We could extend later for multiclass if it makes any sense then.

[fritshermans]

Thanks for looking into this Guillaume! Please let me know if there's anything I can do now ☺️

[glemaitre]

All good. Thanks @fritshermans.

It should be released in the day :).

[fritshermans]

Thanks a lot Guillaume! I'm very happy with this! 🍾