API: run LearnerRanker using single model instead of CrossFit

RELEASE_NOTES.rst

@@ -7,6 +7,19 @@ FACET 2.0
 2.0.0
 ~~~~~
 
+``facet.data``
+^^^^^^^^^^^^^^
+
+- API: class :class:`.RangePartitioner` supports new optional arguments ``lower_bound``
+  and ``upper_bound`` in method :meth:`~.RangePartitioner.fit` and no longer accepts
+  them in the class initializer
+
+``facet.inspection``
+^^^^^^^^^^^^^^^^^^^^
+
+- API: :class:`.LearnerInspector` no longer uses learner crossfits and instead inspects
+  models using a single pass of SHAP calculations, usually leading to performance gains
+  of up to a factor of 50
 - API: return :class:`.LearnerInspector` matrix outputs as :class:`.Matrix` instances
 - API: diagonals of feature synergy, redundancy, and association matrices are now
   ``nan`` instead of 1.0
@@ -15,14 +28,34 @@ FACET 2.0
   row and column order of :class:`.Matrix` objects returned by the corresponding
   ``feature_…_matrix`` methods of :class:`.LearnerInspector`, minimizing the distance
   between adjacent leaves
-  The old sorting behaviour of FACET 1.x can be restored using method
+  The old sorting behaviour of FACET 1 can be restored using method
   :meth:`.LinkageTree.sort_by_weight`
-- API: class :class:`.RangePartitioner` adds optional arguments ``lower_bound`` and
-  ``upper_bound`` to method :meth:`~.RangePartitioner.fit` and no longer accepts them
-  in the class initializer
+
+``facet.selection``
+^^^^^^^^^^^^^^^^^^^
+
+- API: :class:`.ModelSelector` replaces FACET 1 class ``LearnerRanker``, and now
+  supports any CV searcher that supports `scikit-learn`'s CV search API, including
+  `scikit-learn`'s native searchers such as :class:`.GridSearchCV` or
+  :class:`.RandomizedSearchCV`
+- API: new classes :class:`.ParameterSpace` and :class:`MultiParameterSpace` offer an
+  a more convenient and robust mechanism for declaring options or distributions for
+  hyperparameter tuning
+
+``facet.simulation``
+^^^^^^^^^^^^^^^^^^^^
+
+- API: simulations no longer depend on learner crossfits and instead are carried out
+  as a single pass on the full dataset, using the *standard error of mean predictions*
+  to obtain confidence intervals that less conservative yet more realistic
 - VIZ: minor tweaks to simulation plots and reports generated by
   :class:`.SimulationDrawer`
 
+Other
+^^^^^
+
+- API: class ``LearnerCrossfit`` is no longer used in FACET 2 and has been removed
+
 
 FACET 1.2
 ---------
@@ -93,7 +126,7 @@ by the :class:`.LearnerInspector`.
   across matrices as an indication of confidence for each calculated value.
 - API: Method :meth:`.LearnerInspector.shap_plot_data` now returns SHAP values for the
   positive class of binary classifiers.
-- API: Increase efficiency of :class:`.LearnerRanker` parallelization by adopting the
+- API: Increase efficiency of :class:`.ModelSelector` parallelization by adopting the
   new :class:`pytools.parallelization.JobRunner` API provided by :mod:`pytools`
 - BUILD: add support for :mod:`shap` 0.38 and 0.39
 

environment.yml

@@ -27,7 +27,7 @@ dependencies:
   - m2r ~= 0.2
   - pluggy ~= 0.13
   - pre-commit ~= 2.7
-  - pydata-sphinx-theme ~= 0.4.0
+  - pydata-sphinx-theme ~= 0.7
   - pytest ~= 5.2
   - pytest-cov ~= 2.8
   - pyyaml ~= 5.1