Library Updates (#311)

* update whats next for binary perf est

* upd remaining perf est docs

* remove unneded conf band variabls

docs/tutorials/compare_estimated_and_realized_performance.rst

@@ -20,6 +20,8 @@ When the :term:`targets<Target>` become available, the quality of estimations pr
 
 The beginning of the code below is similar to the one in :ref:`tutorial on
 performance calculation with binary classification data<binary-performance-calculation>`.
+while this tutorial uses the **roc_auc** metric, any metric estimated and calculated by NannyML can
+be used for comparison.
 
 For simplicity this guide is based on a synthetic dataset included in the library, where the monitored model
 predicts whether a customer will repay a loan to buy a car.

docs/tutorials/performance_estimation/binary_performance_estimation/business_value_estimation.rst

@@ -160,8 +160,5 @@ What's next
 -----------
 
 The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
-When the target values become available they can be :ref:`compared with the estimated
-results<compare_estimated_and_realized_performance>`.
-
-You can learn more about the Confidence Based Performance Estimation and its limitations in the
-:ref:`How it Works page<performance-estimation-deep-dive>`.
+When the target values become available we can
+:ref:`compared realized and estimated business value results<compare_estimated_and_realized_performance>`.

docs/tutorials/performance_estimation/binary_performance_estimation/confusion_matrix_estimation.rst

@@ -161,8 +161,5 @@ What's next
 -----------
 
 The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
-When the target values become available they can be :ref:`compared with the estimated
-results<compare_estimated_and_realized_performance>`.
-
-You can learn more about the Confidence Based Performance Estimation and its limitations in the
-:ref:`How it Works page<performance-estimation-deep-dive>`.
+When the target values become available we can
+:ref:`compared realized and estimated confusion matrix results<compare_estimated_and_realized_performance>`.

docs/tutorials/performance_estimation/binary_performance_estimation/custom_metric_estimation.rst

@@ -135,11 +135,5 @@ What's next
 -----------
 
 The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
-When the target values become available they can be :ref:`compared with the estimated
-results<compare_estimated_and_realized_performance>`.
-
-You can learn more about the Confidence Based Performance Estimation and its limitations in the
-:ref:`How it Works page<performance-estimation-deep-dive>`.
-
-And if targets are available or become available, you can learn more about *calculating* confusion
-matrix elements in the :ref:`confusion-matrix-calculation` tutorial.
+When the target values become available we can
+:ref:`compared realized and estimated custom performance metric results<compare_estimated_and_realized_performance>`.

docs/tutorials/performance_estimation/binary_performance_estimation/standard_metric_estimation.rst

@@ -157,8 +157,5 @@ What's next
 -----------
 
 The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
-When the target values become available they can be :ref:`compared with the estimated
-results<compare_estimated_and_realized_performance>`.
-
-You can learn more about the Confidence Based Performance Estimation and its limitations in the
-:ref:`How it Works page<performance-estimation-deep-dive>`.
+When the target values become available we can
+:ref:`compared realized and estimated performance results<compare_estimated_and_realized_performance>`.

docs/tutorials/performance_estimation/multiclass_performance_estimation.rst

@@ -140,7 +140,5 @@ What's next
 -----------
 
 The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
-When the target results become available they can be :ref:`compared with the estimated results<compare_estimated_and_realized_performance>`.
-
-You can learn more about the Confidence Based Performance Estimation and its limitations in the
-:ref:`How it Works page<performance-estimation-deep-dive>`.
+When the target values become available we can
+:ref:`compared realized and performance results<compare_estimated_and_realized_performance>`.

docs/tutorials/performance_estimation/regression_performance_estimation.rst

@@ -164,11 +164,8 @@ What's next
 -----------
 
 The :ref:`Data Drift<data-drift>` functionality can help us to understand whether data drift is causing the performance problem.
-When the target values become available they can be :ref:`compared with the estimated
-results<compare_estimated_and_realized_performance>`.
-
-You can learn more about Direct Error Estimation and its limitations in the
-:ref:`How it Works page<performance-estimation-deep-dive>`.
+When the target values become available we can
+:ref:`compared realized and estimated performance results<compare_estimated_and_realized_performance>`.
 
 
 .. _LGBMRegressor defaults: https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html

nannyml/performance_estimation/confidence_based/cbpe.py

@@ -284,8 +284,6 @@ def __init__(
  )
  )
 
- self.confidence_upper_bound = 1
- self.confidence_lower_bound = 0
  self.needs_calibration: bool = False
 
  if calibrator is None: