C03.3 Evaluacion Modelos Clasificacion.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"logo.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer_datos = datasets.load_breast_cancer()\n",
    "\n",
    "cancer_df = pd.DataFrame(cancer_datos[\"data\"],\n",
    "                           columns=cancer_datos[\"feature_names\"]\n",
    "                          )\n",
    "\n",
    "cancer_df[\"objetivo\"] = cancer_datos.target\n",
    "cancer_df[\"objetivo\"] = cancer_df[\"objetivo\"].replace({0:1, 1:0})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer_df[\"objetivo\"].value_counts(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En primer lugar creamos un modelo simple de Regresión Logística."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = cancer_df[cancer_datos.feature_names]\n",
    "y = cancer_df[\"objetivo\"]\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "modelo = LogisticRegression(solver=\"liblinear\")\n",
    "\n",
    "modelo.fit(X_train, y_train)\n",
    "\n",
    "predicciones = modelo.predict(X_test)\n",
    "clases_reales = y_test\n",
    "predicciones_probabilidades = modelo.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def tupla_clase_prediccion(y_real, y_pred):\n",
    "    return list(zip(y_real, y_pred))\n",
    "\n",
    "tupla_clase_prediccion(clases_reales, predicciones)[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Conceptos de Clasificación binaria**\n",
    "\n",
    "En clasificación binaria, tenemos el concepto de casos negativos (clase 0, en el caso del dataset de cancer de mama serian los casos donde el cancer es benigno) y casos positivos (clase 1, en el caso del dataset de cancer de mama serían los casos donde el cancer es maligno). Positivo y negativo no significa que el resultado sea bueno o malo, simplemente que la detección de un cancer maligno se active o no.\n",
    "\n",
    "- Casos positivos: Casos donde la clase es 1 (cánceres malignos)\n",
    "- Casos negativos: Casos donde la clase es 0 (cánceres benignos)\n",
    "\n",
    "Esto nos lleva a computar 4 tipos de observaciones, explicados como ejemplos del dataset del cancer de mama.\n",
    "\n",
    "- Verdaderos Positivos(True positives), serían las imágenes con un cancer maligno que se detectan como cancer maligno.\n",
    "- Falsos Positivos (False positives), serían los cánceres benignos que se detectan como un cancer maligno.\n",
    "- Verdaderos Negativos(True Negatives), serían los canceres benignos que se clasifican como cánceres benignos.\n",
    "- Falsos Negativos(False Negatives), serían los canceres malignos que se clasifican como cánceres benignos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"ml13.png\"> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def VP(clases_reales, predicciones):\n",
    "    par_clase_prediccion = tupla_clase_prediccion(clases_reales, predicciones)\n",
    "    return len([obs for obs in par_clase_prediccion if obs[0]==1 and obs[1]==1])\n",
    "\n",
    "def VN(clases_reales, predicciones):\n",
    "    par_clase_prediccion = tupla_clase_prediccion(clases_reales, predicciones)\n",
    "    return len([obs for obs in par_clase_prediccion if obs[0]==0 and obs[1]==0])\n",
    "    \n",
    "def FP(clases_reales, predicciones):\n",
    "    par_clase_prediccion = tupla_clase_prediccion(clases_reales, predicciones)\n",
    "    return len([obs for obs in par_clase_prediccion if obs[0]==0 and obs[1]==1])\n",
    "\n",
    "def FN(clases_reales, predicciones):\n",
    "    par_clase_prediccion = tupla_clase_prediccion(clases_reales, predicciones)\n",
    "    return len([obs for obs in par_clase_prediccion if obs[0]==1 and obs[1]==0])\n",
    "\n",
    "\n",
    "print(\"\"\"\n",
    "Verdaderos Positivos: {}\n",
    "Verdaderos Negativos: {}\n",
    "Falsos Positivos: {}\n",
    "Falsos Negativos: {}\n",
    "\"\"\".format(\n",
    "    VP(clases_reales, predicciones),\n",
    "    VN(clases_reales, predicciones),\n",
    "    FP(clases_reales, predicciones),\n",
    "    FN(clases_reales, predicciones)    \n",
    "))    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ratios de clasificación"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exactitud (Accuracy)**\n",
    "\n",
    "La exactitud es una medida general de como se comporta el modelo, mide simplemente el porcentaje de casos que se han clasificado correctamente.\n",
    "\n",
    "$$Exactitud=\\frac{Número~de~observaciones~correctamente~clasificadas}{Número~de~observaciones~totales}= \\frac{VP+VN}{VP+VN+FP+FN}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def exactitud(clases_reales, predicciones):\n",
    "    vp = VP(clases_reales, predicciones)\n",
    "    vn = VN(clases_reales, predicciones)\n",
    "    return (vp+vn) / len(clases_reales)\n",
    "\n",
    "exactitud(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import metrics\n",
    "metrics.accuracy_score(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Precisión (Precission)**\n",
    "\n",
    "La precisión indica la habilidad del modelo para clasificar como positivos los casos que son positivos.\n",
    "\n",
    "$$Precisión=\\frac{Número~de~observaciones~positivas~correctamente~clasificadas}{Número~de~observaciones~clasificadas~como~positivas}= \\frac{VP}{VP+FP}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def precision(clases_reales, predicciones):\n",
    "    vp = VP(clases_reales, predicciones)\n",
    "    fp = FP(clases_reales, predicciones)\n",
    "    return vp / (vp+fp)\n",
    "\n",
    "precision(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metrics.average_precision_score(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exhaustividad o sensibilidad(Recall o True Positive Rate)**\n",
    "\n",
    "La sensibilidad nos da una medida de la habilidad del modelo para encontrar todos los casos positivos. La sensibilidad se mide en función de una clase.\n",
    "\n",
    "$$Sensibilidad=\\frac{Número~de~observaciones~positivas~clasificadas~como~positivas}{Número~de~observaciones~positivas~totales}= \\frac{VP}{VP+FN}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sensibilidad(clases_reales, predicciones):\n",
    "    vp = VP(clases_reales, predicciones)\n",
    "    fn = FN(clases_reales, predicciones)\n",
    "    return vp / (vp+fn)\n",
    "\n",
    "sensibilidad(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metrics.recall_score(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Matriz de confusion**\n",
    "\n",
    "La matriz de confusión es una forma muy sencilla de comparar como ha clasificado cada observación el modelo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "confusion_matrix(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Puntuación F1 (F1 score)**\n",
    "\n",
    "La puntuación F1 es una media ponderada entre la sensibilidad (que intenta obtener cuantos mas verdaderos positivos independientemente de los falsos positivos) y la precisión (que intenta obtener solo verdaderos positivos que sean casos claros para limitar los falsos positivos).\n",
    "\n",
    "La puntuación F1 se define como la media harmónica de la precisión y la sensibilidad:\n",
    "\n",
    "$$F1=2*\\frac{1}{\\frac{1}{precisión}+\\frac{1}{sensibilidad}}=2*\\frac{precisión*sensibilidad}{precisión+sensibilidad}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def puntuacion_f1(clases_reales, predicciones):\n",
    "    precision_preds = precision(clases_reales, predicciones)\n",
    "    sensibilidad_preds = sensibilidad(clases_reales, predicciones)\n",
    "    return 2*(precision_preds*sensibilidad_preds)/(precision_preds+sensibilidad_preds)\n",
    "\n",
    "puntuacion_f1(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metrics.f1_score(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Ratio de Falsos Positivos (Ratio de Falsa Alarma o FPR)**\n",
    "\n",
    "El ratio de falsos positivos nos da una medida de las probabilidades de nuestro modelo de asignar una clase positiva a un caso negativo.\n",
    "\n",
    "Se define como:\n",
    "\n",
    "$$FPR=\\frac{Número~de~observaciones~negativas~clasificadas~como~positivas}{Número~de~observaciones~negativas}= \\frac{FP}{FP+TN}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fpr(clases_reales, predicciones):\n",
    "    return (FP(clases_reales, predicciones) / (\n",
    "             FP(clases_reales, predicciones) + VN(clases_reales, predicciones)\n",
    "             )\n",
    "           )\n",
    "fpr(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**¿Cómo clasifica un modelo?**\n",
    "\n",
    "Un modelo como la regresión lineal funciona prediciendo distancias a una \"linea de decision\" que se convierten en probabilidades para cada clase. Pero a la hora de la verdad al modelo le suele interesar sólo saber que clase predice el modelo. En general esto se hace decidiendo un umbral *(threshold)* y clasificando los casos con menor probabilidad como clase negativa y  mayor probabilidad como clase positiva."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"ml14.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame({\"clase_real\":clases_reales,\n",
    "                   \"clase_pred\": predicciones,\n",
    "                   \"probabilidades_0\":modelo.predict_proba(X_test)[:,0],\n",
    "                    \"probabilidades_1\":modelo.predict_proba(X_test)[:,1],\n",
    "                  })\n",
    "df[\"sum_probas\"] = df.probabilidades_0 + df.probabilidades_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como el modelo no tiene ningún motivo para elegir un umbral determinado (sólo sabe probabilidades) elige 0.5 por defecto."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.query(\"probabilidades_1>0.5 & clase_pred==0\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.query(\"probabilidades_0>0.5 & clase_pred==1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Curva Precisión (Precission-Recall Curve)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def probabilidades_a_clases(predicciones_probabilidades, umbral=0.5):\n",
    "    predicciones = np.zeros([len(predicciones_probabilidades), ])\n",
    "    predicciones[predicciones_probabilidades[:,1]>=umbral] = 1\n",
    "    return predicciones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predicciones_probabilidades[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "probabilidades_a_clases(predicciones_probabilidades, umbral=0.90)[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import widgets, fixed, interact\n",
    "@interact(umbral=widgets.FloatSlider(min=0.01, max=0.99, step=0.01, value=0.01))\n",
    "def evaluar_umbral(umbral):\n",
    "    predicciones_en_umbral = probabilidades_a_clases(predicciones_probabilidades, umbral)\n",
    "    sensibilidad_umbral = metrics.recall_score(clases_reales, predicciones_en_umbral)\n",
    "    fpr_umbral = fpr(clases_reales, predicciones_en_umbral)\n",
    "    precision_umbral = precision(clases_reales, predicciones_en_umbral) \n",
    "    print( \"\"\"\n",
    "    Precision: {:.3f}\n",
    "    Sensibilidad:{:.3f}\n",
    "    Ratio de Alarma: {:.3f}\n",
    "    \"\"\".format(\n",
    "        precision_umbral,\n",
    "        sensibilidad_umbral,\n",
    "        fpr_umbral\n",
    "    ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluar_umbral(umbral):\n",
    "    predicciones_en_umbral = probabilidades_a_clases(predicciones_probabilidades, umbral)\n",
    "    precision_umbral = precision(clases_reales, predicciones_en_umbral)\n",
    "    sensibilidad_umbral = metrics.recall_score(clases_reales, predicciones_en_umbral)\n",
    "    fpr_umbral = fpr(clases_reales, predicciones_en_umbral)\n",
    "    return precision_umbral, sensibilidad_umbral, fpr_umbral\n",
    "\n",
    "\n",
    "rango_umbral = np.linspace(0., 1., 1000)\n",
    "sensibilidad_umbrales = []\n",
    "precision_umbrales = []\n",
    "fpr_umbrales = []\n",
    "\n",
    "for umbral in rango_umbral:\n",
    "    precision_umbral, sensibilidad_umbral, fpr_umbral = evaluar_umbral(umbral)\n",
    "    precision_umbrales.append(precision_umbral)\n",
    "    sensibilidad_umbrales.append(sensibilidad_umbral)\n",
    "    fpr_umbrales.append(fpr_umbral)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(sensibilidad_umbrales, precision_umbrales);\n",
    "plt.ylabel(\"Precision\")\n",
    "plt.xlabel(\"Ratio de Verdaderos positivos (sensibilidad)\")\n",
    "plt.title(\"Curva Precision-Recall\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def grafica_precision_recall(clases_reales, predicciones_probabilidades):\n",
    "    precision_, recall_, _ = metrics.precision_recall_curve(\n",
    "        clases_reales, predicciones_probabilidades[:,1])\n",
    "\n",
    "    plt.step(recall_, precision_, color='b', alpha=0.2,\n",
    "         where='post')\n",
    "    plt.fill_between(recall_, precision_, step='post', alpha=0.2,\n",
    "                 color='b')\n",
    "\n",
    "    plt.xlabel('Recall')\n",
    "    plt.ylabel('Precision')\n",
    "    plt.ylim([0.0, 1.05])\n",
    "    plt.xlim([0.0, 1.05])\n",
    "    plt.title('Curva Precision-Recall');\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "grafica_precision_recall(clases_reales, predicciones_probabilidades)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Area bajo la curva (Area Under the Curve, ROC-AUC)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(fpr_umbrales, sensibilidad_umbrales);\n",
    "plt.xlabel(\"Ratio de Falsos positivos (FPR)\")\n",
    "plt.ylabel(\"Ratio de Verdaderos positivos (sensibilidad)\")\n",
    "plt.title(\"Curva ROC\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metrics.roc_auc_score(clases_reales, predicciones)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def grafica_curva_auc(clases_reales, predicciones, predicciones_probabilidades):\n",
    "    fpr, tpr, _ = metrics.roc_curve(clases_reales, predicciones_probabilidades[:,1])\n",
    "    roc_auc = metrics.roc_auc_score(clases_reales, predicciones)\n",
    "    plt.figure()\n",
    "\n",
    "    plt.plot(fpr, tpr, color='darkorange',\n",
    "         lw=2, label='Curva ROC (area = %0.2f)' % roc_auc)\n",
    "    plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--', label=\"estimador aleatorio\")\n",
    "    \n",
    "    plt.xlim([0.0, 1.0])\n",
    "    plt.ylim([0.0, 1.05])\n",
    "    plt.xlabel('FPR')\n",
    "    plt.ylabel('TPR (recall)')\n",
    "    plt.title('Curva ROC')\n",
    "    plt.legend(loc=\"lower right\")\n",
    "    plt.show();\n",
    "\n",
    "grafica_curva_auc(clases_reales, predicciones, predicciones_probabilidades)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluar_modelo(clases_reales, predicciones, probabilidades):\n",
    "    exactitud = metrics.accuracy_score(clases_reales, predicciones)\n",
    "    precision = metrics.average_precision_score(clases_reales, predicciones)\n",
    "    sensibilidad = metrics.recall_score(clases_reales, predicciones)\n",
    "    roc_auc = metrics.roc_auc_score(clases_reales, predicciones)\n",
    "    f1 = metrics.f1_score(clases_reales, predicciones)\n",
    "    print(\"\"\"\n",
    "    Exactitud: {:.3f}\n",
    "    Precisión: {:.3f}\n",
    "    Sensibilidad: {:.3f}\n",
    "    Area bajo curva (AUC): {:.3f}\n",
    "    Puntuación F1: {:.3f}\n",
    "    \"\"\".format(\n",
    "        exactitud, \n",
    "        precision,\n",
    "        sensibilidad,\n",
    "        roc_auc,\n",
    "        f1\n",
    "    ))\n",
    "    \n",
    "evaluar_modelo(clases_reales, predicciones, predicciones_probabilidades)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer_df.objetivo.value_counts(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Como decidir un Umbral de decision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import metrics\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "cancer_datos = datasets.load_breast_cancer()\n",
    "\n",
    "cancer_df = pd.DataFrame(cancer_datos[\"data\"],\n",
    "                           columns=cancer_datos[\"feature_names\"]\n",
    "                          )\n",
    "\n",
    "cancer_df[\"objetivo\"] = cancer_datos.target\n",
    "cancer_df[\"objetivo\"] = cancer_df[\"objetivo\"].replace({0:1, 1:0})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = cancer_df[cancer_datos.feature_names]\n",
    "y = cancer_df[\"objetivo\"]\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "modelo = LogisticRegression(solver=\"liblinear\")\n",
    "\n",
    "modelo.fit(X_train, y_train)\n",
    "\n",
    "predicciones = modelo.predict(X_test)\n",
    "clases_reales = y_test\n",
    "predicciones_probabilidades = modelo.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "probas = modelo.predict_proba(X_test)[:5]\n",
    "probas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "umbral_decision = 0.5\n",
    "\n",
    "probas[:,1]>=umbral_decision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "umbral_decision = 0.1\n",
    "\n",
    "probas[:,1]>=umbral_decision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(coste_fp, coste_fn):\n",
    "    return np.exp(coste_fp) / (np.exp(coste_fn)+np.exp(coste_fp))\n",
    "\n",
    "coste_fn = 1\n",
    "coste_fp = 2\n",
    "softmax(coste_fp, coste_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import widgets, interact\n",
    "\n",
    "@interact\n",
    "def calculo_umbral(\n",
    "    coste_fp=widgets.FloatSlider(min=1, max=10, step=0.1, value=1),\n",
    "    coste_fn=widgets.FloatSlider(min=1, max=10, step=0.1, value=1),\n",
    "):\n",
    "    return softmax(coste_fp, coste_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "coste_fn = 10\n",
    "coste_fp = 1\n",
    "umbral_decision = calculo_umbral(coste_fp, coste_fn)\n",
    "print(umbral_decision)\n",
    "decisiones = probabilidades_a_clases(probas, umbral_decision)\n",
    "decisiones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class BusinessLogisticRegression(LogisticRegression):\n",
    "        \n",
    "    def decision_de_negocio(self, X, coste_fp=1, coste_fn=1, *args, **kwargs):\n",
    "        probs = self.predict_proba(X)\n",
    "        umbral_decision = calculo_umbral(coste_fp, coste_fn)\n",
    "        print(\"Umbral de decision: {}\".format(umbral_decision))\n",
    "        decisiones = probabilidades_a_clases(probs, umbral_decision)\n",
    "        return decisiones\n",
    "        \n",
    "modelo_negocio = BusinessLogisticRegression(solver=\"liblinear\")\n",
    "\n",
    "modelo_negocio.fit(X_train, y_train)        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "modelo_negocio.predict(X_test[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "modelo_negocio.predict_proba(X_test[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "modelo_negocio.decision_de_negocio(X_test[:5], 1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@interact(\n",
    "    coste_fp=widgets.FloatSlider(min=1.,max=10.,step=.1,value=1.),\n",
    "    coste_fn=widgets.FloatSlider(min=1.,max=10.,step=.1,value=1.)\n",
    ")\n",
    "def decision_negocio(coste_fp, coste_fn):\n",
    "    predicciones = modelo_negocio.decision_de_negocio(X_test, coste_fp, coste_fn)\n",
    "    print(confusion_matrix(clases_reales, predicciones))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "widgets": {
   "state": {
    "faecd23e356c4a88b55f22e7d579b0f3": {
     "views": [
      {
       "cell_index": 34
      }
     ]
    }
   },
   "version": "1.2.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}