Add neural networks and bug fixes (#79)

* fix kfold and reader seeds * fix deprecated series append * add FillnaMean and QuantileTransformer * fix verbose * add debug option in preset * fix weight in metric func * add grad clipping * add nn feature pipeline * add linear with logit losses * add neural networks * add DLOptunaTuner * add nn in tabular preset * add nn in text preset * fix verbose * update modules rst * add nn tutorial * add tps nn solution link * add verbose_bar option * add default_nn_params_v2 * reformat * fix cb custom multitask metric * fix links * add nn in TabularUtilizedAutoML * reformat tutorial * add freeze_defaults param * add docs * add params on input * fix mapping when targets are sorted right * add zero_grad to the end of loop
sb-ai-lab · Mar 16, 2023 · f2ef45e · f2ef45e
1 parent 54666e4
commit f2ef45e
Show file tree

Hide file tree

Showing 55 changed files with 4,883 additions and 305 deletions.
diff --git a/README.md b/README.md
@@ -123,6 +123,8 @@ LighAutoML framework has a lot of ready-to-use parts and extensive customization
 - [Tabular Playground Series February 2021 competition solution](https://www.kaggle.com/alexryzhkov/lightautoml-tabulardata-love)
 - [Interpretable WhiteBox solution](https://www.kaggle.com/simakov/lama-whitebox-preset-example)
 - [Custom ML pipeline elements inside existing ones](https://www.kaggle.com/simakov/lama-custom-automl-pipeline-example)
+- [Custom ML pipeline elements inside existing ones](https://www.kaggle.com/simakov/lama-custom-automl-pipeline-example)
+- [Tabular Playground Series November 2022 competition solution with Neural Networks](https://www.kaggle.com/code/mikhailkuz/lightautoml-nn-happiness)
 
 ### Google Colab tutorials and [other examples](examples/):
 
@@ -134,6 +136,7 @@ LighAutoML framework has a lot of ready-to-use parts and extensive customization
 - [`Tutorial_6_custom_pipeline.ipynb`](https://colab.research.google.com/github/AILab-MLTools/LightAutoML/blob/master/examples/tutorials/Tutorial_6_custom_pipeline.ipynb) - shows how to create your own pipeline from specified blocks: pipelines for feature generation and feature selection, ML algorithms, hyperparameter optimization etc.
 - [`Tutorial_7_ICE_and_PDP_interpretation.ipynb`](https://colab.research.google.com/github/AILab-MLTools/LightAutoML/blob/master/examples/tutorials/Tutorial_7_ICE_and_PDP_interpretation.ipynb) - shows how to obtain local and global interpretation of model results using ICE and PDP approaches.
 - [`Tutorial_8_CV_preset.ipynb`](https://colab.research.google.com/github/AILab-MLTools/LightAutoML/blob/master/examples/tutorials/Tutorial_8_CV_preset.ipynb) - example of using TabularCVAutoML preset in CV multi-class classification task.
+- [`Tutorial_9_neural_networks.ipynb`](https://colab.research.google.com/github/AILab-MLTools/LightAutoML/blob/master/examples/tutorials/Tutorial_9_neural_networks.ipynb) - example of using Tabular preset with neural networks.
 
 
 **Note 1**: for production you have no need to use profiler (which increase work time and memory consomption), so please do not turn it on - it is in off state by default

diff --git a/docs/pages/Kaggle_Kernels.rst b/docs/pages/Kaggle_Kernels.rst
@@ -10,3 +10,4 @@ Kaggle Kernels
 * `Tabular Playground Series February 2021 competition solution <https://www.kaggle.com/alexryzhkov/lightautoml-tabulardata-love>`_
 * `Interpretable WhiteBox solution <https://www.kaggle.com/simakov/lama-whitebox-preset-example>`_
 * `Custom ML pipeline elements inside existing ones <https://www.kaggle.com/simakov/lama-custom-automl-pipeline-example>`_
+* `Tabular Playground Series November 2022 competition solution with Neural Networks <https://www.kaggle.com/code/mikhailkuz/lightautoml-nn-happiness>`_
diff --git a/docs/pages/Python-API.rst b/docs/pages/Python-API.rst
@@ -12,6 +12,7 @@ Python-API
     modules/image
     modules/ml_algo
     modules/ml_algo.tuning
+    modules/ml_algo.torch_based
     modules/pipelines
     modules/pipelines.selection
     modules/pipelines.features

diff --git a/docs/pages/Tutorials.rst b/docs/pages/Tutorials.rst
@@ -14,3 +14,4 @@ Tutorials
     tutorials/Tutorial_6_custom_pipeline.nblink
     tutorials/Tutorial_7_ICE_and_PDP_interpretation.nblink
     tutorials/Tutorial_8_CV_preset.nblink
+    tutorials/Tutorial_9_neural_networks.nblink
diff --git a/docs/pages/modules/ml_algo.rst b/docs/pages/modules/ml_algo.rst
@@ -47,6 +47,24 @@ Boosted Trees
     ~boost_lgbm.BoostLGBM
     ~boost_cb.BoostCB
 
+
+Neural Networks
+-------------------------
+
+.. currentmodule:: lightautoml.ml_algo.torch_based
+
+.. autosummary::
+    :toctree: ./generated
+    :nosignatures:
+    :template: classtemplate.rst
+
+    ~nn_models.MLP
+    ~nn_models.DenseLightModel
+    ~nn_models.DenseModel
+    ~nn_models.ResNetModel
+    ~nn_models.SNN
+
+
 WhiteBox
 -------------------------
 

diff --git a/docs/pages/modules/ml_algo.torch_based.rst b/docs/pages/modules/ml_algo.torch_based.rst
@@ -0,0 +1,24 @@
+.. role:: hidden
+    :class: hidden-section
+
+lightautoml.ml_algo
+===================
+
+Torch utils.
+
+Pooling Strategies
+------------------------------
+
+.. currentmodule:: lightautoml.ml_algo.torch_based.nn_models
+
+.. autosummary::
+    :toctree: ./generated
+    :nosignatures:
+    :template: classtemplate.rst
+
+    SequenceAbstractPooler
+    SequenceClsPooler
+    SequenceMaxPooler
+    SequenceSumPooler
+    SequenceAvgPooler
+    SequenceIndentityPooler
diff --git a/docs/pages/modules/ml_algo.tuning.rst b/docs/pages/modules/ml_algo.tuning.rst
@@ -31,3 +31,4 @@ Tuning with Optuna
     :template: classtemplate.rst
 
     OptunaTuner
+    DLOptunaTuner
diff --git a/docs/pages/modules/pipelines.features.rst b/docs/pages/modules/pipelines.features.rst
@@ -91,3 +91,16 @@ Text Feature Pipelines
     TextAutoFeatures
     NLPTFiDFFeatures
     TextBertFeatures
+
+
+Feature Pipelines for Neural Networks Models
+------------------------------------------------------
+
+.. currentmodule:: lightautoml.pipelines.features.torch_pipeline
+
+.. autosummary::
+    :toctree: ./generated
+    :nosignatures:
+    :template: classtemplate.rst
+
+    TorchSimpleFeatures
diff --git a/docs/pages/modules/text.rst b/docs/pages/modules/text.rst
@@ -53,24 +53,6 @@ Tokenizers
     ~tokenizer.SimpleEnTokenizer
 
 
-Pooling Strategies
-------------------------------
-
-.. currentmodule:: lightautoml.text
-
-.. autosummary::
-    :toctree: ./generated
-    :nosignatures:
-    :template: classtemplate.rst
-
-    ~sentence_pooling.SequenceAbstractPooler
-    ~sentence_pooling.SequenceClsPooler
-    ~sentence_pooling.SequenceMaxPooler
-    ~sentence_pooling.SequenceSumPooler
-    ~sentence_pooling.SequenceAvgPooler
-    ~sentence_pooling.SequenceIndentityPooler
-
-
 Utils
 ------------------------------
 

diff --git a/docs/pages/modules/transformers.rst b/docs/pages/modules/transformers.rst
@@ -39,10 +39,12 @@ Numeric
 
     NaNFlags
     FillnaMedian
+    FillnaMean
     FillInf
     LogOdds
     StandardScaler
     QuantileBinning
+    QuantileTransformer
 
 
 Categorical

diff --git a/docs/pages/tutorials/Tutorial_9_neural_networks.nblink b/docs/pages/tutorials/Tutorial_9_neural_networks.nblink
@@ -0,0 +1,3 @@
+{
+    "path": "../../../examples/tutorials/Tutorial_9_neural_networks.ipynb"
+}
Original file line number	Diff line number	Diff line change
Expand Up		@@ -31,3 +31,4 @@ Tuning with Optuna
		:template: classtemplate.rst

		OptunaTuner
		DLOptunaTuner