Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add augmentations inside the fit method #363

Merged
merged 3 commits into from
Mar 23, 2022
Merged

feat: add augmentations inside the fit method #363

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c1e9c4e
feat: add augmentations inside the fit method
Optimox Mar 3, 2022
92c0b81
chore: poetry update
Optimox Mar 8, 2022
efe0b97
Merge branch 'develop' into feat/data-augmentation
Optimox Mar 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ NO_COLOR=\\e[39m
OK_COLOR=\\e[32m
ERROR_COLOR=\\e[31m
WARN_COLOR=\\e[33m
PORT=8889
Optimox marked this conversation as resolved.
Show resolved Hide resolved
PORT=8887
.SILENT: ;
default: help; # default target

Expand Down
6 changes: 6 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,12 @@ A complete example can be found within the notebook `pretraining_example.ipynb`.

/!\ : current implementation is trying to reconstruct the original inputs, but Batch Normalization applies a random transformation that can't be deduced by a single line, making the reconstruction harder. Lowering the `batch_size` might make the pretraining easier.

# Data augmentation on the fly

It is now possible to apply custom data augmentation pipeline during training.
Templates for ClassificationSMOTE and RegressionSMOTE have been added in `pytorch-tabnet/augmentations.py` and can be used as is.


# Easy saving and loading

It's really easy to save and re-load a trained model, this makes TabNet production ready.
Expand Down
15 changes: 13 additions & 2 deletions census_example.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,16 @@
"max_epochs = 100 if not os.getenv(\"CI\", False) else 2"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pytorch_tabnet.augmentations import ClassificationSMOTE\n",
"aug = ClassificationSMOTE(p=0.2)"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -225,10 +235,11 @@
" batch_size=1024, virtual_batch_size=128,\n",
" num_workers=0,\n",
" weights=1,\n",
" drop_last=False\n",
" drop_last=False,\n",
" augmentations=aug, #aug, None\n",
" )\n",
" save_history.append(clf.history[\"valid_auc\"])\n",
" \n",
"\n",
"assert(np.all(np.array(save_history[0]==np.array(save_history[1]))))"
]
},
Expand Down
8 changes: 6 additions & 2 deletions forest_example.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@
"metadata": {},
"outputs": [],
"source": [
"max_epochs = 5 if not os.getenv(\"CI\", False) else 2"
"max_epochs = 50 if not os.getenv(\"CI\", False) else 2"
]
},
{
Expand All @@ -248,12 +248,16 @@
},
"outputs": [],
"source": [
"from pytorch_tabnet.augmentations import ClassificationSMOTE\n",
"aug = ClassificationSMOTE(p=0.2)\n",
"\n",
"clf.fit(\n",
" X_train=X_train, y_train=y_train,\n",
" eval_set=[(X_train, y_train), (X_valid, y_valid)],\n",
" eval_name=['train', 'valid'],\n",
" max_epochs=max_epochs, patience=100,\n",
" batch_size=16384, virtual_batch_size=256\n",
" batch_size=16384, virtual_batch_size=256,\n",
" augmentations=aug\n",
") "
]
},
Expand Down
Loading