Implementation of the "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" paper.
Explore the docs »
View results
· Report Bug
· Request Feature
Table of Contents
The Lottery Ticket Hypothesis:
A randomly-initialized, dense neural network contains a subnetwork that is initialized such that—when trained in isolation—it can match the test accuracy of the original network after training for at most the same number of iterations
I found the Lottery Ticket hypothesis fascinating so I decided to re-implement the paper (fully for fun).
Key features:
- Code and results with the
Lenet-300-100architecture on MNIST dataset - Code and results with the
Conv-2architecture, variants of VGG (Simonyan & Zisserman, 2014) on CIFAR10 dataset - Code and results with the
Conv-4architecture, variants of VGG (Simonyan & Zisserman, 2014) on CIFAR10 dataset - Code and results with the
Conv-6architecture, variants of VGG (Simonyan & Zisserman, 2014) on CIFAR10 dataset
The paper also experiments with Resnet-18 and VGG-19 which I didn't had time to on include (yet).
If you would like to add any of those models, please consider to fork this repo and to create a pull request.
To get a local copy up and running follow these simple example steps.
Make sure to install the python dependencies : python3 -m pip install requirements.txt
(having access to a GPU will greatly increase the training speed but it's not mandatory)
Each folder corresponds to one of the main experiments described in the paper:
To reproduce the experiments, simply follow the insctructions described in each README.md file.
- Add a results section for each model architecture
- Plot the evolution of iteration for early-stopping
- Plot the evolution of iteration for early-stopping with weight resetting
- Plot the graph based on the mean of five exeperiments
- Add the min and max values in each plots
- Add experiments with
Conv-2on CIFAR10 - Add experiments with
Conv-4on CIFAR10 - Add experiments with
Conv-6on CIFAR10 - Add experiments with
Resnet-18on CIFAR10 - Add experiments with
VGG-19on CIFAR10
See the open issues for a full list of proposed features and known issues.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make would improve this project, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/my-feature) - Commit your Changes (
git commit -m 'feat: my new feature) - Push to the Branch (
git push) - Open a Pull Request
Please try to follow Conventional Commits.
Distributed under the MIT License. See LICENSE.txt for more information.
Valentin De Matos - @ThytuVDM - vltn.dematos@gmail.com
Project Link: https://github.com/Thytu/The-Lottery-Ticket-Hypothesis