Top 1 AICovidCough VietNam 2021 Challenge - Fruit AI

Vietnamese | English

1. Setup Envs - Requirements

  # Using anaconda3
  conda create -n aicv115m python=3.6
  conda install pytorch=1.7.0 torchvision=0.8.1 torchaudio=0.7.0 cudatoolkit=10.1 -c pytorch
  !pip install -r "requirements.txt"
  %cd ./modules/pytorch-image-models/
  !pip install .

2. Directory & file structures

aicv115m_challenge_solution
│   .gitignore
│   README.md
│   requirements.txt
|   docker-compose.yml # file config env local for Docker
|   Dockerfile         # file setup library for Docker
│   process.py         # Basic prediction function for 1 file aduio
│   serve.py           # main script to start Flask API
│   main.py            # train - create old submission - create new submission
│
└───assert/
|   |   audio
|   |   metadata
│
└───data/ - datasets are saved here
│   │   aicv115m-data-desc.pdf
│   │   aicv115m_extra_public_1235samples
|   |   |--   extra_public_train_1235samples.csv
|   |   |--   new_1235_audio_files
|   |   
│   │   aicv115m_final_private_test
|   |   |--   private_test_sample_submission.csv
|   |   |--   private_test_audio_files
|   |   
│   │   aicv115m_final_public_test
|   |   |--   public_test_sample_submission.csv
|   |   |--   public_test_audio_files
|   |   
│   │   aicv115m_final_public_train
|   |   |--   public_train_medical_condition.csv
|   |   |--   public_train_metadata.csv
|   |   |--   public_train_audio_files
|   |   
|   |   new_submission
|   |   old_submission
│
└───weights/ - trained models are saved here
|   |   logs            # Chứa các weight new train từ model (Trước khi train khuyến cáo nên xóa các file trong đây)
│   │   1.OnlyModelBase # File chứa weights model 4khz-8khz-48khz Stage1
|   |   
│   |   OldPretrained   # File chứa weights pretrained của fruitai cho 4khz-8khz-48khz Stage2
|   |   |--   Hybrid_4hz_8hz   # Chứa các fold từ 0 - 4
|   |   |--   Hybrid_8hz_8hz   # Chứa các fold từ 0 - 4
|   |   |--   Hybrid_48hz_32hz # Chứa các fold từ 0 - 4
|   |   
|   |   PANNnets
|   |   |-- Cnn14_8k_mAP=0.416.pth
|   |   |-- Cnn14_16k_mAP=0.438.pth
|   |   |-- Cnn14_mAP=0.431.pth
|   |   |-- Wavegram_Logmel_Cnn14_mAP=0.439.pth
|   
└───configs/
│   │   __init__.py
│   │   ...
└───modules/
│   │   __init__.py
│   │   ...
|   |
└───scripts/ - command use for train, create submission, run service
│   │   run_train.sh      # Training
│   │   run_submission.sh # Tạo Submmission từ weights new
│   │   run_oldsubmission.sh # Tạo Old Submmission từ weights FruitAI
│   │   run_service.sh  

Important things:

1. Modify path of dataset same structure-of-path above
2. main.py contains functions: Training - Create New Submission - Create Old Submission
3. Config gpu-id and batch-size of: Gem4hz.yaml - Gem4hz_CNN8hz.yaml Gem8hz.yaml - Gem8hz_CNN8hz.yaml - Gem48hz.yaml - Gem48hz_Wave32hz.yaml

3. Training

Before start training process, you have to follow these stage:

1. Modify GPU-id all files in folder ./configs (Default: gpus: [0,1] - Recommend train with multi-gpu will have the best score GPU: 1070 và 1080ti)
2. Depend on memory gpu, channge batch-size in run_train.sh
3. You can change K-folds by NUM_FOLDS in run_train.sh to have the best result
4. Before train the Stage-2, remove all folders in ./weights/logs/
5. Download pretrained of PANNnets and put it in direction like desciption above (https://zenodo.org/record/3987831)

chmod +x scripts/run_train.sh
./scripts/run_train.sh

4. Create Old Submission

This stage will use the Old Weights of team FruitLab for creating submission

1. Download pretrained of FruitLab team: Please contact oggyfaker@gmail.com
2. All file csv of each khz will be in ./data/old_submission
3. Last file for submission is results.csv

%cd aicv115m_challenge_solution/
chmod +x scripts/run_oldsubmission.sh
./scripts/run_oldsubmission.sh

5. Create New Submission

This stage will use the New Weights for create submission

1. Depend on K-fold you config , set NUM_FOLDS same value
2. Modify all value of NewAlpha in Gem4hz_CNN8hz.yaml Gem8hz_CNN8hz.yaml Gem48hz_Wave32hz.yaml
3. All file csv of each khz will be in ./data/new_submission
4. Last file for submission is results.csv

%cd aicv115m_challenge_solution/
chmod +x scripts/run_submission.sh
./scripts/run_submission.sh

6. Build and run with Docker

This stage will init flask api (localhost must has GPU)

1. Change GPU-id in docker-compose.yml

%cd aicv115m_challenge_solution/
chmod +x ./scripts/run_service.sh
./scripts/run_service.sh