- Model code are all in the
./Model/
folder - These include models such as..
- Check
Parser.py
for the list of hyperparameters available for each model
- We are using the implicit library for Weighted Matrix Factorization (WMF)
- UserKNN, ItemKNN, RP3beta, and all their associated code are from the RecSys 2019 Deep Learning Evaluation GitHub repository
- Mult-VAE is written in Tensorflow, by referencing the notebook provided by the paper's authors
Thank you!! :)
-
Option 1 (Run using specific hyperparameter settings and/or your own grid search)
-
For UserKNN, ItemKNN, RP3beta, and WMF:
train.py
- E.g.
python3 train.py -d "ML-100K" -m "UserKNNCF"
- E.g.
-
For Mult-VAE:
MultiVAE_train.py
- E.g.
python3 MultiVAE_train.py -d "ML-100K" -n_epochs 200 -num_hidden 1 -beta 0.1
- For Mult-VAE, we used the following shell scripts for the experiments in our paper:
- ML-100K:
script_ML100K_MultiVAE.sh
- Amazon (Video Games):
script_AmazonVideoGames_MultiVAE.sh
- ML-100K:
- E.g.
-
-
Option 2 (Bayesian Hyperparameter Optimization, using scikit-optimize)
- We used this option to automatically tune the hyperparameters for UserKNN, ItemKNN, RP3beta, and WMF in our paper
- The code is in
hyperOpt_train.py
- Examples
- ML-100K:
script_ML100K.sh
- Amazon (Video Games):
script_AmazonVideoGames.sh
- ML-100K:
- The results are stored in the
./logs/
folder- Results for a particular dataset and model can be found in
./logs/{dataset}/{model}/
- Results for a particular dataset and model can be found in
- There are some helper files to 'sort' and 'gather' those results, i.e.
sort_results.py
,gather_results.py
, andutilities_results.py
- For example, if we consider the Recall @ 10 metric, there are 3 files in the
./logs/
folder:___results_summary___Rec_10.txt
shows the model performance___results_summary___Rec_10__bar.png
contains the bar plot___results_summary___Rec_10__table.png
shows the relative performance of Model X (row) over Model Y (column)- E.g. for the Amazon (Electronics) dataset in Cluster 1, the value at (Row 1, Column 2) indicates the relative improvement of RP3beta over ItemKNN
- Values in light green (also indicated with a *) are statistically significant with a p-value < 0.05
- Values in dark green (also indicated with a **) are statistically significant with a p-value < 0.01