tuning the downstream MLPF model [pytorch]

[jpata]

• switch MLPF optimizer to AdamW [pytorch]
• use focal loss [pytorch]
• separate decoding layers for different regression components [pytorch]
• added main19.cc for generating CLIC events with PU
• switch CLIC jet clustering to ee_genkt [TF] [pytorch]

[jpata]

Stopped due to patience=20.

Resubmitted with higher dropout, patience=50.

python3 ssl_pipeline.py --data_split_mode domain_adaptation \
  --prefix_VICReg pytorch_${SLURM_JOB_ID} --prefix_mlpf MLPF_test \
  --train_mlpf --native --n_epochs_VICReg 0 --batch_size_mlpf 100 \
  --n_epochs_mlpf 500 --patience 50 --width_mlpf 512 --embedding_dim 512 --lr 0.00005 --nearest 32 --evaluate

[jpata]

Choosing the best (gray line) out of 3 trainings with the same hyperparameters, I'm able to get the pytorch MLPF model to reproduce native PF (and the TF model) in terms of jet response.

This was run with the following options:

  python3 ssl_pipeline.py --data_split_mode domain_adaptation \
  --prefix_VICReg pytorch_${SLURM_JOB_ID} --prefix_mlpf MLPF_test \
  --train_mlpf --native --n_epochs_VICReg 0 --batch_size_mlpf 100 \
  --n_epochs_mlpf 500 --patience 50 --width_mlpf 256 --embedding_dim 256 --lr 0.00005 --num_convs_mlpf 3 --nearest 32 --evaluate