To split the data heterogeneously onto 6 local workers and preprocess it, run:
python preprocessing.py --data covtype --cond 1e-30 --it_max 5000 --n 6
Then, run DIANA+ (quant+)
python main.py --data covtype --alg SD-DIANA-plus --n 6
We support 11 algorithms in total:
Baselines:
- SD-DCGD: DCGD (Khirirat et al. [2018]) + standard quantization (Alistarh et al. [2017])
- SD-DIANA: DIANA (Mishchenko et al. [2019]) + standard quantization (Alistarh et al. [2017])
- BL-DCGD: DCGD (Khirirat et al. [2018]) + block quantization (Alistarh et al. [2017])
- BL-DIANA: DIANA (Mishchenko et al. [2019]) + block quantization (Alistarh et al. [2017])
Proposed Algorithms:
- SD-DCGD-plus-fnl: DCGD+ (Algorithm 1) + standard quantization (Alistarh et al. [2017])
- SD-DCGD-plus: DCGD+ (Algorithm 1) + quantization with varying steps (Section 5)
- SD-DIANA-plus-fnl: DIANA+ (Algorithm 2) + standard quantization (Alistarh et al. [2017])
- SD-DIANA-plus: DIANA+ (Algorithm 2) + quantization with varying steps (Section 5)
- BL-DCGD-plus: DCGD+ (Algorithm 1) + block quantization (Section 4)
- BL-DIANA-plus: DIANA+ (Algorithm 2) + block quantization (Section 4)
- BL-DIANA-plus-fnl: DIANA+ (Algorithm 2) + block quantization (Alistarh et al. [2017])
- Python 3.6+
- MPI4PY
To reproduce the figures on covtype dataset in our paper, run:
bash covtype.sh