BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model

Introduction

BiSHop leverages a sparse Hopfield model with adaptable sparsity, enhanced by column-wise and row-wise modules. It's specifically designed to address challenges in processing rotationally invariant and sparse tabular data.

Installation

Install the Environment

For setting up Conda environments and installing necessary packages, refer to the commands provided below (Please install PyTorch according to the specific version of CUDA on your system).

conda create -n BiSHop python=3.10
conda activate BiSHop
pip3 install torch --index-url https://download.pytorch.org/whl/cu121 # please install based on corresponding version
pip3 install -r requirements.txt

Download Code and Datasets

To clone the project repository to your local machine, execute the following command:

git clone https://github.com/MAGICS-LAB/Bi-SHop.git
cd Bi-SHop

For the datasets for Baseline I, please first create the datasets folder

mkdir datasets

Please download them from the link below and place them in the datasets directory: Download Baseline I Datasets

Reproduce Experiments

Recording Runs

To record run details, update your API key in utils/wandb_api_key.txt and use the --record argument.

Baseline I

Run the file launch_sweep.py and change to the dataset_name in Baseline I.

python launch_baseline1.py --data [name of the data] --project [name of the wandb sweep project]

For example

python launch_baseline1.py --data adult --project bishop_baseline1

Baseline II

Run the file launch_sweep.py and change to the dataset_ID in Baseline II.

python launch_baseline2.py --data [data_id] --project [name of the wandb sweep project]

For example

python launch_baseline2.py --data 361110 --project bishop_baseline2

Start Running

Upon initiating the process, you'll receive a prompt for the Wandb agent that reads: wandb: Run sweep agent with: wandb agent [Agent Name]. To proceed, execute the following command:

wandb agent [Agent Name]

Reproduce Baselines and Ablation

To reproduce benchmark results, please checkout other available Branches.

Citation

If you find our work useful, please consider citing our paper:

@inproceedings{xu2024bishop,
  title={BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model},
  author={Xu, Chenwei and Huang, Yu-Chao and Hu, Jerry Yao-Chieh and Li, Weijian and Gilani, Ammar and Goan, Hsi-Sheng and Liu, Han},
  booktitle={Forty-first International Conference on Machine Learning (ICML)},
  year={2024},
  url={https://arxiv.org/abs/2404.03830}
}