Awesome_Modern_Hopfield_Networks

Paper list for Modern Hopfield Networks

Useful Materials

• scholarpedia.org/article/Hopfield_network

• Discrete Hopfield Network, September 20, 2015

Year 2024

• Sargolzaei, Saleh, and Luis Rueda. "Improving Out-of-Distribution Data Handling and Corruption Resistance via Modern Hopfield Networks." arXiv preprint arXiv:2408.11309 (2024). [Paper]

• Ganjidoost, Ehsan, Mallory Snow, and Jeff Orchard. "Online Training of Hopfield Networks using Predictive Coding." arXiv preprint arXiv:2406.14723 (2024). [Paper]

• Chateau-Laurent, Hugo, and Frederic Alexandre. "Contextual Control of Hopfield Networks in a Hippocampal Model." Proceedings of the Annual Meeting of the Cognitive Science Society. Vol. 46. 2024. [Paper]

• Chaudhry, H., Zavatone-Veth, J., Krotov, D., & Pehlevan, C. (2024). Long sequence hopfield memory. Advances in Neural Information Processing Systems, 36. [Paper]

• Santos, S., Niculae, V., McNamee, D., & Martins, A. F. (2024). Sparse and Structured Hopfield Networks. arXiv preprint arXiv:2402.13725. [Paper]

• McAlister H, Robins A, Szymanski L. Prototype Analysis in Hopfield Networks with Hebbian Learning[J]. Neural Computation, 2024: 1-43. [Paper]

• Nicolini, C., Gopalan, M., Staiano, J., & Lepri, B. (2024). Hopfield Networks for Asset Allocation. arXiv preprint arXiv:2407.17645. [Paper]

• McAlister, Hayden, Anthony Robins, and Lech Szymanski. "Improved Robustness and Hyperparameter Selection in Modern Hopfield Networks." arXiv preprint arXiv:2407.08742 (2024). [Paper]

• Alonso, Nicholas, and Jeffrey L. Krichmar. "A sparse quantized Hopfield network for online-continual memory." Nature Communications 15.1 (2024): 3722. [Paper]

• Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory, Xueyan Niu, Bo Bai, Lei Deng, Wei Han [Paper]

• [arXiv:2405.08766] Energy-based Hopfield Boosting for Out-of-Distribution Detection, Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, Sepp Hochreiter [Paper] [Code]

• Long-term Frame-Event Visual Tracking: Benchmark Dataset and Baseline, Xiao Wang, Ju Huang, Shiao Wang, Chuanming Tang, Bo Jiang, Yonghong Tian, Jin Tang, Bin Luo, arXiv:2403.05839 [Paper] [Code] [DemoVideo]

• Nonparametric Modern Hopfield Models, Jerry Yao-Chieh Hu, Bo-Yu Chen, Dennis Wu, Feng Ruan, Han Liu [Paper] [Code]

• BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model, Chenwei Xu, Yu-Chao Huang, Jerry Yao-Chieh Hu, Weijian Li, Ammar Gilani, Hsi-Sheng Goan, Han Liu [Paper] [Code]

• "Reconstructing creative thoughts: Hopfield neural networks." Checiu, Denisa, Mathias Bode, and Radwa Khalil. Neurocomputing (2024): 127324. [Paper]

• On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis, Jerry Yao-Chieh Hu, Thomas Lin, Zhao Song, Han Liu [Paper]

• "Sequential memory with temporal predictive coding." Tang, Mufeng, Helen Barron, and Rafal Bogacz. Advances in Neural Information Processing Systems 36 (2024). [Paper]

• Dense Hopfield Networks in the Teacher-Student Setting, Robin Thériault, Daniele Tantari [Paper]

Year 2023

• Outlier-Efficient Hopfield Layers for Large Transformer-Based Models, Jerry Yao-Chieh Hu, Pei-Hsuan Chang, Robin Luo, Hong-Yu Chen, Weijian Li, Wei-Po Wang, Han Liu [Paper]

• Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models, Dennis Wu, Jerry Yao-Chieh Hu, Teng-Yun Hsiao, Han Liu [Paper]

• STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction, Dennis Wu, Jerry Yao-Chieh Hu, Weijian Li, Bo-Yu Chen, Han Liu [Paper]

• Neuron-Astrocyte Associative Memory, arXiv:2311.08135, Leo Kozachkov, Jean-Jacques Slotine, Dmitry Krotov [Paper]

• Seidl, Philipp, et al. "Improving few-and zero-shot reaction template prediction using modern hopfield networks." Journal of chemical information and modeling 62.9 (2022): 2111-2120. [Paper] [Code] [Video Tutorial]

• [NIPS-2023] On Sparse Modern Hopfield Model, Jerry Yao-Chieh Hu, [Paper]

• Jeremy Lu, Jonah Wu, Understanding Hopfield Neural Networks, [Paper]

• Farah Aymen Mounir, Surveying Hopfeild Neural Network and its Applications, [Paper]

• Malyaban Bal , Abhronil Sengupta, Sequence Learning Using Equilibrium Propagation, [Paper] [Code]

• Benjamin Hoover, et al, Energy Transformer, [Paper] [Code]

• Hamza Tahir Chaudhry, Long Sequence Hopfield Memory [Paper]

• Ota, Toshihiro, and Masato Taki. "iMixer: hierarchical Hopfield network implies an invertible, implicit and iterative MLP-Mixer." arXiv preprint arXiv:2304.13061 (2023). [Paper]

• Auer, Andreas, et al. "Conformal prediction for time series with Modern Hopfield Networks." arXiv preprint arXiv:2303.12783 (2023). [Paper]

• Ota, Toshihiro, et al. "Learning with Partial Forgetting in Modern Hopfield Networks." International Conference on Artificial Intelligence and Statistics. PMLR, 2023. [Paper]

• Chang S, Kopp M, Ghamisi P. Dsfer-Net: A Deep Supervision and Feature Retrieval Network for Bitemporal Change Detection Using Modern Hopfield Networks[J]. arXiv preprint arXiv:2304.01101, 2023. [Paper] [Code]

• Krotov, Dmitry. "A new frontier for Hopfield networks." Nature Reviews Physics (2023): 1-2. [Paper]

• Burns, Thomas F., and Tomoki Fukai. "Simplicial Hopfield networks." The Eleventh International Conference on Learning Representations. 2022. [Paper]

• [Neural Computation] Ota, Toshihiro, and Ryo Karakida. "Attention in a family of Boltzmann machines emerging from modern Hopfield networks." Neural Computation 35.8 (2023): 1463-1480. [Paper] [Code]

• [ICLR 2022] Zhang, Jinsong, et al. "Out-of-Distribution Detection based on In-Distribution Data Patterns Memorization with Modern Hopfield Energy." The Eleventh International Conference on Learning Representations. 2022. [Paper] [Code]

Year 2022

• Sanchez-Fernandez, Ana, et al. "Contrastive learning of image-and structure-based representations in drug discovery." ICLR2022 Machine Learning for Drug Discovery. 2022. [Paper]

• [NIPS 2022] Fürst, Andreas, et al. "Cloob: Modern hopfield networks with infoloob outperform clip." Advances in neural information processing systems 35 (2022): 20450-20468. [Paper] [Code]

• Seidl, Philipp, et al. "Improving few-and zero-shot reaction template prediction using modern hopfield networks." Journal of chemical information and modeling 62.9 (2022): 2111-2120. [Paper] [Code] [Video]

• [NeurIPS 2022] Iatropoulos, Georgios, Johanni Brea, and Wulfram Gerstner. "Kernel Memory Networks: A Unifying Framework for Memory Modeling." Advances in Neural Information Processing Systems 35 (2022): 35326-35338. [Paper]

• Millidge B, Salvatori T, Song Y, et al. Universal hopfield networks: A general framework for single-shot associative memory models[C]//International Conference on Machine Learning. PMLR, 2022: 15561-15583. [Paper] [Code]

• [IEEE TIP-2023] Xu, Yonghao, et al. "Txt2Img-MHN: Remote sensing image generation from text using modern Hopfield networks." arXiv preprint arXiv:2208.04441 (2022). [Paper] [Code]

• Seidl, Philipp, et al. "Improving few-and zero-shot reaction template prediction using modern hopfield networks." Journal of chemical information and modeling 62.9 (2022): 2111-2120. [Paper] [Code]

• [NIPS-2022] Fürst, Andreas, et al. "Cloob: Modern hopfield networks with infoloob outperform clip." Advances in neural information processing systems 35 (2022): 20450-20468. [Paper] [Code]

• Schäfl, Bernhard, et al. "Hopular: Modern hopfield networks for tabular data." arXiv preprint arXiv:2206.00664 (2022). [Paper] [Code] [Blog]

Year 2021 and Before

• [ICLR 2021] Ramsauer, Hubert, et al. "Hopfield Networks is All You Need." International Conference on Learning Representations. 2021 [Paper] [Code] [Project Page]

• [NIPS-2020] Widrich, Michael, et al. "Modern hopfield networks and attention for immune repertoire classification." Advances in Neural Information Processing Systems 33 (2020): 18832-18845. [Paper] [Code]

• [Workshop NeurIPS 2021] Widrich, Michael, et al. "Modern hopfield networks for return decomposition for delayed rewards." Deep RL Workshop NeurIPS 2021. 2021.

• [Frontiers in big Data (2022)] Liang, Yuchen, Dmitry Krotov, and Mohammed J. Zaki. "Modern Hopfield Networks for graph embedding." Frontiers in big Data 5 (2022): 1044709. [Paper]

• Seidl, Philipp, et al. "Modern hopfield networks for few-and zero-shot reaction template prediction." arXiv preprint arXiv:2104.03279 (2021).

• [Workshop NeurIPS 2021] Modern Hopfield Networks for Sample-Efficient Return Decomposition from Demonstrations, [Paper]