Skip to content

Latest commit

 

History

History
88 lines (77 loc) · 2.8 KB

README.md

File metadata and controls

88 lines (77 loc) · 2.8 KB
Raw

LLMmap (Fingerprinting For Large Language Models)

Basic functionalities implemented. Better and wider range of options will be added as the project evolves (sorry for all this latency).

Requiriments

pip install -r requirements.txt

Download pre-trained models

At the moment, we provide two open and closed-set pre-trained models for the supported LLMs below.

To use the models:

  • Download the archive here.
  • unzip data.zip inside the repository.

To test out the models, use the main_interactive.py script:

usage: main_interactive.py [-h] [--inference_model_path INFERENCE_MODEL_PATH] [--gpus GPUS]

Interactive session for LLM fingeprinting

options:
  -h, --help            show this help message and exit
  --inference_model_path INFERENCE_MODEL_PATH
                        Path inference model to use
  --gpus GPUS           Comma-separated list of GPUs to use (e.g., "0,1,2,3")

Supported LLMs

  • CohereForAI/aya-23-35B
  • CohereForAI/aya-23-8B
  • Deci/DeciLM-7B-instruct
  • HuggingFaceH4/zephyr-7b-beta
  • NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
  • Qwen/Qwen2-1.5B-Instruct
  • Qwen/Qwen2-72B-Instruct
  • Qwen/Qwen2-7B-Instruct
  • abacusai/Smaug-Llama-3-70B-Instruct
  • claude-3-5-sonnet-20240620
  • claude-3-haiku-20240307
  • claude-3-opus-20240229
  • google/gemma-1.1-2b-it
  • google/gemma-1.1-7b-it
  • google/gemma-2-27b-it
  • google/gemma-2-9b-it
  • google/gemma-2b-it
  • google/gemma-7b-it
  • gpt-3.5-turbo
  • gpt-4-turbo-2024-04-09
  • gpt-4o-2024-05-13
  • gradientai/Llama-3-8B-Instruct-Gradient-1048k
  • internlm/internlm2_5-7b-chat
  • meta-llama/Llama-2-7b-chat-hf
  • meta-llama/Meta-Llama-3-70B-Instruct
  • meta-llama/Meta-Llama-3-8B-Instruct
  • meta-llama/Meta-Llama-3.1-70B-Instruct
  • meta-llama/Meta-Llama-3.1-8B-Instruct
  • microsoft/Phi-3-medium-128k-instruct
  • microsoft/Phi-3-medium-4k-instruct
  • microsoft/Phi-3-mini-128k-instruct
  • microsoft/Phi-3-mini-4k-instruct
  • microsoft/Phi-3.5-MoE-instruct
  • mistralai/Mistral-7B-Instruct-v0.1
  • mistralai/Mistral-7B-Instruct-v0.2
  • mistralai/Mistral-7B-Instruct-v0.3
  • mistralai/Mixtral-8x7B-Instruct-v0.1
  • nvidia/Llama3-ChatQA-1.5-8B
  • openchat/openchat-3.6-8b-20240522
  • openchat/openchat_3.5
  • togethercomputer/Llama-2-7B-32K-Instruct
  • upstage/SOLAR-10.7B-Instruct-v1.0

Paper

Paper available here. To cite it:

@misc{pasquini2024llmmapfingerprintinglargelanguage,
      title={LLMmap: Fingerprinting For Large Language Models}, 
      author={Dario Pasquini and Evgenios M. Kornaropoulos and Giuseppe Ateniese},
      year={2024},
      eprint={2407.15847},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2407.15847}, 
}