Skip to content

@llm-jp llm-jp

Change the repository type filter

All

    Repositories list

    38 repositories

    • tuning-competition

      Public
      CSS
      •0•0•0•0•Updated Nov 18, 2024Nov 18, 2024

    • llm-jp-eval

      Public
      Python
      •
      Apache License 2.0
      •38•100•4•6•Updated Nov 18, 2024Nov 18, 2024

    • awesome-japanese-llm

      Public
      日本語LLMまとめ - Overview of Japanese LLMs
      japanesegenerative-modeljapanese-languagelanguage-modelslanguage-modelgenerative-modelsmultimodalvision-and-languagevision-languagefoundation-models
      TypeScript
      •
      Apache License 2.0
      •31•1k•2•0•Updated Nov 16, 2024Nov 16, 2024

    • FastChat2

      Public
      An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
      Python
      •
      Apache License 2.0
      •4.6k•0•0•1•Updated Nov 15, 2024Nov 15, 2024

    • text2dataset

      Public
      Easily turn large English text datasets into Japanese text datasets using open LLMs.
      Python
      •
      MIT License
      •0•14•6•0•Updated Nov 14, 2024Nov 14, 2024

    • scripts

      Public
      Shell
      •
      Apache License 2.0
      •1•1•4•9•Updated Nov 11, 2024Nov 11, 2024

    • Toxicity_Filter_fastText

      Public
      A fastText-based classifier for removing toxic texts from corpora, trained on automatically generated labeled data.
      Python
      •0•0•0•0•Updated Nov 9, 2024Nov 9, 2024

    • Megatron-LM

      Public
      Python
      •
      Other
      •0•3•0•1•Updated Nov 7, 2024Nov 7, 2024

    • Toxicity_Filter

      Public
      A DeBERTa-based toxicity filter trained on LLM-jp Toxicity Dataset
      Python
      •0•0•0•0•Updated Nov 5, 2024Nov 5, 2024

    • llm-jp-data-contamination

      Public
      The data contamination repository for the LLM-JP project. Forked from @eiei7
      Python
      •0•1•0•0•Updated Nov 3, 2024Nov 3, 2024

    • nii-sakura-cluster-configs

      Public
      Shell
      •0•3•0•0•Updated Oct 26, 2024Oct 26, 2024

    • moe-recipes

      Public
      Shell
      •0•1•0•0•Updated Oct 24, 2024Oct 24, 2024

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •27k•0•0•0•Updated Oct 14, 2024Oct 14, 2024

    • llm-leaderboard

      Public
      Project of llm evaluation to Japanese tasks
      Python
      •36•0•0•0•Updated Aug 11, 2024Aug 11, 2024

    • FastChat

      Public
      An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
      Python
      •
      Apache License 2.0
      •4.6k•0•0•0•Updated Aug 9, 2024Aug 9, 2024

    • experiments

      Public
      Issue-Only Pretrain Task Management Repository
      0•0•50•0•Updated Aug 8, 2024Aug 8, 2024

    • llm-jp-tokenizer

      Public
      Roff
      •
      Apache License 2.0
      •8•31•1•0•Updated Aug 7, 2024Aug 7, 2024

    • llmjp_topic_model

      Public
      Python
      •0•0•0•0•Updated Jul 19, 2024Jul 19, 2024

    • modelwg

      Public
      LLM-jp Model-WG Working Directory
      Shell
      •
      Apache License 2.0
      •0•6•0•4•Updated Jul 9, 2024Jul 9, 2024

    • DeepSpeed-MII

      Public
      MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
      Python
      •
      Apache License 2.0
      •175•0•0•0•Updated Jul 5, 2024Jul 5, 2024

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •4.6k•0•0•0•Updated Jun 21, 2024Jun 21, 2024

    • nii-geniac-gcp-lustre

      Public
      GENIAC用のLustreクラスタ構築スクリプト
      Jinja
      •0•0•0•0•Updated Jun 20, 2024Jun 20, 2024

    • nii-geniac-hpc-toolkit

      Public
      HCL
      •0•1•0•0•Updated Jun 13, 2024Jun 13, 2024

    • llm-jp-sft

      Public
      Python
      •
      Apache License 2.0
      •15•51•0•1•Updated Jun 13, 2024Jun 13, 2024

    • memorization-analysis

      Public
      Python
      •0•0•3•1•Updated Jun 7, 2024Jun 7, 2024

    • nii-geniac-gcp-maintenance

      Public
      Shell
      •0•0•0•0•Updated Jun 7, 2024Jun 7, 2024

    • open-llm-leaderboard-local

      Public
      Open LLM Leaderboard のローカル実行用スクリプト
      Python
      •1•3•0•1•Updated Feb 24, 2024Feb 24, 2024

    • llm-jp-model-playground

      Public
      Interactive application to verify multiple LLMs
      TypeScript
      •
      MIT No Attribution
      •2•13•11•2•Updated Feb 20, 2024Feb 20, 2024

    • llm-jp-dpo

      Public
      Python
      •
      Apache License 2.0
      •3•5•0•0•Updated Feb 6, 2024Feb 6, 2024

    • llm-jp-corpus

      Public
      Python
      •
      Apache License 2.0
      •4•41•3•0•Updated Feb 2, 2024Feb 2, 2024