Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add base model qlora fintuning config file and optimize deduplicate.py #128

Merged
merged 8 commits into from
Mar 23, 2024

Conversation

chg0901
Copy link
Collaborator

@chg0901 chg0901 commented Mar 23, 2024

optimize deduplicate.py

  1. Add time print information
  2. save duplicate dataset as well
  3. remove print(content)

Terminal print information in seconds

(tf2_9) D:\github_repos\EmoLLM\datasets>python deduplicate.py
file name: ./FatherLikeBF/qwen9747_zhipuai8857_final_merge_all.json
print_len_seen_hashes=500 Time:  15.8351 15.8351
print_len_seen_hashes=1000 Time:  40.83554 56.67064
print_len_duplicate=500 Time:  89.68914 89.68914
print_len_seen_hashes=1500 Time:  87.62722 144.29787
print_len_duplicate=1000 Time:  105.06385 194.75299
print_len_duplicate=1500 Time:  120.15144 314.90443
print_len_seen_hashes=2000 Time:  199.31009 343.60795
print_len_duplicate=2000 Time:  146.22297 461.1274

add base model qlora fintuning config file: internlm2_7b_base_qlora_e10

add full finetune code from internlm2

other 2 configs for base model

update cli_internlm2.py with three methods to download or load model

  1. download model in openxlab (6-8M/s)
  2. download model in modelscope (20-30M/s)
  3. offline model

create upload_modelscope.py

add base model and update personal contributions

Create README_internlm2_7b_base_qlora.md

InternLM2 7B Base QLoRA fine-tuning guide

Add time print information
save duplicate dataset as well
remove print(content)
 three methods to load model

1. download model in openxlab
2. download model in modelscope
3. offline model
InternLM2 7B Base QLoRA 微调指南
@aJupyter aJupyter changed the base branch from main to dev March 23, 2024 11:17
@aJupyter aJupyter merged commit a12a7ef into SmartFlowAI:dev Mar 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants