Skip to content

Commit

Permalink
update picture
Browse files Browse the repository at this point in the history
  • Loading branch information
wuzijun committed Sep 25, 2024
1 parent 7c9675b commit 938f951
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 4 deletions.
2 changes: 1 addition & 1 deletion Journey/Day04/Day04.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ litgpt pretrain --config Experiments/configs/microstories.yaml

我的模型其实已经训练了一段时间,show一下训练过程中的图表:

![wandb](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/image.png)
![pretrain_wandb](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/pretrain_wandb.png)

## 小结
1. 详细介绍了`litgpt`的预训练模型配置文件。
Expand Down
2 changes: 1 addition & 1 deletion Journey/Day13/Day13.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ if __name__ == "__main__":

训练过程中的指标波动比较大,简单贴个图:

![](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/image.png)
![](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/dpo_train.png)

`DPO`的原理细节这里就不展开了,大家可以自行参考论文和开源的代码实现。

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,15 @@ Memory: 9347MiB / 64195MiB
```
### 预训练
详细参数请参考 [pretrain.yaml](./Experiments/configs/microstories/pretrain.yaml)
![](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/image.png)
![pretrain_wandb](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/pretrain_wandb.png)

### 指令微调
详细参数请参考 [sft.yaml](./Experiments/configs/microstories/sft.yaml)
![image_v2](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/image_v2.png)

### DPO
训练脚本参考[dpo_train.py](./Journey/Day13/dpo_train.py)
![](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/image.png)
![](https://erxuanyi-1257355350.cos.ap-beijing.myqcloud.com/dpo_train.png)


## 目录结构
Expand Down

0 comments on commit 938f951

Please sign in to comment.