refactor scripts/finetune.py into new cli modules #550
Conversation
|
|
||
| validate_config(cfg) | ||
|
|
||
| normalize_config(cfg) |
There was a problem hiding this comment.
Maybe this name should be setup_config? normalize sounds confusing
| print(tokenizer.decode(generated["sequences"].cpu().tolist()[0])) | ||
|
|
||
|
|
||
| def choose_config(path: Path): |
There was a problem hiding this comment.
Should this be removed? I think this causes problem on multi-gpu.
| streamer=streamer, | ||
| ) | ||
| print("=" * 40) | ||
| print(tokenizer.decode(generated["sequences"].cpu().tolist()[0])) |
There was a problem hiding this comment.
can we skip special tokens here?
|
|
||
| LOG.info("running merge of LoRA with base model") | ||
| model = model.merge_and_unload() | ||
| model.to(dtype=torch.float16) |
There was a problem hiding this comment.
this should be set to bf16 if set, else fp16. We can close the other PR that deals with this.
| cli_args: TrainerCliArgs, | ||
| ): | ||
| model, tokenizer = load_model_and_tokenizer(cfg=cfg, cli_args=cli_args) | ||
| safe_serialization = cfg.save_safetensors is True |
There was a problem hiding this comment.
This should be moved to the "normalize config" function.
| cli_args: TrainerCliArgs, | ||
| ): | ||
| model, _ = load_model_and_tokenizer(cfg=cfg, cli_args=cli_args) | ||
| safe_serialization = cfg.save_safetensors is True |
There was a problem hiding this comment.
Same here. If moved to "normalize config", this can be changed.
| dataset_meta = load_datasets(cfg=parsed_cfg, cli_args=parsed_cli_args) | ||
| if parsed_cli_args.prepare_ds_only: | ||
| return | ||
| train(cfg=parsed_cfg, cli_args=parsed_cli_args, dataset_meta=dataset_meta) |
There was a problem hiding this comment.
safe tensor fix inside train.py also.
| ) | ||
|
|
||
| dataset_meta = load_datasets(cfg=parsed_cfg, cli_args=parsed_cli_args) | ||
| if parsed_cli_args.prepare_ds_only: |
There was a problem hiding this comment.
Should this be a separate command?
Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
* refactor scripts/finetune.py into new cli modules * continue to support scripts/finetune.py * update readme with updated cli commands * Update scripts/finetune.py Co-authored-by: NanoCode012 <kevinvong@rocketmail.com> --------- Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
WIP