Model directory as a parameter to encode.py #23

erickrf · 2019-08-19T21:45:02Z

I included a CLI argument to encode.py allowing to specify the model directory. The default was kept as models.

…ve LF line endings and all files stay unix on commit

Add note about setting PYTHONIOENCODING=UTF-8 env var for running examples

Example will `tee` stdout to `/tmp/samples` from conditional and unconditional generation scripts.

add description for flags

added python download script and modified requirements to add the modules needed. Tested in Windows Version 10.0.17134 Build 17134 and Ubuntu 18.04.1 LTS

This write-up was loosely inspired in part by Mitchell et al.’s work on [Model Cards for Model Reporting](https://arxiv.org/abs/1810.03993). Adding such model usage sections could be good practice in general for open source research projects with potentially broad applications.

This enables multi-GPU or distributed training using Horovod

Added instructions and script for distributed training with Horovod

… are enough contiguous tokens to sample from. This can be disabled with --combine 0.

…w training the 345M model.

Add toposort to requirements

…acting to typos.

…ional_samples.py.

WuTheFWasThat and others added 30 commits February 17, 2019 17:24

update README

5b64684

reorganize and add temp 0.7

6dab221

add license

aae26ab

add conditional samples

fc0ee6d

separate out tensorflow install

825aa3d

shuffle headings

92ce9f2

more warning

bf43e73

instructinos mention git clone

23ed990

Add a Dockerfile and document usage in README

99af6d7

fixed unconditional sampling reproducibility issue

2cf46d9

fixed seed arg to ensure reproducibility in conditional-samples model

946facf

update readme

b6f943d

add conditional samples with default settings

a3aa7de

add .gitattributes file to ensure files copied to docker container ha…

68bf7a0

…ve LF line endings and all files stay unix on commit

Minor: update readme

c5b9c89

Add note about setting PYTHONIOENCODING=UTF-8 env var for running examples

Minor: update readme

c314dda

Example will `tee` stdout to `/tmp/samples` from conditional and unconditional generation scripts.

Add documentation for help flags (nshepperd#81)

ed49f03

add description for flags

slight fix to batch size description

9d1e704

updates

0465394

Add finetuning code.

d1fc873

chmod +x

1fba31f

Add finetuning instructions

dfca3cf

Fix sample generation with batch_size greater than 1.

9423776

Python download script (nshepperd#89)

8eb6793

added python download script and modified requirements to add the modules needed. Tested in Windows Version 10.0.17134 Build 17134 and Ubuntu 18.04.1 LTS

update download stuff

ed0dedc

add contributors md and move dev docs out

79a246a

fix for windows (thanks to chrothenbach)

8637828

Add training script with Horovod support

3e18729

This enables multi-GPU or distributed training using Horovod

Fix typo in train command in README

ec16bad

WuTheFWasThat and others added 29 commits March 18, 2019 14:27

Update CONTRIBUTORS.md

d14501a

Merge pull request nshepperd#2 from tlkh/finetuning

ef62678

Added instructions and script for distributed training with Horovod

autoformat

c465071

Combine input text files with <|endoftext|> delimiter to ensure there…

1e32b10

… are enough contiguous tokens to sample from. This can be disabled with --combine 0.

Write losses to summary file for tensorboard.

3a3ce65

Add learning rate as command line flag.

d5b387b

Use argparse instead of fire in train.py.

b106d0a

Fix encode.py

2044d13

Add gradient accumulation with default of 5 minibatches

a359a34

Merge remote-tracking branch 'origin/master' into finetuning

8738950

Turn off gradient accumulation by default, it shouldn't be needed.

eda8777

updates for 345M model

0503b1b

reference dataset

b5ef71a

remove samples

dd75299

Add gradient checkpointing and another optimization necessary to allo…

47df6da

…w training the 345M model.

Add "validation" loss calculation.

c46ed99

Add toposort to requirements

941a762

Merge pull request nshepperd#3 from Tenoke/finetuning

13c5412

Add toposort to requirements

Add option to use SGD for optimizer

3985cc7

Record learning rate in tensorboard logs

7fc2a44

Add text in README for --optimizer flag

a464925

Reduce default learning rate of train.py.

ae535b6

Merge remote-tracking branch 'origin/master' into finetuning

2d4fd0c

New feature: add noise to network inputs to regularize against overre…

6a77a7b

…acting to typos.

Add top-p sampling

87fe3d7

Add top_p to interactive_conditional_samples.py and generate_uncondit…

e99ee37

…ional_samples.py.

fix typo in top_p

2b24145

Fix top_p sampling for batch_size>1

6c1f21d

Model directory as a parameter to encode.py

2b12ad8

nshepperd force-pushed the finetuning branch from 29ce412 to 89cb310 Compare October 31, 2022 06:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Model directory as a parameter to encode.py #23

Model directory as a parameter to encode.py #23

erickrf commented Aug 19, 2019

Model directory as a parameter to encode.py #23

Are you sure you want to change the base?

Model directory as a parameter to encode.py #23

Conversation

erickrf commented Aug 19, 2019