Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
klshuster authored Aug 2, 2021
1 parent 72a404b commit 3b90417
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion projects/hallucination/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ The following commands demonstrate how to train some of the models above; we int
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a T5-Large RAG-Turn Doc-Then-Turn model with DPR Retrieval on WoW

Expand All @@ -98,6 +99,7 @@ The following commands demonstrate how to train some of the models above; we int
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a BlenderBot-2.7B RAG Sequence Model with DPR-Poly Retrieval on WoW

Expand All @@ -113,6 +115,7 @@ For the BlenderBot model, we add extra positions to the encoder, so that we can
--model-parallel True --optimizer adam \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a BART-Large FiD Model, with a DPR Retriever initialized from a DPR Model trained with RAG.

Expand All @@ -127,6 +130,7 @@ This is the **BART FiD RAG** model specified above.
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

#### Train a T5-Base FiD Model, using a smaller index for debug purposes.

Expand All @@ -141,4 +145,5 @@ We provide a smaller FAISS index comprising ~3k documents, which encompasses all
--log-every-n-secs 30 --lr-scheduler reduceonplateau --lr-scheduler-patience 1 \
--model-parallel True --optimizer adam --text-truncate 512 --truncate 512 \
--learningrate 1e-05 --validation-metric-mode min --validation-every-n-epochs 0.25 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--validation-max-exs 1000 --validation-metric ppl --validation-patience 5 \
--model-file /path/to/saved/model

0 comments on commit 3b90417

Please sign in to comment.