Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Commit

Permalink
[docs] Add FAQ question about evaling fixed responses (#4120)
Browse files Browse the repository at this point in the history
  • Loading branch information
stephenroller authored Oct 28, 2021
1 parent 86aa8e4 commit 9f0c8d5
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions docs/source/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,19 @@ correctly set. When loading a pretrained checkpoint, all of the parameters for
the model itself will be loaded from the model's `.opt` file, but all
task-specific parameters will need to be re-specified.

If results differ by a few small decimal places, this can often be attributed
to differences in hardware or software environment.

## I want to generate a lot of responses to fixed utterances

The easiest way to do this is to [create a
teacher](tutorial_task) in ParlAI Dialog Format. Then, use
`eval_model` with world logging to store all the responses:

```
parlai eval_model -t fromfile:parlaiformat --fromfile-datapath yourtextfile.txt \
-mf some_model_file --world-logs outputfile
```

## Why is my generative model's perplexity so high (>1000) when evaluating?

Expand Down

0 comments on commit 9f0c8d5

Please sign in to comment.