Skip to content

Commit

Permalink
Update paper_topics.txt
Browse files Browse the repository at this point in the history
  • Loading branch information
s-jse authored Mar 1, 2024
1 parent 0d8e454 commit a355213
Showing 1 changed file with 5 additions and 9 deletions.
14 changes: 5 additions & 9 deletions configs/paper_topics.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,15 @@
2. Shows new powerful test set contamination or membership inference methods for language models. Test set contamination is the phenomenon where a language model observes a benchmark dataset during pretraining.
- Relevant: test statistics that can detect contamination of benchmarks in language models. statistics that can provide guarantees are more interesting. membership inference methods that are general enough to apply to language models are also relevant.
- Not relevant: any papers that do not consider language models, or that do not consider test set contamination.
3. Shows a significant advance in the performance of diffusion language models.
- Relevant: papers that study language models that are also diffusion models. Continuous diffusions are even more relevant, while discrete diffusions are less so.
- Not relevant: papers about image diffusions like DALL-E or Stable Diffusion, or papers that do not explicitly mention language models or applications to text.
4. Describes new paradigms to evaluating open-ended text generation. Evaluating the outputs of language models is hard, especially in open-ended settings like for chatbots.
3. Describes new paradigms to evaluating open-ended text generation. Evaluating the outputs of language models is hard, especially in open-ended settings like for chatbots.
- Relevant: papers that fundamentally rethink language model evaluation -- especially by accounting for subjectivity or using adversaries.
- Not relevant: specific evaluations for specific tasks, identifying new properties or flaws of language models, or simply collecting new data.
5. Conducts surveys or provides data into real-world usage and safety properties of language models.
4. Conducts surveys or provides data into real-world usage and safety properties of language models.
- Relevant: papers that create new datasets or surveys on real-world usage of language models.
- Not relevant: papers that apply language models to new real-world tasks.
6. Studies 'scaling laws' in the context of neural networks. Scaling laws refer to the very clear power-law relationship between the size or computational power used to train a model and the performance of that model.
5. Studies 'scaling laws' in the context of neural networks. Scaling laws refer to the very clear power-law relationship between the size or computational power used to train a model and the performance of that model.
- Relevant: theoretical or conceptual explanation behind scaling laws for language models.
- Not relevant: papers that have experiments at different model scales (but do not explicitly fit a scaling law) or papers that mention scaling laws, but the scaling laws are not the central subject of the paper

In suggesting papers to your friend, remember that he enjoys papers on statistical machine learning, and generative modeling in natural language processing.
Your friend also likes learning about surprising empirical results in language models, as well as clever statistical tricks.
He does not want to read papers that are about primarily applications of methods to specific domains.
In suggesting papers to your friend, remember that he likes learning about surprising empirical results in language models, as well as clever practical tricks.
He does not want to read papers that are primarily about applications of methods to the medical or law domains.

0 comments on commit a355213

Please sign in to comment.