Add docs for dqn.py

[vwxyzjn]

Description

This PR adds documentation for dqn.py and couple documentation fixes for dqn_atari.py

Types of changes

• Documentation

Checklist:

• I've read the CONTRIBUTION guide (required).
• I have ensured pre-commit run --all-files passes (required).
• I have updated the documentation and previewed the changes via mkdocs serve.
• I have updated the tests accordingly (if applicable).

If you are adding new algorithms or your change could result in performance difference, you may need to (re-)run tracked experiments. See #137 as an example PR.

• I have contacted @vwxyzjn to obtain access to the openrlbenchmark W&B team (required).
• I have tracked applicable experiments in openrlbenchmark/cleanrl with --capture-video flag toggled on (required).
• I have added additional documentation and previewed the changes via mkdocs serve.
  • I have explained note-worthy implementation details.
  • I have explained the logged metrics.
  • I have added links to the original paper and related papers (if applicable).
  • I have added links to the PR related to the algorithm.
  • I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
  • I have added the learning curves (in PNG format with width=500 and height=300).
  • I have added links to the tracked experiments.
• I have updated the tests accordingly (if applicable).

[gitpod-io]

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/vwxyzjn/cleanrl/AuRtE9dc44G88RDWW5CapYTxT3Pe
✅ Preview: https://cleanrl-git-dqnpy-docs-addition-vwxyzjn.vercel.app

[vwxyzjn]

This PR is ready for review.

[vwxyzjn]

Additional note: I realized it would be pretty hard to run benchmark experiments based on some master's version. From now on, we should do something like

gh pr checkout 157
poetry install

in the reproducibility instructions.

[dosssman]

Looking good over here too.
I noticed there is no dqn_atari.py benchmarks in the docs though. Will it be alright ?

[vwxyzjn]

Looking good over here too. I noticed there is no dqn_atari.py benchmarks in the docs though. Will it be alright ?

The dqn_atari.py docs are already added from #124.

[vwxyzjn]

Let me know if this addresses your concern :)