Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remap action to fit gym's action space #313

Merged
merged 15 commits into from
Mar 21, 2021

Conversation

ChenDRAG
Copy link
Collaborator

See #312 for details.

@ChenDRAG ChenDRAG requested a review from Trinkle23897 March 18, 2021 12:33
@ChenDRAG
Copy link
Collaborator Author

ChenDRAG commented Mar 18, 2021

The validity of this implementation is verified by my version of PPO as stated in #307.
I didn't check tests and docs, so it won't pass tests. Will fix this tomorrow.

@ChenDRAG ChenDRAG marked this pull request as draft March 18, 2021 12:37
@Trinkle23897 Trinkle23897 marked this pull request as ready for review March 19, 2021 13:32
@Trinkle23897 Trinkle23897 requested a review from danagi March 19, 2021 13:33
Copy link
Collaborator

@danagi danagi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the action of SAC mapped in forward and then mapped again in map_action?

@Trinkle23897
Copy link
Collaborator

Is the action of SAC mapped in forward and then mapped again in map_action?

just once, the code in forward is to correct the log_prob for entropy term

danagi
danagi previously approved these changes Mar 19, 2021
Copy link
Collaborator

@danagi danagi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great job

@Trinkle23897 Trinkle23897 linked an issue Mar 20, 2021 that may be closed by this pull request
@codecov-io
Copy link

codecov-io commented Mar 21, 2021

Codecov Report

Merging #313 (ecff62b) into master (0c7117d) will increase coverage by 0.01%.
The diff coverage is 97.22%.

❗ Current head ecff62b differs from pull request most recent head 60dee40. Consider uploading reports for the commit 60dee40 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #313      +/-   ##
==========================================
+ Coverage   93.91%   93.92%   +0.01%     
==========================================
  Files          51       51              
  Lines        3270     3278       +8     
==========================================
+ Hits         3071     3079       +8     
  Misses        199      199              
Flag Coverage Δ
unittests 93.92% <97.22%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
tianshou/policy/modelfree/a2c.py 86.20% <ø> (ø)
tianshou/policy/modelfree/sac.py 84.88% <91.66%> (-0.84%) ⬇️
tianshou/data/collector.py 94.93% <100.00%> (+0.04%) ⬆️
tianshou/policy/base.py 79.56% <100.00%> (+2.32%) ⬆️
tianshou/policy/modelfree/ddpg.py 98.68% <100.00%> (-0.10%) ⬇️
tianshou/policy/modelfree/discrete_sac.py 87.69% <100.00%> (ø)
tianshou/policy/modelfree/pg.py 97.36% <100.00%> (ø)
tianshou/policy/modelfree/ppo.py 97.59% <100.00%> (+1.07%) ⬆️
tianshou/policy/modelfree/td3.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c7117d...60dee40. Read the comment docs.

@Trinkle23897 Trinkle23897 merged commit 4d92952 into thu-ml:master Mar 21, 2021
@ChenDRAG ChenDRAG deleted the clip_action branch March 21, 2021 10:28
@Trinkle23897 Trinkle23897 mentioned this pull request Apr 1, 2021
8 tasks
BFAnas pushed a commit to BFAnas/tianshou that referenced this pull request May 5, 2024
Co-authored-by: Trinkle23897 <trinkle23897@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants