Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify the definition of AbstractTraces #14

Merged

Conversation

findmyway
Copy link
Member

No description provided.

@codecov-commenter
Copy link

codecov-commenter commented May 22, 2022

Codecov Report

Merging #14 (344b3b6) into main (c2db98d) will increase coverage by 50.66%.
The diff coverage is 78.35%.

@@             Coverage Diff             @@
##             main      #14       +/-   ##
===========================================
+ Coverage   30.06%   80.73%   +50.66%     
===========================================
  Files           9        8        -1     
  Lines         316      301       -15     
===========================================
+ Hits           95      243      +148     
+ Misses        221       58      -163     
Impacted Files Coverage Δ
src/trajectory.jl 80.76% <0.00%> (-10.90%) ⬇️
src/traces.jl 83.91% <80.99%> (-1.80%) ⬇️
src/common/CircularArraySARTTraces.jl 100.00% <100.00%> (+100.00%) ⬆️
src/common/CircularArraySLARTTraces.jl 100.00% <100.00%> (+100.00%) ⬆️
src/patch.jl 100.00% <100.00%> (ø)
src/samplers.jl 100.00% <100.00%> (ø)
src/common/sum_tree.jl 66.21% <0.00%> (+66.21%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2db98d...344b3b6. Read the comment docs.

@findmyway findmyway requested a review from HenriDeh May 22, 2022 13:26
@findmyway
Copy link
Member Author

This might affect #12 .

The main change here is in the traces.jl file. I replaced the Trace with LastDimSlices. Generally speaking, each trace now can be any AbstractVector.

@findmyway findmyway marked this pull request as draft May 23, 2022 02:19
src/episodes.jl Outdated Show resolved Hide resolved
Copy link
Member

@HenriDeh HenriDeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these changes are not too problematic for #12.

  • If I understand well the new Trace, the idea is that the container (parent) is, for example, a CircularArrayBuffer, but Trace make it look like a Vector and will return a view when fetched. I think an example in the docstring would help understand this better. I would also add unit tests that combine both because this will probably be the main use case.
  • I'd do the same for Traces, MultiplexTraces and MergedTraces.
  • I think MultiplexTraces is a really nice addition.
  • I see the point of MergedTraces, it is like a Traces but that can accommodate different types of AbstractTraces. I think in practice, because of MultiplexTraces, it is always MergedTraces that will be used. Theferefore I'm wondering if Traces should not become what MergedTraces is at the moment.

There are plenty of changes in this PR, it's a bit hard to get a hang of the new design just by looking at the code. So I'll wait for this to be merged to check if #12 works well with it. But overall it seems nice. I may have new suggestions in the future after fiddling with this a bit longer.

I think a good addition right now would be unit tests this with non-scalar traces and with episodes. I think this may highlight potential caveats and also serve as a proto-documentation of examples.

@findmyway
Copy link
Member Author

findmyway commented May 23, 2022

I think these changes are not too problematic for #12.

  • If I understand well the new Trace, the idea is that the container (parent) is, for example, a CircularArrayBuffer, but Trace make it look like a Vector and will return a view when fetched. I think an example in the docstring would help understand this better. I would also add unit tests that combine both because this will probably be the main use case.

Sure, I'll add exhaustive tests to make sure the correctness for CircularArrayBuffers.

  • I'd do the same for Traces, MultiplexTraces and MergedTraces.

Will do.

  • I think MultiplexTraces is a really nice addition.

😄

  • I see the point of MergedTraces, it is like a Traces but that can accommodate different types of AbstractTraces. I think in practice, because of MultiplexTraces, it is always MergedTraces that will be used. Theferefore I'm wondering if Traces should not become what MergedTraces is at the moment.

This is worth rethinking. I'll give it a try.

(Done)

There are plenty of changes in this PR, it's a bit hard to get a hang of the new design just by looking at the code. So I'll wait for this to be merged to check if #12 works well with it. But overall it seems nice. I may have new suggestions in the future after fiddling with this a bit longer.

🤗

I think a good addition right now would be unit tests this with non-scalar traces and with episodes. I think this may highlight potential caveats and also serve as a proto-documentation of examples.

OK

@findmyway findmyway changed the title Use StructArrays.jl to avoid duplicate codes Unify the definition of AbstractTraces May 25, 2022
@findmyway findmyway marked this pull request as ready for review May 25, 2022 05:35
Copy link
Member

@HenriDeh HenriDeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good !

src/traces.jl Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
findmyway and others added 3 commits May 25, 2022 18:41
Co-authored-by: Henri Dehaybe <47037088+HenriDeh@users.noreply.github.com>
Co-authored-by: Henri Dehaybe <47037088+HenriDeh@users.noreply.github.com>
@findmyway findmyway merged commit 1bb46a9 into JuliaReinforcementLearning:main May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants