Feature/improvements batch #1181
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 8 commits
July 20, 2024 16:24
1. apply_array_func for applying array operations recursively. Use it in to_numpy and to_torch 2. isnull, hasnull, dropnull 3. set_array_at_key for setting a subarray at a desired index inplace Added extensive tests for the new methods
A typo led to None instead of arr being returned
|
@dantp-ai Pls have a look if you have time, you also worked a lot with Batch in the past |
|
When review is done I'd extend the PR description and add a changelog entry. With this and buffer improvements merged, I think we should release version 1.1 |
added 3 commits
July 30, 2024 12:49
Removed part of the tests of cat_ that were handling incompatible batches
Trinkle23897
approved these changes
Jul 31, 2024
added 2 commits
July 31, 2024 19:14
needed now due to the stricter Batch.cat_
|
Thanks for the review @Trinkle23897 ! Some more fixes in tests were needed, now CI should run through. I'd merge then |
|
I'll merge now and will update the changelog later |
dantp-ai
added a commit
to dantp-ai/tianshou
that referenced
this pull request
Aug 2, 2024
* Seems that batch slicing leads to slightly different floats some of the time (See: thu-ml#1181)
This was referenced Aug 2, 2024
MischaPanch
pushed a commit
that referenced
this pull request
Aug 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains several important extensions and improvements of Batch.
isnull,hasnullanddropnull, which helps finding errors early.schemafrom a batch. This was used inBatch.cat_to perform additional input validation (we now make sure there that the structures are the same when concatenating batches). This input validation is a breaking change! Some tests that concatenated incompatible batches were removed. Eventually, we can add aget_schemamethod to the batch that will retrieve metainfo like shapes and datatypes. For now, this is delegated to the user who can use the newapply_values_transformDistributionwhich were not properly sliced, inviting bugs and errors in user code. This is now fixed - albeit not in a pretty way (torch doesn't allow slicing the objects natively)The new code was extensively tested and documented.