-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backport sort and finalize fixes #1859
Merged
Merged
Commits on Sep 30, 2024
-
Fixes for sort and finalize (#1763)
Fixes: #1737 #1736 #1735 <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing --> --------- Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
Configuration menu - View commit details
-
Copy full SHA for 946cbf9 - Browse repository at this point
Copy the full SHA 946cbf9View commit details -
Fix symbol list bug on compaction (#1324)
- Only adds a symbol list entry on append if there is no previous version. - Fixes `finalize_staged_data` to write to the symbol list. - Adds a python test for the new behavior.
Configuration menu - View commit details
-
Copy full SHA for c430234 - Browse repository at this point
Copy the full SHA c430234View commit details -
More sort and finalize fixes (#1799)
<!--Example: Fixes #1234. See also #3456.--> Fixes #1738 Fixes #1781 Fixes #1466 Fixes #1795 Fixes #1797 Fixes #1807 Fixes #1828 A notable change is that staged writes no longer validate the index is sorted. The validation is done at the moment compact_incompletes/finalize_staged_data/sort_and_finalize_staged_data is called. This is because sort_and_finalize_staged_data does not require the segments to be sorted, but the call for adding a staged segment is the same. We should add a separate call for that. Note also that all incomplete keys for a symbol are deleted if any of the finalize calls fail. The other option is to leave the segments. In that case the user will have the responsibility of calling `delete_staged_data`. <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing --> --------- Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
Configuration menu - View commit details
-
Copy full SHA for 0fe71d6 - Browse repository at this point
Copy the full SHA 0fe71d6View commit details -
Fix row slicing with sort_and_finalize throw on column slicing when t…
…he column group is larger than the segment column size (#1838) #### Reference Issues/PRs <!--Example: Fixes #1234. See also #3456.--> #### What does this implement or fix? Closes #1851 Closes #1848 #### Any other comments? #### Checklist <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing --> --------- Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
Configuration menu - View commit details
-
Copy full SHA for 624b432 - Browse repository at this point
Copy the full SHA 624b432View commit details -
Remove read_dataframe_merged. It was removed with #1698.
Vasil Pashov committedSep 30, 2024 Configuration menu - View commit details
-
Copy full SHA for 8eadb29 - Browse repository at this point
Copy the full SHA 8eadb29View commit details
Commits on Oct 1, 2024
-
Add drop_staged_data_on_failure flag for sort and finalize Add unit tests to make sure it behaves as expected Add drop_staged_data_on_failure flag for compact incompletes Add tests for drop_staged_data_on_failure for compact_incompletes Add documentation for finalize_staged_data and sort_and_finalize_staged_data Stylistic change on checking whether to delete incomplete keys Don't throw in destructor Remove unused headers Rename drop_staged_data_on_failure to delete_staged_data_on_failure Address review comments Fix conda build Removes spaces from `environment_unix.yml` which new versions of mamba are not processing correctly. Fix row slicing with sort_and_finalize throw on column slicing when the column group is larger than the segment column size (#1838) <!--Example: Fixes #1234. See also #3456.--> Closes #1851 Closes #1848 <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing --> --------- Co-authored-by: Vasil Pashov <vasil.pashov@man.com> Fix incomplete segments raii when exception is thrown before read incompletes to pipeline Fix docstrings
Vasil Pashov committedOct 1, 2024 Configuration menu - View commit details
-
Copy full SHA for 79dd396 - Browse repository at this point
Copy the full SHA 79dd396View commit details -
Vasil Pashov committed
Oct 1, 2024 Configuration menu - View commit details
-
Copy full SHA for 9adfa72 - Browse repository at this point
Copy the full SHA 9adfa72View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.