compact_incomplete with append=True should fail if dynamic schema is false and incomplete segment fields do not exactly match the existing fields
#1466
Assignees
Labels
bug
Something isn't working
Comments
alexowens90
added a commit
that referenced
this issue
Apr 9, 2024
10 tasks
vasil-pashov
added a commit
that referenced
this issue
Sep 30, 2024
<!--Example: Fixes #1234. See also #3456.--> Fixes #1738 Fixes #1781 Fixes #1466 Fixes #1795 Fixes #1797 Fixes #1807 Fixes #1828 A notable change is that staged writes no longer validate the index is sorted. The validation is done at the moment compact_incompletes/finalize_staged_data/sort_and_finalize_staged_data is called. This is because sort_and_finalize_staged_data does not require the segments to be sorted, but the call for adding a staged segment is the same. We should add a separate call for that. Note also that all incomplete keys for a symbol are deleted if any of the finalize calls fail. The other option is to leave the segments. In that case the user will have the responsibility of calling `delete_staged_data`. <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing --> --------- Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
vasil-pashov
added a commit
that referenced
this issue
Sep 30, 2024
<!--Example: Fixes #1234. See also #3456.--> Fixes #1738 Fixes #1781 Fixes #1466 Fixes #1795 Fixes #1797 Fixes #1807 Fixes #1828 A notable change is that staged writes no longer validate the index is sorted. The validation is done at the moment compact_incompletes/finalize_staged_data/sort_and_finalize_staged_data is called. This is because sort_and_finalize_staged_data does not require the segments to be sorted, but the call for adding a staged segment is the same. We should add a separate call for that. Note also that all incomplete keys for a symbol are deleted if any of the finalize calls fail. The other option is to leave the segments. In that case the user will have the responsibility of calling `delete_staged_data`. <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details> <!-- Thanks for contributing a Pull Request to ArcticDB! Please ensure you have taken a look at: - ArcticDB's Code of Conduct: https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md - ArcticDB's Contribution Licensing: https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing --> --------- Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: