Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calling sort_and_finalize_staged_data with staged segment of DataFrame with 0 rows fails #1736

Closed
Tracked by #1679
vasil-pashov opened this issue Aug 1, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@vasil-pashov
Copy link
Collaborator

vasil-pashov commented Aug 1, 2024

Describe the bug
When the staging area is consisted of DataFrames without any rows calling sort_and_finalize_staged_data fails.

Steps to reproduce

import pandas as pd
import numpy as np
import arcticdb as adb

ac = adb.Arctic("lmdb://test")
lib = ac.get_library("test", create_if_missing=True)
df = pd.DataFrame({"col": []}, index=pd.DatetimeIndex([]))
lib.write("sym", df, staged=True)
lib.sort_and_finalize_staged_data("sym")

Ouput:

Traceback (most recent call last):
  File "...test.py", line 8, in <module>
    lib.write("sym", df, staged=True)
  File "...arcticdb\version_store\library.py", line 461, in write
    return self._nvs.write(
  File "...arcticdb\version_store\_store.py", line 583, in write
    self.version_store.write_parallel(symbol, item, norm_meta, udm)
arcticdb_ext.exceptions.InternalException: E_ASSERTION_FAILURE Allocate data called with zero size

Expected behavior
This should succeed and be a noop.

Related issues
This could be related to: #1735, #1737

OS, Python, Arctic versions

Python: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
OS: Windows-10-10.0.22631-SP0
ArcticDB: 4.5.0rc1
@vasil-pashov vasil-pashov changed the title Dataframe without any rows (but existing columns) fails with E_ASSERTION_FAILURE Allocate data called with zero size Calling sort_and_finalize_staged_data with staged segment of DataFrame with 0 rows fails Aug 1, 2024
@vasil-pashov vasil-pashov self-assigned this Aug 1, 2024
@vasil-pashov vasil-pashov added the bug Something isn't working label Aug 1, 2024
vasil-pashov added a commit that referenced this issue Aug 21, 2024
#### Reference Issues/PRs
Fixes: #1737 #1736 #1735

#### What does this implement or fix?

#### Any other comments?

#### Checklist

<details>
  <summary>
   Checklist for code changes...
  </summary>
 
- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
vasil-pashov added a commit that referenced this issue Sep 30, 2024
Fixes: #1737 #1736 #1735

<details>
  <summary>
   Checklist for code changes...
  </summary>

- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
vasil-pashov added a commit that referenced this issue Sep 30, 2024
Fixes: #1737 #1736 #1735

<details>
  <summary>
   Checklist for code changes...
  </summary>

- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->

---------

Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant