Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix/1507/disallow appending row range to datetime index with dynamic schema #1517

Commits on Apr 2, 2024

  1. Implement empty index for 0-rowed columns (#1429)

    #### Reference Issues/PRs
    Closes: #1428 
    
    #### What does this implement or fix?
    Create an empty-index type. This required change in the Python and in
    the C++ layer.
    * In the C++ layer an new index type was added.
    (IndexDescriptor::EMPTY). It does not allocate a filed in the storage
    (similar to how row range index does not allocate a field). The checks
    for index compatibility are relaxed, the empty index is compatible with
    all other index types and it gets overridden the first time a non-empty
    index is written (either through update or append). On write we check if
    the dataframe contains 0 rows and if so it gets assigned an empty index.
    * The logic in the python layer is dodgy and needs discussion. In the
    current state the normalization metadata and the index descriptor are
    stored separately. There is one proto message describing both DateTime
    index and Ranged Index. The current change made it so that in case of 0
    rows the python layer passes RowRange index to the C++ layer which
    checks if there are any rows in the DF. If there are rows Row range
    index is used, otherwise empty index is used. Note the
    `is_not_range_index` proto field. IMO it needs some refactoring in
    further PRs. It's used in the python layer to check if the first column
    is index or not.
    #### Any other comments?
    
    Merge this after: #1436
    
    #### Checklist
    
    <details>
      <summary>
       Checklist for code changes...
      </summary>
     
    - [ ] Have you updated the relevant docstrings, documentation and
    copyright notice?
    - [ ] Is this contribution tested against [all ArcticDB's
    features](../docs/mkdocs/docs/technical/contributing.md)?
    - [ ] Do all exceptions introduced raise appropriate [error
    messages](https://docs.arcticdb.io/error_messages/)?
     - [ ] Are API changes highlighted in the PR description?
    - [ ] Is the PR labelled as enhancement or bug so it appears in
    autogenerated release notes?
    </details>
    
    <!--
    Thanks for contributing a Pull Request to ArcticDB! Please ensure you
    have taken a look at:
    - ArcticDB's Code of Conduct:
    https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
    - ArcticDB's Contribution Licensing:
    https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
    -->
    
    ---------
    
    Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
    vasil-pashov and Vasil Pashov authored Apr 2, 2024
    Configuration menu
    Copy the full SHA
    a14eb48 View commit details
    Browse the repository at this point in the history
  2. Dev/vasil.pashov/index file refactor (#1472)

    Refactor index.hpp
    
    Move the functions in a .cpp file. Add explicit template instantiations
    for the CRTP base class in the cpp file.
    
    #### Checklist
    
    <details>
      <summary>
       Checklist for code changes...
      </summary>
     
    - [ ] Have you updated the relevant docstrings, documentation and
    copyright notice?
    - [ ] Is this contribution tested against [all ArcticDB's
    features](../docs/mkdocs/docs/technical/contributing.md)?
    - [ ] Do all exceptions introduced raise appropriate [error
    messages](https://docs.arcticdb.io/error_messages/)?
     - [ ] Are API changes highlighted in the PR description?
    - [ ] Is the PR labelled as enhancement or bug so it appears in
    autogenerated release notes?
    </details>
    
    <!--
    Thanks for contributing a Pull Request to ArcticDB! Please ensure you
    have taken a look at:
    - ArcticDB's Code of Conduct:
    https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
    - ArcticDB's Contribution Licensing:
    https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
    -->
    
    ---------
    
    Co-authored-by: Vasil Pashov <vasil.pashov@man.com>
    vasil-pashov and Vasil Pashov authored Apr 2, 2024
    Configuration menu
    Copy the full SHA
    e6c4f28 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. Feature flag the empty type using library config

    Vasil Pashov committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    2effe0c View commit details
    Browse the repository at this point in the history
  2. Make empty index default to datetime to preserve behavior

    Vasil Pashov committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    2d1512f View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Feature flag empty index

    Vasil Pashov committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    0e1f11c View commit details
    Browse the repository at this point in the history
  2. Add comments

    Vasil Pashov committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    f8a8a66 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. Configuration menu
    Copy the full SHA
    87ef604 View commit details
    Browse the repository at this point in the history
  2. Resolve review comments

    Vasil Pashov committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    3732304 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. Configuration menu
    Copy the full SHA
    6fbe507 View commit details
    Browse the repository at this point in the history
  2. Bugfix 1507: Prevent appending row-range indexed dfs to datetime inde…

    …xed dfs with dynamic schema
    alexowens90 committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    a550780 View commit details
    Browse the repository at this point in the history