Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Libraries written by 4.5.0 are not backwards compatible for s3 https stоrages #1846

Closed
G-D-Petrov opened this issue Sep 24, 2024 · 0 comments · Fixed by #1840
Closed

Libraries written by 4.5.0 are not backwards compatible for s3 https stоrages #1846

G-D-Petrov opened this issue Sep 24, 2024 · 0 comments · Fixed by #1840
Assignees
@G-D-Petrov
Labels
bug Something isn't working

Comments

@G-D-Petrov
Copy link
Collaborator

Describe the bug

Client using 4.5.0 are no longer writing the https flag in the library config, which is needed by the older clients to correctly for their http requests.

We need to change it, so:

  • the https' value is written when creating the library - so the libraries are backwards compatible and can be read by older clients
  • the https' flag is overridden when the config is read - so the newer clients can read libraries created by 4.5.0

Steps/Code to Reproduce

ac = Arctic("s3s://...")
lib = ac.create_library("test")
lib.write("sym", pd.DataFrame())

config = ac._library_manager.get_library_config("test")
s3_storage = get_s3_storage_config(config)
assert s3_storage.https

Expected Results

The assert check should succeed as the https flag's value should reflect that the library was created with s3s/https in mind.

OS, Python Version and ArcticDB Version

Python: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
OS: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35 ArcticDB: 4.5.0

Backend storage used

S3

Additional Context

No response
@G-D-Petrov G-D-Petrov added the bug Something isn't working label Sep 24, 2024
@G-D-Petrov G-D-Petrov self-assigned this Sep 24, 2024
This was referenced Sep 24, 2024
Merged
Merged
@G-D-Petrov G-D-Petrov changed the title Libraries written by 4.5.0 are not backwards compatible for s3 https stprages Libraries written by 4.5.0 are not backwards compatible for s3 https stоrages Sep 24, 2024
G-D-Petrov added a commit that referenced this issue Sep 24, 2024
@G-D-Petrov
Fix bug where libraries created with 4.5.0 are unreadable by newer ve…
44a5a67 
…rsions (#1842)

#### Reference Issues/PRs
Needed to fix #1846 on 4.5.1rc

#### What does this implement or fix?
Fixes a bug where newer versions are not able to read libraries created
by 4.5.0

#### Any other comments?

#### Checklist

<details>
  <summary>
   Checklist for code changes...
  </summary>
 
- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->
G-D-Petrov added a commit that referenced this issue Sep 30, 2024
@G-D-Petrov
Don't override the https value for backwards compatibility (#1840)
70400ec 
#### Reference Issues/PRs
Fixes #1846 on master

#### What does this implement or fix?
Overriding the value also wipes it from the storage config.
Older clients have no way of overriding the value and they rely on the
value in storage.
So don't override it for now, as it breaks backwards compatibility.

#### Any other comments?
I've tested the change manually and libraries created after the change
can be read correctly by older versions.

#### Checklist

<details>
  <summary>
   Checklist for code changes...
  </summary>
 
- [ ] Have you updated the relevant docstrings, documentation and
copyright notice?
- [ ] Is this contribution tested against [all ArcticDB's
features](../docs/mkdocs/docs/technical/contributing.md)?
- [ ] Do all exceptions introduced raise appropriate [error
messages](https://docs.arcticdb.io/error_messages/)?
 - [ ] Are API changes highlighted in the PR description?
- [ ] Is the PR labelled as enhancement or bug so it appears in
autogenerated release notes?
</details>

<!--
Thanks for contributing a Pull Request to ArcticDB! Please ensure you
have taken a look at:
- ArcticDB's Code of Conduct:
https://github.com/man-group/ArcticDB/blob/master/CODE_OF_CONDUCT.md
- ArcticDB's Contribution Licensing:
https://github.com/man-group/ArcticDB/blob/master/docs/mkdocs/docs/technical/contributing.md#contribution-licensing
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@G-D-Petrov G-D-Petrov

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't override the https value for backwards compatibility man-group/ArcticDB
1 participant
@G-D-Petrov