Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Inputs handle non-aligned and non-padded buffer #276

Closed
charles-paperman opened this issue Sep 20, 2023 · 3 comments · Fixed by #337
Closed

Make Inputs handle non-aligned and non-padded buffer #276

charles-paperman opened this issue Sep 20, 2023 · 3 comments · Fixed by #337
Assignees
Labels
area: performance Performance improvements type: feature New feature or request
Milestone

Comments

@charles-paperman
Copy link
Collaborator

For some buffer, it is impossible to padd and to aligned without performing a full copy.
The inputs layer could:

  • Detect that the start address is not aligned, build a custom first block to overpass the alignement padding artificially the begining with null byte
  • Detect that the end address is not aligned, build a custom last block in the same spirit.

Additional context
This is useful when you don't have any control on the origin of the data: python or when your data is a substring of something else and you want to avoid copying.

@charles-paperman charles-paperman added the type: feature New feature or request label Sep 20, 2023
@github-actions github-actions bot added the acceptance: triage Waiting for owner's input label Sep 20, 2023
@github-actions
Copy link

Tagging @V0ldek for notifications

@V0ldek
Copy link
Member

V0ldek commented Sep 20, 2023

This is doable, but overhead has to be measured.

We already have proof-of-concept code for handling the last block separately when it comes to padding. We could split off the first block and treat it separately as well. The question is how big of a performance impact this would have on small inputs, so we need some small benches to test this.

@V0ldek V0ldek added this to the v1.0.0 milestone Sep 20, 2023
@github-actions github-actions bot added acceptance: go ahead Reviewed, implementation can start and removed acceptance: triage Waiting for owner's input labels Sep 20, 2023
@V0ldek V0ldek added mod: engine area: performance Performance improvements labels Sep 20, 2023
@V0ldek V0ldek removed the mod: engine label Oct 4, 2023
@V0ldek V0ldek self-assigned this Oct 5, 2023
@V0ldek V0ldek moved this from Todo to In Progress in Active rsonpath development Oct 5, 2023
@V0ldek V0ldek mentioned this issue Oct 11, 2023
3 tasks
@V0ldek V0ldek mentioned this issue Oct 29, 2023
3 tasks
V0ldek added a commit that referenced this issue Oct 30, 2023
- Padding and alignment is now handled automatically by the input types,
allowing them to work safely without copying the entire input. The overhead is now
limited to the padding, which is at most 256 bytes in total.
- [`BorrowedBytes`](https://docs.rs/rsonpath-lib/0.8.4/rsonpath/input/borrowed/struct.BorrowedBytes.html) is now safe to construct.
- [`OwnedBytes`](https://docs.rs/rsonpath-lib/0.8.4/rsonpath/input/owned/struct.OwnedBytes.html) no longer copies
the entire source on construction.

Ref: #276
@github-project-automation github-project-automation bot moved this from In Progress to Merged in Active rsonpath development Oct 30, 2023
@github-actions github-actions bot removed the acceptance: go ahead Reviewed, implementation can start label Oct 30, 2023
@V0ldek
Copy link
Member

V0ldek commented Oct 30, 2023

Released in v0.8.4

@V0ldek V0ldek moved this from Merged to Released in Active rsonpath development Oct 30, 2023
V0ldek added a commit that referenced this issue Oct 30, 2023
- Padding and alignment is now handled automatically by the input types,
allowing them to work safely without copying the entire input. The overhead is now
limited to the padding, which is at most 256 bytes in total.
- [`BorrowedBytes`](https://docs.rs/rsonpath-lib/0.8.4/rsonpath/input/borrowed/struct.BorrowedBytes.html) is now safe to construct.
- [`OwnedBytes`](https://docs.rs/rsonpath-lib/0.8.4/rsonpath/input/owned/struct.OwnedBytes.html) no longer copies
the entire source on construction.

Ref: #276
@V0ldek V0ldek moved this from Todo to Released in Active rq development Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: performance Performance improvements type: feature New feature or request
Projects
Status: Released
Development

Successfully merging a pull request may close this issue.

2 participants