Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edit boundary blocks on the server side to drop non-requested data #238

Open
brainstorm opened this issue Apr 10, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@brainstorm
Copy link
Member

Currently the htsget protocol assumes that some returned data blocks will contain reads that were not requested. This was an accepted trade-off in the spec but for some privacy usecases, those extra bytes should be dropped on the server side instead of the current situation where (presumably) the client just ignores those. From @mmalenic's input on the topic:

Basically, this occurs because the htsget-rs server only calculates byte ranges based on the index file (e.g .cram.crai). Since the index file does not contain all possible positions of the file that can be sliced, the server returns the smallest byte ranges that include the request. However, this means that often there are additional bytes included that aren't part of the request.

To address this we could have a strict mode which which would allow the htsget-rs server to inspect the file and only return the data that is required by the request. This would ensure that no additional data is present past the requested reference name, start and end regions.

@brainstorm brainstorm added the enhancement New feature or request label Apr 15, 2024
@mmalenic
Copy link
Member

Noting here that this doesn't only apply to Crypt4GH. It also applies to the bytes returned in BGZF files, which need to align to BGZF blocks.

@mmalenic mmalenic changed the title Decrypt and re-encrypt boundary blocks on the server side to drop non-requested data Edit boundary blocks on the server side to drop non-requested data May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants