Skip to content

added checksum support CRC32, CRC32C, SHA1, SHA256 and CRC64NVME #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

added checksum support CRC32, CRC32C, SHA1, SHA256 and CRC64NVME #195

wants to merge 2 commits into from

Conversation

@HJLebbink
Copy link
Member

@HJLebbink HJLebbink commented Nov 21, 2025 •
edited
Loading

Adds checksum support (CRC32, CRC32C, SHA1, SHA256, CRC64NVME) with automatic verification and hardware acceleration.

Key Changes

Performance

  • CRC32: 63x faster (2.2ms → 34.6µs per MB) via crc32fast hardware acceleration
  • SHA256: 8% faster via ring with AES-NI by default
  • All algorithms hardware-accelerated where available

Features

  • Automatic checksum verification on downloads
  • Checksum support for PUT, GET, COPY, APPEND operations
  • Multipart upload checksum capture

@HJLebbink HJLebbink requested a review from Copilot November 21, 2025 15:02
@HJLebbink HJLebbink self-assigned this Nov 21, 2025
@HJLebbink HJLebbink added the enhancement Used in release doc generation label Nov 21, 2025
Copilot finished reviewing on behalf of HJLebbink November 21, 2025 15:06

This comment was marked as outdated.

Sign in to view

@HJLebbink HJLebbink force-pushed the feature/hashes branch 2 times, most recently from d878166 to 10bd659 Compare November 21, 2025 15:22
@HJLebbink HJLebbink changed the title added CRC32, CRC32C, SHA1, SHA256 and CRC64NVME added checksum support CRC32, CRC32C, SHA1, SHA256 and CRC64NVME Nov 21, 2025
@HJLebbink HJLebbink requested a review from Copilot November 21, 2025 17:49
Copilot finished reviewing on behalf of HJLebbink November 21, 2025 17:54

This comment was marked as resolved.

Sign in to view

@HJLebbink HJLebbink force-pushed the feature/hashes branch 2 times, most recently from ff6d09c to 2c94f6f Compare November 21, 2025 21:13
@HJLebbink HJLebbink requested a review from Copilot November 21, 2025 21:16
Copilot finished reviewing on behalf of HJLebbink November 21, 2025 21:20

This comment was marked as resolved.

Sign in to view

@HJLebbink HJLebbink requested a review from Copilot November 21, 2025 21:53
Copilot finished reviewing on behalf of HJLebbink November 21, 2025 21:55

This comment was marked as resolved.

Sign in to view

@HJLebbink HJLebbink requested a review from Copilot November 21, 2025 22:21
@HJLebbink HJLebbink marked this pull request as ready for review November 21, 2025 22:22
Copilot finished reviewing on behalf of HJLebbink November 21, 2025 22:25
Copilot AI reviewed Nov 21, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

@HJLebbink HJLebbink force-pushed the feature/hashes branch 4 times, most recently from 0073240 to cb4a746 Compare November 22, 2025 23:35
@HJLebbink HJLebbink requested a review from Copilot November 22, 2025 23:36
Copilot finished reviewing on behalf of HJLebbink November 22, 2025 23:39
Copilot AI reviewed Nov 22, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

@HJLebbink HJLebbink force-pushed the feature/hashes branch 2 times, most recently from 09edad0 to e874d8c Compare November 23, 2025 17:43
@HJLebbink
Fix incorrect part size in multipart copy
23f97c3 
  The bug was using size (remaining bytes) instead of length (actual part size) when constructing PartInfo in the
  multipart copy loop. This would record wrong sizes for each part - especially problematic for the last part.
@HJLebbink
added CRC32, CRC32C, SHA1, SHA256 and CRC64NVME
d52f7a6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@HJLebbink HJLebbink

Labels

enhancement Used in release doc generation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@HJLebbink