Add parameterized CRC-32 and CRC-64

[bartonjs]

Add parameterized CRC-32 and CRC-64. Currently, they are just done as
byte-by-byte table lookup, except for the Crc32 and Crc64 parameter
sets, which defer to the already-optimized implementations, and CRC-32/C,
which uses x86 and ARM intrinsics.

Generalized vectorization will be done as a followup.

Fixes #123164, #85222, and #78063.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io-hashing, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

Adds support for computing CRC-32 and CRC-64 using caller-specified parameter sets, while keeping the existing optimized implementations for the default CRC-32 / CRC-64 variants (and CRC-32C via intrinsics where available). This expands System.IO.Hashing to cover additional CRC variants requested in #123164.

Changes:

• Introduce Crc32ParameterSet / Crc64ParameterSet types with factory creation and a small set of well-known presets (e.g., CRC-32, CRC-32C, CRC-64, CRC-64/NVME).
• Add parameterized overloads/constructors on Crc32 and Crc64 that accept a parameter set, and route hashing through the parameter set implementation.
• Add extensive new test coverage for parameterized CRC-32/CRC-64 variants and known-answer tests for well-known parameter sets.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32.cs	Adds Crc32ParameterSet constructor/property and parameterized Hash/TryHash/HashToUInt32 overloads; routes computation through parameter set.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc64.cs	Adds Crc64ParameterSet constructor/property and parameterized Hash/TryHash/HashToUInt64 overloads; routes computation through parameter set.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.cs	Introduces core CRC-32 parameter set type (properties, Create, finalization, byte-order write).
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.Table.cs	Implements table-based CRC-32 update for reflected and forward modes.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.WellKnown.cs	Adds well-known CRC-32 and CRC-32C parameter sets, including intrinsic-based CRC-32C path.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc64ParameterSet.cs	Introduces core CRC-64 parameter set type (properties, Create, finalization, byte-order write).
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc64ParameterSet.Table.cs	Implements table-based CRC-64 update for reflected and forward modes.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc64ParameterSet.WellKnown.cs	Adds well-known CRC-64 (ECMA-182) and CRC-64/NVME parameter sets.
src/libraries/System.IO.Hashing/src/System.IO.Hashing.csproj	Adds new parameter set sources to the build.
src/libraries/System.IO.Hashing/ref/System.IO.Hashing.cs	Updates the public reference surface for new parameter set types and overloads.
src/libraries/System.IO.Hashing/tests/System.IO.Hashing.Tests.csproj	Includes new test sources for parameterized CRC testing.
src/libraries/System.IO.Hashing/tests/Crc32Tests.cs	Adds argument-null validation tests for new CRC-32 parameterSet overloads/ctor.
src/libraries/System.IO.Hashing/tests/Crc32ParameterSetTests.cs	Adds known-answer tests for several CRC-32 parameter sets.
src/libraries/System.IO.Hashing/tests/Crc32Tests_Parameterized.cs	Adds parameterized driver-based test framework for CRC-32 parameter sets.
src/libraries/System.IO.Hashing/tests/Crc32Tests_ParameterSet_Crc32.cs	Adds CRC-32 parameter set driver + singleton/value tests.
src/libraries/System.IO.Hashing/tests/Crc32Tests_ParameterSet_Crc32C.cs	Adds CRC-32C parameter set driver + singleton/value tests.
src/libraries/System.IO.Hashing/tests/Crc32Tests_ParameterSet_Custom.cs	Adds custom CRC-32 parameter set drivers using Create(...).
src/libraries/System.IO.Hashing/tests/Crc64ParameterSetTests.cs	Adds known-answer tests for several CRC-64 parameter sets.
src/libraries/System.IO.Hashing/tests/Crc64Tests_Parameterized.cs	Adds parameterized driver-based test framework for CRC-64 parameter sets.
src/libraries/System.IO.Hashing/tests/Crc64Tests_Parameterized_Crc64.cs	Adds CRC-64 parameter set driver + singleton/value tests.
src/libraries/System.IO.Hashing/tests/Crc64Tests_Parameterized_Custom.cs	Adds a custom CRC-64 parameter set driver using Create(...).
src/libraries/System.IO.Hashing/tests/Crc64Tests_Parameterized_Nvme.cs	Adds CRC-64/NVME parameter set driver + singleton/value tests.

[stephentoub]

Code Review: PR #124117 - Add parameterized CRC-32 and CRC-64

Holistic Assessment

Motivation: This PR addresses issue #123164 by adding parameterized CRC-32 and CRC-64 support, allowing users to specify custom polynomials, initial values, final XOR values, and reflection settings. This is a valuable addition as different protocols use different CRC parameters.

Approach: The implementation introduces Crc32ParameterSet and Crc64ParameterSet abstract classes that encapsulate CRC parameters and computation logic. Well-known parameter sets (IEEE 802.3, CRC-32C, ECMA-182, NVMe) are provided as static properties, and custom parameter sets can be created via a factory method. The existing Crc32 and Crc64 classes are extended with constructor and static method overloads that accept parameter sets.

Net Positive: ✅ Yes - This is a well-structured, backward-compatible extension that significantly increases the utility of the hashing library.

---

Detailed Findings

✅ Correctness

1. Known-answer tests: Comprehensive testing against the CRC catalogue at reveng.sourceforge.io with multiple parameter sets (AIXM, AUTOSAR, BZIP2, ISCSI, CKSUM, etc.).

2. Residue verification: Tests properly verify the CRC residue property which validates the algorithm correctness.

3. Optimized paths preserved: The existing vectorized implementations for CRC-32 (IEEE 802.3) and CRC-64 (ECMA-182) are correctly preserved through dedicated parameter set classes that delegate to the optimized Update methods.

4. Hardware acceleration for CRC-32C: x86 SSE4.2 and ARM CRC32 intrinsics are correctly used when available.

⚠️ Issues to Address

1. Missing XML documentation on Crc64ParameterSet.Crc64 and Crc64ParameterSet.Nvme properties (Crc64ParameterSet.WellKnown.cs lines 8-12):
   The Crc32ParameterSet has full XML documentation for its static properties, but Crc64ParameterSet.Crc64 and Crc64ParameterSet.Nvme lack XML doc comments. For API consistency and IntelliSense support, these should be documented similarly to their CRC-32 counterparts.

2. Inconsistent [CLSCompliant(false)] attribute on Crc64.Hash overload (ref/System.IO.Hashing.cs line 63-64):
   The Crc64.Hash(Crc64ParameterSet parameterSet, ReadOnlySpan<byte> source, Span<byte> destination) method has [CLSCompliant(false)] in the ref assembly, but the corresponding Crc32.Hash overload (line 23) does not have this attribute. This appears to be an inconsistency - neither should need it since they just return int and take spans. Please verify this is intentional.

3. Base class declared with different modifiers in different files (Crc32ParameterSet.cs line 8 vs ref assembly line 34):
   In Crc32ParameterSet.cs, the class is declared as public partial class Crc32ParameterSet but the ref assembly and Crc32ParameterSet.WellKnown.cs declare it as public abstract partial class Crc32ParameterSet. The main file is missing the abstract keyword. While this may compile (partial classes merge modifiers), it should be consistent for clarity.

💡 Suggestions (Non-blocking)

1. Consider caching well-known Nvme parameter set with a dedicated class:
   Crc64ParameterSet.Nvme creates the parameter set via Create(), which generates a new lookup table each time. Consider using the field ??= pattern with a dedicated class (like Ieee8023ParameterSet) if NVMe CRC is expected to be used frequently, to avoid repeated table generation.

2. Switch expression simplification (Crc32ParameterSet.cs lines 59-65):
   csharp Crc32ParameterSet set = reflectValues switch { false => new ForwardTableBasedCrc32(polynomial, initialValue, finalXorValue), _ => new ReflectedTableBasedCrc32(polynomial, initialValue, finalXorValue), }; return set;
   Could be simplified to:
   csharp return reflectValues ? new ReflectedTableBasedCrc32(polynomial, initialValue, finalXorValue) : new ForwardTableBasedCrc32(polynomial, initialValue, finalXorValue);

3. Test file BOM characters: The new test files have Unicode BOM (\u{feff}) at the start. This is generally fine but worth noting for consistency with other test files in the repository.

---

Cross-cutting Analysis

• API Surface: The new APIs follow the existing patterns in the library (static Hash methods, TryHash, instance-based streaming). The parameter sets are immutable and thread-safe.

• Performance: Default/well-known parameter sets use optimized code paths. Custom parameter sets use byte-by-byte table lookup which is reasonable for the initial implementation; the PR description mentions vectorization as a planned follow-up.

• Backward Compatibility: ✅ Existing APIs continue to work unchanged with the same behavior.

---

Test Quality

✅ Excellent test coverage including:

• Parameter set property verification
• Known-answer tests from the CRC catalogue
• Residue tests validating mathematical properties
• Null argument validation
• Singleton verification for well-known parameter sets
• Tests for both reflected and forward (non-reflected) CRC variants

---

Summary

Verdict: ✅ Approve with minor changes

This is a well-designed, thoroughly-tested feature addition. The implementation correctly preserves existing optimizations while adding flexible parameterization. Please address the missing XML documentation on Crc64ParameterSet properties and verify the [CLSCompliant(false)] inconsistency before merging.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 4 comments.

[stephentoub]

🤖 Copilot Code Review — PR #124117

Holistic Assessment

Motivation: ✅ This PR addresses a genuine need identified in issue #123164 (api-approved). Multiple CRC variants exist (NVMe, CRC-32C, CKSUM, BZIP2, etc.) and users currently must implement their own CRC calculations for non-standard variants. The request to support parameterized CRCs is well-justified.

Approach: ✅ The design is sound. It introduces Crc32ParameterSet and Crc64ParameterSet classes that encapsulate CRC parameters (polynomial, initial value, final XOR, reflection). Well-known parameter sets (Crc32, Crc32C, Crc64, Nvme) use optimized implementations, while custom sets use table-based lookup. The polymorphic dispatch via internal Update methods is clean.

Summary: ✅ LGTM with minor suggestions. This is a well-structured, backward-compatible feature addition with comprehensive test coverage. The code correctly preserves existing optimizations for standard CRC variants. The few observations below are minor and non-blocking.

---

Detailed Findings

✅ Correctness — Verified

1. Table generation algorithms are correct for both forward (MSB-first) and reflected (LSB-first) CRCs.
2. Known-answer tests validate against the CRC catalogue at reveng.sourceforge.io with multiple parameter sets (AIXM, AUTOSAR, BZIP2, ISCSI, CKSUM, MEF, etc.).
3. CRC residue property is correctly tested, validating the mathematical property that CRC(data || CRC(data)) produces a fixed value.
4. Optimized paths preserved: The existing vectorized implementations for CRC-32 (IEEE 802.3) and CRC-64 (ECMA-182) are correctly delegated through Ieee8023ParameterSet and Ecma182ParameterSet.
5. CRC-32C hardware acceleration correctly uses SSE4.2 and ARM CRC32 intrinsics when available.

✅ API Design — Consistent

• The new APIs follow the existing patterns in the library (static Hash methods, TryHash, instance-based streaming).
• Parameter sets are immutable and thread-safe for concurrent use.
• Null validation is consistently applied with ArgumentNullException.ThrowIfNull.
• The ParameterSet property on Crc32/Crc64 instances allows introspection of the parameters being used.

✅ Thread Safety — Acceptable

The field ??= ... pattern used for lazy singleton initialization (e.g., Crc32ParameterSet.Crc32, Crc64ParameterSet.Nvme) is a benign race pattern. Multiple threads could potentially create duplicate instances during first access, but since the instances are immutable and functionally equivalent, this is harmless. This is a common pattern in the runtime for avoiding lock overhead on rarely-accessed static properties.

✅ Test Coverage — Excellent

• Parameterized tests cover forward and reflected CRC variants
• Known-answer tests from the CRC catalogue
• Residue tests validating mathematical properties
• Null argument validation tests
• Singleton verification for well-known parameter sets
• Multiple custom parameter sets tested (CKSUM, MEF, CD-ROM EDC, NVMe, WE)

💡 Minor Suggestions (Non-blocking)

1. Consider using BitOperations.ReverseBits — The manual ReverseBits implementations in Crc32ParameterSet.cs and Crc64ParameterSet.cs could potentially use System.Numerics.BitOperations.ReverseBits if available in the target framework. This is a minor code simplification opportunity.

2. Limitation documented — The current design assumes ReflectInput == ReflectOutput (single ReflectValues property). This covers the vast majority of standard CRCs but prevents supporting rare "crossed-endian" variants like CRC-12/UMTS. The comment in the Finalize method correctly notes this as a future consideration.

---

Cross-cutting Analysis

• Backward Compatibility: ✅ All existing APIs continue to work unchanged with identical behavior.
• Performance: Default/well-known parameter sets use optimized code paths. Custom parameter sets use byte-by-byte table lookup, which is reasonable. The PR description notes vectorization for custom sets is planned as a follow-up.
• Documentation: ✅ XML doc comments are comprehensive on all new public APIs.

---

Review performed with code-review skill, models: Claude Sonnet 4, Gemini 3 Pro

[bartonjs]

@stephentoub By the way, your additional code reviewer skill is hallucinating:

Consider using BitOperations.ReverseBits — The manual ReverseBits implementations in Crc32ParameterSet.cs and Crc64ParameterSet.cs could potentially use System.Numerics.BitOperations.ReverseBits if available in the target framework.

That isn't public API, and doesn't exist in core as non-public API. Maybe it used to as non-public... but it's definitely not something that can be called.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated no new comments.

[bartonjs]

/ba-g timeout