Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data refactor #2560

Merged
merged 15 commits into from
Dec 30, 2024
Merged

Data refactor #2560

merged 15 commits into from
Dec 30, 2024

Conversation

dirvine
Copy link
Member

@dirvine dirvine commented Dec 20, 2024

Add Pointer Type and Refactor LinkedList Structure

Changes

  • refactor Transacions -> LinkedList (can be single linked or doubly linked)
  • Introduces new Pointer type for network addressing
  • Makes LinkedList outputs optional
  • Adds validation and storage handling for network records
  • Improves split record handling in networking layer

Details

New Pointer Type

  • Added Pointer and PointerAddress types for network addressing
  • Implemented signature verification and validation for Pointer records
  • Added storage and validation handling in ant-node
  • Added new log markers for tracking Pointer operations
  • Introduced InvalidSignature error variant for validation failures

LinkedList Refactoring

  • Made outputs field optional via Option<Vec<(PublicKey, LinkedListContent)>>
  • Updated constructors and signature calculation to handle optional outputs
  • Modified transaction tests to use .into() for empty output vectors
  • Maintains backward compatibility with existing functionality

Networking Layer Updates

  • Enhanced split record handling for both Pointer and LinkedList records
  • Added proper signature verification for split records
  • Implemented count-based selection for handling duplicate records
  • Updated record storage validation to handle new types

Testing

  • Added tests for Pointer validation and storage
  • Updated transaction tests for optional outputs
  • All existing tests pass (excluding known ant-evm issues)

Impact

These changes improve the flexibility of the network addressing system while making
LinkedList transactions more versatile. The Pointer type provides a new way to handle
network addressing, while optional outputs in LinkedList allow for simpler transaction
structures when needed.

Breaking Changes

None. All changes maintain backward compatibility with existing functionality.

* Add new Pointer type for network addressing
* Add PointerAddress type and related network address handling
* Implement signature verification for Pointer records
* Add validation and storage handling for Pointer records in ant-node
* Add log markers for Pointer operations
* Make LinkedList outputs field optional by wrapping in Option<Vec>
* Update LinkedList constructors and signature calculation to handle optional outputs
* Fix transaction tests to use .into() for empty output vectors
* Update ant-networking to handle split records for both Pointer and optional outputs
* Maintain signature verification and count-based selection for split records
* Add InvalidSignature error variant to handle Pointer validation failures

This change introduces a new Pointer type for network addressing while also making
LinkedList transactions more flexible by allowing them to be created without outputs.
The networking layer has been updated to handle both changes, including proper
split record handling, signature verification, and storage validation. All changes
maintain backward compatibility with existing functionality.
@dirvine dirvine marked this pull request as ready for review December 24, 2024 15:06
autonomi/python/examples/autonomi_pointers.py Dismissed Show dismissed Hide dismissed
autonomi/python/examples/autonomi_pointers.py Dismissed Show dismissed Hide dismissed
autonomi/python/examples/autonomi_pointers.py Dismissed Show dismissed Hide dismissed
ant-node/Cargo.toml Outdated Show resolved Hide resolved
Copy link
Member

@grumbach grumbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚢 ✅

Comment on lines +32 to +37
pub enum PointerTarget {
ChunkAddress(ChunkAddress),
LinkedListAddress(LinkedListAddress),
PointerAddress(PointerAddress),
ScratchpadAddress(ScratchpadAddress),
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we omitted registers here on purpose?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this include DataMap, not just for files but arbitrary metadata?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed +1 @happybeing

@jacderida jacderida merged commit f92438c into maidsafe:main Dec 30, 2024
20 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants