Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can not decode types with variable length encoding. #181

Closed
recmo opened this issue Sep 30, 2020 · 1 comment · Fixed by #221
Closed

Can not decode types with variable length encoding. #181

recmo opened this issue Sep 30, 2020 · 1 comment · Fixed by #221

Comments

@recmo
Copy link

recmo commented Sep 30, 2020

I have an event that contains a public key as an elliptic curve point, which looks roughly like this:

enum PublicKey {
   Zero, // default
   Point {
      x: U256.
      y: U256,
   }
}

When I register it using register_type_size<PublicKey>("PublicKey")*, I can successfully decode events containing Zero values, but non-zero values fail with:

Error: Scale codec error: Not enough data to fill buffer

On the other hand, if I force the type_size to the second case (which is 65 bytes encoded) using register_type_size<(u8, [u8;32], [u8;32])>("PublicKey") then I can sucessfully decode non-zero values but the Zero value fails with the above error.

How do I get event decoding to work in all cases?

*: PS: I'd like to use the #[module] macro, but it doesn't allow me to add types other than trait associated ones?

@h4x3rotab
Copy link
Contributor

Currently subxt can only handle fix-size types. In register_type_size, it get the type by counting the byte length of T::encode(Default::default()).

h4x3rotab added a commit to Phala-Network/substrate-subxt that referenced this issue Jan 14, 2021
Fixes paritytech#196, paritytech#181, #28

## Dyanmic sized types

Before this change, the event decoder assume all the event types
have fixed sizes. Some counterexamples are: Hashes, AuthorityList.

In this change, instead of decoding by skipping the fixed-length bytes,
we introduce `type_segmenter` registry which decodes the raw event
bytes with the actual scale codec. So variable length types can be
handled correctly.

## New attribute for pallet type definition

In the past, trait associated type is the only way to add types to
the EventsDecoder implementation of a pallet. But in reality it's
common that the events in a pallet references some types not defined
in the trait associated types. Some examples are: `IdentificationTuple`
and `SessionIndex` in Session pallet.

In this change, we introduce more attributes to add the types:

```rust
#[module]
trait Pallet: System {
    #![event_type(SomeType)]
    #![event_alias(TypeNameAlias = SomeType)]
    #![event_alias(SomeOtherAlias = TypeWithAssociatedTypes<T>)]
}
```

## Tested

Compile with `nightly-2020-10-01`; smoke test to sync a full
Phala bockchain.
h4x3rotab added a commit to Phala-Network/substrate-subxt that referenced this issue Jan 14, 2021
Fixes paritytech#196, paritytech#181, #28

## Dyanmic sized types

Before this change, the event decoder assume all the event types
have fixed sizes. Some counterexamples are: Hashes, AuthorityList.

In this change, instead of decoding by skipping the fixed-length bytes,
we introduce `type_segmenter` registry which decodes the raw event
bytes with the actual scale codec. So variable length types can be
handled correctly.

## New attribute for pallet type definition

In the past, trait associated type is the only way to add types to
the EventsDecoder implementation of a pallet. But in reality it's
common that the events in a pallet references some types not defined
in the trait associated types. Some examples are: `IdentificationTuple`
and `SessionIndex` in Session pallet.

In this change, we introduce more attributes to add the types:

```rust
#[module]
trait Pallet: System {
    #![event_type(SomeType)]
    #![event_alias(TypeNameAlias = SomeType)]
    #![event_alias(SomeOtherAlias = TypeWithAssociatedTypes<T>)]
}
```

## Tested

Compile with `nightly-2020-10-01`; smoke test to sync a full
Phala bockchain.
ascjones pushed a commit that referenced this issue Jan 20, 2021
* Refactor event type decoding hand declartion

Fixes #196, #181, #28

## Dyanmic sized types

Before this change, the event decoder assume all the event types
have fixed sizes. Some counterexamples are: Hashes, AuthorityList.

In this change, instead of decoding by skipping the fixed-length bytes,
we introduce `type_segmenter` registry which decodes the raw event
bytes with the actual scale codec. So variable length types can be
handled correctly.

## New attribute for pallet type definition

In the past, trait associated type is the only way to add types to
the EventsDecoder implementation of a pallet. But in reality it's
common that the events in a pallet references some types not defined
in the trait associated types. Some examples are: `IdentificationTuple`
and `SessionIndex` in Session pallet.

In this change, we introduce more attributes to add the types:

```rust
#[module]
trait Pallet: System {
    #![event_type(SomeType)]
    #![event_alias(TypeNameAlias = SomeType)]
    #![event_alias(SomeOtherAlias = TypeWithAssociatedTypes<T>)]
}
```

## Tested

Compile with `nightly-2020-10-01`; smoke test to sync a full
Phala bockchain.

* Format code

* Make rustfmt::skip an outer attribute

* Ignore the sample code

* Alias the event segmenter closure

* Copy AuthorityList from sp_finality_grandpa

* Remove the unused static event type size

* Make segmenter as a trait, resue grandpa::Public

* Wrap PhantomData in struct TypeMarker
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants