Error when encoding/decoding non-explicitely tagged vecs

[orthecreedence]

Using rasn v0.9.5. I recently decided to switch the serialization on the project I'm working on from #[rasn(tag(explicit(N)))] to #[rasn(tag(N)) and I'm now having an issue with deserialization. I reduced my case down to two examples:

    #[test]
    fn ser_sequence_vec2() {
        #[derive(Debug, Clone, AsnType, Encode, Decode)]
        #[rasn(choice)]
        pub enum MultisigPolicySignature {
            #[rasn(tag(0))]
            Key {
                #[rasn(tag(0))]
                key: String,
            },
        }

        #[derive(AsnType, Clone, Debug, Decode, Encode)]
        struct Transaction {
            #[rasn(tag(0))]
            signatures: Vec<MultisigPolicySignature>,
        }

        #[derive(AsnType, Clone, Debug, Decode, Encode)]
        struct Transactions {
            #[rasn(tag(0))]
            transactions: Vec<Transaction>,
        }

        let transactions1 = Transactions {
            transactions: vec![
                Transaction {
                    signatures: vec![
                        MultisigPolicySignature::Key {
                            key: String::from("key"),
                        }
                    ],
                }
            ],
        };

        let ser1 = rasn::der::encode(&transactions1).unwrap();
        println!("ser1: {:?}", ser1);
        let transactions1_2: Transactions = rasn::der::decode(&ser1[..]).unwrap();
        println!("trans2: {:?}", transactions1_2);
        assert_eq!(transactions1_2.transactions.len(), 1);
    }

This test gives the output:

running 1 test
ser1: [48, 13, 160, 11, 48, 9, 160, 7, 48, 5, 128, 3, 107, 101, 121]
trans2: Transactions { transactions: [] }
thread 'util::ser::tests::ser_sequence_vec2' panicked at 'assertion failed: `(left == right)`
  left: `0`,
 right: `1`', src/util/ser.rs:428:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test util::ser::tests::ser_sequence_vec2 ... FAILED

Notice the transactions [] ...it deserializes the outer vec as empty. This serializes to https://lapo.it/asn1js/#MA2gCzAJoAcwBYADa2V5 which I think is correct, so the issue might be in decoding.

This can be reduced even more:

    #[test]
    fn ser_sequence_vec2() {
        #[derive(Debug, Clone, AsnType, Encode, Decode)]
        #[rasn(choice)]
        pub enum MultisigPolicySignature {
            #[rasn(tag(0))]
            Key {
                #[rasn(tag(0))]
                key: String,
            },
        }

        #[derive(AsnType, Clone, Debug, Decode, Encode)]
        struct Transactions {
            #[rasn(tag(0))]
            transactions: Vec<MultisigPolicySignature>,
        }

        let transactions1 = Transactions {
            transactions: vec![
                MultisigPolicySignature::Key {
                    key: String::from("key"),
                },
            ],
        };

        let ser1 = rasn::der::encode(&transactions1).unwrap();
        println!("ser1: {:?}", ser1);
        let transactions1_2: Transactions = rasn::der::decode(&ser1[..]).unwrap();
        println!("trans2: {:?}", transactions1_2);
        assert_eq!(transactions1_2.transactions.len(), 1);
    }

Output:

running 1 test
ser1: [48, 9, 160, 7, 48, 5, 128, 3, 107, 101, 121]
thread 'util::ser::tests::ser_sequence_vec2' panicked at 'called `Result::unwrap()` on an `Err` value: FieldError { name: "Transactions.transactions", error: "Unexpected extra data found: length `7` bytes" }', src/util/ser.rs:416:74
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test util::ser::tests::ser_sequence_vec2 ... FAILED

This serializes to https://lapo.it/asn1js/#MAmgBzAFgANrZXk which again looks correct to me.

Interestingly, this can be "fixed" in both examples by updating MultisigPolicySignature to either remove the tag specification or make it explicit:

        // works
        pub enum MultisigPolicySignature {
            Key {
                #[rasn(tag(0))]
                key: String,
            },
        }

        // works
        pub enum MultisigPolicySignature {
            #[rasn(tag(explicit(0)))]
            Key {
                #[rasn(tag(0))]
                key: String,
            },
        }

I'm guessing this has something to do with abiguity when dealing with SEQUENCE vs SEQUENCE OF, but not sure if there's an easy fix. I can try to pick at it and open a PR, but I am definitely moving out of my comfort space, so some guidance would be appreciated. Thank you!

[orthecreedence]

Debugging this a bit more (couldn't help myself!) I modded decode_sequence_of:

    fn decode_sequence_of<D: Decode>(
        &mut self,
        tag: Tag,
        _: Constraints,
    ) -> Result<Vec<D>, Self::Error> {
        self.parse_constructed_contents(tag, true, |decoder| {
            let mut items = Vec::new();

            match D::decode(decoder) {
                Ok(item) => items.push(item),
                Err(e) => panic!("SEQ: {:?} {} -- {:?}", tag, e, decoder.input),
            }
            //while let Ok(item) = D::decode(decoder) {
                //items.push(item);
            //}

            Ok(items)
        })
    }

(Obviously this is just for debugging, I have no intention of throwing in panics everywhere). This spits out:

thread 'util::ser::tests::ser_sequence_vec2' panicked at 'SEQ: Tag { class: Context, value: 0 } No valid `CHOICE` variant for `MultisigPolicySignature` -- [48, 5, 128, 3, 107, 101, 121]', /var/www/stamp/tmp/rasn/src/ber/de.rs:557:27

That input [48 5 128 ...] decodes to https://lapo.it/asn1js/#MAWAA2tleQ, which I think (not an ASN1 expert) is enough info to decode the enum, so there might be something missing in the derive macro. Going to keep picking at it.

[XAMPPRocky]

Thank you for your issue!

Going to keep picking at it.

Please do :)

[orthecreedence]

I think I'm about as far as I can get...any guidance would be appreciated =].

Seems the problem lies in macros/rc/enum.rs::impl_decode(), which generates the following:

if rasn::TagTree::tag_contains(& tag, &[rasn::TagTree::Leaf(rasn :: Tag :: new(rasn :: types :: Class :: Context, 0))]) {                                                    
    return {                                                                                                     
        let decode_fn = | decoder : & mut D | -> core :: result :: Result < _,D :: Error > {
            #[derive(rasn :: AsnType, rasn :: Decode, rasn :: Encode)] struct InnerMultisigPolicySignature { #[rasn(tag(0))] key : String, } ;
            let inner = < InnerMultisigPolicySignature > :: decode_with_tag(decoder, rasn :: Tag :: new(rasn :: types :: Class :: Context, 0)) ? ;
            Ok(Self :: Key { key : inner.key })
        } ;
        (decode_fn) (decoder)
    }
}
Err(rasn :: de :: Error :: no_valid_choice("MultisigPolicySignature"))

The tag value is coming through as Tag { class: Universal, value: 16 }, which is obviously not class: Context, value: 0 so I'm not sure what the solve is...maybe allow a sequence there as well as a tag? Seems the tag we're looking for is embedded in the sequence itself. However, I'm not sure how to structure this in the code or if it would affect anything else. Any pointers?

I also managed to reduce the test case even further, eliminating the outer container and just using Vec<MultisigPolicySignature>:

    #[test]
    fn decode_implicit_tagged_vec_choice() {
        #[derive(Debug, Clone, AsnType, Encode, Decode)]
        #[rasn(choice)]
        pub enum MultisigPolicySignature {
            #[rasn(tag(0))]
            Key {
                #[rasn(tag(0))]
                key: String,
            },
        }

        let transactions2 = vec![
            MultisigPolicySignature::Key {
                key: String::from("key1"),
            },
        ];

        let ser2 = rasn::der::encode(&transactions2).unwrap();
        let transactions2_2: Vec<MultisigPolicySignature> = rasn::der::decode(&ser2[..]).unwrap();
        assert_eq!(transactions2_2.len(), 1);
    }

Thanks!