Skip to content
This repository has been archived by the owner on Jan 22, 2025. It is now read-only.

sdk: Add try_from_slice_unchecked for Borsh #16098

Merged
merged 4 commits into from
Mar 26, 2021
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 106 additions & 2 deletions sdk/program/src/borsh.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
#![allow(clippy::integer_arithmetic)]
//! Borsh utils
use borsh::schema::{BorshSchema, Declaration, Definition, Fields};
use std::collections::HashMap;
use {
borsh::{
maybestd::io::Error,
schema::{BorshSchema, Declaration, Definition, Fields},
BorshDeserialize,
},
std::collections::HashMap,
};

/// Get packed length for the given BorchSchema Declaration
fn get_declaration_packed_len(
Expand Down Expand Up @@ -54,3 +60,101 @@ pub fn get_packed_len<S: BorshSchema>() -> usize {
let schema_container = S::schema_container();
get_declaration_packed_len(&schema_container.declaration, &schema_container.definitions)
}

/// Deserializes without checking that the entire slice has been consumed
///
/// Normally, `try_from_slice` checks the length of the final slice to ensure
/// that the deserialization uses up all of the bytes in the slice.
///
/// Note that there is a potential issue with this function. Any buffer greater than
/// or equal to the expected size will properly deserialize. For example, if the
/// user passes a buffer destined for a different type, the error won't get caught
/// as easily.
pub fn try_from_slice_unchecked<T: BorshDeserialize>(data: &[u8]) -> Result<T, Error> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about a little test for this guy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unchecked sounds potentially dangerous, should folks be cautious using this function and if so what should they pay attention to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, that's a good point. The main difference is that this function eliminates a check that the slice has been totally read, so it lets you work with overallocated buffers. The normal try_from_slice returns an error if there's data left in the buffer after deserialization. How could we clarify that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a downside to always using the "unchecked" version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The length check ensures that whatever you're expecting to deserialize uses up all of the bytes in the buffer, so you could potentially run into an issue if you always use unchecked. Any buffer that's big enough could work to deserialize your type, so if you pass the wrong buffer with unchecked, the error wouldn't get caught.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any buffer greater than or equal to the expected size would work, correct? Maybe call it "unsized", "unchecked" is fine but should probably include something to effect of the description you have above as what the "unchecked" means practically to the developer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct -- I'll rename the function and add the description then

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the good description you have provided I think "unchecked" makes sense, I can go either way but "unchecked" is more consistent with rust norms

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also consistent with the PR that I put in for Borsh JS :-D

let mut data_mut = data;
let result = T::deserialize(&mut data_mut)?;
Ok(result)
}

#[cfg(test)]
mod tests {
use {
super::*,
borsh::{maybestd::io::ErrorKind, BorshSchema, BorshSerialize},
std::mem::size_of,
};

#[derive(BorshSerialize, BorshDeserialize, BorshSchema)]
enum TestEnum {
NoValue,
Value(u32),
StructValue {
#[allow(dead_code)]
number: u64,
#[allow(dead_code)]
array: [u8; 8],
},
}

#[derive(BorshSerialize, BorshDeserialize, BorshSchema)]
struct TestStruct {
pub array: [u64; 16],
pub number: u128,
pub tuple: (u8, u16),
pub enumeration: TestEnum,
}

#[derive(Debug, PartialEq, BorshSerialize, BorshDeserialize, BorshSchema)]
struct Child {
pub data: [u8; 64],
}

#[derive(Debug, PartialEq, BorshSerialize, BorshDeserialize, BorshSchema)]
struct Parent {
pub data: Vec<Child>,
}

#[test]
fn unchecked_deserialization() {
let data = vec![
Child { data: [0u8; 64] },
Child { data: [1u8; 64] },
Child { data: [2u8; 64] },
];
let parent = Parent { data };

// exact size, both work
let mut byte_vec = vec![0u8; 4 + get_packed_len::<Child>() * 3];
let mut bytes = byte_vec.as_mut_slice();
parent.serialize(&mut bytes).unwrap();
let deserialized = Parent::try_from_slice(&byte_vec).unwrap();
assert_eq!(deserialized, parent);
let deserialized = try_from_slice_unchecked::<Parent>(&byte_vec).unwrap();
assert_eq!(deserialized, parent);

// too big, only unchecked works
let mut byte_vec = vec![0u8; 4 + get_packed_len::<Child>() * 10];
let mut bytes = byte_vec.as_mut_slice();
parent.serialize(&mut bytes).unwrap();
let err = Parent::try_from_slice(&byte_vec).unwrap_err();
assert_eq!(err.kind(), ErrorKind::InvalidData);
let deserialized = try_from_slice_unchecked::<Parent>(&byte_vec).unwrap();
assert_eq!(deserialized, parent);
}

#[test]
fn packed_len() {
assert_eq!(
get_packed_len::<TestEnum>(),
size_of::<u8>() + size_of::<u64>() + size_of::<u8>() * 8
);
assert_eq!(
get_packed_len::<TestStruct>(),
size_of::<u64>() * 16
+ size_of::<u128>()
+ size_of::<u8>()
+ size_of::<u16>()
+ get_packed_len::<TestEnum>()
);
}
}