Convenience API for skipping data?

[Gankra]

I'm working on a format where I have several arrays in my data stream that I don't want to deserialize right away, but instead jump over and deserialize on demand later (element-by-element).

It would be nice if there was methods like:

// returns bytes skipped over
bincode::skip_over<T: Deserialize>(&mut Reader, count: usize) -> Result<usize>;

Bincode might be able to implement this better than I could naively, by using things like fixed-serialized-size info to skip over everything without any parsing in O(1). In general this API needs to parse (enums), but it can basically be like "it only parses as much as is needed to skip over the data".

[samlh]

https://serde.rs/ignored-any.html may be relevant

[Gankra]

I'm using bincode, so I need to give the fields a type.

[Xaeroxe]

@gankro serde 1.0 introduces zero copy deserialization, so as soon support for 1.0 is finished you should be able to use that to similar effect.

[Gankra]

Zero copy deserialization only works on bytes and strs; my slices contain actual types that need to be materialized into a new buffer.

[Xaeroxe]

I see. Well in the mean time maybe you could use borrowed_bytes with a wrapper struct and interpret the bytes lazily when needed? I'm not sure if that's easy or doable I'm just thinking out loud.

[TyOverby]

@Xaeroxe: that's what I suggested in #serde a few days ago. Unfortunately it would require serializing extra "length" bytes into the output, so for lots of tiny structs this isn't an option.

[Xaeroxe]

Hmm I wonder if it's possible to do this using generics so the length is coded into the executable rather than the memory representation. I might look into that.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.