Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way to get Stream<Token=char> from io::Read? #329

Open
tailhook opened this issue Nov 13, 2021 · 1 comment
Open

Is there a way to get Stream<Token=char> from io::Read? #329

tailhook opened this issue Nov 13, 2021 · 1 comment

Comments

@tailhook
Copy link

Hi!

I'm seeing examples of how to parse and decode bytes from io::Read, but still not clear is now to do the same to have char stream (and also preferably with byte positioner). I find using chars easier for some use cases. Is there any built-in or easily accessible way to do that, or do I have to implement custom stream for that?

@Marwes
Copy link
Owner

Marwes commented Nov 15, 2021

The Read stream

combine/src/stream/read.rs

Lines 157 to 175 in fdd63e2

pub struct Stream<R> {
bytes: Bytes<R>,
}
impl<R: Read> StreamOnce for Stream<R> {
type Token = u8;
type Range = &'static [u8];
type Position = usize;
type Error = Error;
#[inline]
fn uncons(&mut self) -> Result<u8, StreamErrorFor<Self>> {
match self.bytes.next() {
Some(Ok(b)) => Ok(b),
Some(Err(err)) => Err(Error::Io(err)),
None => Err(Error::EndOfInput),
}
}
}
uses https://doc.rust-lang.org/std/io/trait.Read.html#method.bytes internally however there is no equivalent for reading char directly from an io::Read so you would need to implement that first.

If at all possible I would recommend against parsing directly from a io::Read as it is significantly slower than reading from an in memory buffer. If you need streaming you may want to look into using https://docs.rs/combine/4.6.2/combine/macro.decode.html and partial parsing instead, it may be easier to create an input stream that contains a &[u8], yet returns char/&str (might be a good addition to combine itself actually).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants