Feature Request: peek(io::IO, x)

[kmsquire]

As discussed here: https://groups.google.com/forum/?fromgroups=#!topic/julia-users/kDKL1dS4L0w

Basically, I'd like to be able to peek at the first few characters of an IO stream, without removing the characters from the stream. In particular, it needs to work with streams that are not files and therefore not seekable.

[JeffBezanson]

All we have so far is peek of a byte and peekchar, both only on files. The next step is to implement whatever kinds of peeking we want for IOBuffer, then other streams will operate out of that.

[kmsquire]

I thought I had seen these when I implemented gzip support, but missed them this time because they aren't exported.

Right now, for IOStream, both mimic the c interface: peek reads a byte but returns an Int32, and returns -1 on failure, and peekchar returns char(-1) on failure. It was trivial to implement this for IOBuffer, although it doesn't address the full request here.

Unfortunately, returning -1 doesn't generalize. The obvious solution is to throw an EOFError when trying to peek beyond the end of the file/buffer, but then the interface is inconsistent with the current (unexported but standard) implementation.

Thoughts?

CC: @vtjnash

[JeffBezanson]

Ah yes, the fact that they're not exported probably means I wasn't happy with the API. There is also a reason why very few (any?) environments have peek for anything more than a byte. We probably have to throw EOFError to make it work in general.

[kmsquire]

A google search revealed the following:

1. Python offers to peek at a specified number of bytes for buffered streams, although it may not return the number of requested bytes.

http://docs.python.org/2/library/io.html#buffered-streams

2. More interestingly, java has a mark()/reset() mechanism that lets you read up to a specified maximum number of bytes from a buffered input stream, then reset the buffer to the mark:

http://docs.oracle.com/javase/6/docs/api/java/io/InputStream.html#mark(int)

I think I like the java version, but not sure about the implications. Thoughts?

[kmsquire]

Actually, either way would be fine. The python version is nice in that it's simple, and closer in spirit to julia.

[vtjnash]

The Java implementation seems much cooler to me, since you don't need to reimplement all of the read methods. In python, everything is just a string buffer, so it's not really a problem to peek N bytes and then parse. In Julia, it seems better to me to be able to do multi-type reads (like Java) without needing to create a separate read-ahead/peek buffer. The Java-like implementation could even give you file-like seek access on a stream, by expressing locations relative to the mark.

[kmsquire]

That's fine--I'm okay with either one. I'll work on it.

[StefanKarpinski]

The Java mark interface does seem rather nice. I can see having to specify a limit beforehand as being constraining sometimes, however. If we add a unmark method, then you can have mark without a limit as long as the programmer is careful always unmark later. A nice interface might be mark with a do block, i.e.:

mark(io) do
  # can reset io in here
end

In fact, that kind of makes me wonder if it wouldn't make more sense to simply wrap the bare io object with another stream that buffers and allows resetting, although I feel like that might not always work well enough.

[kmsquire]

A wrapper was what I was leaning toward. Perhaps there could be two versions, one with a fixed buffer size, and one which spills to disk after the buffer fills (for non-file streams).

[kmsquire]

This can be closed now that #3656 is merged.