process_frame() refactor broke 2-pass encoding

[rzumer]

45ef0af causes only one frame to be encoded in my 2-pass encoding tests (happens on the first pass, so could also occur for single-pass). After reverting this commit only, master works as expected.

<derf> I mean, by the time you're at Result<Option<Vec<...>>, ...>, and all of different possibilities (Ok(None/Some(empty/full))/Err) have different meanings for the caller, maybe it is time to start thinking about an enum or something to tell people what is actually going on.
<derf> That patch collapsed two of the meanings (None and an empty Vec), but didn't actually preserve the existing code's behavior in those cases.