Rollup merge of rust-lang#99703 - dtolnay:tokenstreamsizehint, r=petr…

…ochenkov

Expose size_hint() for TokenStream's iterator

The iterator for `proc_macro::TokenStream` is a wrapper around a `Vec` iterator:

https://github.com/rust-lang/rust/blob/babff2211e3ae9ef52852dc1b01f3eacdd94c12e/library/proc_macro/src/lib.rs#L363-L371

so it can cheaply provide a perfectly precise size hint, with just a pointer subtraction:

https://github.com/rust-lang/rust/blob/babff2211e3ae9ef52852dc1b01f3eacdd94c12e/library/alloc/src/vec/into_iter.rs#L170-L177

I need the size hint in syn (https://github.com/dtolnay/syn/blob/1.0.98/src/buffer.rs) to reduce allocations when converting TokenStream into syn's internal TokenBuffer representation.

Aside from `size_hint`, the other non-default methods in `std::vec::IntoIter`'s `Iterator` impl are `advance_by`, `count`, and `__iterator_get_unchecked`. I've included `count` in this PR since it is trivial. I did not include `__iterator_get_unchecked` because it is spoopy and I did not feel like dealing with that. Lastly, I did not include `advance_by` because that requires `feature(iter_advance_by)` (rust-lang#77404) and I noticed this comment at the top of libproc_macro:

https://github.com/rust-lang/rust/blob/babff2211e3ae9ef52852dc1b01f3eacdd94c12e/library/proc_macro/src/lib.rs#L20-L22

library/proc_macro/src/lib.rs

@@ -382,6 +382,14 @@ pub mod token_stream {
  bridge::TokenTree::Literal(tt) => TokenTree::Literal(Literal(tt)),
  })
  }
+
+ fn size_hint(&self) -> (usize, Option<usize>) {
+ self.0.size_hint()
+ }
+
+ fn count(self) -> usize {
+ self.0.count()
+ }
  }
 
  #[stable(feature = "proc_macro_lib2", since = "1.29.0")]