Panic in rustdoc: tokenization error for markdown bullet points.

[pierzchalski]

The following code causes cargo to panic when run with cargo doc:

/// A header to give rustdoc an anchor point
///
///     * a `code` token in a bullet point more than 3 spaces deeper.
pub struct Test(usize);

cargo doc panics with:

<stdin>:1:5: 1:6 error: unknown start of token: `
<stdin>:1 * a `code` token in a bullet point more than 3 spaces deeper.

thread '<unnamed>' panicked at 'Box<Any>', ../src/libsyntax/parse/lexer/mod.rs:1438
stack backtrace:
   1:     0x7fe1e3017610 - std::sys::backtrace::tracing::imp::write::h9fb600083204ae7f
   2:     0x7fe1e3024d0b - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::hca543c34f11229ac
   3:     0x7fe1e3024943 - std::panicking::default_hook::hc2c969e7453d080c
   4:     0x7fe1e2fe98bf - std::sys_common::unwind::begin_unwind_inner::h30e12d15ce2b2e25
   5:     0x7fe1db76ea27 - std::sys_common::unwind::begin_unwind::h8827897210a88096
   6:     0x7fe1db8e6dc3 - syntax::parse::lexer::StringReader::advance_token::h92adf4275488d206
   7:     0x7fe1db82ce86 - _<parse..lexer..StringReader<'a> as parse..lexer..Reader>::next_token::h6ac0af3b935ce0f5
   8:     0x7fe1e35ee799 - rustdoc::html::highlight::write_source::hdf6c06b0745e4cd3
   9:     0x7fe1e35edd4a - rustdoc::html::highlight::render_with_highlighting::h3034e707544efe70
  10:     0x7fe1e35ff5f4 - rustdoc::html::markdown::render::block::h51e60e78173225c0
  11:     0x7fe1e3684444 - parse_block
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/rt/hoedown/src/document.c:1803
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/rt/hoedown/src/document.c:2479
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/rt/hoedown/src/document.c:2433
  12:     0x7fe1e3684c02 - hoedown_document_render
                        at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/rt/hoedown/src/document.c:2894
  13:     0x7fe1e35fe27b - rustdoc::html::markdown::render::h583ff8c87af19db5
  14:     0x7fe1e309660d - core::fmt::write::h218cb9026097c841
  15:     0x7fe1e3634d7d - _<html..render..Item<'a> as std..fmt..Display>::fmt::hb4df979c6c2505c8
  16:     0x7fe1e309658f - core::fmt::write::h218cb9026097c841
  17:     0x7fe1e361c252 - std::io::Write::write_fmt::h0247a1638aa03f22
  18:     0x7fe1e362e426 - rustdoc::html::render::Context::item::render::h299acd2a78322578
  19:     0x7fe1e36294c1 - rustdoc::html::render::Context::item::_$u7b$$u7b$closure$u7d$$u7d$::h7daa4740b19738f3
  20:     0x7fe1e3556da7 - rustdoc::html::render::run::h8c90ad33a5f6502c
  21:     0x7fe1e3534436 - rustdoc::main_args::h56d504458f0db355
  22:     0x7fe1e352fe6a - std::sys_common::unwind::try::try_fn::h5afa0fb66b64d38b
  23:     0x7fe1e3014dab - __rust_try
  24:     0x7fe1e3014d3d - std::sys_common::unwind::inner_try::h47a4d9cd4a369dcd
  25:     0x7fe1e35302a8 - _<F as std..boxed..FnBox<A>>::call_box::h31b423b2780022d9
  26:     0x7fe1e3022ea4 - std::sys::thread::Thread::new::thread_start::h6f266e069bf4ec2b
  27:     0x7fe1db013423 - start_thread
  28:     0x7fe1e2c81cbc - clone
  29:                0x0 - <unknown>
error: Could not document `rustdoc_crash`.

This panic disappears if:

• The bullet point doesn't contain a ``` or \ character.
• The difference in whitespace of the first non-bullet line and the first bullet line is strictly less than 4 characters (for example, in the above sample the first line has a single space, while the bullet has 5. If the bullet had 4 spaces, there would be no panic).

[alexcrichton]

I believe this is a dupe of #12834 (as markdown interprets that bullet as code which rustdoc tries to parse as Rust, but thanks for the report!