Rewrite doc_markdown to use pulldown-cmark

[mcarton]

Rewrite doc_markdown to use pulldown-cmark.

This should fix a number of false positives and true negatives. It should mean that the lint can be extended more easily in the futur.

WIP: I have not got the spans quite right yet. The best I can do is line-wise (actually attribute-wise) spans, while the current lint report the actual problematic word.

Fix #1469.

[killercup]

Yes! This is gonna be good

[mcarton]

This is ready to be merged.

[mcarton]

Appveyor build looks to have failed because of caching problem.

[mcarton]

The windows build is failing because of

------stderr------------------------------
error[E0464]: multiple matching crates for `pulldown_cmark`
  --> clippy_lints/src/lib.rs:55:1
   |
55 | extern crate pulldown_cmark;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidates:
   = note: path: \\?\C:\projects\rust-clippy\clippy_tests\target\debug\deps\libpulldown_cmark-2b392b6bad2227f1.rlib
   = note: crate name: pulldown_cmark
   = note: path: \\?\C:\Users\appveyor\.rustup\toolchains\nightly-i686-pc-windows-gnu\lib\rustlib\i686-pc-windows-gnu\lib\libpulldown_cmark-650d3ff121d3d129.rlib
   = note: crate name: pulldown_cmark
error[E0463]: can't find crate for `pulldown_cmark`
  --> clippy_lints/src/lib.rs:55:1
   |
55 | extern crate pulldown_cmark;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
error: aborting due to previous error(s)

Any idea what is going on?

[oli-obk]

Rustc is distributing its own pulldown... somehow we're getting a conflict with the distributed one. Do we set any unusual path env var values in appveyor?

[mcarton]

Looks like we do. Cc @llogiq, I've no idea how either Appveyor or Windows work.

[mcarton]

This works \o/
Any objection to merging?

[Manishearth]

No, but I would like a quick performance comparison.

[oli-obk]

You can remove my two commits then. Seems they had no effect and were messy to begin with

[mcarton]

Before:
diesel: (3767 lines starting with //[/!])

time: 0.036; rss: 114MB	early lint checks
time: 0.036; rss: 113MB	early lint checks
time: 0.036; rss: 113MB	early lint checks

serde: (3111 lines starting with //[/!])

time: 0.034; rss: 111MB	early lint checks
time: 0.034; rss: 111MB	early lint checks
time: 0.034; rss: 111MB	early lint checks

clippy_lints: (10809 lines starting with //[/!])

time: 0.034; rss: 178MB	early lint checks
time: 0.029; rss: 178MB	early lint checks
time: 0.030; rss: 182MB	early lint checks

After:
diesel:

time: 0.038; rss: 113MB	early lint checks
time: 0.039; rss: 114MB	early lint checks
time: 0.036; rss: 114MB	early lint checks

serde:

time: 0.034; rss: 111MB	early lint checks
time: 0.033; rss: 113MB	early lint checks
time: 0.037; rss: 113MB	early lint checks

clippy_lints:

time: 0.033; rss: 179MB	early lint checks
time: 0.030; rss: 183MB	early lint checks
time: 0.030; rss: 181MB	early lint checks

Does not look like there is much of a difference. Any idea of a biggish crate with a lot of docs?
(It also indirectly found dead links in diesel) 😄

[Manishearth]

(feel free to merge, fwiw)