Skip to content

rustdoc does not expand escape sequences in doc fragments of macros properly #25943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
thelink2012 opened this issue Jun 1, 2015 · 0 comments
Closed

rustdoc does not expand escape sequences in doc fragments of macros properly #25943

thelink2012 opened this issue Jun 1, 2015 · 0 comments
Labels
T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@thelink2012
Copy link

All of the following I'm going to describe happens only when expanding doc comments in a macro, using meta fragments, in other contexts it works fine, so it indeed is a thing! A sample of doc comments expansion in a macro can be found at #25929 so I'll skip samples and go direct to the point.

If I use a escape sequence of usual strings it seems to expand it right there on rustdoc.

I have found the issue by trying to do something like the following:

 /// `\0`

When I run cargo doc I get this panic:

PS C:\Projects\forks\iup-rust> cargo doc --verbose
       Fresh libc v0.1.6
   Compiling iup v0.0.2 (file:///C:/Projects/forks/iup-rust)
     Running `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dependency=C:\Projects\for
ks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc=C:\Projects\forks\iup-
rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-
d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib --e
xtern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib`
       Fresh iup-sys v0.0.3
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: NulError(6, [60, 99, 111, 100, 101, 62, 0,
60, 47, 99, 111, 100, 101, 62])', C:/bot/slave/stable-dist-rustc-win-32/build/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c8fcc9f - html::toc::TocBuilder::new::hb7ed7be44c6fc916q6r
   7: 0x6ca514d4 - hoedown_buffer_printf
   8: 0x6ca5185d - hoedown_buffer_printf
   9: 0x76ddb106 - free
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Any', C:/bot/slave/stable-dist-rustc-win-32/bu
ild/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c99b2cc - main::h854d123b237a9daepdu
   7: 0x6313c59c - rust_try
   8:   0x401645 - main
   9:   0x4013de
  10: 0x765b7c04 - BaseThreadInitThunk
  11: 0x76fcad1f - RtlInitializeExceptionChain
Could not document `iup`.

Caused by:
  Process didn't exit successfully: `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dep
endency=C:\Projects\forks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc
=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\tar
get\debug\deps\liblibc-d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16
fee22cb642ccd4.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib` (exi
t code: 101)

When using the \0 outside a code block it does not happen, but if I try a hexadecimal literal outside the code block instead it still panics.

/// \x0

I get

PS C:\Projects\forks\iup-rust> cargo doc --verbose
   Compiling iup v0.0.2 (file:///C:/Projects/forks/iup-rust)
     Running `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dependency=C:\Projects\for
ks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc=C:\Projects\forks\iup-
rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-
d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib --e
xtern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib`
       Fresh libc v0.1.6
       Fresh iup-sys v0.0.3
thread '<unnamed>' panicked at 'lexer should have rejected a bad character escape \x0 Action generated when the text is
edited, but before its value is actually changed.', C:/bot/slave/stable-dist-rustc-win-32/build/src/libcore\option.rs:33
0
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6a1f6a42 - parse::char_lit::h76da3a3875da6b37cXT
   7: 0x6a1f86d2 - parse::str_lit::h85149c8921a41c28h2T
   8: 0x6a1f59b7 - parse::parser::Parser<'a>::lit_from_token::h68fc0ae22c986b30yQG
   9: 0x6a1fa317 - parse::parser::Parser<'a>::parse_lit::h9f70e38391de0fa2yUG
  10: 0x6a2482b4 - parse::attr::Parser<'a>.ParserAttr::parse_meta_item::h1c98783168c92ce14kT
  11: 0x6a423862 - ext::tt::macro_parser::parse_nt::h25dc8b6a526655dfPZf
  12: 0x6a13ad50 - ext::tt::macro_parser::parse::ha0f605ccc4fd9aa7GJf
  13: 0x6a137b8f - ast::TokenTree::parse::h79e7e0468b7d782bdCl
  14: 0x6a42563a - ext::tt::macro_rules::MacroRulesMacroExpander.TTMacroExpander::expand::h96b4da2fbcd98be3Xeg
  15: 0x6a38044b - ext::expand::expand_item_mac::h6351873f0995efe7tib
  16: 0x6a37030d - ext::expand::expand_item::hf68f0012c3b848a1scb
  17: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  18: 0x6a379e6f - ext::expand::expand_item::hf68f0012c3b848a1scb
  19: 0x6a379ae6 - ext::expand::expand_item::hf68f0012c3b848a1scb
  20: 0x6a374757 - ext::expand::expand_item::hf68f0012c3b848a1scb
  21: 0x6a3cfc65 - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  22: 0x6a3cfacf - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  23: 0x6a37173b - ext::expand::expand_item::hf68f0012c3b848a1scb
  24: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  25: 0x6a379e6f - ext::expand::expand_item::hf68f0012c3b848a1scb
  26: 0x6a379ae6 - ext::expand::expand_item::hf68f0012c3b848a1scb
  27: 0x6a374757 - ext::expand::expand_item::hf68f0012c3b848a1scb
  28: 0x6a3cfc65 - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  29: 0x6a3cfacf - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  30: 0x6a37173b - ext::expand::expand_item::hf68f0012c3b848a1scb
  31: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  32: 0x6a379e6f - ext::expand::expand_item::hf68f0012c3b848a1scb
  33: 0x6a379ae6 - ext::expand::expand_item::hf68f0012c3b848a1scb
  34: 0x6a374757 - ext::expand::expand_item::hf68f0012c3b848a1scb
  35: 0x6a3cfc65 - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  36: 0x6a3cfacf - ext::expand::PatIdentRenamer<'a>.Folder::fold_mac::h289ea2cd95f4c5b4CMb
  37: 0x6a37173b - ext::expand::expand_item::hf68f0012c3b848a1scb
  38: 0x6a36ccd8 - ext::expand::expand_item::hf68f0012c3b848a1scb
  39: 0x6a3d6f01 - ext::expand::expand_crate::ha0bcd83c140e9790Y9b
  40: 0x6cd67820 - driver::collect_crate_metadata::h5feaa081cce22dd0E1a
  41: 0x6cd13930 - driver::phase_2_configure_and_expand::he8982b2f8dc553f7Wsa
  42: 0x6c8cdb2e - core::run_core::h54f67f861d00dcf7wnj
  43: 0x6c9b22c6 - usage::h7829c7327eb005efpgu
  44: 0x6c9b16fb - usage::h7829c7327eb005efpgu
  45: 0x63100e18 - sys::process2::Command::cwd::h292693c1b697cec3Yeu
  46: 0x765b7c04 - BaseThreadInitThunk
  47: 0x76fcad1f - RtlInitializeExceptionChain
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: "rustc failed"', C:/bot/slave/stable-dist-r
ustc-win-32/build/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c9a88c8 - usage::h7829c7327eb005efpgu
   7: 0x6c99fef1 - main_args::hacd7685d13170c8aChu
   8: 0x6c99c239 - main::h854d123b237a9daepdu
   9: 0x6c99b6ab - main::h854d123b237a9daepdu
  10: 0x63100e18 - sys::process2::Command::cwd::h292693c1b697cec3Yeu
  11: 0x765b7c04 - BaseThreadInitThunk
  12: 0x76fcad1f - RtlInitializeExceptionChain
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Any', C:/bot/slave/stable-dist-rustc-win-32/bu
ild/src/libcore\result.rs:729
stack backtrace:
   1: 0x63102bdf - rt::unwind::register::ha3686cf8cb60783c3wv
   2: 0x630c56fc - rt::unwind::begin_unwind_inner::h6a42beee3bbc278bduv
   3: 0x630c611f - rt::unwind::begin_unwind_fmt::h2fc1fc2db73eb310Rsv
   4: 0x63102640 - rust_begin_unwind
   5: 0x6311cf28 - panicking::panic_fmt::h52b0d659b0d05042wwy
   6: 0x6c99b2cc - main::h854d123b237a9daepdu
   7: 0x6313c59c - rust_try
   8:   0x401645 - main
   9:   0x4013de
  10: 0x765b7c04 - BaseThreadInitThunk
  11: 0x76fcad1f - RtlInitializeExceptionChain
Could not document `iup`.

Caused by:
  Process didn't exit successfully: `rustdoc src\lib.rs --crate-name iup -o C:\Projects\forks\iup-rust\target\doc -L dep
endency=C:\Projects\forks\iup-rust\target\debug -L dependency=C:\Projects\forks\iup-rust\target\debug\deps --extern libc
=C:\Projects\forks\iup-rust\target\debug\deps\liblibc-d8d1720964a096ed.rlib --extern libc=C:\Projects\forks\iup-rust\tar
get\debug\deps\liblibc-d8d1720964a096ed.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16
fee22cb642ccd4.rlib --extern iup_sys=C:\Projects\forks\iup-rust\target\debug\deps\libiup_sys-16fee22cb642ccd4.rlib` (exi
t code: 101)

Interesting but let's try something that is not a null character to see what happens:

/// \x61

Ha! I get a a in my html output!

Meta

PS C:\Projects\forks\iup-rust> rustc --version --verbose
rustc 1.0.0 (a59de37e9 2015-05-13) (built 2015-05-14)
binary: rustc
commit-hash: a59de37e99060162a2674e3ff45409ac73595c0e
commit-date: 2015-05-13
build-date: 2015-05-14
host: i686-pc-windows-gnu
release: 1.0.0
PS C:\Projects\forks\iup-rust> cargo --version --verbose
cargo 0.2.0-nightly (efb482d 2015-04-30) (built 2015-04-30)
PS C:\Projects\forks\iup-rust> rustdoc --version --verbose
rustdoc 1.0.0 (a59de37e9 2015-05-13) (built 2015-05-14)
binary: rustdoc
commit-hash: a59de37e99060162a2674e3ff45409ac73595c0e
commit-date: 2015-05-13
build-date: 2015-05-14
host: i686-pc-windows-gnu
release: 1.0.0
@thelink2012 thelink2012 changed the title rustdoc does not expand doc fragments of macros properly rustdoc does not expand escape sequences in doc fragments of macros properly Jun 1, 2015
@steveklabnik steveklabnik added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Jun 8, 2015
@barosl barosl mentioned this issue Jul 4, 2015
Merged
barosl added a commit to barosl/rust that referenced this issue Jul 9, 2015
@barosl
Preserve escape sequences in documentation comments on macro expansion
5c60d1d 
Escape sequences in documentation comments must not be parsed as a
normal string when expanding a macro, otherwise some innocent but
invalid-escape-sequence-looking comments will trigger an ICE.

Although this commit replaces normal string literals with raw string
literals in macro expansion, this shouldn't be much a problem
considering documentation comments are converted into attributes before
being passed to a macro anyways.

Fixes rust-lang#25929.
Fixes rust-lang#25943.
Manishearth added a commit to Manishearth/rust that referenced this issue Jul 17, 2015
@Manishearth
Rollup merge of rust-lang#26777 - barosl:macro-doc-escapes, r=pnkfelix
22aa163 
Escape sequences in documentation comments must not be parsed as a normal string when expanding a macro, otherwise some innocent but invalid-escape-sequence-looking comments will trigger an ICE.

Although this commit replaces normal string literals with raw string literals in macro expansion, this shouldn't be much a problem considering documentation comments are converted into attributes before being passed to a macro anyways.

Fixes rust-lang#25929.
Fixes rust-lang#25943.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@steveklabnik @thelink2012