lexer: Disallow bare CR in raw byte strings #60793

Xanewok · 2019-05-13T18:53:27Z

Handles bare CR ~~but doesn't translate \r\n to \n yet in raw strings yet~~ and translates CRLF to LF in raw strings.

As a side-note I think it'd be good to change the unescape_ to return plain iterators to reduce some boilerplate (e.g. has_error could benefit from collecting Result<T> and aborting early on errors) but will do that separately, unless I missed something here that prevents it.

@matklad @petrochenkov thoughts?

rust-highfive · 2019-05-13T18:53:38Z

r? @pnkfelix

(rust_highfive has picked a reviewer for you, use r? to override)

Xanewok · 2019-05-13T18:54:38Z

src/libsyntax/lib.rs

@@ -18,6 +18,7 @@
 #![feature(step_trait)]
 #![feature(try_trait)]
 #![feature(unicode_internals)]
+#![feature(rustc_private)]


The changes from the first commit (so cargo check works) were of drive-by nature, can remove them if needed

Yes, please remove.

Done. Out of curiosity, why should we skip this? Is it because it adds noise and is somewhat obvious? (after all we're in the compiler code)

Less noise and impossible to forget something if it's added automatically.
rustc crates are not supposed to be built in isolation, outside of the rustc build system, so boilerplate common for all rustc crates is regularly moved to the workspace or rustbuild.
The rustc_private features in particular were moved in #41847.

IIRC, x.py check should provide the same functionality as cargo check.

src/libsyntax/parse/unescape.rs

src/test/ui/parser/raw-byte-string-literals.rs

matklad · 2019-05-13T20:52:04Z

As a side-note I think it'd be good to change the unescape_ to return plain iterators to reduce some boilerplate (e.g. has_error could benefit from collecting Result and aborting early on errors) but will do that separately, unless I missed something here that prevents it.

I think changing unescape_ to usual, external iterators, will make implementation trickier and slower in the common case: one would have to reify the state, instead of keeping it implicitly on the stack. If I am mistaking and the impl turns out not significantly worse, I am all for flipping to external iterators!

However, I think here it's counter-intiutively more important to preserve implementation simplicity (that's where all interesting bits are) rather than interface simplicity (interface is a glue code, which is isolated), so if internal iteration is easier to implement, I'd keep it.

src/libsyntax/parse/literal.rs

petrochenkov · 2019-05-15T15:06:46Z

Closes #60604.

I'd keep #60604 open until the \r\n -> \n translation is moved to the file loading stage (or a decision that we don't want to move it is made).

petrochenkov · 2019-05-15T15:07:40Z

The direction seems right (same as #60261 basically).
I'll wait with review until this fully ready.

r? @petrochenkov

bors · 2019-05-23T16:17:41Z

☔ The latest upstream changes (presumably #60965) made this pull request unmergeable. Please resolve the merge conflicts.

It was commented out as part of rust-lang@8a8e497. Done probably by accident, since the code in question was moved to a match arm, along with newly introduced logic to detect bare CRs in raw strings.

Xanewok · 2019-06-08T18:10:10Z

(...) will make implementation trickier and slower in the common case: one would have to reify the state, instead of keeping it implicitly on the stack

@matklad maybe we could use iter::from_fn?

Rebased the PR and would like for this to land and then land a separate one which translates \r\n -> \n if possible.
@petrochenkov this should be ready for review now

rust-highfive · 2019-06-08T18:23:55Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:2bd09510:start=1560017242083832836,finish=1560017331892705772,duration=89808872936
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:04:12] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:12] tidy error: /checkout/src/libsyntax/parse/literal.rs:7: line longer than 100 chars
[00:04:13] tidy error: /checkout/src/test/ui/parser/raw-byte-string-literals.rs:3: CR character
[00:04:17] some tidy checks failed
[00:04:17] 
[00:04:17] 
[00:04:17] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:17] 
[00:04:17] 
[00:04:17] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:17] Build completed unsuccessfully in 0:01:16
---
travis_time:end:05adf9bc:start=1560017599003094722,finish=1560017599008008025,duration=4913303
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:04afd132
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:02450e10
travis_time:start:02450e10
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:058c44c0
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-06-08T19:29:31Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:13322194:start=1560018496448215263,finish=1560018585083139017,duration=88634923754
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[00:58:16] .................................................................................................... 3800/5657
[00:58:19] .........................................................ii......................................... 3900/5657
[00:58:22] ..............................................................................i..................... 4000/5657
[00:58:24] .................................................................................................... 4100/5657
[00:58:26] ..........................................i..........................................F.............. 4200/5657
[00:58:41] .................................................................................................... 4400/5657
[00:58:47] .................................................................................................... 4500/5657
[00:58:50] .................................................................................................... 4600/5657
[00:58:54] .................................................................................................... 4700/5657
---
[00:59:32] 
[00:59:32] ---- [ui] ui/parser/raw-byte-string-literals.rs stdout ----
[00:59:32] diff of stderr:
[00:59:32] 
[00:59:32] 1 error: bare CR not allowed in string, use \r instead
[00:59:32] +   --> $DIR/raw-byte-string-literals.rs:4:9
[00:59:32] 3    |
[00:59:32] 4 LL |     br"a
";
---
[00:59:32] 10 LL |     br"é";
[00:59:32] 11    |        ^
[00:59:32] 
[00:59:32] 12 
[00:59:32] 13 error: found invalid character; only `#` is allowed in raw string delimitation: ~
[00:59:32] +   --> $DIR/raw-byte-string-literals.rs:6:6
[00:59:32] 15    |
[00:59:32] 15    |
[00:59:32] 16 LL |     br##~"a"~##;
[00:59:32] 
[00:59:32] 
[00:59:32] The actual stderr differed from the expected stderr.
[00:59:32] Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/parser/raw-byte-string-literals/raw-byte-string-literals.stderr
[00:59:32] Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/parser/raw-byte-string-literals/raw-byte-string-literals.stderr
[00:59:32] To update references, rerun the tests and pass the `--bless` flag
[00:59:32] To only update this specific test, also pass `--test-args parser/raw-byte-string-literals.rs`
[00:59:32] error: 1 errors occurred comparing output.
[00:59:32] status: exit code: 1
[00:59:32] status: exit code: 1
[00:59:32] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/parser/raw-byte-string-literals.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/parser/raw-byte-string-literals" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "continue-parse-after-error" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/parser/raw-byte-string-literals/auxiliary" "-A" "unused"
[00:59:32] ------------------------------------------
[00:59:32] 
[00:59:32] ------------------------------------------
[00:59:32] stderr:
[00:59:32] stderr:
[00:59:32] ------------------------------------------
[00:59:32] error: bare CR not allowed in string, use \r instead
[00:59:32]    |
[00:59:32] LL |     br"a
[00:59:32] LL |     br"a
"; //~ ERROR bare CR not allowed in string
[00:59:32] 
[00:59:32] error: raw byte string must be ASCII
[00:59:32]   --> /checkout/src/test/ui/parser/raw-byte-string-literals.rs:5:8
[00:59:32]    |
[00:59:32]    |
[00:59:32] LL |     br"é";  //~ ERROR raw byte string must be ASCII
[00:59:32] 
[00:59:32] 
[00:59:32] error: found invalid character; only `#` is allowed in raw string delimitation: ~
[00:59:32]    |
[00:59:32]    |
[00:59:32] LL |     br##~"a"~##;  //~ ERROR only `#` is allowed in raw string delimitation
[00:59:32] 
[00:59:32] error: aborting due to 3 previous errors
[00:59:32] 
[00:59:32] 
---
[00:59:32] thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:521:22
[00:59:32] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[00:59:32] 
[00:59:32] 
[00:59:32] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[00:59:32] 
[00:59:32] 
[00:59:32] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[00:59:32] Build completed unsuccessfully in 0:55:36
---
travis_time:end:05477f8e:start=1560022168009779236,finish=1560022168014750157,duration=4970921
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:23233db1
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:243bd2dc
travis_time:start:243bd2dc
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:2b0162c8
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-06-08T20:54:41Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:03079aba:start=1560023244962839850,finish=1560023342162488535,duration=97199648685
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[01:01:33] .....................ii............................................................................. 1400/2925
[01:01:45] .................................................................................................... 1500/2925
[01:01:54] .........................................................................i.......i.................. 1600/2925
[01:02:08] .................................................................................................... 1700/2925
[01:02:22] ..............................F..................................................................... 1800/2925
[01:02:46] ....i.......................................................................i....................... 2000/2925
[01:03:12] .................................................................................................... 2100/2925
[01:03:34] ........................................................................................test [run-pass] run-pass/mpsc_stress.rs has been running for over 60 seconds
[01:03:35] ............ 2200/2925
---
[01:05:24] ------------------------------------------
[01:05:24] stderr:
[01:05:24] ------------------------------------------
[01:05:24] thread 'main' panicked at 'assertion failed: `(left == right)`
[01:05:24]   left: `"string\r\nliteral"`,
[01:05:24]  right: `"string\nliteral"`', /checkout/src/test/run-pass/lexer-crlf-line-endings-string-literal-doc-comment.rs:32:5
[01:05:24] 
[01:05:24] ------------------------------------------
[01:05:24] 
[01:05:24] 
---
[01:05:24] test result: FAILED. 2915 passed; 1 failed; 9 ignored; 0 measured; 0 filtered out
[01:05:24] 
[01:05:24] 
[01:05:24] 
[01:05:24] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-pass" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-pass" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:05:24] 
[01:05:24] 
[01:05:24] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:05:24] Build completed unsuccessfully in 1:01:29
---
travis_time:end:0b1d0db8:start=1560027277780394948,finish=1560027277840754842,duration=60359894
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:20f16e6d
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:1104b394
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

src/libsyntax/parse/lexer/mod.rs

src/libsyntax/parse/unescape.rs

Xanewok · 2019-06-09T14:12:30Z

Applied feedback and actually implemented CRLF -> LF rewrite in raw byte strings (forgot about the byte variant 🤦‍♂️ ) in 735ac05#diff-a8037ce27e05cd3a29b041f9ec00f94eR176

matklad · 2019-06-09T14:42:16Z

LGTM now, let's wait for @petrochenkov review!

src/test/ui/parser/raw-byte-string-literals.stderr

petrochenkov · 2019-06-10T14:05:50Z

r=me with #60793 (comment) fixed

petrochenkov · 2019-06-10T15:35:27Z

@bors r+

bors · 2019-06-10T15:35:28Z

📌 Commit 630d5f3 has been approved by petrochenkov

bors · 2019-06-10T23:32:19Z

⌛ Testing commit 630d5f3 with merge 5e2c110...

@matklad

lexer: Disallow bare CR in raw byte strings Handles bare CR ~but doesn't translate `\r\n` to `\n` yet in raw strings yet~ and translates CRLF to LF in raw strings. As a side-note I think it'd be good to change the `unescape_` to return plain iterators to reduce some boilerplate (e.g. `has_error` could benefit from collecting `Result<T>` and aborting early on errors) but will do that separately, unless I missed something here that prevents it. @matklad @petrochenkov thoughts?

bors · 2019-06-11T02:25:31Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: petrochenkov
Pushing 5e2c110 to master...

rust-highfive · 2019-06-11T02:26:39Z

📣 Toolstate changed by #60793!

Tested on commit 5e2c110.
Direct link to PR: #60793

💔 rls on linux: test-pass → test-fail (cc @Xanewok, @rust-lang/infra).

@Xanewok

Tested on commit rust-lang/rust@5e2c110. Direct link to PR: <rust-lang/rust#60793> 💔 rls on linux: test-pass → test-fail (cc @Xanewok, @rust-lang/infra).

@matklad

…klad Add unit tests for unescaping raw (byte) strings Adds unit tests for functionality introduced in rust-lang#60793. r? @matklad @petrochenkov

rust-highfive assigned pnkfelix May 13, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 13, 2019

Xanewok commented May 13, 2019

View reviewed changes

matklad reviewed May 13, 2019

View reviewed changes

src/libsyntax/parse/unescape.rs Outdated Show resolved Hide resolved

src/libsyntax/parse/unescape.rs Outdated Show resolved Hide resolved

src/test/ui/parser/raw-byte-string-literals.rs Show resolved Hide resolved

petrochenkov self-assigned this May 13, 2019

petrochenkov reviewed May 15, 2019

View reviewed changes

src/libsyntax/parse/literal.rs Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 15, 2019

petrochenkov unassigned pnkfelix May 15, 2019

Xanewok force-pushed the raw-string-cleanup branch 2 times, most recently from 9085079 to 03e4521 Compare June 8, 2019 18:03

Xanewok added 3 commits June 8, 2019 20:05

Clean up minor bits

5b17a5e

Separate a scan_raw_string (similar raw_byte variant)

b8e3533

Remove redundant, commented out code

08ede49

It was commented out as part of rust-lang@8a8e497. Done probably by accident, since the code in question was moved to a match arm, along with newly introduced logic to detect bare CRs in raw strings.

Xanewok force-pushed the raw-string-cleanup branch from 03e4521 to 6d7bc3b Compare June 8, 2019 18:05

Xanewok marked this pull request as ready for review June 8, 2019 18:06

Xanewok added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 8, 2019

Xanewok mentioned this pull request Jun 8, 2019

Improve raw string diagnostic #61255

Closed

Xanewok force-pushed the raw-string-cleanup branch from 7e5ef6d to e5d5a08 Compare June 8, 2019 19:46

Validate and transcribe raw strings via unescape module

cab7e7f

matklad reviewed Jun 9, 2019

View reviewed changes

src/libsyntax/parse/lexer/mod.rs Show resolved Hide resolved

matklad reviewed Jun 9, 2019

View reviewed changes

src/libsyntax/parse/unescape.rs Outdated Show resolved Hide resolved

Xanewok added 4 commits June 9, 2019 14:15

Add test for raw byte CRLF translation

8cd51ff

Add a doc comment for scan_raw_string

3c1d352

Actually translate CRLF in raw byte strings and unify unescape impl

735ac05

Use char byte calculation using existing iterator

63dc7da

petrochenkov reviewed Jun 10, 2019

View reviewed changes

src/test/ui/parser/raw-byte-string-literals.stderr Outdated Show resolved Hide resolved

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 10, 2019

Don't suggest using \r in raw strings

630d5f3

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 10, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 11, 2019

bors merged commit 630d5f3 into rust-lang:master Jun 11, 2019

Xanewok deleted the raw-string-cleanup branch June 20, 2019 08:44

Xanewok mentioned this pull request Jun 20, 2019

Add unit tests for unescaping raw (byte) strings #61996

Merged

ehuss mentioned this pull request Jun 26, 2019

Character and string token definitions need updating. rust-lang/reference#626

Open

6 tasks

matklad mentioned this pull request Jul 22, 2019

Raw Byte strings do not handle \r #60604

Closed

petrochenkov mentioned this pull request Sep 11, 2019

convert \r\n -> \n in include_str! macro #63681

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lexer: Disallow bare CR in raw byte strings #60793

lexer: Disallow bare CR in raw byte strings #60793

Xanewok commented May 13, 2019 •

edited

Loading

rust-highfive commented May 13, 2019

Xanewok May 13, 2019

petrochenkov May 15, 2019

Xanewok Jun 8, 2019

petrochenkov Jun 10, 2019

petrochenkov Jun 10, 2019

matklad commented May 13, 2019

petrochenkov commented May 15, 2019

petrochenkov commented May 15, 2019 •

edited

Loading

bors commented May 23, 2019

Xanewok commented Jun 8, 2019 •

edited

Loading

rust-highfive commented Jun 8, 2019

rust-highfive commented Jun 8, 2019

rust-highfive commented Jun 8, 2019

Xanewok commented Jun 9, 2019

matklad commented Jun 9, 2019

petrochenkov commented Jun 10, 2019

petrochenkov commented Jun 10, 2019

bors commented Jun 10, 2019

bors commented Jun 10, 2019

bors commented Jun 11, 2019

rust-highfive commented Jun 11, 2019

lexer: Disallow bare CR in raw byte strings #60793

lexer: Disallow bare CR in raw byte strings #60793

Conversation

Xanewok commented May 13, 2019 • edited Loading

rust-highfive commented May 13, 2019

Xanewok May 13, 2019

Choose a reason for hiding this comment

petrochenkov May 15, 2019

Choose a reason for hiding this comment

Xanewok Jun 8, 2019

Choose a reason for hiding this comment

petrochenkov Jun 10, 2019

Choose a reason for hiding this comment

petrochenkov Jun 10, 2019

Choose a reason for hiding this comment

matklad commented May 13, 2019

petrochenkov commented May 15, 2019

petrochenkov commented May 15, 2019 • edited Loading

bors commented May 23, 2019

Xanewok commented Jun 8, 2019 • edited Loading

rust-highfive commented Jun 8, 2019

rust-highfive commented Jun 8, 2019

rust-highfive commented Jun 8, 2019

Xanewok commented Jun 9, 2019

matklad commented Jun 9, 2019

petrochenkov commented Jun 10, 2019

petrochenkov commented Jun 10, 2019

bors commented Jun 10, 2019

bors commented Jun 10, 2019

bors commented Jun 11, 2019

rust-highfive commented Jun 11, 2019

Xanewok commented May 13, 2019 •

edited

Loading

petrochenkov commented May 15, 2019 •

edited

Loading

Xanewok commented Jun 8, 2019 •

edited

Loading