rustfmt fails with errors "left behind trailing whitespace"

[sdht0]

I have rustfmt-preview installed on the nightly toolchain (rustc 1.30.0-nightly (3edb355 2018-08-03)) using rustup.

Running cargo +nightly fmt on the crate: https://github.com/frankmcsherry/differential-dataflow results in numerous "left behind trailing whitespace" errors. Some sample errors:

internal error: left behind trailing whitespace
  --> [..]/differential-dataflow/src/algorithms/prefix_sum.rs:70
   |
70 |         .iterate(|ranges| 
   |                          ^
warning: rustfmt may have failed to format. See previous 1 errors.

  --> [..]/differential-dataflow/examples/seb.rs:37
   |
37 |       let cooccurrences = 
   |                          ^
  --> [..]/differential-dataflow/examples/seb.rs:44
   |
44 |       let row_sums = 
   |                     ^

  --> [..]/differential-dataflow/examples/dataflow.rs:35
   |
35 |             let (tweet_input, tweets) = scope.new_collection(); 
   |                                                                ^

[topecongiro]

rustfmt is failing due to this line.

[topecongiro]

wrap_str checks the width of formatted code, and veto if it exceeds max width. Currently it does not distinguish between normal code and comments. I think that it should skip checking the width of the comment.

https://github.com/rust-lang-nursery/rustfmt/blob/b2a80f99426363c092968ef1b4e771c159d7aa1b/src/utils.rs#L350-L358

[sdht0]

Thank you!

[sdht0]

Hi. I think this bug still exists.

% rustfmt -V    
rustfmt 0.99.2-nightly (5c9a2b6 2018-08-07)

% cargo fmt
internal error: left behind trailing whitespace
  --> [...]/differential-dataflow/src/algorithms/prefix_sum.rs:69
   |
69 |     unit_ranges.iterate(|ranges| 
   |                                 ^

warning: rustfmt may have failed to format. See previous 1 errors.

[topecongiro]

Thanks! Looks like there was another issue when there is comment between closure's argument list and its body.

E.g.

|foo|
// comment
foo + 3;

[sdht0]

You're welcome. Before rustfmt 1.0, is there any plan to run https://github.com/rust-lang-nursery/crater but for rustfmt? That would actually highlight any bugs like this and also ones that result in compilation/test failures.

[scampi]

With the file below, the error is a bit different, but also more informative:

fn main() {
    let toto = |foo|
        // my comment
        foo + 3;
}

The error:

internal error: not formatted because a comment would be lost
 --> /home/yfful/documents/code/rustfmt/issue2896.rs:2
  |
2 |     let toto = |foo|
  |
  = note: set `error_on_unformatted = false` to suppress the warning against comments or string literals

Tested with rustfmt 0.99.2-nightly (2db2327a 2018-08-16).

[sdht0]

Does not look related, but rustfmt is also failing to format the long lines https://github.com/frankmcsherry/differential-dataflow/blob/master/src/trace/implementations/ord.rs#L210 and https://github.com/frankmcsherry/differential-dataflow/blob/master/src/trace/implementations/ord.rs#L301. The rest of the file seems to have been correctly formatted. Should I open a new bug report?

[sdht0]

Are bugs like this not going to be fixed before the 1.0.0 release? rustfmt 1.0.0-stable (4a4e508 2018-11-29) still has this bug and I guess the other open issues marked bug.

[nrc]

It will not - what's on beta is what will be in the 1.0. We can fix this post-1.0, since the warning can be fixed without being backwards incompatible

[ogiekako]

Could be a separate issue, but rustfmt fails with the following code.

fn main() {
    if 0 == ' ' 
    /* x */ as i32 {
    }
}

error:

error[internal]: left behind trailing whitespace
 --> /playground/src/main.rs:2:2:15
  |
2 |     if 0 == ' ' 
  |                ^
  |

warning: rustfmt has failed to format. See previous 1 errors.

Tested with rustfmt 1.2.2-nightly (5274b49 2019-04-24).

[Boscop]

I also got 5 of these errors today after cargo fix and after updating my rustfmt to rustfmt 1.4.11-nightly (18382352 2019-12-03).

[idubrov]

Same issue: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=699a67c3936eb4c43b5d5533f500ea5c

[davidBar-On]

Pull request #4391 should resolve the issue. Seem to be because when a comment at the end of a line is moved to a new line, there is no trimming of the blanks before the comment. The PR splits the result line, strip whitespaces from the end of each line and recombine the lines.

[mikechambers]

fyi, here is another example where I am hitting this issue

Error:

error[internal]: left behind trailing whitespace
  --> /Users/mesh/Exercism/rust/beer-song/src/lib.rs:11:11:0

With the error occurring on the long Strings. (manually wrapping the strings fixes the issue)

Here is the complete code:

pub fn verse(n: u32) -> String {
    let count: u32 = n;

    let mut out:String  = match count {
        2..=u32::MAX => {
            let s = if count > 2 {
                "s"
            } else {
                ""
            };
    
            format!(
                "{count} bottles of beer on the wall, {count} bottles of beer.\nTake one down and pass it around, {n_count} bottle{s} of beer on the wall.",
                count = count,
                n_count = count - 1,
                s = s
            )
        },
        1 => {
            String::from(
                "1 bottle of beer on the wall, 1 bottle of beer.\nTake it down and pass it around, no more bottles of beer on the wall.",
            )
        },
        0 => {
            String::from(
                "No more bottles of beer on the wall, no more bottles of beer.\nGo to the store and buy some more, 99 bottles of beer on the wall.",
            )
        }
    };

    out.push_str("\n");

    out
}

pub fn sing(start: u32, end: u32) -> String {
    let mut song_verses: Vec<String> = Vec::new();

    for i in (end..start + 1).rev() {
        song_verses.push(verse(i));
    }

    song_verses.join("\n")
}

[davidBar-On]

@mikechambers, the error happens because line 11 is not empty but starts with whitespaces. rustfmt does not trim lines that includes only whitespaces (I don't know if this is by design). The error doesn't seem to be related to the long strings.

[davepacheco]

rustfmt does not trim lines that includes only whitespaces (I don't know if this is by design).

@davidBar-On While digging a bit into #4663, I found that doesn't seem to be true. For example:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=df8ccaeebc257eb892fc014fc7586424

That code has trailing whitespace on line 2. If you reformat the code, rustfmt removes that whitespace and produces no errors. Did I misunderstand what you meant?

[davidBar-On]

@davepacheco, I retried to format both the code in this issue and the code in #4663, and found that when using edition = "2018" in rustfmt.toml (current default) there is no error. However, with edition = "2015" the error about the trailing whitespace occurs in both cases. The rustplay link you provided sets edition=2018. but if you change it to edition=2015 the error occurs.

It seems that the whitespace error was fixed during last few months (as far as I remember, when I wrote the note on Nov., the error still occured).

[calebcartwright]

rustfmt does not trim lines that includes only whitespaces (I don't know if this is by design)
While digging a bit into #4663, I found that doesn't seem to be true. For example

Perhaps I can provide some clarity here... rustfmt is an AST-driven pretty printer type of code formatter which means that it visits and has to emit something for each node in the tree. In cases where rustfmt fails to format a particular node in the AST (which can happen for myriad reasons), the raw contents within the span of that node are emitted back out (i.e. the original snippet exactly as it was in the original input). This is done to prevent deleting parts of your code just because rustfmt failed to format those parts.

If the original input contained trailing whitespace in one of these failed-to-format cases, then accordingly the output/emitted formatting will too, and that in turn later triggers the left behind trailing whitespace error. When you see this it just about always means that there's a bug in how rustfmt is handling that particular kind of AST node, and you just happen to have trailing whitespace somewhere within the associated span. So rustfmt does remove any trailing whitespace as part of successful formatting, but in cases of failed formatting there may be some whitespace left behind depending on the original input.

Additionally, though that left behind trailing whitespace error message is always the same in these cases, the bugs/causes and fixes are entirely unrelated (e.g. a case of this error in one place, like a match expression, is completely different than seeing the same error message in a different place, like a local stmt). As such, when a bug is fixed for one case, it typically has zero impact on another.

[oilaba]

I am still getting this error on rustfmt 1.4.25.

[char-ptr]

I am still getting this error on rustfmt 1.5.1-nightly (495b2166 2022-07-03)

[davidBar-On]

It seems that the issue was fixed in version Two. With version = "One" in rustfmt.toml I get the error, but with version = "Two" there is no error.

[ytmimi]

@davidBar-On and @pozm, would you mind sharing the code snippets that you're using to test / trigger the issue respectively. I want to make sure we're all on the same page here. As mentioned in #2896 (comment) there are potentially many different reasons for rustfmt to inform you that it "left behind trailing whitespace", so you each might be running into a different underlying issue.

[davidBar-On]

@ytmimi, I used the code from @mikechambers comment above.

[calebcartwright]

Unfortunately I believe this issue has outlived its utility, and has become a pile on that's sprawled in too many directions to be an efficient means of addressing anything.

The original report, and thus sole scope, of this issue was utilizing a different component, which no longer exists, from ~4 years ago. If someone wants to go back to the original cases mentioned in the OP, and see whether or not the different syntactical scenarios are actually still reproducible with modern day rustfmt then that would be great. Any such reproducible cases should be cross referenced against other such open issues, and if there's not something that already exists then a new issue can be opened.

The same applies for the different scenarios surfaced in comments over the years.

Given the above, I'm going to close and lock this so that nothing gets lost.