Incremental text sync panics

[matklad]

            0ms - infer:wait (319 calls)
            2ms - parse_macro_query (64 calls)
thread 'main' panicked at 'assertion failed: self.is_char_boundary(n)', /home/matklad/.rustup/toolchains/beta-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/macros/mod.rs:34:9
stack backtrace:
   21ms - handle_inlay_hints
       20ms - inlay_hints
            6ms - Semantics::analyze2 (307 calls)
            0ms - crate_def_map:wait (42 calls)
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1069
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1504
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:218
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:511
  11: rust_begin_unwind
             at src/libstd/panicking.rs:419
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:111
  13: core::panicking::panic
             at src/libcore/panicking.rs:54
  14: alloc::string::String::replace_range
  15: rust_analyzer::main_loop::apply_document_changes
  16: ra_vfs::Vfs::change_file_overlay

cc @lnicola, not sure how to repro this, but I've seen this a couple of times.

[lnicola]

Oof :-(. Were you using multiple cursors or doing anything fancy?

[matklad]

Yeah, not sure what was the problem. I delibiratelly tried doing weird things, but no, I can't repro :(

[lnicola]

I might have found a repro:

enum Enum {
    A,
    B,
}

struct Foo {
    f: Option<Enum>,
}

fn f(foo: &Foo) {
    match foo.f {
        None => (),
        Some(En) => (),
    }
}

<|>fn main() {}

Press Ctrl-Home and cut.

[lnicola]

Actually, it sometimes crashes in a different place:

thread '<unnamed>' panicked at 'index out of bounds: the len is 2 but the index is 12', crates/ra_ide_db/src/line_index.rs:88:9
  14: ra_ide_db::line_index::LineIndex::offset
  15: rust_analyzer::main_loop::handlers::handle_code_action

It might be caused by a bad incremental edit, but I think I've also seen it on the first edit after restarting the server.

I wonder if we might be using a stale line index for diagnostics after applying the edit.

[matklad]

Seems like there might be just two independent crashes? I also thought that this is some runaway assist, but there's rust_analyzer::main_loop::apply_document_changes in my stack trace.

[lnicola]

I still can't reproduce this reliably, but it seems to happen when using undo and redo.

[lnicola]

It might be a different issue, but I have a repro. Type a letter here:

// <|>ț
fn main() {}

It seems to be caused by the non-ASCII character, so maybe again we have two different issues.

[lnicola]

It looks like this:

[crates/rust-analyzer/src/main_loop.rs:670] &old_text = "// ț\n"
[crates/rust-analyzer/src/main_loop.rs:671] &content_changes = [
    TextDocumentContentChangeEvent {
        range: Some(
            Range {
                start: Position {
                    line: 0,
                    character: 3,
                },
                end: Position {
                    line: 0,
                    character: 3,
                },
            },
        ),
        range_length: Some(
            0,
        ),
        text: "a",
    },
]
[crates/rust-analyzer/src/main_loop.rs:699] &line_index = LineIndex {
    newlines: [
        0,
        6,
    ],
    utf16_lines: {
        0: [
            Utf16Char {
                start: 3,
                end: 5,
            },
        ],
    },
}
[crates/rust-analyzer/src/main_loop.rs:701] &range = 4..4
thread 'main' panicked at 'assertion failed: self.is_char_boundary(n)'

We're inserting the new character right in the middle of that "ț". I'm not sure yet what utf16_lines is supposed to contain (but it seems correct), so maybe there's a bug in the LineIndex mapping of columns?

[lnicola]

Filed #4276, but if you didn't run into this while editing comments in Cyrillic or with emojis, there might still be other bugs around.

[matklad]

but if you didn't run into this while editing comments in Cyrillic or with emojis, there might still be other bugs around.

I actually realized why it might be hard to reproduce -- the point at which we get a panic might not be the point where we hit the bug. Rather, we can apply a change successfully, but wrongly, and it'll blow-up down the line during unrelated edit.

, but if you didn't run into this while editing comments in Cyrillic or with emojis, there might still be other bugs around.

Yeah, I think I don't actually have non-ascii text in the code I was working with... So looks like is_char_boundary(n) is actually n > text.len()

[matklad]

... which means that #4276 can't be a fix here? Urgh, non-constructive proofs of bugs' existence are the worst =/

[lnicola]

Do you want to disable incremental sync until this is fixed? But I'm afraid it will never get fixed if we do that 😛.

Maybe we should catch the panic and log the original text and list of changes, just in case.