Incomplete highlights for complex queries (tree-sitter match limit not large enough?).

[JaagupAverin]

Check for existing issues

• Completed

Describe the bug / provide steps to reproduce it

Currently the match limit for Tree-Sitter is assigned at 64 in zed/crates/language/src/syntax_map.rs:1820:

impl QueryCursorHandle {
    pub fn new() -> Self {
        let mut cursor = QUERY_CURSORS.lock().pop().unwrap_or_default();
        cursor.set_match_limit(64);
        QueryCursorHandle(Some(cursor))
    }
}

This limit appears to cause issues quite easily in Python, for example this code:

class Foo:
    def __init__(self) -> None:
        self.a = 1
        """Docstring A"""
        self.b = 2
        """Docstring B"""
        self.c = 3
        """Docstring C"""
        self.d = 4
        """Docstring D"""
        self.e = 5
        """Docstring E"""

is highlighted as follows:

(i.e. after a certain token, the highlighting only semi-works - the attributes and strings are captured as expected, but the string.doc capture stops being applied)
By doubling the match limit to 128, the highlighting works better, but ofc I'm only sweeping the problem under the carpet:

I don't know if this limit can be increased easily without performance impact (probably would need to increase it to at least 1024 or so to cover even the most complicated situations?), or in fact the issue lies somewhere else entirely, so I won't be attempting a PR myself but I'm happy if this gets a solution soon :)

Environment

Zed Preview 0.166.1 546b49c

If applicable, add screenshots or screencasts of the incorrect state / behavior

No response

If applicable, attach your Zed.log file to this issue.

No response

[notpeter]

pub fn set_match_limit(&mut self, limit: u32)

Set the maximum number of in-progress matches for this cursor. The limit must be > 0 and <= 65536.

Source: set_match_limit docs.

---

Neovim uses 256. See: runtime/lua/vim/treesitter/query.lua. Previously it was 64; and 32 before that.

• fix(treesitter): bump match limit up neovim/neovim#18397
• fix(treesitter): raise ts_match_limit to 256 neovim/neovim#22497
• vim.treesitter query problem neovim/neovim#22055
• Treesitter: ts_match_limit of 256 is not always enough neovim/neovim#26325

Helix uses 256 too. See: helix-core/src/syntax.rs. Previously it was 64.

• improve performance of tree sitter query captures (for text object motions in particular) helix-editor/helix#4707
• Bump tree-sitter pattern matching limit to 256 helix-editor/helix#4830
• Tree-sitter match limit is insufficient sometimes helix-editor/helix#6509

[0x2CA]

Simply increasing this value will lead to performance problems, which needs to be balanced between effect and performance, different languages have different balance points, I think it may need to be modified freely according to the different languages will be better, so that most languages keep the value low to have good performance is more important, some languages allow a little sacrifice of performance for the sake of better effect. In a more exaggerated case, if the user wants to be able to configure a larger value.

Of course the time concepts of the other comments make more sense in lieu of unclear values, and it's better to let the user configure the maximum accepted delay time