Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pygls 1.0.0a0 #230

Merged
merged 8 commits into from
Dec 5, 2022
Merged

pygls 1.0.0a0 #230

merged 8 commits into from
Dec 5, 2022

Conversation

dimbleby
Copy link
Contributor

pygls is going 1.0, and the main change is that they're migrating away from their homegrown pydantic models for the protocol and using https://pypi.org/project/lsprotocol/ (https://github.com/microsoft/lsprotocol) - which seems to be autogenerated (and uses cattrs).

that turns out not to be a tremendously impactful change: a few things get renamed in more-or-less obvious ways, the validation gets slightly stricter in a couple of places, that's about it.

I don't expect you'll want to merge / publish before they actually go 1.0.0. But if and when you're happy that this seems anyway not worse than before, I expect they'd be glad to hear about it in openlawlibrary/pygls#273

@dimbleby dimbleby changed the title pygls 1.0.0a pygls 1.0.0a0 Oct 18, 2022
@pappasam
Copy link
Owner

Thanks for the PR! Tried testing it with my current client (coc.nvim) and it breaks with the following deserialization message:

ERROR:pygls.protocol:Unable to deserialize message
  + Exception Group Traceback (most recent call last):
  |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/pygls/protocol.py", line 347, in _deserialize_message
  |     return converter.structure(data, request_type)
  |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 309, in structure
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |   File "<cattrs generated structure lsprotocol.types.InitializeRequest>", line 26, in structure_InitializeRequest
  |     if errors: raise __c_cve('While structuring ' + 'InitializeRequest', errors, __cl)
  | cattrs.errors.ClassValidationError: While structuring InitializeRequest (1 sub-exception)
  +-+---------------- 1 ----------------
    | Exception Group Traceback (most recent call last):
    |   File "<cattrs generated structure lsprotocol.types.InitializeRequest>", line 10, in structure_InitializeRequest
    |     res['params'] = __c_structure_params(o['params'], __c_type_params)
    |   File "<cattrs generated structure lsprotocol.types.InitializeParams>", line 63, in structure_InitializeParams
    |     if errors: raise __c_cve('While structuring ' + 'InitializeParams', errors, __cl)
    | cattrs.errors.ClassValidationError: While structuring InitializeParams (1 sub-exception)
    | Structuring class InitializeRequest @ attribute params
    +-+---------------- 1 ----------------
      | Exception Group Traceback (most recent call last):
      |   File "<cattrs generated structure lsprotocol.types.InitializeParams>", line 5, in structure_InitializeParams
      |     res['capabilities'] = __c_structure_capabilities(o['capabilities'], __c_type_capabilities)
      |   File "<cattrs generated structure lsprotocol.types.ClientCapabilities>", line 40, in structure_ClientCapabilities
      |     if errors: raise __c_cve('While structuring ' + 'ClientCapabilities', errors, __cl)
      | cattrs.errors.ClassValidationError: While structuring ClientCapabilities (1 sub-exception)
      | Structuring class InitializeParams @ attribute capabilities
      +-+---------------- 1 ----------------
        | Exception Group Traceback (most recent call last):
        |   File "<cattrs generated structure lsprotocol.types.ClientCapabilities>", line 12, in structure_ClientCapabilities
        |     res['text_document'] = __c_structure_text_document(o['textDocument'], __c_type_text_document)
        |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 574, in _structure_optional
        |     return self._structure_func.dispatch(other)(obj, other)
        |   File "<cattrs generated structure lsprotocol.types.TextDocumentClientCapabilities>", line 184, in structure_TextDocumentClientCapabilities
        |     if errors: raise __c_cve('While structuring ' + 'TextDocumentClientCapabilities', errors, __cl)
        | cattrs.errors.ClassValidationError: While structuring TextDocumentClientCapabilities (1 sub-exception)
        | Structuring class ClientCapabilities @ attribute text_document
        +-+---------------- 1 ----------------
          | Exception Group Traceback (most recent call last):
          |   File "<cattrs generated structure lsprotocol.types.TextDocumentClientCapabilities>", line 144, in structure_TextDocumentClientCapabilities
          |     res['semantic_tokens'] = __c_structure_semantic_tokens(o['semanticTokens'], __c_type_semantic_tokens)
          |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 574, in _structure_optional
          |     return self._structure_func.dispatch(other)(obj, other)
          |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilities>", line 54, in structure_SemanticTokensClientCapabilities
          |     if errors: raise __c_cve('While structuring ' + 'SemanticTokensClientCapabilities', errors, __cl)
          | cattrs.errors.ClassValidationError: While structuring SemanticTokensClientCapabilities (1 sub-exception)
          | Structuring class TextDocumentClientCapabilities @ attribute semantic_tokens
          +-+---------------- 1 ----------------
            | Exception Group Traceback (most recent call last):
            |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilities>", line 5, in structure_SemanticTokensClientCapabilities
            |     res['requests'] = __c_structure_requests(o['requests'], __c_type_requests)
            |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilitiesRequestsType>", line 16, in structure_SemanticTokensClientCapabilitiesRequestsType
            |     if errors: raise __c_cve('While structuring ' + 'SemanticTokensClientCapabilitiesRequestsType', errors, __cl)
            | cattrs.errors.ClassValidationError: While structuring SemanticTokensClientCapabilitiesRequestsType (1 sub-exception)
            | Structuring class SemanticTokensClientCapabilities @ attribute requests
            +-+---------------- 1 ----------------
              | Traceback (most recent call last):
              |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilitiesRequestsType>", line 12, in structure_SemanticTokensClientCapabilitiesRequestsType
              |     res['full'] = __c_structure_full(o['full'], __c_type_full)
              |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 377, in _structure_error
              |     raise StructureHandlerNotFoundError(msg, type_=cl)
              | cattrs.errors.StructureHandlerNotFoundError: Unsupported type: typing.Union[bool, lsprotocol.types.SemanticTokensClientCapabilitiesRequestsTypeFullType1, NoneType]. Register a structure hook for it.
              | Structuring class SemanticTokensClientCapabilitiesRequestsType @ attribute full
              +------------------------------------

Validation of this breaks:

[Trace - 12:05:09 PM] Sending request 'initialize - (0)'.
Params: {
    "processId": 1105947,
    "rootPath": "/home/sroeca/src/Personal/jedi-language-server",
    "rootUri": "file:///home/sroeca/src/Personal/jedi-language-server",
    "capabilities": {
        "workspace": {
            "applyEdit": true,
            "workspaceEdit": {
                "documentChanges": true,
                "resourceOperations": [
                    "create",
                    "rename",
                    "delete"
                ],
                "failureHandling": "undo",
                "normalizesLineEndings": true,
                "changeAnnotationSupport": {
                    "groupsOnLabel": false
                }
            },
            "didChangeConfiguration": {
                "dynamicRegistration": true
            },
            "didChangeWatchedFiles": {
                "dynamicRegistration": true,
                "relativePatternSupport": true
            },
            "codeLens": {
                "refreshSupport": true
            },
            "executeCommand": {
                "dynamicRegistration": true
            },
            "configuration": true,
            "fileOperations": {
                "dynamicRegistration": true,
                "didCreate": true,
                "didRename": true,
                "didDelete": true,
                "willCreate": true,
                "willRename": true,
                "willDelete": true
            },
            "semanticTokens": {
                "refreshSupport": true
            },
            "inlayHint": {
                "refreshSupport": true
            },
            "inlineValue": {
                "refreshSupport": true
            },
            "diagnostics": {
                "refreshSupport": true
            },
            "symbol": {
                "dynamicRegistration": true,
                "symbolKind": {
                    "valueSet": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        9,
                        10,
                        11,
                        12,
                        13,
                        14,
                        15,
                        16,
                        17,
                        18,
                        19,
                        20,
                        21,
                        22,
                        23,
                        24,
                        25,
                        26
                    ]
                },
                "tagSupport": {
                    "valueSet": [
                        1
                    ]
                },
                "resolveSupport": {
                    "properties": [
                        "location.range"
                    ]
                }
            },
            "workspaceFolders": true
        },
        "textDocument": {
            "publishDiagnostics": {
                "relatedInformation": true,
                "versionSupport": true,
                "tagSupport": {
                    "valueSet": [
                        1,
                        2
                    ]
                },
                "codeDescriptionSupport": true,
                "dataSupport": true
            },
            "synchronization": {
                "dynamicRegistration": true,
                "willSave": true,
                "willSaveWaitUntil": true,
                "didSave": true
            },
            "completion": {
                "dynamicRegistration": true,
                "contextSupport": true,
                "completionItem": {
                    "snippetSupport": true,
                    "commitCharactersSupport": true,
                    "documentationFormat": [
                        "markdown",
                        "plaintext"
                    ],
                    "deprecatedSupport": true,
                    "preselectSupport": true,
                    "insertReplaceSupport": true,
                    "tagSupport": {
                        "valueSet": [
                            1
                        ]
                    },
                    "resolveSupport": {
                        "properties": [
                            "documentation",
                            "detail",
                            "additionalTextEdits"
                        ]
                    },
                    "labelDetailsSupport": true,
                    "insertTextModeSupport": {
                        "valueSet": [
                            1,
                            2
                        ]
                    }
                },
                "completionItemKind": {
                    "valueSet": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        9,
                        10,
                        11,
                        12,
                        13,
                        14,
                        15,
                        16,
                        17,
                        18,
                        19,
                        20,
                        21,
                        22,
                        23,
                        24,
                        25
                    ]
                },
                "insertTextMode": 2,
                "completionList": {
                    "itemDefaults": [
                        "commitCharacters",
                        "editRange",
                        "insertTextFormat",
                        "insertTextMode"
                    ]
                }
            },
            "hover": {
                "dynamicRegistration": true,
                "contentFormat": [
                    "markdown",
                    "plaintext"
                ]
            },
            "signatureHelp": {
                "dynamicRegistration": true,
                "contextSupport": true,
                "signatureInformation": {
                    "documentationFormat": [
                        "markdown",
                        "plaintext"
                    ],
                    "activeParameterSupport": true,
                    "parameterInformation": {
                        "labelOffsetSupport": true
                    }
                }
            },
            "references": {
                "dynamicRegistration": true
            },
            "definition": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "documentHighlight": {
                "dynamicRegistration": true
            },
            "documentSymbol": {
                "dynamicRegistration": true,
                "symbolKind": {
                    "valueSet": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        9,
                        10,
                        11,
                        12,
                        13,
                        14,
                        15,
                        16,
                        17,
                        18,
                        19,
                        20,
                        21,
                        22,
                        23,
                        24,
                        25,
                        26
                    ]
                },
                "hierarchicalDocumentSymbolSupport": true,
                "tagSupport": {
                    "valueSet": [
                        1
                    ]
                },
                "labelSupport": true
            },
            "codeAction": {
                "dynamicRegistration": true,
                "isPreferredSupport": true,
                "disabledSupport": true,
                "dataSupport": true,
                "honorsChangeAnnotations": false,
                "resolveSupport": {
                    "properties": [
                        "edit"
                    ]
                },
                "codeActionLiteralSupport": {
                    "codeActionKind": {
                        "valueSet": [
                            "",
                            "quickfix",
                            "refactor",
                            "refactor.extract",
                            "refactor.inline",
                            "refactor.rewrite",
                            "source",
                            "source.organizeImports"
                        ]
                    }
                }
            },
            "codeLens": {
                "dynamicRegistration": true
            },
            "formatting": {
                "dynamicRegistration": true
            },
            "rangeFormatting": {
                "dynamicRegistration": true
            },
            "onTypeFormatting": {
                "dynamicRegistration": true
            },
            "rename": {
                "dynamicRegistration": true,
                "prepareSupport": true,
                "honorsChangeAnnotations": true,
                "prepareSupportDefaultBehavior": 1
            },
            "documentLink": {
                "dynamicRegistration": true,
                "tooltipSupport": true
            },
            "typeDefinition": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "implementation": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "declaration": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "colorProvider": {
                "dynamicRegistration": true
            },
            "foldingRange": {
                "dynamicRegistration": true,
                "rangeLimit": 5000,
                "lineFoldingOnly": true,
                "foldingRangeKind": {
                    "valueSet": [
                        "comment",
                        "imports",
                        "region"
                    ]
                },
                "foldingRange": {
                    "collapsedText": false
                }
            },
            "selectionRange": {
                "dynamicRegistration": true
            },
            "callHierarchy": {
                "dynamicRegistration": true
            },
            "linkedEditingRange": {
                "dynamicRegistration": true
            },
            "semanticTokens": {
                "dynamicRegistration": true,
                "tokenTypes": [
                    "namespace",
                    "type",
                    "class",
                    "enum",
                    "interface",
                    "struct",
                    "typeParameter",
                    "parameter",
                    "variable",
                    "property",
                    "enumMember",
                    "event",
                    "function",
                    "method",
                    "macro",
                    "keyword",
                    "modifier",
                    "comment",
                    "string",
                    "number",
                    "regexp",
                    "decorator",
                    "operator"
                ],
                "tokenModifiers": [
                    "declaration",
                    "definition",
                    "readonly",
                    "static",
                    "deprecated",
                    "abstract",
                    "async",
                    "modification",
                    "documentation",
                    "defaultLibrary"
                ],
                "formats": [
                    "relative"
                ],
                "requests": {
                    "range": true,
                    "full": {
                        "delta": true
                    }
                },
                "multilineTokenSupport": false,
                "overlappingTokenSupport": false,
                "serverCancelSupport": true,
                "augmentsSyntaxTokens": true
            },
            "inlayHint": {
                "dynamicRegistration": true,
                "resolveSupport": {
                    "properties": [
                        "tooltip",
                        "textEdits",
                        "label.tooltip",
                        "label.location",
                        "label.command"
                    ]
                }
            },
            "inlineValue": {
                "dynamicRegistration": true
            },
            "diagnostic": {
                "dynamicRegistration": true,
                "relatedDocumentSupport": true
            },
            "typeHierarchy": {
                "dynamicRegistration": true
            }
        },
        "window": {
            "showMessage": {
                "messageActionItem": {
                    "additionalPropertiesSupport": true
                }
            },
            "showDocument": {
                "support": true
            },
            "workDoneProgress": true
        },
        "general": {
            "regularExpressions": {
                "engine": "ECMAScript",
                "version": "ES2020"
            },
            "markdown": {
                "parser": "marked",
                "version": "4.0.10"
            },
            "positionEncodings": [
                "utf-16"
            ],
            "staleRequestSupport": {
                "cancel": true,
                "retryOnContentModified": [
                    "textDocument/inlayHint",
                    "textDocument/semanticTokens/full",
                    "textDocument/semanticTokens/range",
                    "textDocument/semanticTokens/full/delta"
                ]
            }
        }
    },
    "initializationOptions": {
        "enable": true,
        "startupMessage": false,
        "trace": {
            "server": "verbose"
        },
        "jediSettings": {
            "autoImportModules": [
                "pygls"
            ],
            "caseInsensitiveCompletion": true,
            "debug": false
        },
        "executable": {
            "args": [],
            "command": "jedi-language-server"
        },
        "codeAction": {
            "nameExtractFunction": "jls_extract_def",
            "nameExtractVariable": "jls_extract_var"
        },
        "completion": {
            "disableSnippets": false,
            "resolveEagerly": false,
            "ignorePatterns": []
        },
        "diagnostics": {
            "enable": true,
            "didOpen": true,
            "didChange": true,
            "didSave": true
        },
        "hover": {
            "enable": true,
            "disable": {
                "class": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "function": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "instance": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "keyword": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "module": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "param": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "path": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "property": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "statement": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                }
            }
        },
        "workspace": {
            "extraPaths": [],
            "symbols": {
                "maxSymbols": 20,
                "ignoreFolders": [
                    ".nox",
                    ".tox",
                    ".venv",
                    "__pycache__",
                    "venv"
                ]
            }
        }
    },
    "trace": "verbose",
    "workspaceFolders": [
        {
            "uri": "file:///home/sroeca/src/Personal/jedi-language-server",
            "name": "jedi-language-server"
        }
    ],
    "locale": "en_US",
    "clientInfo": {
        "name": "coc.nvim",
        "version": "0.0.82"
    }
}

@pappasam
Copy link
Owner

pappasam commented Oct 18, 2022

Based on the spec, it doesn't look like there's anything obviously wrong with the request: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#semanticTokensClientCapabilities

Relevant section:

"semanticTokens": {
    "requests": {
        "range": true,
        "full": {
            "delta": true
        }
    }
}

@karthiknadig
Copy link
Contributor

karthiknadig commented Oct 18, 2022

This looks like a missed cattrs hook for this particular case in lsprotocol. Created microsoft/lsprotocol#95 to track it.

@karthiknadig
Copy link
Contributor

I just published https://pypi.org/project/lsprotocol/2022.0.0a6/ with a fix for the above issue.

@pappasam for context lsprotocol is a package that is generated from the LSP spec. It is currently in alpha, and attempts to take the drudgery out of writing out types and constraints for LSP. Since this is auto-generated there are going to be some issues especially with cattrs hooks for some of the special cases in LSP. If you run into any bugs please file them on https://github.com/microsoft/lsprotocol/issues , with the error stack like from above.

@pappasam
Copy link
Owner

It works for me now! I'll play with it some more and monitor until the 1.0 release, thanks @dimbleby and @karthiknadig

@dimbleby
Copy link
Contributor Author

I manually added a py.typed to my local lsprotocol to see the effect, and sure enough mypy offered some errors. I've pushed a commit that resolves them.

(The analogous changes could be made on main: they should still be correct, even if the pygls typing is not good enough to tell us that)

@dimbleby
Copy link
Contributor Author

dimbleby commented Dec 3, 2022

@pappasam pygls has now gone 1.0.0, and I've updated accordingly - so far as I know this is good to go.

They snuck in a last-minute change to make the name and version both mandatory on the LanguageServer, that required a small refactor. Otherwise not much is different here since you last looked at it.

lsprotocol didn't yet declare itself non-alpha - there was some discussion (from here) about whether to wait for it but the view seems to be that it's ready enough and will likely stop calling itself alpha shortly. I don't think this project needs to wait for that, but it wouldn't be crazy to do so.

Copy link
Owner

@pappasam pappasam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Now that pygls relies on lsprotocol, I'll look into removing pydantic before doing a new release

@pappasam pappasam merged commit 7ef13b3 into pappasam:main Dec 5, 2022
@pappasam
Copy link
Owner

pappasam commented Dec 6, 2022

Note: I migrated to cattrs locally, but the error messages were not very user friendly. I'm just going to keep pydantic for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants