Create language diagnostic support

[aaroncrespo]

Inline errors etc.

[RLovelett]

I would love it if SourceKit provided the information somewhere. That way it would be nice and parseable. I'm trying to ascertain if SourceKit provides such an API. (Though if that is not possible then compilation is the only avenue left to us).

From there it should merely be a process of converting the Swift diagnostic format to the VS Code Diagnostic format and sending it to the editor.

[RLovelett]

🎉 Found it! It was already documented in Protocols.md.

Source

/// Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam ut nisi finibus, sodales risus
/// vel, hendrerit risus. Ut sed magna ultricies, congue odio commodo, posuere risus. Cum sociis
/// natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In feugiat lacus
/// elementum, maximus quam quis, suscipit neque. Curabitur pretium efficitur felis sit amet
/// facilisis. Cras tincidunt pulvinar neque, eu varius sapien gravida quis. Suspendisse potenti.
struct Foo {

    /// Cras tincidunt pulvinar neque, eu varius sapien gravida quis.
    let bar: Int

    /// Suspendisse potenti.
    let baz: Bool

    /// Cras tristique facilisis metus, ac bibendum lorem efficitur at. Vivamus finibus rhoncus
    /// ipsum molestie convallis. Sed ac sem at nisl consectetur ornare. Aenean purus magna,
    /// consequat id sodales sed, rhoncus eu augue. Lorem ipsum dolor sit amet, consectetur
    /// adipiscing elit.
    ///
    /// - parameter qux: Sed dapibus nisi at velit interdum, et.
    init(qux: String) {
        self.bar = 3
        self.baz = false
    }

}

ley Y = Food(qud: 3)

For example, running a source file that looks like above through that SourceKit request returns:

Request

{
  key.request: source.request.docinfo,
  key.sourcefile: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift"
}

Response

Skip to key.diagnostics...

{
  key.annotations: [
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 0,
      key.length: 96
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 96,
      key.length: 97
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 193,
      key.length: 96
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 289,
      key.length: 93
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 382,
      key.length: 98
    },
    {
      key.kind: source.lang.swift.syntaxtype.keyword,
      key.offset: 480,
      key.length: 6
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 487,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 498,
      key.length: 66
    },
    {
      key.kind: source.lang.swift.syntaxtype.keyword,
      key.offset: 568,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 572,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.ref.struct,
      key.name: "Int",
      key.usr: "s:Si",
      key.offset: 577,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 586,
      key.length: 25
    },
    {
      key.kind: source.lang.swift.syntaxtype.keyword,
      key.offset: 615,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 619,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.ref.struct,
      key.name: "Bool",
      key.usr: "s:Sb",
      key.offset: 624,
      key.length: 4
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 634,
      key.length: 92
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 730,
      key.length: 89
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 823,
      key.length: 88
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 915,
      key.length: 21
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 940,
      key.length: 4
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment,
      key.offset: 948,
      key.length: 61
    },
    {
      key.kind: source.lang.swift.syntaxtype.doccomment.field,
      key.offset: 954,
      key.length: 9
    },
    {
      key.kind: source.lang.swift.syntaxtype.keyword,
      key.offset: 1013,
      key.length: 4
    },
    {
      key.kind: source.lang.swift.syntaxtype.parameter,
      key.offset: 1018,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 1018,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.ref.struct,
      key.name: "String",
      key.usr: "s:SS",
      key.offset: 1023,
      key.length: 6
    },
    {
      key.kind: source.lang.swift.ref.var.local,
      key.name: "self",
      key.offset: 1041,
      key.length: 4
    },
    {
      key.kind: source.lang.swift.ref.var.instance,
      key.name: "bar",
      key.usr: "s:vV8__main__3Foo3barSi",
      key.offset: 1046,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.number,
      key.offset: 1052,
      key.length: 1
    },
    {
      key.kind: source.lang.swift.ref.var.local,
      key.name: "self",
      key.offset: 1062,
      key.length: 4
    },
    {
      key.kind: source.lang.swift.ref.var.instance,
      key.name: "baz",
      key.usr: "s:vV8__main__3Foo3bazSb",
      key.offset: 1067,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.keyword,
      key.offset: 1073,
      key.length: 5
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 1089,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 1093,
      key.length: 1
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 1097,
      key.length: 4
    },
    {
      key.kind: source.lang.swift.syntaxtype.identifier,
      key.offset: 1102,
      key.length: 3
    },
    {
      key.kind: source.lang.swift.syntaxtype.number,
      key.offset: 1107,
      key.length: 1
    }
  ],
  key.entities: [
    {
      key.kind: source.lang.swift.decl.struct,
      key.name: "Foo",
      key.usr: "s:V8__main__3Foo",
      key.doc.full_as_xml: "<Class file=\"/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift\" line=\"6\" column=\"8\"><Name>Foo</Name><USR>s:V8__main__3Foo</USR><Declaration>struct Foo</Declaration><Abstract><Para>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam ut nisi finibus, sodales risus vel, hendrerit risus. Ut sed magna ultricies, congue odio commodo, posuere risus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In feugiat lacus elementum, maximus quam quis, suscipit neque. Curabitur pretium efficitur felis sit amet facilisis. Cras tincidunt pulvinar neque, eu varius sapien gravida quis. Suspendisse potenti.</Para></Abstract></Class>",
      key.offset: 480,
      key.length: 606,
      key.fully_annotated_decl: "<decl.struct><syntaxtype.keyword>struct</syntaxtype.keyword> <decl.name>Foo</decl.name></decl.struct>",
      key.entities: [
        {
          key.kind: source.lang.swift.decl.var.instance,
          key.name: "bar",
          key.usr: "s:vV8__main__3Foo3barSi",
          key.doc.full_as_xml: "<Other file=\"/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift\" line=\"9\" column=\"9\"><Name>bar</Name><USR>s:vV8__main__3Foo3barSi</USR><Declaration>let bar: Int</Declaration><Abstract><Para>Cras tincidunt pulvinar neque, eu varius sapien gravida quis.</Para></Abstract></Other>",
          key.fully_annotated_decl: "<decl.var.instance><syntaxtype.keyword>let</syntaxtype.keyword> <decl.name>bar</decl.name>: <decl.var.type><ref.struct usr=\"s:Si\">Int</ref.struct></decl.var.type></decl.var.instance>"
        },
        {
          key.kind: source.lang.swift.decl.var.instance,
          key.name: "baz",
          key.usr: "s:vV8__main__3Foo3bazSb",
          key.doc.full_as_xml: "<Other file=\"/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift\" line=\"12\" column=\"9\"><Name>baz</Name><USR>s:vV8__main__3Foo3bazSb</USR><Declaration>let baz: Bool</Declaration><Abstract><Para>Suspendisse potenti.</Para></Abstract></Other>",
          key.fully_annotated_decl: "<decl.var.instance><syntaxtype.keyword>let</syntaxtype.keyword> <decl.name>baz</decl.name>: <decl.var.type><ref.struct usr=\"s:Sb\">Bool</ref.struct></decl.var.type></decl.var.instance>"
        },
        {
          key.kind: source.lang.swift.decl.function.constructor,
          key.name: "init(qux:)",
          key.usr: "s:FV8__main__3FoocFT3quxSS_S0_",
          key.doc.full_as_xml: "<Function file=\"/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift\" line=\"20\" column=\"5\"><Name>init(qux:)</Name><USR>s:FV8__main__3FoocFT3quxSS_S0_</USR><Declaration>init(qux: String)</Declaration><Abstract><Para>Cras tristique facilisis metus, ac bibendum lorem efficitur at. Vivamus finibus rhoncus ipsum molestie convallis. Sed ac sem at nisl consectetur ornare. Aenean purus magna, consequat id sodales sed, rhoncus eu augue. Lorem ipsum dolor sit amet, consectetur adipiscing elit.</Para></Abstract><Parameters><Parameter><Name>qux</Name><Direction isExplicit=\"0\">in</Direction><Discussion><Para>Sed dapibus nisi at velit interdum, et.</Para></Discussion></Parameter></Parameters></Function>",
          key.offset: 1013,
          key.length: 70,
          key.fully_annotated_decl: "<decl.function.constructor><syntaxtype.keyword>init</syntaxtype.keyword>(<decl.var.parameter><decl.var.parameter.argument_label>qux</decl.var.parameter.argument_label>: <decl.var.parameter.type><ref.struct usr=\"s:SS\">String</ref.struct></decl.var.parameter.type></decl.var.parameter>)</decl.function.constructor>",
          key.entities: [
            {
              key.kind: source.lang.swift.decl.var.local,
              key.keyword: "qux",
              key.name: "qux",
              key.offset: 1023,
              key.length: 6
            }
          ]
        }
      ]
    }
  ],
  key.diagnostics: [
    {
      key.line: 27,
      key.column: 1,
      key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
      key.severity: source.diagnostic.severity.error,
      key.description: "use of unresolved identifier 'ley'",
      key.ranges: [
        {
          key.offset: 1089,
          key.length: 3
        }
      ]
    },
    {
      key.line: 27,
      key.column: 4,
      key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
      key.severity: source.diagnostic.severity.error,
      key.description: "consecutive statements on a line must be separated by ';'",
      key.fixits: [
        {
          key.offset: 1092,
          key.length: 0,
          key.sourcetext: ";"
        }
      ]
    },
    {
      key.line: 27,
      key.column: 5,
      key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
      key.severity: source.diagnostic.severity.error,
      key.description: "use of unresolved identifier 'Y'",
      key.ranges: [
        {
          key.offset: 1093,
          key.length: 1
        }
      ]
    },
    {
      key.line: 27,
      key.column: 9,
      key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
      key.severity: source.diagnostic.severity.error,
      key.description: "use of unresolved identifier 'Food'",
      key.ranges: [
        {
          key.offset: 1097,
          key.length: 4
        }
      ],
      key.diagnostics: [
        {
          key.line: 6,
          key.column: 8,
          key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
          key.severity: source.diagnostic.severity.note,
          key.description: "did you mean 'Foo'?",
          key.fixits: [
            {
              key.offset: 1097,
              key.length: 4,
              key.sourcetext: "Foo"
            }
          ]
        },
        {
          key.line: 27,
          key.column: 9,
          key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
          key.severity: source.diagnostic.severity.note,
          key.description: "did you mean 'Void'? (Swift.Void)"
        },
        {
          key.line: 27,
          key.column: 9,
          key.filepath: "/Users/ryan/Source/node-sourcekit/test/fixtures/example.swift",
          key.severity: source.diagnostic.severity.note,
          key.description: "did you mean 'Bool'? (Swift.Bool)"
        }
      ]
    }
  ]
}

Now I need to implement it in node-sourcekit.

[aaroncrespo]

Looks like this needs package.swift support so you can fill out the compiler args?

[RLovelett]

Yes and no. There exists a functionality to get the compiler args for a given workspace.

But what exists right now is not robust at all. It is merely a list of the files in the workspace (minus the file you are interrogating).

I say all of that to say: not having SwiftPM/Package.swift support is not a blocker for this feature. And considering an interface is already in place to query for such information I would imagine whatever SwiftPM/Package.swift support was created in the future would simply provide the same interface and could easily be deprecate/replace this existing functionality.

[aaroncrespo]

Yeah, I guess for a start using swift PM to get a list of files in the Sources dir is not that different from getting the list of files in the workspace (for now)

[codeflows]

So the current status is that vscode-swift uses SourceKitten, but node-sourcekit will be a replacement at some point? Or is it used in some branch already? Inline errors would be awesome. :)

From a normal IDE user's standpoint SourceKit crashes (or used to crash) a lot. I wonder if anybody's tried to write a more lightweight syntax checker simply on top of Swift's own Parser/Lexer, to catch and highlight errors that way. Of course the deeper code analysis could still happen via SourceKit.

[aaroncrespo]

There is a limitation in vsode wrt to native plugin node modules. For now no node-sourcekit.

From a normal IDE user's standpoint SourceKit crashes (or used to crash) a lot.

I haven't really seen many crashes in any swift writing I've done in the past year and a half.

I wonder if anybody's tried to write a more lightweight syntax checker simply on top of Swift's own Parser/Lexer, to catch and highlight errors that way

Thats pretty much SourceKit. I don't know how you could get more light weight though. Swift type checking and resolution is very modern and not so simple. SourceKit is swifts paser/lexer. It generates the AST for the compiler. Writing that while keeping up with swift's evolution is already a challenge for sourcekit.

[RLovelett]

So the current status is that vscode-swift uses SourceKitten, but node-sourcekit will be a replacement at some point?

@aaroncrespo is right there is an upstream issue with VS Code that is a blocker. At least for now in moving to node-sourcekit. See: microsoft/vscode#658 for more information.

Because of that road-block I'm now looking at implementing the VS Code language server protocol natively in Swift. Very early days and don't really have anything to show yet but I think that would be cool to write the Swift language server...in Swift.

SourceKit is OSS now. With that if we, the community, are experiencing crashes etc then I think the best course of action is to fix the problems there rather than make a fork or some new Parser/Lexer. Beyond that I have no interest in trying to keep up with parsing this language. 🙃

I actually think projects like this, that also stand on SourceKit, will help make SourceKit stable for everyone eventually.

[codeflows]

Thanks for the comments @aaroncrespo & @RLovelett ! Trying to wrap my head around all of this.

And yeah, luckily SourceKit has been more stable recently :) I just had it crash on Friday for the first time in a long while and that sparked my comments. I bet it'll keep improving.

Writing the language server in Swift definitely sounds like a good idea! I'll keep following the progress and help if I can.