Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(swift) Revamped keywords and added support for operators #2908

Merged
merged 20 commits into from
Dec 15, 2020
Merged

(swift) Revamped keywords and added support for operators #2908

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
60a32b2
(swift) Revamped keywords and added support for operators. Also fixes…
svanimpe Dec 4, 2020
5d1e17c
Update changelog.
svanimpe Dec 4, 2020
4adf9ed
(swift) Small fix for when the input starts with a try!? operator.
svanimpe Dec 5, 2020
2bb4994
(swift) Small fix for when the input starts with a try!? operator (pa…
svanimpe Dec 5, 2020
8a7c3f9
Merge branch 'master' into swift-operators
svanimpe Dec 5, 2020
86b7c81
(swift) Big grammar improvements.
svanimpe Dec 10, 2020
09be110
(swift) Update author and changelog.
svanimpe Dec 10, 2020
1026ac0
Linter fixes
svanimpe Dec 11, 2020
da01153
Highlight literals as literals again.
svanimpe Dec 11, 2020
2a1163e
More keyword fixes.
svanimpe Dec 11, 2020
f8510bd
Remove returnBegin and fix relevances
svanimpe Dec 11, 2020
b5e2767
Small move
svanimpe Dec 12, 2020
71ce42e
explain dot operators
joshgoebel Dec 13, 2020
d249413
use built in keywords functionality
joshgoebel Dec 15, 2020
0d679d1
avoid two files with same name
joshgoebel Dec 15, 2020
f5f3043
remove kw? case as it is handled by modes
joshgoebel Dec 15, 2020
1e2c56d
simplify available rule
joshgoebel Dec 15, 2020
1e55113
existing keywords are sufficient
joshgoebel Dec 15, 2020
821d247
Merge branch 'master' into swift-operators
joshgoebel Dec 15, 2020
ea6d8df
Merge branch 'master' into swift-operators
joshgoebel Dec 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -310,3 +310,4 @@ Contributors:
- Kyle Brown <kylebrown9@github>
- Marcus Ortiz <mportiz08@gmail.com>
- Guillaume Grossetie <ggrossetie@yuzutech.fr>
- Steven Van Impe <steven.vanimpe@icloud.com>
3 changes: 2 additions & 1 deletion CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ New Languages:
Language improvements:

- enh(makefile): Add `make` as an alias (#2883) [tripleee][]
- enh(swift) Revamped keywords and added support for operators (#2908) [Steven Van Impe][]

Grammar improvements:

Expand All @@ -24,7 +25,7 @@ Grammar improvements:
[Oldes Huhuman]: https://github.com/Oldes
[Josh Goebel]: https://github.com/joshgoebel
[tripleee]: https://github.com/tripleee

[Steven Van Impe]: https://github.com/svanimpe/

## Version 10.4.0

Expand Down
134 changes: 103 additions & 31 deletions src/languages/swift.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,39 +7,52 @@ Website: https://swift.org
Category: common, system
*/

import { either } from "../lib/regex";

export default function(hljs) {
var SWIFT_KEYWORDS = {
// override the pattern since the default of of /\w+/ is not sufficient to
// capture the keywords that start with the character "#"
$pattern: /[\w#]+/,
keyword: '#available #colorLiteral #column #else #elseif #endif #file ' +
'#fileLiteral #function #if #imageLiteral #line #selector #sourceLocation ' +
'_ __COLUMN__ __FILE__ __FUNCTION__ __LINE__ Any as as! as? associatedtype ' +
'associativity break case catch class continue convenience default defer deinit didSet do ' +
'dynamic dynamicType else enum extension fallthrough false fileprivate final for func ' +
'get guard if import in indirect infix init inout internal is lazy left let ' +
'mutating nil none nonmutating open operator optional override postfix precedence ' +
'prefix private protocol Protocol public repeat required rethrows return ' +
'right self Self set some static struct subscript super switch throw throws true ' +
'try try! try? Type typealias unowned var weak where while willSet',
literal: 'true false nil',
built_in: 'abs advance alignof alignofValue anyGenerator assert assertionFailure ' +
'bridgeFromObjectiveC bridgeFromObjectiveCUnconditional bridgeToObjectiveC ' +
'bridgeToObjectiveCUnconditional c compactMap contains count countElements countLeadingZeros ' +
'debugPrint debugPrintln distance dropFirst dropLast dump encodeBitsAsWords ' +
'enumerate equal fatalError filter find getBridgedObjectiveCType getVaList ' +
'indices insertionSort isBridgedToObjectiveC isBridgedVerbatimToObjectiveC ' +
'isUniquelyReferenced isUniquelyReferencedNonObjC join lazy lexicographicalCompare ' +
'map max maxElement min minElement numericCast overlaps partition posix ' +
'precondition preconditionFailure print println quickSort readLine reduce reflect ' +
'reinterpretCast reverse roundUpToAlignment sizeof sizeofValue sort split ' +
'startsWith stride strideof strideofValue swap toString transcode ' +
'underestimateCount unsafeAddressOf unsafeBitCast unsafeDowncast unsafeUnwrap ' +
'unsafeReflect withExtendedLifetime withObjectAtPlusZero withUnsafePointer ' +
'withUnsafePointerToObject withUnsafeMutablePointer withUnsafeMutablePointers ' +
'withUnsafePointer withUnsafePointers withVaList zip'
};
// https://docs.swift.org/swift-book/ReferenceManual/LexicalStructure.html#ID413
// https://docs.swift.org/swift-book/ReferenceManual/zzSummaryOfTheGrammar.html
const SWIFT_KEYWORDS = {
// Override the pattern since the default of /\w+/ is not sufficient to capture
// the keywords that start with a number sign (#) or that have parentheses.
$pattern: /#?\w+(\(\w+\))?/,
keyword:
// Reserved keywords. These require backticks to use as an identifier:
'_ Any as associatedtype break case catch class continue default defer deinit do else enum extension ' +
joshgoebel marked this conversation as resolved.
Show resolved Hide resolved
'fallthrough fileprivate fileprivate(set) for func guard if import in init inout internal internal(set) is ' +
'let operator precedencegroup private private(set) protocol public public(set) repeat rethrows return ' +
'self Self static struct subscript super switch throw throws try typealias var where while ' +
// This list includes as, is, and try, even though they're technically operators.
// The remaining operators as?, as!, try?, and try! are declared in the OPERATOR mode.

// Keywords that begin with a number sign:
'#available #colorLiteral #column #dsohandle #else #elseif #endif #error #file #fileID #fileLiteral #filePath ' +
'#function #if #imageLiteral #keyPath #line #selector #sourceLocation #warning ' +

// Keywords reserved in particular contexts. These can be used as identifiers:
'convenience didSet dynamic final get infix indirect lazy mutating nonmutating open open(set) ' +
'optional override postfix prefix Protocol required set some Type unowned unowned(safe) unowned(unsafe) weak willSet ',
// Ideally, these keywords should only be matched in specific modes, as they can result in false positives.
// For example, the following keywords are commented out because they're only used in
// precedencegroup declarations, so they will likely result in mostly false positives:
// 'assignment associativity higherThan left lowerThan none right '
literal: 'true false nil',
built_in: 'abs advance alignof alignofValue anyGenerator assert assertionFailure ' +
'bridgeFromObjectiveC bridgeFromObjectiveCUnconditional bridgeToObjectiveC ' +
'bridgeToObjectiveCUnconditional c compactMap contains count countElements countLeadingZeros ' +
'debugPrint debugPrintln distance dropFirst dropLast dump encodeBitsAsWords ' +
'enumerate equal fatalError filter find getBridgedObjectiveCType getVaList ' +
'indices insertionSort isBridgedToObjectiveC isBridgedVerbatimToObjectiveC ' +
'isUniquelyReferenced isUniquelyReferencedNonObjC join lazy lexicographicalCompare ' +
'map max maxElement min minElement numericCast overlaps partition posix ' +
'precondition preconditionFailure print println quickSort readLine reduce reflect ' +
'reinterpretCast reverse roundUpToAlignment sizeof sizeofValue sort split ' +
'startsWith stride strideof strideofValue swap toString transcode ' +
'underestimateCount unsafeAddressOf unsafeBitCast unsafeDowncast unsafeUnwrap ' +
'unsafeReflect withExtendedLifetime withObjectAtPlusZero withUnsafePointer ' +
'withUnsafePointerToObject withUnsafeMutablePointer withUnsafeMutablePointers ' +
'withUnsafePointer withUnsafePointers withVaList zip'
};

var TYPE = {
className: 'type',
Expand Down Expand Up @@ -98,13 +111,72 @@ export default function(hljs) {
};
SUBST.contains = [NUMBER];

// https://docs.swift.org/swift-book/ReferenceManual/LexicalStructure.html#ID418
const operatorHead = either(
/[/=\-+!*%<>&|^~?]/,
/[\u00A1–\u00A7]/,
/[\u00A9\u00AB]/,
/[\u00AC\u00AE]/,
/[\u00B0\u00B1]/,
/[\u00B6\u00BB\u00BF\u00D7\u00F7]/,
/[\u2016–\u2017]/,
/[\u2020–\u2027]/,
/[\u2030–\u203E]/,
/[\u2041–\u2053]/,
/[\u2055–\u205E]/,
/[\u2190–\u23FF]/,
/[\u2500–\u2775]/,
/[\u2794–\u2BFF]/,
/[\u2E00–\u2E7F]/,
/[\u3001–\u3003]/,
/[\u3008–\u3020]/,
/[\u3030]/
);
const operatorCharacter = either(
operatorHead,
/[\u0300–\u036F]/,
/[\u1DC0–\u1DFF]/,
/[\u20D0–\u20FF]/,
/[\uFE00–\uFE0F]/,
/[\uFE20–\uFE2F]/,
// TODO: The following characters are also allowed, but the regex doesn't work as intended.
// For example, it also matches -10 as an operator.
// /[\u{E0100}–\u{E01EF}]/u
Copy link
Member

@joshgoebel joshgoebel Dec 5, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem right. From what I'm reading this is the full range of what MIGHT be used for defining a custom operator... just because something could potentially be an operator doesn't mean it most assuredly IS an operator and should be highlighted everywhere as an operator... correct?

This "wideness" might make sense in the context of something like func [x] (a,b) where [x] is an operator being defined (and we know that by context), but I don't think it makes sense in general. Pretty sure in the standard case we should only highlight well-known built-in operators that Swift supports, like >, <, etc...

Or do those UTF-8 ranges have NO other uses in Swift at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these characters are used elsewhere in the grammar, such as <GenericParam>, or OptionalType?, but other than that, they're always operators.

I assume the set of "head" character for operators and other identifiers is disjunct. For example, if you look at the range A1-FF, the ones that are missing from the list here are listed explicitly as valid head characters for regular (non-operator) identifiers:

\u00A8\u00AA\u00AD\u00AF\u00B2–\u00B5\u00B7–\u00BA
\u00BC–\u00BE\u00C0–\u00D6\u00D8–\u00F6\u00F8–\u00FF

So I think this should work just fine, as long as the cases where operators are used as punctuation are handled by specific modes. In the case of generics and optional types, that's already the case.

joshgoebel marked this conversation as resolved.
Show resolved Hide resolved
);
const OPERATOR = {
className: 'operator',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these operators or keywords?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this question about as? as! try? try!?

If so, the grammar specifies them as operators and keywords at the same time, so I'm highlighting them as keywords, which is what other grammars do too, although most don't bother highlighting the question or exlamation mark. I preferred to include that in the match, otherwise it would be matched as a postfix operator.

Copy link
Member

@joshgoebel joshgoebel Dec 5, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are keywords then you should remove "operator", it's not serving any purpose. We can't have them be both (not desirable).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll add them to list of "special" keywords (like the ones that have parentheses) instead, so I don't have to change the keywords $pattern to allow question marks or exclamation points.

My goal here was to group these keywords under the "operator" umbrella as well, in case a future grammar says "any operator is allowed here", which would then also include these keywords.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My goal here was to group these keywords under the "operator" umbrella as well, in case a future grammar says "any operator is allowed here", which would then also include these keywords.

I think you may be over thinking that. :)

variants: [
{ // TODO: Replace with negative look-behind when available.
className: 'keyword',
begin: /\s(?=as[?!]?\s)/,
excludeBegin: true,
joshgoebel marked this conversation as resolved.
Show resolved Hide resolved
end: /as[?!]?(?=\s)/
},
{ // TODO: Replace with negative look-behind when available.
className: 'keyword',
begin: /\s(?=is\s)/,
excludeBegin: true,
end: /is(?=\s)/
joshgoebel marked this conversation as resolved.
Show resolved Hide resolved
},
{ // TODO: Replace with negative look-behind when available.
className: 'keyword',
begin: /(^|[^.])(?=\btry[?!]?\s)/,
excludeBegin: true,
end: /try[?!]?(?=\s)/
},
{ begin: `${operatorHead}${operatorCharacter}*` },
{ begin: `\\.(\\.|${operatorCharacter})+` }
]
};

return {
name: 'Swift',
keywords: SWIFT_KEYWORDS,
contains: [
STRING,
hljs.C_LINE_COMMENT_MODE,
BLOCK_COMMENT,
OPERATOR,
OPTIONAL_USING_TYPE,
TYPE,
NUMBER,
Expand Down
2 changes: 1 addition & 1 deletion test/markup/swift/multiline-string.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<span class="hljs-keyword">var</span> string = <span class="hljs-string">&quot;&quot;&quot;
<span class="hljs-keyword">var</span> string <span class="hljs-operator">=</span> <span class="hljs-string">&quot;&quot;&quot;
var a = not actually code
&quot;&quot;&quot;</span>
10 changes: 5 additions & 5 deletions test/markup/swift/numbers.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@


<span class="hljs-comment">// expressions containing numeric literals</span>
+<span class="hljs-number">0</span>
+-<span class="hljs-number">1</span>
<span class="hljs-number">2</span>-<span class="hljs-number">3</span>
-<span class="hljs-number">10</span>.magnitude
fn(-<span class="hljs-number">5</span>)
<span class="hljs-operator">+</span><span class="hljs-number">0</span>
<span class="hljs-operator">+-</span><span class="hljs-number">1</span>
<span class="hljs-number">2</span><span class="hljs-operator">-</span><span class="hljs-number">3</span>
<span class="hljs-operator">-</span><span class="hljs-number">10</span>.magnitude
fn(<span class="hljs-operator">-</span><span class="hljs-number">5</span>)
<span class="hljs-number">0x2</span>.p2

<span class="hljs-comment">// expressions not containing numeric literals</span>
Expand Down
109 changes: 109 additions & 0 deletions test/markup/swift/operators.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
x <span class="hljs-keyword">is</span> y
x.is(y)
x <span class="hljs-keyword">as</span> y
x <span class="hljs-keyword">as?</span> y
x <span class="hljs-keyword">as!</span> y
a.as(y)
<span class="hljs-keyword">try</span> x
<span class="hljs-keyword">try?</span> x
<span class="hljs-keyword">try!</span> x
x.try(y)

<span class="hljs-operator">!</span><span class="hljs-literal">true</span>
<span class="hljs-operator">~</span>x
<span class="hljs-operator">+</span><span class="hljs-number">1</span>
<span class="hljs-operator">-</span><span class="hljs-number">1</span>
<span class="hljs-operator">..&lt;</span><span class="hljs-number">1</span>
<span class="hljs-operator">...</span><span class="hljs-number">1</span>
<span class="hljs-number">0</span><span class="hljs-operator">...</span>
a<span class="hljs-operator">?</span>
a<span class="hljs-operator">!</span>

a <span class="hljs-operator">&lt;&lt;</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&lt;&lt;</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&gt;&gt;</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&gt;&gt;</span><span class="hljs-number">1</span>
a <span class="hljs-operator">*</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">*</span><span class="hljs-number">1</span>
a <span class="hljs-operator">/</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">/</span><span class="hljs-number">1</span>
a <span class="hljs-operator">%</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">%</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&amp;*</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&amp;*</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&amp;</span> b
a<span class="hljs-operator">&amp;</span>b
a <span class="hljs-operator">+</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">+</span><span class="hljs-number">1</span>
a <span class="hljs-operator">-</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">-</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&amp;+</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&amp;+</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&amp;-</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&amp;-</span><span class="hljs-number">1</span>
a <span class="hljs-operator">|</span> b
a<span class="hljs-operator">|</span>b
a <span class="hljs-operator">^</span> b
a<span class="hljs-operator">^</span>b
<span class="hljs-number">0</span> <span class="hljs-operator">..&lt;</span> <span class="hljs-number">1</span>
<span class="hljs-number">0</span><span class="hljs-operator">..&lt;</span><span class="hljs-number">1</span>
<span class="hljs-number">0</span> <span class="hljs-operator">...</span> <span class="hljs-number">1</span>
<span class="hljs-number">0</span><span class="hljs-operator">...</span><span class="hljs-number">1</span>
a <span class="hljs-operator">??</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">??</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&lt;</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&lt;</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&lt;=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&lt;=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&gt;</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&gt;</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&gt;=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&gt;=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">==</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">==</span><span class="hljs-number">1</span>
a <span class="hljs-operator">!=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">!=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">===</span> b
a<span class="hljs-operator">===</span>b
a <span class="hljs-operator">!==</span> b
a<span class="hljs-operator">!==</span>b
a <span class="hljs-operator">~=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">~=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">.==</span> b
a<span class="hljs-operator">.==</span>b
a <span class="hljs-operator">.!=</span> b
a<span class="hljs-operator">.!=</span>b
a <span class="hljs-operator">.&lt;</span> b
a<span class="hljs-operator">.&lt;</span>b
a <span class="hljs-operator">.&lt;=</span> b
a<span class="hljs-operator">.&lt;=</span>b
a <span class="hljs-operator">.&gt;</span> b
a<span class="hljs-operator">.&gt;</span>b
a <span class="hljs-operator">.&gt;=</span> b
a<span class="hljs-operator">.&gt;=</span>b
<span class="hljs-literal">true</span> <span class="hljs-operator">&amp;&amp;</span> <span class="hljs-literal">false</span>
<span class="hljs-literal">true</span><span class="hljs-operator">&amp;&amp;</span><span class="hljs-literal">false</span>
<span class="hljs-literal">true</span> <span class="hljs-operator">||</span> <span class="hljs-literal">false</span>
<span class="hljs-literal">true</span><span class="hljs-operator">||</span><span class="hljs-literal">false</span>
a <span class="hljs-operator">=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">*=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">*=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">/=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">/=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">%=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">%=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">+=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">+=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">-=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">-=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&lt;&lt;=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&lt;&lt;=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&gt;&gt;=</span> <span class="hljs-number">1</span>
a<span class="hljs-operator">&gt;&gt;=</span><span class="hljs-number">1</span>
a <span class="hljs-operator">&amp;=</span> b
a<span class="hljs-operator">&amp;=</span>b
a <span class="hljs-operator">|=</span> b
a<span class="hljs-operator">|=</span>b
a <span class="hljs-operator">^=</span> b
a<span class="hljs-operator">^=</span>b
Loading