Skip to content

Make indentation explicit in context-free syntax #11219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 27, 2021
Merged

Make indentation explicit in context-free syntax #11219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
336dd68
Make indentation explicit in context-free syntax
odersky Jan 26, 2021
fcfb579
Update text in indentation.md
odersky Jan 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 54 additions & 37 deletions docs/docs/internals/syntax.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ layout: doc-page
title: "Scala 3 Syntax Summary"
---

The following descriptions of Scala tokens uses literal characters `‘c’` when
The following description of Scala tokens uses literal characters `‘c’` when
referring to the ASCII fragment `\u0000` – `\u007F`.

_Unicode escapes_ are used to represent the Unicode character with the given
_Unicode escapes_ are used to represent the [Unicode character](https://www.w3.org/International/articles/definitions-characters/) with the given
hexadecimal code:

```ebnf
Expand All @@ -17,6 +17,7 @@ hexDigit ::= ‘0’ | … | ‘9’ | ‘A’ | … | ‘F’ | ‘a’ |
Informal descriptions are typeset as `“some comment”`.

### Lexical Syntax

The lexical syntax of Scala is given by the following grammar in EBNF
form.

Expand All @@ -28,9 +29,9 @@ letter ::= upper | lower “… and Unicode categories Lo, Lt, Nl”
digit ::= ‘0’ | … | ‘9’
paren ::= ‘(’ | ‘)’ | ‘[’ | ‘]’ | ‘{’ | ‘}’ | ‘'(’ | ‘'[’ | ‘'{’
delim ::= ‘`’ | ‘'’ | ‘"’ | ‘.’ | ‘;’ | ‘,’
opchar ::= “printableChar not matched by (whiteSpace | upper | lower |
letter | digit | paren | delim | opchar | Unicode_Sm |
Unicode_So)”
opchar ::= “printableChar not matched by (whiteSpace | upper |
lower | letter | digit | paren | delim | opchar |
Unicode_Sm | Unicode_So)”
printableChar ::= “all characters in [\u0020, \u007F] inclusive”
charEscapeSeq ::= ‘\’ (‘b’ | ‘t’ | ‘n’ | ‘f’ | ‘r’ | ‘"’ | ‘'’ | ‘\’)

Expand All @@ -42,7 +43,6 @@ plainid ::= alphaid
| op
id ::= plainid
| ‘`’ { charNoBackQuoteOrNewline | UnicodeEscape | charEscapeSeq } ‘`’
| INT // interpolation id, only for quasi-quotes
idrest ::= {letter | digit} [‘_’ op]
quoteId ::= ‘'’ alphaid

Expand Down Expand Up @@ -83,29 +83,49 @@ comment ::= ‘/*’ “any sequence of characters; nested comments ar

nl ::= “new line character”
semi ::= ‘;’ | nl {nl}
colonEol ::= ": at end of line that can start a template body"
```


## Optional Braces

The lexical analyzer also inserts `indent` and `outdent` tokens that represent regions of indented code [at certain points](../reference/other-new-features-indentation.html)

In the context-free productions below we use the notation `<<< ts >>>`
to indicate a token sequence `ts` that is either enclosed in a pair of braces `{ ts }` or that constitutes an indented region `indent ts outdent`. Analogously, the
notation `:<<< ts >>>` indicates a token sequence `ts` that is either enclosed in a pair of braces `{ ts }` or that constitutes an indented region `indent ts outdent` that follows
a `:` at the end of a line.


```
<<< ts >>> ::= ‘{’ ts ‘}’
| indent ts outdent
:<<< ts >>> ::= [nl] ‘{’ ts ‘}’
| `:` indent ts outdent

## Keywords

### Regular keywords

```
abstract case catch class def do else enum
export extends false final finally for given if
implicit import lazy match new null object package
private protected override return super sealed then throw
trait true try type val var while with
yield
: = <- => <: :> # @
=>> ?=>
abstract case catch class def do else
enum export extends false final finally for
given if implicit import lazy match new
null object override package private protected return
sealed super then throw trait true try
type val var while with yield
: = <- => <: :> #
@ =>> ?=>
```

### Soft keywords

```
derives end extension inline infix opaque open transparent using | * + -
derives end extension infix inline opaque open transparent using | * + -
```

See the [separate section on soft keywords](./soft-modifier.md) for additional
details on where a soft keyword is recognized.

## Context-free Syntax

The context-free syntax of Scala is given by the following EBNF
Expand Down Expand Up @@ -147,10 +167,9 @@ FunArgTypes ::= InfixType
| FunParamClause
FunParamClause ::= ‘(’ TypedFunParam {‘,’ TypedFunParam } ‘)’
TypedFunParam ::= id ‘:’ Type
MatchType ::= InfixType `match` ‘{’ TypeCaseClauses ‘}’
MatchType ::= InfixType `match` <<< TypeCaseClauses >>>
InfixType ::= RefinedType {id [nl] RefinedType} InfixOp(t1, op, t2)
RefinedType ::= WithType {[nl] Refinement} RefinedTypeTree(t, ds)
WithType ::= AnnotType {‘with’ AnnotType} (deprecated)
RefinedType ::= AnnotType {[nl] Refinement} RefinedTypeTree(t, ds)
AnnotType ::= SimpleType {Annotation} Annotated(t, annot)

SimpleType ::= SimpleLiteral SingletonTypeTree(l)
Expand Down Expand Up @@ -209,7 +228,7 @@ PostfixExpr ::= InfixExpr [id]
InfixExpr ::= PrefixExpr
| InfixExpr id [nl] InfixExpr InfixOp(expr, op, expr)
| InfixExpr MatchClause
MatchClause ::= ‘match’ ‘{’ CaseClauses ‘}’ Match(expr, cases)
MatchClause ::= ‘match’ <<< CaseClauses >>> Match(expr, cases)
PrefixExpr ::= [‘-’ | ‘+’ | ‘~’ | ‘!’] SimpleExpr PrefixOp(expr, op)
SimpleExpr ::= SimpleRef
| Literal
Expand All @@ -218,9 +237,8 @@ SimpleExpr ::= SimpleRef
| ‘$’ ‘{’ Block ‘}’
| Quoted
| quoteId -- only inside splices
| ‘new’ ConstrApp {‘with’ ConstrApp} New(constr | templ)
[[colonEol] TemplateBody
| ‘new’ [colonEol] TemplateBody
| ‘new’ ConstrApp {‘with’ ConstrApp} [TemplateBody] New(constr | templ)
| ‘new’ TemplateBody
| ‘(’ ExprsInParens ‘)’ Parens(exprs)
| SimpleExpr ‘.’ id Select(expr, id)
| SimpleExpr ‘.’ MatchClause
Expand All @@ -237,7 +255,7 @@ ParArgumentExprs ::= ‘(’ [‘using’] ExprsInParens ‘)’
| ‘(’ [ExprsInParens ‘,’] PostfixExpr ‘:’ ‘_’ ‘*’ ‘)’ exprs :+ Typed(expr, Ident(wildcardStar))
ArgumentExprs ::= ParArgumentExprs
| BlockExpr
BlockExpr ::= ‘{’ (CaseClauses | Block) ‘}’
BlockExpr ::= <<< (CaseClauses | Block) >>>
Block ::= {BlockStat semi} [BlockResult] Block(stats, expr?)
BlockStat ::= Import
| {Annotation {nl}} [‘implicit’ | ‘lazy’] Def
Expand Down Expand Up @@ -356,7 +374,6 @@ EndMarkerTag ::= id | ‘if’ | ‘while’ | ‘for’ | ‘match’ |
RefineDcl ::= ‘val’ ValDcl
| ‘def’ DefDcl
| ‘type’ {nl} TypeDcl
| INT
Dcl ::= RefineDcl
| ‘var’ VarDcl
ValDcl ::= ids ‘:’ Type PatDef(_, ids, tpe, EmptyTree)
Expand All @@ -372,7 +389,7 @@ Def ::= ‘val’ PatDef
| ‘type’ {nl} TypeDcl
| TmplDef
PatDef ::= ids [‘:’ Type] ‘=’ Expr
| Pattern2 [‘:’ Type] ‘=’ Expr PatDef(_, pats, tpe?, expr)
| Pattern2 [‘:’ Type] ‘=’ Expr PatDef(_, pats, tpe?, expr)
VarDef ::= PatDef
| ids ‘:’ Type ‘=’ ‘_’
DefDef ::= DefSig [‘:’ Type] ‘=’ Expr DefDef(_, name, tparams, vparamss, tpe, expr)
Expand All @@ -386,23 +403,23 @@ ClassDef ::= id ClassConstr [Template]
ClassConstr ::= [ClsTypeParamClause] [ConstrMods] ClsParamClauses with DefDef(_, <init>, Nil, vparamss, EmptyTree, EmptyTree) as first stat
ConstrMods ::= {Annotation} [AccessModifier]
ObjectDef ::= id [Template] ModuleDef(mods, name, template) // no constructor
EnumDef ::= id ClassConstr InheritClauses [colonEol] EnumBody
EnumDef ::= id ClassConstr InheritClauses EnumBody
GivenDef ::= [GivenSig] (AnnotType [‘=’ Expr] | StructuralInstance)
GivenSig ::= [id] [DefTypeParamClause] {UsingParamClause} ‘:’ -- one of `id`, `DefParamClause`, `UsingParamClause` must be present
StructuralInstance ::= ConstrApp {‘with’ ConstrApp} ‘with’ TemplateBody
Extension ::= ‘extension’ [DefTypeParamClause] ‘(’ DefParam ‘)’
{UsingParamClause}] ExtMethods
ExtMethods ::= ExtMethod | [nl] ‘{’ ExtMethod {semi ExtMethod ‘}’
ExtMethods ::= ExtMethod | [nl] <<< ExtMethod {semi ExtMethod} >>>
ExtMethod ::= {Annotation [nl]} {Modifier} ‘def’ DefDef
Template ::= InheritClauses [colonEol] [TemplateBody] Template(constr, parents, self, stats)
Template ::= InheritClauses [TemplateBody]
InheritClauses ::= [‘extends’ ConstrApps] [‘derives’ QualId {‘,’ QualId}]
ConstrApps ::= ConstrApp ({‘,’ ConstrApp} | {‘with’ ConstrApp})
ConstrApp ::= SimpleType1 {Annotation} {ParArgumentExprs} Apply(tp, args)
ConstrApp ::= SimpleType1 {Annotation} {ParArgumentExprs}
ConstrExpr ::= SelfInvocation
| ‘{’ SelfInvocation {semi BlockStat} ‘}’
| <<< SelfInvocation {semi BlockStat} >>>
SelfInvocation ::= ‘this’ ArgumentExprs {ArgumentExprs}

TemplateBody ::= [nl] ‘{’ [SelfType] TemplateStat {semi TemplateStat} ‘}’
TemplateBody ::= :<<< [SelfType] TemplateStat {semi TemplateStat} >>>
TemplateStat ::= Import
| Export
| {Annotation [nl]} {Modifier} Def
Expand All @@ -414,12 +431,12 @@ TemplateStat ::= Import
SelfType ::= id [‘:’ InfixType] ‘=>’ ValDef(_, name, tpt, _)
| ‘this’ ‘:’ InfixType ‘=>’

EnumBody ::= [nl] ‘{’ [SelfType] EnumStat {semi EnumStat} ‘}’
EnumBody ::= :<<< [SelfType] EnumStat {semi EnumStat} >>>
EnumStat ::= TemplateStat
| {Annotation [nl]} {Modifier} EnumCase
EnumCase ::= ‘case’ (id ClassConstr [‘extends’ ConstrApps]] | ids)

TopStatSeq ::= TopStat {semi TopStat}
TopStats ::= TopStat {semi TopStat}
TopStat ::= Import
| Export
| {Annotation [nl]} {Modifier} Def
Expand All @@ -428,8 +445,8 @@ TopStat ::= Import
| PackageObject
| EndMarker
|
Packaging ::= ‘package’ QualId [nl | colonEol] ‘{’ TopStatSeq ‘}’ Package(qid, stats)
PackageObject ::= ‘package’ ‘object’ ObjectDef object with package in mods.
Packaging ::= ‘package’ QualId :<<< TopStats >>>
PackageObject ::= ‘package’ ‘object’ ObjectDef

CompilationUnit ::= {‘package’ QualId semi} TopStatSeq Package(qid, stats)
CompilationUnit ::= {‘package’ QualId semi} TopStats
```
5 changes: 2 additions & 3 deletions docs/docs/reference/other-new-features/indentation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,13 @@ layout: doc-page
title: Optional Braces
---

As an experimental feature, Scala 3 enforces some rules on indentation and allows
some occurrences of braces `{...}` to be optional.
It can be turned off with the compiler flag `-noindent`.
Scala 3 enforces some rules on indentation and allows some occurrences of braces `{...}` to be optional:

- First, some badly indented programs are flagged with warnings.
- Second, some occurrences of braces `{...}` are made optional. Generally, the rule
is that adding a pair of optional braces will not change the meaning of a well-indented program.

These changescan can be turned off with the compiler flag `-noindent`.
### Indentation Rules

The compiler enforces two rules for well-indented programs, flagging violations as warnings.
Expand Down
33 changes: 21 additions & 12 deletions docs/docs/reference/syntax.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,23 @@ comment ::= ‘/*’ “any sequence of characters; nested comments ar

nl ::= “new line character”
semi ::= ‘;’ | nl {nl}
colonEol ::= ": at end of line that can start a template body"
```

## Optional Braces

The lexical analyzer also inserts `indent` and `outdent` tokens that represent regions of indented code [at certain points](../reference/other-new-features-indentation.html)

In the context-free productions below we use the notation `<<< ts >>>`
to indicate a token sequence `ts` that is either enclosed in a pair of braces `{ ts }` or that constitutes an indented region `indent ts outdent`.
to indicate a token sequence `ts` that is either enclosed in a pair of braces `{ ts }` or that constitutes an indented region `indent ts outdent`. Analogously, the
notation `:<<< ts >>>` indicates a token sequence `ts` that is either enclosed in a pair of braces `{ ts }` or that constitutes an indented region `indent ts outdent` that follows
a `:` at the end of a line.


```
<<< ts >>> ::= ‘{’ ts ‘}’
| indent ts outdent
:<<< ts >>> ::= [nl] ‘{’ ts ‘}’
| `:` indent ts outdent

## Keywords

Expand Down Expand Up @@ -226,8 +235,8 @@ SimpleExpr ::= SimpleRef
| ‘$’ ‘{’ Block ‘}’
| Quoted
| quoteId -- only inside splices
| ‘new’ ConstrApp {‘with’ ConstrApp} [[colonEol] TemplateBody
| ‘new’ [colonEol] TemplateBody
| ‘new’ ConstrApp {‘with’ ConstrApp} [TemplateBody]
| ‘new’ TemplateBody
| ‘(’ ExprsInParens ‘)’
| SimpleExpr ‘.’ id
| SimpleExpr ‘.’ MatchClause
Expand All @@ -242,7 +251,7 @@ ParArgumentExprs ::= ‘(’ [‘using’] ExprsInParens ‘)’
| ‘(’ [ExprsInParens ‘,’] PostfixExpr ‘:’ ‘_’ ‘*’ ‘)’
ArgumentExprs ::= ParArgumentExprs
| BlockExpr
BlockExpr ::= ‘{’ (CaseClauses | Block) ‘}’
BlockExpr ::= <<< (CaseClauses | Block) >>>
Block ::= {BlockStat semi} [BlockResult]
BlockStat ::= Import
| {Annotation {nl}} [‘implicit’ | ‘lazy’] Def
Expand Down Expand Up @@ -384,23 +393,23 @@ ClassDef ::= id ClassConstr [Template]
ClassConstr ::= [ClsTypeParamClause] [ConstrMods] ClsParamClauses
ConstrMods ::= {Annotation} [AccessModifier]
ObjectDef ::= id [Template]
EnumDef ::= id ClassConstr InheritClauses [colonEol] EnumBody
GivenDef ::= [GivenSig] (Type [‘=’ Expr] | StructuralInstance)
EnumDef ::= id ClassConstr InheritClauses EnumBody
GivenDef ::= [GivenSig] (AnnotType [‘=’ Expr] | StructuralInstance)
GivenSig ::= [id] [DefTypeParamClause] {UsingParamClause} ‘:’ -- one of `id`, `DefParamClause`, `UsingParamClause` must be present
StructuralInstance ::= ConstrApp {‘with’ ConstrApp} ‘with’ TemplateBody
Extension ::= ‘extension’ [DefTypeParamClause] ‘(’ DefParam ‘)’
{UsingParamClause}] ExtMethods
ExtMethods ::= ExtMethod | [nl] ‘{’ ExtMethod {semi ExtMethod ‘}’
ExtMethods ::= ExtMethod | [nl] <<< ExtMethod {semi ExtMethod} >>>
ExtMethod ::= {Annotation [nl]} {Modifier} ‘def’ DefDef
Template ::= InheritClauses [colonEol] [TemplateBody]
Template ::= InheritClauses [TemplateBody]
InheritClauses ::= [‘extends’ ConstrApps] [‘derives’ QualId {‘,’ QualId}]
ConstrApps ::= ConstrApp ({‘,’ ConstrApp} | {‘with’ ConstrApp})
ConstrApp ::= SimpleType1 {Annotation} {ParArgumentExprs}
ConstrExpr ::= SelfInvocation
| <<< SelfInvocation {semi BlockStat} >>>
SelfInvocation ::= ‘this’ ArgumentExprs {ArgumentExprs}

TemplateBody ::= [nl] ‘{’ [SelfType] TemplateStat {semi TemplateStat} ‘}’
TemplateBody ::= :<<< [SelfType] TemplateStat {semi TemplateStat} >>>
TemplateStat ::= Import
| Export
| {Annotation [nl]} {Modifier} Def
Expand All @@ -412,7 +421,7 @@ TemplateStat ::= Import
SelfType ::= id [‘:’ InfixType] ‘=>’
| ‘this’ ‘:’ InfixType ‘=>’

EnumBody ::= [nl] ‘{’ [SelfType] EnumStat {semi EnumStat} ‘}’
EnumBody ::= :<<< [SelfType] EnumStat {semi EnumStat} >>>
EnumStat ::= TemplateStat
| {Annotation [nl]} {Modifier} EnumCase
EnumCase ::= ‘case’ (id ClassConstr [‘extends’ ConstrApps]] | ids)
Expand All @@ -426,7 +435,7 @@ TopStat ::= Import
| PackageObject
| EndMarker
|
Packaging ::= ‘package’ QualId [nl | colonEol] ‘{’ TopStats‘}’
Packaging ::= ‘package’ QualId :<<< TopStats >>>
PackageObject ::= ‘package’ ‘object’ ObjectDef

CompilationUnit ::= {‘package’ QualId semi} TopStats
Expand Down