-
Notifications
You must be signed in to change notification settings - Fork 185
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve documentation and polish several details
- Simplify `github:` expansino - New "Developer guide" section with a comprehensive tutorial for writing a syntactic linters and semantic rewrite. - Upgrade to scalafmt v1.5.1 and use javadoc formatting for docstrings - Docstrings for SymbolInfo, SType and STree (still needs more work)
- Loading branch information
Showing
58 changed files
with
2,078 additions
and
958 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
--- | ||
id: api | ||
title: API Documentation | ||
--- | ||
|
||
The Scalafix public API documentation is composed of several packages. | ||
|
||
## Scalafix v1 | ||
|
||
Latest Scaladoc: | ||
[v@VERSION@](https://static.javadoc.io/ch.epfl.scala/scalafix-core_2.12/@VERSION@/scalafix/v1/index.html) | ||
|
||
The Scalafix v1 API is available through `import scalafix.v1._`. Key data | ||
structures include: | ||
|
||
- `Patch`: to describe source code rewrites such as removing or replacing tokens | ||
and trees. | ||
- `Diagnostic`: a linter error message that can be reported at a source file | ||
location. | ||
- `SyntacticRule`: super class for all syntactic rules. | ||
- `SyntacticDocument`: context about a single source file containing syntactic | ||
information such as trees/tokens. | ||
- `SemanticRule`: super class for all semantic rules. | ||
- `SemanticDocument`: context about a single source file containing syntactic | ||
information such as trees/tokens and semantic information such as | ||
symbols/types/synthetics. | ||
- `Symbol`: a unique identifier for a definition. For example, the `String` | ||
class has the symbol `java/lang/String#`. The type alias to `String` in the | ||
Scala standard library has the symbol `scala/Predef.Symbol#`. | ||
- `SymbolInfo`: metadata about a `Symbol`, including accessibility | ||
(private/protected/...), kind (val/def/class/trait/...), properties | ||
(final/abstract/implicit), language (Scala/Java) and type signature. | ||
- `SType`: an ADT that encodes the full Scala type system. | ||
- `STree`: an ADT that encodes synthetic tree nodes such as inferred `.apply`, | ||
inferred type parameters, implicit arguments and implicit conversions. | ||
|
||
## Scalafix v0 | ||
|
||
Latest Scaladoc: | ||
[v@VERSION@](https://static.javadoc.io/ch.epfl.scala/scalafix-core_2.12/@VERSION@/scalafix/v0/index.html) | ||
|
||
This is a legacy API that exists to smoothen the migration for existing Scalafix | ||
v0.5 rules. If you are writing a new Scalafix rule, please use the v1 API. | ||
|
||
## Scalameta Trees | ||
|
||
Latest Scaladoc: | ||
[v@SCALAMETA@](https://static.javadoc.io/org.scalameta/trees_2.12/@SCALAMETA@/scala/meta/index.html) | ||
|
||
The type `scala.meta.Tree` is a | ||
["lossless syntax tree"](http://www.oilshell.org/blog/2017/02/11.html) for the | ||
Scala language syntax. A Scalameta Tree preserves more details than the Scala | ||
compiler syntax trees. The detailed syntax tree nodes in Scalameta are great for | ||
fine-grained manipulation source code like is required in Scalafix. | ||
|
||
The tree API is available through `import scala.meta._`. Key data structures | ||
include: | ||
|
||
- `Tree`: the supertype of all tree nodes. Key methods include: | ||
- `pos: Position`: the source code position of this tree node | ||
- `symbol: Symbol`: extension method made available via | ||
`import scalafix.v1._`, requires an implicit `SemanticDocument`. Returns the | ||
the symbol of this tree node | ||
- `syntax: String`: the pretty-printed tree node | ||
- `structure: String`: the raw structure of this tree node, useful for | ||
figuring out how to pattern match against a particular tree node. | ||
- `Term`: the supertype for all term nodes in "term position". Terms are the | ||
parts of programs that get executed at runtime such as `run(42)` in | ||
`val x: String = run(42)`. | ||
- `Type`: the supertype for all type nodes in "type position". Type are the | ||
parts of programs that only exist at compile time, such as `String` in | ||
`val x: String = ""`. | ||
- `Stat`: the supertype for all tree nodes that can appear in "statement | ||
position". Statement position is the | ||
- `Defn`: the supertype for all definitions | ||
|
||
## Scalameta Tokens | ||
|
||
Latest Scaladoc: | ||
[v@SCALAMETA@](https://static.javadoc.io/org.scalameta/tokens_2.12/@SCALAMETA@/scala/meta/tokens/Token.html) | ||
|
||
The type `scala.meta.Token` is a representation of all lexical tokens in the | ||
Scala language. Each token kind has it's own type, such as `Token.Space`, and | ||
`Token.KwClass` ("keyword `class`"). | ||
|
||
The token API is available through `import scala.meta._`. Key data structures | ||
include: | ||
|
||
- `Token`: the supertype for all token kinds. Sub-types of `Token` include | ||
`Token.Space`, `Token.KwClass` ("keyword `class`"), `Token.LeftParen` and | ||
`Token.Colon`. | ||
- `Tokens`: a sequence of tokens with efficient collection operations such as | ||
slice, take and drop. It's important to keep in mind that a normal source file | ||
can have tens of thousands of tokens so operations like `filter` may perform | ||
badly. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
--- | ||
id: before-you-begin | ||
title: Before you write code | ||
--- | ||
|
||
Before you dive right into the implementation details of your rule, it's good to | ||
ask yourself the following questions first. | ||
|
||
## Do you want a linter or rewrite? | ||
|
||
- Rewrites can automatically fix problems, which is ideal but means rewrites can | ||
only address problems that have obvious solutions. | ||
|
||
- Linters highlight problematic code without providing a fix, which means they | ||
demand more interaction from the user but can at the same time target a larger | ||
problem space than rewrites. | ||
|
||
## Is your rule syntactic or semantic? | ||
|
||
- Syntactic rules are simple to run since they don't require compilation but | ||
syntactic rules are limited since they don't have access to symbols and types. | ||
- Semantic rules are are slower and more complicated to run since they require | ||
compilation but at the same time semantic rules are able to perform more | ||
advanced analysis since they have access to symbols and types. | ||
|
||
## What is an acceptable false-positive rate? | ||
|
||
- If you are doing a one-off migration script for your company codebase then it | ||
might be OK to manually fix trickier corner cases. | ||
- If your linter catches critical production bugs it's maybe OK that it has a | ||
few false positives. If your linter catches low-importance bugs like enforcing | ||
a code style then false positives are not acceptable. | ||
|
||
## What diff does the rewrite produce? | ||
|
||
Before implementing a rule, it's good to manually migrate/refactor a few | ||
examples first. Manual refactoring is great to discover corner and estimate how | ||
complicated the rule needs to be. | ||
|
||
## Who will use your rule? | ||
|
||
The target audience/users of your rule can impact the implementation the rule. | ||
If you are the only end-user of the rule, then you can maybe take shortcuts and | ||
worry less about rare corner cases that may be easier to fix manually. If your | ||
rule is intended to be used by the entire Scala community, then you might want | ||
to be more careful with corner cases. | ||
|
||
## What codebase does your rule target? | ||
|
||
Is your rule specific to a particular codebase or is the rule intended to be | ||
used for any arbitrary project? It's easier to validate your a rule if it will | ||
only run on a single codebase. You may not even need tests, since the codebase | ||
is the only test. If your rule is intended to be used in any random codebase, | ||
you need to put more effort into handling corner cases. In general, the smaller | ||
the target domain of your rule, the easier it is to implement a rule. | ||
|
||
## How often will your rule run? | ||
|
||
Are you writing a one-off migration script or will your rule run on every pull | ||
request? A rule that runs on every pull request should ideally have some unit | ||
tests and be documented so that other people can help maintain the rule. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
--- | ||
id: setup | ||
title: Setup | ||
--- | ||
|
||
Scalafix supports writing custom linter and rewrite rules. Custom rules have | ||
access to a rich Scalafix public API and users can run custom rules with the | ||
same UX as built-in rules, including tab completions in the sbt-scalafix plugin. | ||
|
||
## sbt | ||
|
||
The quickest way to get started in sbt is to use the `scalafix.g8` project | ||
template: | ||
|
||
``` | ||
cd reponame # The project you want to implement rules for. | ||
sbt new scalacenter/scalafix.g8 --rule="reponame" --version="v1.0" | ||
cd scalafix | ||
sbt tests/test | ||
``` | ||
|
||
The `--rule=<reponame>` option should match the name of your GitHub repository. | ||
The template generates a `scalafix/` directory with four different sbt projects | ||
|
||
```scala | ||
scalafix | ||
├── rules // rule implementations | ||
├── input // code to be analyzed by rule | ||
├── output // expected output from running input on rules | ||
└── tests // small project where unit tests run | ||
``` | ||
|
||
The `scalafix/` directory is a self-contained sbt build and can live in the same | ||
directory as your existing library. | ||
|
||
To run unit tests, execute `tests/test`. For a productive edit/test/debug | ||
workflow, it's recommended to open an sbt shell and re-run the tests on file | ||
save | ||
|
||
``` | ||
$ sbt | ||
> ~tests/test | ||
``` | ||
|
||
### Import into IntelliJ | ||
|
||
The project generated should import into IntelliJ like a normal project. Input | ||
and output test files are written in plain `*.scala` files and should have full | ||
IDE support. | ||
|
||
### Customizing input and output projects | ||
|
||
The input and output projects are regular sbt projects and can have custom | ||
library dependencies. To add library dependencies to the input project, add the | ||
following sbt settings | ||
|
||
```diff | ||
lazy val input = project.settings( | ||
+ libraryDependencies += "org" %% "name" % "version" | ||
) | ||
``` | ||
|
||
The only requirement is that the input project uses a Scala compiler version | ||
that is supported by | ||
[SemanticDB](https://github.com/scalameta/scalameta/blob/master/semanticdb/semanticdb3/semanticdb3.md) | ||
compiler plugin. Supported Scala compiler versions include: | ||
|
||
- Scala @SCALA211@ | ||
- Scala @SCALA212@ | ||
|
||
The output project can use any Scala compiler version. | ||
|
||
### Running a single test | ||
|
||
To run a single test case instead of the full test suite run the following | ||
command: `tests/testOnly *MySuite -- -z MyRegex`. This will run only test cases | ||
with the names matching the regular expression "MyRegex" inside test suites | ||
matching the regular expression `*MySuite`. See the ScalaTest documentation | ||
section: | ||
[Selecting suites and tests](http://www.scalatest.org/user_guide/using_the_runner#selectingSuitesAndTests) | ||
for more details about the `-z` option. | ||
|
||
## Library API | ||
|
||
To test custom Scalafix rules outside of sbt use the library API of | ||
`scalafix-testkit` directly. Start by adding a dependency on the | ||
[scalafix-testkit](https://search.maven.org/artifact/ch.epfl.scala/scalafix-testkit_@SCALA212@/@VERSION@/jar) | ||
artifact | ||
|
||
``` | ||
coursier fetch ch.epfl.scala:scalafix-testkit_@SCALA212@:@VERSION@ | ||
``` | ||
|
||
Next, create a test suite that extends the class `SemanticRuleSuite` | ||
|
||
```scala | ||
package myproject | ||
class MyTests extends scalafix.testkit.SemanticRuleSuite { | ||
runAllTests() | ||
} | ||
``` | ||
|
||
This setup assumes there will be a resource file `scalafix-testkit.properties` | ||
with instructions about the full classpath of the input sources and source | ||
directories for the input and output sources. To learn more about what values | ||
should go into `scalafix-testkit.properties`, consult the Scaladocs. | ||
|
||
Latest Scaladoc: | ||
[`TestkitProperties`](https://static.javadoc.io/ch.epfl.scala/scalafix-testkit_@SCALA212@/@VERSION@/scalafix/testkit/TestkitProperties.html) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
--- | ||
id: sharing-rules | ||
title: Sharing rules | ||
--- | ||
|
||
## Run the rule from source code | ||
|
||
Running a rule from source code is the simplest way to run a custom rule. | ||
However, rules that are compiled from source have the following limitations: | ||
|
||
- Limited code re-use, rule must be implemented in a single source file | ||
- No dependencies, rules can only use the Scalafix public API | ||
- Slow, rule is re-compiled on every invocation so it's not great for | ||
interactive usage. | ||
- No tab completion in the sbt shell, users need to manually type the path to | ||
the source file to discover the rule | ||
|
||
To run a custom rule from source, start by installing the sbt plugin as | ||
instructed in [here](../users/installation.md#sbt). The SemanticDB compiler | ||
plugin must be enabled to run semantic rules like `NamedLiteralArguments`. | ||
|
||
Next, open an sbt shell | ||
|
||
``` | ||
sbt | ||
> | ||
``` | ||
|
||
You have different options to run the rule from soruce: `file:`, `http:` or | ||
`github:` | ||
|
||
### Using `file:` | ||
|
||
The easiest way to run a custom rule from source code is to use the | ||
`file:/path/to/NamedLiteralArguments.scala` syntax. | ||
|
||
``` | ||
scalafix file:/path/to/NamedLiteralArguments.scala | ||
``` | ||
|
||
### Using `http:` | ||
|
||
Another way to run a rule from source is to publish it as a gist and share the | ||
raw URL | ||
|
||
``` | ||
scalafix https://gist.githubusercontent.com/olafurpg/eeccf32f035d13f0728bc94d8ec0a776/raw/78c81bb7f390eb98178dd26ea03c42bd5a998666/NamedLiteralArguments.scala | ||
``` | ||
|
||
### Using `github:` | ||
|
||
Another way to run custom rules from source is to use the `github:org/repo` | ||
scheme. | ||
|
||
``` | ||
sbt | ||
> scalafix github:olafurpg/named-literal-arguments | ||
... | ||
``` | ||
|
||
The expansion rules for `github:org/repo` is the following | ||
|
||
| Before | After | | ||
| -------------------------------------- | ------------------------------------------------------------------------ | | ||
| `github:org/repo` | `scalafix/rules/src/main/scala/fix/Repo.scala` | | ||
| `github:org/some-repo` | `scalafix/rules/src/main/scala/fix/SomeRepo.scala` | | ||
| `github:org/repo/RuleName` | `scalafix/rules/src/main/scala/fix/RuleName.scala` | | ||
| `github:org/repo/RuleName?sha=HASH125` | (at commit `HASH125`) `scalafix/rules/src/main/scala/fix/RuleName.scala` | | ||
|
||
## Publish your rule to Maven Central | ||
|
||
Branch: | ||
[`step-6`](https://github.com/olafurpg/named-literal-arguments/commit/88f18b16c9dd939a3f1c08672b121ac2bc1c590d) | ||
|
||
The most robust way to share a custom rule is to publish it as a library to | ||
Maven Central. The diff in `step-6` shows the necessary changes to build.sbt to | ||
populate publishing information. The | ||
[sbt-ci-release](https://github.com/olafurpg/sbt-ci-release) readme documents | ||
the further steps to configure gpg and Sonatype. | ||
|
||
Once you have your rule, users can depend on it in the sbt plugin by updating | ||
`scalafixDependencies` | ||
|
||
```scala | ||
// build.sbt | ||
scalafixDependencies in ThisBuild += | ||
"com.geirsson" % "named-literal-arguments_2.12" % "VERSION" | ||
// sbt shell | ||
> scalafix NamedLiteralArguments | ||
``` | ||
|
||
Users of the Scalafix command-line interface can use the `--tool-classpath` flag | ||
|
||
``` | ||
scalafix \ | ||
--tool-classpath $(coursier fetch com.geirsson:named-literal-arguments_2.12:VERSION) \ | ||
-r NamedLiteralArguments \ | ||
--classpath MY_PROJECT_CLASSPATH \ | ||
my-project/src/main/scala | ||
``` | ||
|
||
Don't be intimidating by publishing to Maven Central, it gets easier once you've | ||
done it the first time. The benefits of publishing a rule to Maven Central are | ||
many. | ||
|
||
- Dependencies, you can use custom library dependency to implement your rule | ||
- Fast to run, no need to re-compile the rule on every Scalafix invocation | ||
- Tab completion in sbt, users can tab-complete your rule when using sbt plugin | ||
j |
Oops, something went wrong.