-
Notifications
You must be signed in to change notification settings - Fork 340
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade to Scalameta v3.7 #275
Conversation
I will start working on scalameta/scalameta#1479 today or tomorrow at the latest. |
byLine | ||
} else { | ||
val byCharacter = Integer.compare( | ||
x.range.get.startCharacter, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find it unfortunate that we always call .get on these. What is the reason that are always sure they are nonEmpty, yet make these optional? Since they are positional maybe this could be modeled better with the concept of pos.MIN and pos.MAX and never have an unpositioned item?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guard against isEmpty
above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is what I typically use in these cases:
(x.range, y.range) match {
case (None, _) | (_, None) => 0
case (Some(xRange), Some(yRange)) =>
//...
}
or
(x.range, y.range) match {
case (Some(xRange), Some(yRange)) =>
//...
case _ => 0
}
implicit private val occurrenceOrdering: Ordering[SymbolOccurrence] = | ||
new Ordering[semanticdb3.SymbolOccurrence] { | ||
override def compare(x: SymbolOccurrence, y: SymbolOccurrence): Int = { | ||
if (x.range.isEmpty || x.range.isEmpty) 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Double check on x
, you probably meant y.range.isEmpty
the second time.
startLine = r.startLine, | ||
startColumn = r.startCharacter, | ||
endLine = r.endLine, | ||
endColumn = r.endCharacter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these are the cases in which I wish we had a structural type system
@@ -42,12 +42,12 @@ import monix.reactive.Observable | |||
import monix.reactive.Observer | |||
import monix.reactive.OverflowStrategy | |||
import org.langmeta.inputs.Input | |||
import org.langmeta.internal.semanticdb.XtensionDatabase | |||
import org.langmeta.internal.semanticdb.schema | |||
import scala.meta.internal.semanticdb3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
curiosity: why are semanticdb3
data types internal?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because otherwise we'd have to provide strong compatibility guarantees for them: https://github.com/scalameta/scalameta/blob/master/COMPATIBILITY.md. I don't think we're fully ready to do this right now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking great, the new names make everything so much clearer to me (especially SymbolOccurrence
vs ResolvedName
is a welcome change).
I've found some minor issues and asked some questions. See the comments.
logger.info(s"Loading file $filename") | ||
d.withUri(filename) | ||
.withOccurrences { | ||
// This should be done inside semanticdb-scalac. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there an issue to track this? (I know the comment was already there)
Some(Position(definition.start, definition.end)), | ||
val role = | ||
if (kind.isPackage) s.SymbolOccurrence.Role.REFERENCE | ||
else s.SymbolOccurrence.Role.REFERENCE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if
and else
branches yield the same value
case t: Defn.Var => | ||
pats(t.pats, Kind.METHOD, Property.VAR.value); stop() | ||
case t: Decl.Var => | ||
pats(t.pats, Kind.METHOD, Property.VAR.value); stop() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is is right that Defn.Var/Val
are all Kind.METHOD
?
I've read https://github.com/scalameta/scalameta/blob/master/semanticdb/semanticdb3/semanticdb3.md#symbolinformation-1, it's not clear whether t
is the field or the synthetic method here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The SymbolOccurrence
part of the format is still unspecified at this point. For now, I propose that we do whatever semanticdb-scalac does.
range = None | ||
) | ||
} | ||
result.getOrElse(Hover(Nil, None)) | ||
} | ||
|
||
/** Returns a definition tree for this symbol signature */ | ||
private def getPrettyDefinition( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm glad to see this gone in favor of a scalameta pretty-printer!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree! The current hover output is not helpful but that needs to be fixed upstream.
sym | ||
) | ||
case _ => | ||
} | ||
document.symbols.foreach { | ||
case s.ResolvedSymbol(sym, Some(denot)) => | ||
case denot: semanticdb3.SymbolInformation => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/denot/symbolInfo ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or simply info
. This is how we name related variables in scalameta/scalameta.
): List[LintMessage] = | ||
Patch.lintMessages(patches, ctx) | ||
Patch(patches, ctx, Some(index))._2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor: for readability, I tend to shy away from _N
methods in tuples.
Can we do:
val (_, lintMessage) = Patch(patches, ctx, Some(index))
lintMessage
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
Thanks for the comments! The test suite passes and most things seem to work OK except the remaining TODOs |
NB, a couple test suites are still disabled due to blocking issues. Fixing hover tests will require integrating with metacp and make hover depend on |
new Ordering[semanticdb3.SymbolOccurrence] { | ||
override def compare(x: SymbolOccurrence, y: SymbolOccurrence): Int = { | ||
if (x.range.isEmpty) 0 | ||
else if (y.range.isEmpty) 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (x.range.isEmpty || y.range.isEmpty) 0
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made it this way to make it easier to see diff between x.range
and y.range
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For that reason I like to format such conditions this way:
if (
x.range.isEmpty ||
y.range.isEmpty
) ...
But scalafmt with changes it to this:
if (x.range.isEmpty ||
y.range.isEmpty) ...
😒
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an undocumented align.ifWhileOpenParen = true
to override that @laughedelic I agree that's the most annoying part of align = false
, this is the indentation style used by Scala.js.
We should probably support config style in if conditions though 🤔
cc/ @martijnhoekstra FYI the scalapb protobuf generation is no longer part of the build step, it wasn't required. |
Blocked by scalacenter/scalafix#699 with test failures https://gist.github.com/olafurpg/30729248cc54ccafa3802f87850a683e Help to grind through the reported issues would be appreciated https://github.com/scalameta/scalameta/milestone/26 The tickests are ordered by importance, scalameta/scalameta#1491 is by far most critical |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While manually testing the new hover I hit on the problem that classes with @JsonCodec
don't have SymbolInformation. I opened scalameta/scalameta#1504 to track this
@@ -214,7 +214,7 @@ object HoverTest extends BaseHoverTest { | |||
| List(1) match { case <<x>> :: Nil => } | |||
|} | |||
""".stripMargin, | |||
"val x: Int" | |||
"Int" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Blocked by scalameta/scalameta#1503
OK, this is ready for another round of review. All blocking issues should be addressed now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall I’m a big 👍 on this PR. Feedback is small, I would always like to see more issues vs TODOs (we called them TODONTs in my last gig 😀 ) and continue to dislike the number of times we call Option.get but that may just be a fact of life with Protobuf.
@@ -493,10 +503,11 @@ class MetalsServices( | |||
|
|||
object MetalsServices extends LazyLogging { | |||
lazy val cacheDirectory: AbsolutePath = { | |||
val cacheVersion = "0.1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we just bump this every time we change format?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct, I moved this variable to Mtags
, added a detailed docstring and changed the number to "1"
so upgrades are a simple increment. No need for semantic versioning
) | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This whole file was a pleasure to read.
@@ -35,26 +35,28 @@ class ScalacProvider()(implicit client: JsonRpcClient) extends LazyLogging { | |||
// Looking up by sourceDirectory alone is not enough since | |||
// in sbt it's possible to `sources += file("blah")`, which would | |||
// not be respected if we only went by directories. | |||
.orElse(compilerBySourceDirectory(uri)) | |||
.orElse(compilerBySourceDirectory(uri).map(_._2)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw you and @gabro discussing earlier but don’t remember the result. Do we have a convention to avoid ._N methods? I personally don’t like the readability here as it makes it difficult to see that we are falling back to Global.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I agree _N
methods are hard to read. Updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this might actually be a decent linter rule "Introduce a case class or use pattern matching instead! Overuse of tuples is a code smell"
|
||
import scalafix.patch.TokenPatch._ | ||
// TODO(olafur): make this method public in scalafix API |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about a ticket for this instead of a todo?
Defn.Trait(mods, Type.Name(info.name), Nil, EmptyCtor, EmptyTemplate) | ||
} else if (info.kind.isClass) { | ||
// TODO: include type parameters and primary constructor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I worry about TODO vs tickets. Consider creating issues for these and referencing the issue in the TODO comment.
PrettyType.toType(info.tpe.get, symtab, QualifyStrategy.Readable).tree | ||
val tpe = info.tpe match { | ||
case Some(t) => | ||
if (t.tag.isMethodType) t.methodType.get.returnType.get |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean if t is tagged as a method type that we know that both get calls will succeed here? Is this more protobuf influence? The number of calls we make to get is a continual source of concern for me.
} catch { | ||
case _: InvalidProtocolBufferException => | ||
logger.error( | ||
s"Have you upgraded to SemanticDB v3? Error parsing $semanticdbPath" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does the protobuf have a specific version field we can check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After SemanticDB v3 there is TextDocument.schema :) I tried to implement a smarter thing to detect old payloads but saw the logic would have to be in the critical path, so I opted to catch exceptions instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review @ShaneDelmore I updated the PR and added several more cleanups
@@ -493,10 +503,11 @@ class MetalsServices( | |||
|
|||
object MetalsServices extends LazyLogging { | |||
lazy val cacheDirectory: AbsolutePath = { | |||
val cacheVersion = "0.1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct, I moved this variable to Mtags
, added a detailed docstring and changed the number to "1"
so upgrades are a simple increment. No need for semantic versioning
} catch { | ||
case _: InvalidProtocolBufferException => | ||
logger.error( | ||
s"Have you upgraded to SemanticDB v3? Error parsing $semanticdbPath" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After SemanticDB v3 there is TextDocument.schema :) I tried to implement a smarter thing to detect old payloads but saw the logic would have to be in the critical path, so I opted to catch exceptions instead.
@@ -35,26 +35,28 @@ class ScalacProvider()(implicit client: JsonRpcClient) extends LazyLogging { | |||
// Looking up by sourceDirectory alone is not enough since | |||
// in sbt it's possible to `sources += file("blah")`, which would | |||
// not be respected if we only went by directories. | |||
.orElse(compilerBySourceDirectory(uri)) | |||
.orElse(compilerBySourceDirectory(uri).map(_._2)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I agree _N
methods are hard to read. Updated
@@ -35,26 +35,28 @@ class ScalacProvider()(implicit client: JsonRpcClient) extends LazyLogging { | |||
// Looking up by sourceDirectory alone is not enough since | |||
// in sbt it's possible to `sources += file("blah")`, which would | |||
// not be respected if we only went by directories. | |||
.orElse(compilerBySourceDirectory(uri)) | |||
.orElse(compilerBySourceDirectory(uri).map(_._2)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this might actually be a decent linter rule "Introduce a case class or use pattern matching instead! Overuse of tuples is a code smell"
I switched to this branch, cleaned all cache, did
And then I hit on this:
Trace: `java.util.NoSuchElementException: scala.meta.metals.Configuration#`
|
Thanks for trying it out @laughedelic ! That particular "failed to pretty print" error is caused by scalameta/scalameta#1504 The symbol table does not have an entry for The first error |
I'm not sure if it's a good idea to tone down the logs to skip such cases 🤔 |
When a file contains deprecation messages, semanticdb-scalac appends an empty TextDocument containing only the diagnostics section.
* Disable scalafix * Remove trailing comma * Reformat * Fix scripted tests * Remove unused scalapb build integration
- setup metacp integration - setup symbol table integration - adjust hover expected output
The latest release exposes a better API that made it possible to remove almost all of copy-pasted code from scalafix internals.
- Include JDK classpath - Document blocking issues
SemanticDB v2 payloads will now cause an error to be reported.in the logs.
- Document Mtags version number - Replace TODO with tickets - Simplify handling of class info type in hover - Include primary constructor for hover
I've rebased on top of master. Please refrain from refactoring the codebase to avoid further rebase issues. I haven't been able to address the remaining blocking issues since I've been traveling quite a lot past weeks and the upstream fixes require more focused work than what I can do on an airplane. |
I am fine to keep this PR open longer, I don't mind rebasing on top of master for new changes. My comment above was primarily about refactorings that move stuff around because that unnecessarily conflicts with changes in this PR. |
I don't think we should merge this PR for now. As it is, merging would be a regression and the upgrade doesn't bring immediate benefits feature wise. A better time would be when a stable Scalafix v0.6 is out since then it will arguable be a huge simplification that we no longer have to deal with |
The primary tickets blocking this change are
|
I haven't forgotten this PR! There's been a lot of activity in scalameta/scalameta recently to address the remaining issues blocking this PR. For example, we merged a PR today addressing problem related to incremental compilation scalameta/scalameta#1612 There remain 26 open issues in the v4.0 milestone https://github.com/scalameta/scalameta/milestone/26 and 70 issues have already been closed. The deadline for release is July 9th. I'm gonna close this PR until then :) |
A PR fixing |
This PR upgrades from Scalameta v2.1.7 from January this year to Scalameta v3.7.3 that was released this morning. The SemanticDB schema went through aggressive changes that primarily involved
Position.{start,end}
offsets toRange.{startLine, startCharacter, endLine, endCharacter}
to align with LSP conventions.In addition, the following changes in semanticdb-scalac directly affect Metals
Denotation.signature
is deprecated in favor ofSymbolInformation.tpe
, which affects HoverProvider that used the oldsignature
to show a readable signature. We are blocked by Metap should expose API to pretty-print single SymbolInformation.tpe scalameta#1479 to produce human readable output forSymbolInformation.tpe
.-P:semanticdb:denotations:definitions
is now default, meaning that scalafix depends onmetalsSetup
in order to process the dependency classpath.This PR is primarily about upgrading and making sure the functionality works OK. I'd like to get this merged asap to avoid merge conflicts.
TODOs:
metalsSetup
SymbolInformation.tpe
, blocked by Metap should expose API to pretty-print single SymbolInformation.tpe scalameta#1479SymbolInformation.Kind