Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AST-based Highlighter #197

Merged
merged 42 commits into from
Jan 14, 2025
Merged

AST-based Highlighter #197

merged 42 commits into from
Jan 14, 2025

Conversation

fdodino
Copy link
Contributor

@fdodino fdodino commented Dec 19, 2024

Salió el AST Highlighter nomás! Tomé el trabajo original de Octavio, le corregí errores que iba tirando, le agregué 67 tests y lo probé bastante en profundidad, aun así le agregué una configuración para eventualmente desactivarlo si lo necesitamos.

  • agregar tests unitarios para asegurarnos completitud
  • logré que el tokenProvider no tenga dependencias con vscode, y que en todo caso el highlighter lo convierta a cosas de VSCode si es que no tira la performance para abajo (no parece tirarlo)
  • La extensión que usa el highlighter por regex sigue sirviendo como plan B por si se rompe el highlighter o bien si se rompe el parser y no tenemos AST

@fdodino fdodino marked this pull request as ready for review January 7, 2025 03:24
@fdodino fdodino requested review from ivojawer and PalumboN January 7, 2025 03:24
Copy link
Contributor

@PalumboN PalumboN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VAMOOOOO!! 🚀 💯 🟢 🔴 🟡 🔵 🟤

Dejé algunos comentarios, el que más me interesa es sobre la API, si no hay que usar el EnvironmentProvider que se encarga de pasar de files a AST.

Y altos tests wachoooo!!! 🚥

@@ -1,4 +1,4 @@
<!-- DO NOT REMOVE - contributor_list:data:start:["ivojawer", "fdodino", "PalumboN", "npasserini", "Miranda-03", "FerRomMu", "dependabot[bot]"]:end -->
<!-- DO NOT REMOVE - contributor_list:data:start:["ivojawer", "fdodino", "PalumboN", "npasserini", "Miranda-03", "dependabot[bot]", "FerRomMu"]:end -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Qué onda esto? Bancamos a dependabot[bot] 😆

package.json Outdated Show resolved Hide resolved
Comment on lines 30 to 34
const tokensBuilder = new vscode.SemanticTokensBuilder(legend)
const parsedFile = parse.File(document.fileName)
const textFile = document.getText()
const parsedPackage = parsedFile.tryParse(textFile)
const packageNode = parsedPackage.members[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Esto está bien que se parsee acá? No debería sacar/armar el AST usando el EnvironmentProvider?

Comment on lines 36 to 37
const splittedLines = textFile.split('\n')
const processed = excludeNullish(processCode(packageNode, splittedLines))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acá por qué es necesario las splittedLines?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

contesto por acá para dejar por escrito que highlightea por línea o tira error (sí, es así, nació con el corazón ortiva)

{
const nodeResults = mergeHighlightingResults(processNode(node, textDocument, acumResults.references), processAnnotationsForNode(node, textDocument, acumResults.references))
return mergeHighlightingResults(acumResults, nodeResults)
}, { result: [], references: [{ name: 'console', type: 'Reference' }] }).result
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Qué onda este console acá?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jajaja, tampoco tengo mucha idea pero se rompe el test del program. Hay que investigarlo

Comment on lines 13 to 18
const parsedFile = parse.File(filePath)
const docText = readFileSync(filePath, { encoding: 'utf-8' })
const parsedPackage = parsedFile.tryParse(docText)
const splittedLines = docText.split('\n')
const packageNode = parsedPackage.members[0]
const processed = excludeNullish(processCode(packageNode, splittedLines))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Esta lógica está también en highlighter.ts. Y dejé un comentario.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sí, ésto podríamos refactorizarlo a futuro, pero era la parte del highlighter que usa la dependencia a vscode vs. los tests que yo logré aislar para poder correrlos en forma unitaria

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahí metí un refactor para aprovechar más código (del issue #199)

fdodino and others added 3 commits January 9, 2025 23:49
Co-authored-by: Nahuel Palumbo <nahuel.palumbo@gmail.com>
@fdodino fdodino changed the title Higlighter AST-based Highlighter AST-based Jan 10, 2025
@fdodino fdodino changed the title Highlighter AST-based AST-based Highlighter Jan 10, 2025
@fdodino fdodino merged commit a7766f2 into master Jan 14, 2025
5 of 6 checks passed
@fdodino fdodino deleted the ast-highlighter branch January 14, 2025 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants