Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hovering on a macro should show more details like description and arguments #1178

Closed
1 task
saravmajestic opened this issue May 30, 2024 · 1 comment
Closed
1 task
Assignees
Labels
enhancement New feature or request sweep

Comments

@saravmajestic
Copy link
Collaborator

saravmajestic commented May 30, 2024

Describe the feature

When referencing a macro in dbt model, it will be good to show a hover popup (using vscode extension hover provider. example: src\hover_provider\modelHoverProvider.ts) This hover popup should show description of macro and the arguments. The description and arguments need to be populated in macroMetaMap in src\manifest\parsers\macroParser.ts.

Describe alternatives you've considered

Manually check macro file

Who will benefit?

All users

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@saravmajestic saravmajestic added the enhancement New feature or request label May 30, 2024
@saravmajestic saravmajestic self-assigned this May 30, 2024
Copy link
Contributor

sweep-ai bot commented May 30, 2024

🚀 Here's the PR! #1179

💎 Sweep Pro: You have unlimited Sweep issues

Actions

  • ↻ Restart Sweep

Step 1: 🔎 Searching

Here are the code search results. I'm now analyzing these search results to write the PR.

Relevant files (click to expand). Mentioned files will always appear here.

import { readFileSync } from "fs";
import { provide } from "inversify-binding-decorators";
import { DBTTerminal } from "../../dbt_client/dbtTerminal";
import { MacroMetaMap } from "../../domain";
import { DBTProject } from "../dbtProject";
import { createFullPathForNode } from ".";
@provide(MacroParser)
export class MacroParser {
constructor(private terminal: DBTTerminal) {}
createMacroMetaMap(
macros: any[],
project: DBTProject,
): Promise<MacroMetaMap> {
return new Promise(async (resolve) => {
this.terminal.debug(
"MacroParser",
`Parsing macros for "${project.getProjectName()}" at ${
project.projectRoot
}`,
);
const macroMetaMap: MacroMetaMap = new Map();
if (macros === null || macros === undefined) {
resolve(macroMetaMap);
}
const rootPath = project.projectRoot.fsPath;
// TODO: these things can change so we should recreate them if project config changes
const projectName = project.getProjectName();
const packagePath = project.getPackageInstallPath();
if (packagePath === undefined) {
throw new Error(
"packagePath is not defined in " + project.projectRoot.fsPath,
);
}
for (const key in macros) {
const macro = macros[key];
const { package_name, name, original_file_path } = macro;
const packageName = package_name;
const macroName =
packageName === projectName ? name : `${packageName}.${name}`;
const fullPath = createFullPathForNode(
projectName,
rootPath,
packageName,
packagePath,
original_file_path,
);
if (!fullPath) {
continue;
}
try {
const macroFile: string = readFileSync(fullPath).toString("utf8");
const macroFileLines = macroFile.split("\n");
for (let index = 0; index < macroFileLines.length; index++) {
const currentLine = macroFileLines[index];
if (
currentLine.match(new RegExp(`macro\\s${name}\\(`)) ||
currentLine.match(
new RegExp(`test\\s${name.replace("test_", "")}\\(`),
)
) {
macroMetaMap.set(macroName, {
path: fullPath,
line: index,
character: currentLine.indexOf(name),
uniqueId: key,
});
break;
}
}
} catch (error) {
this.terminal.debug(
"MacroParser",
`File not found at '${fullPath}', probably compiled is outdated.`,
error,
);
}
}
this.terminal.debug(
"MacroParser",
`Returning macros for "${project.getProjectName()}" at ${
project.projectRoot
}`,
macroMetaMap,
);
resolve(macroMetaMap);
});
}

import {
Definition,
DefinitionLink,
DefinitionProvider,
Disposable,
Location,
Position,
ProviderResult,
TextDocument,
Uri,
} from "vscode";
import { MacroMetaMap } from "../domain";
import { DBTProjectContainer } from "../manifest/dbtProjectContainer";
import { ManifestCacheChangedEvent } from "../manifest/event/manifestCacheChangedEvent";
import { isEnclosedWithinCodeBlock, provideSingleton } from "../utils";
import { TelemetryService } from "../telemetry";
@provideSingleton(MacroDefinitionProvider)
export class MacroDefinitionProvider implements DefinitionProvider, Disposable {
private macroToLocationMap: Map<string, MacroMetaMap> = new Map();
private static readonly IS_MACRO = /\w+\.?\w+/;
private disposables: Disposable[] = [];
constructor(
private dbtProjectContainer: DBTProjectContainer,
private telemetry: TelemetryService,
) {
this.disposables.push(
dbtProjectContainer.onManifestChanged((event) =>
this.onManifestCacheChanged(event),
),
);
}
dispose() {
while (this.disposables.length) {
const x = this.disposables.pop();
if (x) {
x.dispose();
}
}
}
provideDefinition(
document: TextDocument,
position: Position,
): ProviderResult<Definition | DefinitionLink[]> {
return new Promise((resolve) => {
const textLine = document.lineAt(position).text;
const range = document.getWordRangeAtPosition(
position,
MacroDefinitionProvider.IS_MACRO,
);
const word = document.getText(range);
if (
range &&
textLine[range.end.character] === "(" &&
isEnclosedWithinCodeBlock(document, range)
) {
const packageName = this.dbtProjectContainer.getPackageName(
document.uri,
);
const macroName =
packageName !== undefined && !word.includes(".")
? `${packageName}.${word}`
: word;
const definition = this.getMacroDefinition(macroName, document.uri);
if (definition !== undefined) {
resolve(definition);
this.telemetry.sendTelemetryEvent("provideMacroDefinition");
return;
}
}
resolve(undefined);
});
}
private onManifestCacheChanged(event: ManifestCacheChangedEvent): void {
event.added?.forEach((added) => {
this.macroToLocationMap.set(
added.project.projectRoot.fsPath,
added.macroMetaMap,
);
});
event.removed?.forEach((removed) => {
this.macroToLocationMap.delete(removed.projectRoot.fsPath);
});
}
private getMacroDefinition(
macroName: string,
currentFilePath: Uri,
): Definition | undefined {
const projectRootpath =
this.dbtProjectContainer.getProjectRootpath(currentFilePath);
if (projectRootpath === undefined) {
return;
}
const macroMap = this.macroToLocationMap.get(projectRootpath.fsPath);
if (macroMap === undefined) {
return;
}
const location = macroMap.get(macroName);
if (location && location.path) {
return new Location(
Uri.file(location.path),
new Position(location.line, location.character),
);
}
return undefined;
}

import { Disposable, languages } from "vscode";
import { DBTPowerUserExtension } from "../dbtPowerUserExtension";
import { provideSingleton } from "../utils";
import { ModelHoverProvider } from "./modelHoverProvider";
import { SourceHoverProvider } from "./sourceHoverProvider";
@provideSingleton(HoverProviders)
export class HoverProviders implements Disposable {
private disposables: Disposable[] = [];
constructor(
private modelHoverProvider: ModelHoverProvider,
private sourceHoverProvider: SourceHoverProvider,
) {
this.disposables.push(
languages.registerHoverProvider(
DBTPowerUserExtension.DBT_SQL_SELECTOR,
this.modelHoverProvider,
),
);
this.disposables.push(
languages.registerHoverProvider(
DBTPowerUserExtension.DBT_SQL_SELECTOR,
this.sourceHoverProvider,
),
);
}
dispose() {
while (this.disposables.length) {
const x = this.disposables.pop();
if (x) {
x.dispose();
}
}
}

import { existsSync, readFileSync } from "fs";
import { provide } from "inversify-binding-decorators";
import * as path from "path";
import { Uri } from "vscode";
import { DBTTerminal } from "../../dbt_client/dbtTerminal";
import { DBTProject } from "../dbtProject";
import { ManifestCacheChangedEvent } from "../event/manifestCacheChangedEvent";
import { DocParser } from "./docParser";
import { GraphParser } from "./graphParser";
import { MacroParser } from "./macroParser";
import { NodeParser } from "./nodeParser";
import { SourceParser } from "./sourceParser";
import { TestParser } from "./testParser";
import { TelemetryService } from "../../telemetry";
import { ExposureParser } from "./exposureParser";
import { MetricParser } from "./metricParser";
@provide(ManifestParser)
export class ManifestParser {
private lastSentParseManifestProps: any;
constructor(
private nodeParser: NodeParser,
private macroParser: MacroParser,
private metricParser: MetricParser,
private graphParser: GraphParser,
private sourceParser: SourceParser,
private testParser: TestParser,
private exposureParser: ExposureParser,
private docParser: DocParser,
private terminal: DBTTerminal,
private telemetry: TelemetryService,
) {}
public async parseManifest(project: DBTProject) {
this.terminal.debug(
"ManifestParser",
`Going to parse manifest for "${project.getProjectName()}" at ${
project.projectRoot
}`,
);
const targetPath = project.getTargetPath();
if (!targetPath) {
this.terminal.debug(
"ManifestParser",
"targetPath should be defined at this stage for project " +
project.projectRoot.fsPath,
);
return;
}
const projectRoot = project.projectRoot;
const manifest = this.readAndParseManifest(projectRoot, targetPath);
if (manifest === undefined) {
const event: ManifestCacheChangedEvent = {
added: [
{
project,
nodeMetaMap: new Map(),
macroMetaMap: new Map(),
metricMetaMap: new Map(),
sourceMetaMap: new Map(),
testMetaMap: new Map(),
graphMetaMap: {
parents: new Map(),
children: new Map(),
tests: new Map(),
metrics: new Map(),
},
docMetaMap: new Map(),
exposureMetaMap: new Map(),
},
],
};
return event;
}
const {
nodes,
sources,
macros,
semantic_models,
parent_map,
child_map,
docs,
exposures,
} = manifest;
const nodeMetaMapPromise = this.nodeParser.createNodeMetaMap(
nodes,
project,
);
const macroMetaMapPromise = this.macroParser.createMacroMetaMap(
macros,
project,
);
const metricMetaMapPromise = this.metricParser.createMetricMetaMap(
semantic_models,
project,
);
const sourceMetaMapPromise = this.sourceParser.createSourceMetaMap(
sources,
project,
);
const testMetaMapPromise = this.testParser.createTestMetaMap(
nodes,
project,
);
const exposuresMetaMapPromise = this.exposureParser.createExposureMetaMap(
exposures,
project,
);
const docMetaMapPromise = this.docParser.createDocMetaMap(docs, project);
const [
nodeMetaMap,
macroMetaMap,
metricMetaMap,
sourceMetaMap,
testMetaMap,
docMetaMap,
exposureMetaMap,
] = await Promise.all([
nodeMetaMapPromise,
macroMetaMapPromise,
metricMetaMapPromise,
sourceMetaMapPromise,
testMetaMapPromise,
docMetaMapPromise,
exposuresMetaMapPromise,
]);
const graphMetaMap = this.graphParser.createGraphMetaMap(
project,
parent_map,
child_map,
nodeMetaMap,
sourceMetaMap,
testMetaMap,
metricMetaMap,
);
const nodeCounts = Object.values(nodes as any[]).reduce((map, node) => {
const key = node.resource_type + "_count";
if (!map.has(key)) {
map.set(key, 0);
}
map.set(key, map.get(key) + 1);
return map;
}, new Map());
const parseManifestProps = {
...Object.fromEntries(nodeCounts.entries()),
sources_count: sourceMetaMap.size,
macros_count: macroMetaMap.size,
};
if (
this.lastSentParseManifestProps === undefined ||
Object.entries(this.lastSentParseManifestProps).toString() !==
Object.entries(parseManifestProps).toString()
) {
// we only sent this event if there is a change in the monitored values
this.telemetry.sendTelemetryEvent(
"parseManifest",
{
project: DBTProject.hashProjectRoot(projectRoot.fsPath),
},
parseManifestProps,
);
this.lastSentParseManifestProps = parseManifestProps;
}
const event: ManifestCacheChangedEvent = {
added: [
{
project,
nodeMetaMap: nodeMetaMap,
macroMetaMap: macroMetaMap,
metricMetaMap: metricMetaMap,
sourceMetaMap: sourceMetaMap,
graphMetaMap: graphMetaMap,
testMetaMap: testMetaMap,
docMetaMap: docMetaMap,
exposureMetaMap: exposureMetaMap,
},
],
};
return event;
}
private readAndParseManifest(projectRoot: Uri, targetPath: string) {
const pathParts = [targetPath];
if (!path.isAbsolute(targetPath)) {
pathParts.unshift(projectRoot.fsPath);
}
const manifestLocation = path.join(...pathParts, DBTProject.MANIFEST_FILE);
this.terminal.debug(
"ManifestParser",
`Reading manifest at ${manifestLocation} for project at ${projectRoot}`,
);
try {
const manifestFile = readFileSync(manifestLocation, "utf8");
return JSON.parse(manifestFile);
} catch (error) {
this.terminal.debug(
"ManifestParser",
`Could not read manifest file at ${manifestLocation}, ignoring error`,
error,
);
}
}
}
export const createFullPathForNode: (
projectName: string,
rootPath: string,
packageName: string,
packagePath: string,
relativeFilePath: string,
) => string | undefined = (
projectName,
rootPath,
packageName,
packagePath,
relativeFilePath,
) => {
if (packageName !== projectName) {
const rootPathWithPackage = path.join(
packagePath,
packageName,
relativeFilePath,
);
if (existsSync(rootPathWithPackage)) {
return rootPathWithPackage;
}
return undefined;
}
return path.join(rootPath, relativeFilePath);

import { MarkdownString } from "vscode";
import { NodeMetaType, SourceMetaType } from "../domain";
export function generateHoverMarkdownString(
node: NodeMetaType | SourceMetaType,
nodeType: string,
): MarkdownString {
const content = new MarkdownString();
content.supportHtml = true;
content.isTrusted = true;
content.appendMarkdown(
`<span style="color:#347890;">(${nodeType})&nbsp;</span><span><strong>${node.name}</strong></span>`,
);
if (node.description !== "") {
content.appendMarkdown(`</br><span>${node.description}</span>`);
}
content.appendText("\n");
content.appendText("\n");
content.appendMarkdown("---");
content.appendText("\n");
content.appendText("\n");
for (const colKey in node.columns) {
const column = node.columns[colKey];
content.appendMarkdown(
`<span style="color:#347890;">(column)&nbsp;</span><span>${column.name} &nbsp;</span>`,
);
if (column.data_type !== null) {
content.appendMarkdown(
`<span>-&nbsp;${column.data_type.toLowerCase()}</span>`,
);
}
if (column.description !== "") {
content.appendMarkdown(
`<br/><span><em>${column.description}</em></span>`,
);
}
content.appendMarkdown("</br>");
}
return content;

import {
CancellationToken,
HoverProvider,
Disposable,
Position,
ProviderResult,
Range,
TextDocument,
Uri,
Hover,
MarkdownString,
} from "vscode";
import { NodeMetaMap } from "../domain";
import { DBTProjectContainer } from "../manifest/dbtProjectContainer";
import { ManifestCacheChangedEvent } from "../manifest/event/manifestCacheChangedEvent";
import { provideSingleton } from "../utils";
import { TelemetryService } from "../telemetry";
import { generateHoverMarkdownString } from "./utils";
import { DBTTerminal } from "../dbt_client/dbtTerminal";
@provideSingleton(ModelHoverProvider)
export class ModelHoverProvider implements HoverProvider, Disposable {
private modelToLocationMap: Map<string, NodeMetaMap> = new Map();
private static readonly IS_REF = /(ref)\([^)]*\)/;
private static readonly GET_DBT_MODEL = /(?!'|")([^(?!'|")]*)(?='|")/gi;
private disposables: Disposable[] = [];
constructor(
private dbtProjectContainer: DBTProjectContainer,
private telemetry: TelemetryService,
private dbtTerminal: DBTTerminal,
) {
this.disposables.push(
dbtProjectContainer.onManifestChanged((event) =>
this.onManifestCacheChanged(event),
),
);
}
dispose() {
while (this.disposables.length) {
const x = this.disposables.pop();
if (x) {
x.dispose();
}
}
}
provideHover(
document: TextDocument,
position: Position,
token: CancellationToken,
): ProviderResult<Hover> {
return new Promise((resolve) => {
const hover = document.getText(document.getWordRangeAtPosition(position));
const word = document.getText(
document.getWordRangeAtPosition(position, ModelHoverProvider.IS_REF),
);
const project = this.dbtProjectContainer.findDBTProject(document.uri);
if (!project) {
this.dbtTerminal.debug(
"modeHoverProvider:provideHover",
"Could not load hover provider, project not found in container for " +
document.uri.fsPath,
);
return;
}
if (word !== undefined && hover !== "ref") {
const dbtModel = word.match(ModelHoverProvider.GET_DBT_MODEL);
if (dbtModel && dbtModel.length === 1) {
const mdString = this.getHoverMarkdownFor(
project.getProjectName(),
dbtModel[0],
document.uri,
);
if (mdString !== undefined) {
const hover = new Hover(mdString, new Range(position, position));
resolve(hover);
}
this.telemetry.sendTelemetryEvent("provideModelHover", {
type: "single",
});
return;
}
if (dbtModel && dbtModel.length === 3) {
const mdString = this.getHoverMarkdownFor(
dbtModel[0],
dbtModel[2],
document.uri,
);
if (mdString !== undefined) {
const hover = new Hover(mdString, new Range(position, position));
resolve(hover);
}
this.telemetry.sendTelemetryEvent("provideModelHover", {
type: "dual",
});
return;
}
}
resolve(undefined);
});
}
private onManifestCacheChanged(event: ManifestCacheChangedEvent): void {
event.added?.forEach((added) => {
this.modelToLocationMap.set(
added.project.projectRoot.fsPath,
added.nodeMetaMap,
);
});
event.removed?.forEach((removed) => {
this.modelToLocationMap.delete(removed.projectRoot.fsPath);
});
}
private getHoverMarkdownFor(
projectName: string,
modelName: string,
currentFilePath: Uri,
): MarkdownString | undefined {
const projectRootpath =
this.dbtProjectContainer.getProjectRootpath(currentFilePath);
if (projectRootpath === undefined) {
return;
}
const nodeMap = this.modelToLocationMap.get(projectRootpath.fsPath);
if (nodeMap === undefined) {
return;
}
const node = nodeMap.get(modelName);
if (node) {
return generateHoverMarkdownString(node, "ref");
}
return undefined;
}

Step 2: ⌨️ Coding

src/manifest/parsers/macroParser.ts

Update the `createMacroMetaMap` method to parse macro descriptions and arguments.
--- 
+++ 
@@ -1,17 +1,26 @@
-for (let index = 0; index < macroFileLines.length; index++) {
-  const currentLine = macroFileLines[index];
-  if (
-    currentLine.match(new RegExp(`macro\\s${name}\\(`)) ||
-    currentLine.match(
-      new RegExp(`test\\s${name.replace("test_", "")}\\(`),
-    )
-  ) {
-    macroMetaMap.set(macroName, {
-      path: fullPath,
-      line: index,
-      character: currentLine.indexOf(name),
-      uniqueId: key,
-    });
+let description = "";
+let arguments = [];
+
+// Parse macro description
+for (let i = index - 1; i >= 0; i--) {
+  const line = macroFileLines[i].trim();
+  if (line.startsWith("/*") || line.startsWith('"""')) {
+    description = line.slice(2, -2).trim();
     break;
   }
-}
+}
+
+// Parse macro arguments
+const argsMatch = currentLine.match(/\((.*)\)/);
+if (argsMatch) {
+  arguments = argsMatch[1].split(",").map(arg => arg.trim());
+}
+
+macroMetaMap.set(macroName, {
+  path: fullPath,
+  line: index,
+  character: currentLine.indexOf(name),
+  uniqueId: key,
+  description,
+  arguments
+});

src/hover_provider/macroHoverProvider.ts

Create a new `MacroHoverProvider` class.
import {
  CancellationToken,
  HoverProvider,
  Hover,
  Position,
  ProviderResult,
  TextDocument,
  MarkdownString
} from "vscode";
import { DBTProjectContainer } from "../manifest/dbtProjectContainer";
import { TelemetryService } from "../telemetry";
import { MacroMetaType } from "../domain";
import { generateMacroHoverMarkdown } from "./utils";

export class MacroHoverProvider implements HoverProvider {
  constructor(
    private dbtProjectContainer: DBTProjectContainer,
    private telemetry: TelemetryService
  ) {}

  provideHover(
    document: TextDocument,
    position: Position,
    token: CancellationToken
  ): ProviderResult<Hover> {
    const hoverText = document.getText(document.getWordRangeAtPosition(position));
    
    const macroNameMatch = hoverText.match(/(\w+)\(/);
    if (!macroNameMatch) {
      return null;
    }

    const macroName = macroNameMatch[1];
    const project = this.dbtProjectContainer.findDBTProject(document.uri);
    if (!project) {
      return null;  
    }

    const macroMeta = project.macroMetaMap.get(macroName);
    if (!macroMeta) {
      return null;
    }

    const hoverContent = this.generateMacroHoverContent(macroMeta);
    return new Hover(hoverContent);
  }

  private generateMacroHoverContent(macroMeta: MacroMetaType): MarkdownString {
    return generateMacroHoverMarkdown(macroMeta);
  }
}

src/hover_provider/index.ts

Register the `MacroHoverProvider` in the `HoverProviders` class.
--- 
+++ 
@@ -2,4 +2,5 @@
 import { DBTPowerUserExtension } from "../dbtPowerUserExtension";
 import { provideSingleton } from "../utils";
 import { ModelHoverProvider } from "./modelHoverProvider";
-import { SourceHoverProvider } from "./sourceHoverProvider";
+import { SourceHoverProvider } from "./sourceHoverProvider";
+import { MacroHoverProvider } from "./macroHoverProvider";

src/hover_provider/utils.ts

Add a new `generateMacroHoverMarkdown` function to the end of the `src/hover_provider/utils.ts` file.
import { MarkdownString } from "vscode";
import { NodeMetaType, SourceMetaType } from "../domain";

export function generateHoverMarkdownString(
  node: NodeMetaType | SourceMetaType,
  nodeType: string,
): MarkdownString {
  const content = new MarkdownString();
  content.supportHtml = true;
  content.isTrusted = true;
  content.appendMarkdown(
    `<span style="color:#347890;">(${nodeType})&nbsp;</span><span><strong>${node.name}</strong></span>`,
  );
  if (node.description !== "") {
    content.appendMarkdown(`</br><span>${node.description}</span>`);
  }
  content.appendText("\n");
  content.appendText("\n");
  content.appendMarkdown("---");
  content.appendText("\n");
  content.appendText("\n");
  for (const colKey in node.columns) {
    const column = node.columns[colKey];
    content.appendMarkdown(
      `<span style="color:#347890;">(column)&nbsp;</span><span>${column.name} &nbsp;</span>`,
    );
    if (column.data_type !== null) {
      content.appendMarkdown(
        `<span>-&nbsp;${column.data_type.toLowerCase()}</span>`,
      );
    }
    if (column.description !== "") {
      content.appendMarkdown(
        `<br/><span><em>${column.description}</em></span>`,
      );
    }
    content.appendMarkdown("</br>");
  }
  return content;
}

import { MacroMetaType } from "../domain";

export function generateMacroHoverMarkdown(macro: MacroMetaType): MarkdownString {
  const content = new MarkdownString();
  content.appendMarkdown(`## ${macro.name}`);
  
  if (macro.description) {
    content.appendMarkdown(`\n\n${macro.description}`);
  }

  content.appendMarkdown("\n\n### Arguments\n\n");
  
  for (const arg of macro.arguments) {
    content.appendMarkdown(`- \`${arg}\`\n`);
  }

  return content;
}

Step 3: 🔄️ Validating

Your changes have been successfully made to the branch sweep/hovering_on_a_macro_should_show_more_det. I have validated these changes using a syntax checker and a linter.


Tip

To recreate the pull request, edit the issue title or description.

This is an automated message generated by Sweep AI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request sweep
Projects
None yet
Development

No branches or pull requests

2 participants