Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.2.1 (official release) #121

Merged
merged 15 commits into from
Sep 23, 2024
Merged
13 changes: 10 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
# Change Log

# 0.2.1
- Added a new command `Kedro: Show logs` for troubleshooting.

# 0.2.0
# 0.2.0 (pre-release)
## Major changes and new features
- Integrated Kedro-Viz flowchart into the extension with a new command `Kedro: Run Kedro Viz`.
- Modified the extension to search pipelines from all `<package_name>/pipelines` folder.

- Added new command`Kedro
## Bug fix
- `Select Environment` actions now update the run environment instead of base.
- Modified status bar to show both environments, i.e.`base + local`, and `base` is no longer selectable.
- Fixed a minor bug where status bar showing incorrect text, i.e. it shows `prod` insteadf of `base + prod`.
- Fixed a bug where "Find reference" return too many matches for catalog dataset, it returns only exact matches now.
- Fixed a bug where `Find reference` return too many matches for catalog dataset, it returns only exact matches now.
- Fixed a bug where namespace dataset navigation is not working properly
- Fixed a bug where navigating on nested parameters should go to the top level key.
- Modify the extension to search pipelines from all <package_name>/pipelines folder.
Expand Down
116 changes: 74 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,25 @@ The extension integrates [Kedro](https://github.com/kedro-org/kedro) projects wi

If you encounter issue, report it in [Github](https://github.com/kedro-org/vscode-kedro/issues) or [Slack](https://slack.kedro.org), we will try to fix ASAP.

- [vscode-kedro](#vscode-kedro)
- [Requirements](#requirements)
- [How to use this extension](#how-to-use-this-extension)
- [Feature](#feature)
- [Go to Definition from pipeline.py to configuration files](#go-to-definition-from-pipelinepy-to-configuration-files)
- [Go to Reference from configuration files to pipeline.py](#go-to-reference-from-configuration-files-to-pipelinepy)
- [Autocompletion in Python](#autocompletion-in-python)
- [Schema Validation](#schema-validation)
- [Hover](#hover)
- [To navigate between the flowchart and the code editor in Kedro-Viz:](#to-navigate-between-the-flowchart-and-the-code-editor-in-kedro-viz)
- [Settings](#settings)
- [Change Configuration Environment](#change-configuration-environment)
- [How to restart a server if there are error](#how-to-restart-a-server-if-there-are-error)
- [Assumptions](#assumptions)
- [Single Kedro Project](#single-kedro-project)
- [Configure Kedro Environment](#configure-kedro-environment)
- [Pipeline Discovery](#pipeline-discovery)
- [Visualisation with Kedro-Viz](#visualisation-with-kedro-viz)
- [Troubleshooting](#troubleshooting)

## Requirements
1. VS Code 1.64.0 or greater
Expand All @@ -12,11 +31,58 @@ If you encounter issue, report it in [Github](https://github.com/kedro-org/vscod
## How to use this extension
1. Install `Kedro` from the extension
2. Select the correct Python interpreter that you use to run the Kedro project with the `> Python: select interpreter` command
3. Open your VSCode workspace at the root of the Kedro project.

p.s. If you can `kedro run` with the environment, you are good to go.

The extension requires `bootstrap_project` in Kedro, you need to make sure you can do `kedro run` without getting any immediate error, otherwise you may get a server panic error.



# Feature
## Go to Definition from pipeline.py to configuration files
Use `Cmd` (Mac)/ `Ctrl` (Window) + `Click` or `F12` to trigger `Go to Definition`
![go to definition](assets/lsp-go-to-definition.gif)

## Go to Reference from configuration files to pipeline.py
- `Cmd` or `Ctrl` (Window) + `Click` on the definition.
- Use `Find Reference`
- Use the shortcut `Shift` + `F12`
![find reference](assets/lsp-find-reference.gif)

**Note:** You can find pipeline reference in all the python files under `<package_name>/pipelines`
```
- pipelines
- sub_pipeline
- pipeline_data_processing.py
- sub_pipeline_1
- pipeline_data_processing_1.py
```

## Autocompletion in Python
Type `"` in any `pipeline.py` and it should trigger the autocompletion list.
![autocompletion](assets/lsp-autocompletion.gif)

## Schema Validation
![schema validation](assets/lsp-schema-validation.gif)

## Hover
Just hover your mouse over any `params:`, datasets or hit the command `Show or Focus Hover`
![hover](assets/lsp-hover.gif)

## To navigate between the flowchart and the code editor in Kedro-Viz:

Navigate to Node Functions:
Click on a node in the Kedro-Viz flowchart, and it will automatically navigate to the corresponding node function in your code.
![navigation to node function](assets/viz-vsc-nav-function-node.gif)


Navigate to DataCatalog:
Clicking on a data node in the flowchart will open the corresponding dataset in the Data Catalog.
![navigation to dataset](assets/viz-vsc-nav-data-node.gif)



## Settings
### Change Configuration Environment
By default, the extension references the configuration loader's base_env (typically `base`). To change the directory where the extension looks for configurations, the extension provides 3 different ways to do this:
Expand All @@ -33,7 +99,10 @@ Click `Output` and select `Kedro` from the dropdown list. It may gives you some
Hit `Cmd` + `Shift` + `P` to open the VSCode command, look for `kedro: restart server` in case it's panic.

## Assumptions
### Configuration Source
### Single Kedro Project
The extension need to identify where is the Kedro project. It assumes the root of the workspace is a Kedro project, i.e. open the project where the `pyproject.toml` is.

### Configure Kedro Environment
Currently, the extension assume the source of configuration is in the `base_env` defined by the config loader (if you didn't speficy, [usually it is `conf/base`](https://docs.kedro.org/en/stable/configuration/configuration_basics.html#configuration-loading)).

This mean that if the configuration is overrided by the `default_run_env`(usually it is `local`), the extension may fails to resolve to the correct location.
Expand Down Expand Up @@ -68,45 +137,8 @@ This will launch Kedro-Viz and display your pipeline visually within the extensi

**Note:** To update the Kedro-Viz flowchart after making any changes to your Kedro project, please hit `Cmd` + `Shift` + `P` to open the VSCode command and look for `kedro: restart server`.

## Troubleshooting
In VSCode, you can check the `Output` tab next to your terminal and select `Kedro` to read the logs.

# Feature
## Go to Definition from pipeline.py to configuration files
Use `Cmd` (Mac)/ `Ctrl` (Window) + `Click` or `F12` to trigger `Go to Definition`
![go to definition](assets/lsp-go-to-definition.gif)

## Go to Reference from configuration files to pipeline.py
- `Cmd` or `Ctrl` (Window) + `Click` on the definition.
- Use `Find Reference`
- Use the shortcut `Shift` + `F12`
![find reference](assets/lsp-find-reference.gif)

**Note:** You can find pipeline reference in all the python files under `<package_name>/pipelines`
```
- pipelines
- sub_pipeline
- pipeline_data_processing.py
- sub_pipeline_1
- pipeline_data_processing_1.py
```

## Autocompletion in Python
Type `"` in any `pipeline.py` and it should trigger the autocompletion list.
![autocompletion](assets/lsp-autocompletion.gif)

## Schema Validation
![schema validation](assets/lsp-schema-validation.gif)

## Hover
Just hover your mouse over any `params:`, datasets or hit the command `Show or Focus Hover`
![hover](assets/lsp-hover.gif)

## To navigate between the flowchart and the code editor in Kedro-Viz:

Navigate to Node Functions:
Click on a node in the Kedro-Viz flowchart, and it will automatically navigate to the corresponding node function in your code.
![navigation to node function](assets/viz-vsc-nav-function-node.gif)


Navigate to DataCatalog:
Clicking on a data node in the flowchart will open the corresponding dataset in the Data Catalog.
![navigation to dataset](assets/viz-vsc-nav-data-node.gif)
You can either run the `Kedro: Show logs` command or click `Output` and select `Kedro` from the dropdown list. It may gives you some hints and report back if you think this is a bug.
![VSCode Output log](assets/troubleshooting.png)
Binary file added assets/troubleshooting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion bundled/tool/lsp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,8 @@ def get_project_data_from_viz(lsClient):
data = get_kedro_project_json_data()
return data
except Exception as e:
print(f"An error occurred: {e}")
print(f"Kedro-Viz: {e}")
log_error(f"Kedro-Viz: {e}")
finally:
print("Execution completed.")
return data
Expand Down
5 changes: 5 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,11 @@
"command": "kedro.runKedroViz",
"category": "kedro",
"title": "Run Kedro Viz"
},
{
"command": "kedro.showOutputChannel",
"category": "kedro",
"title": "Show logs"
}
]
},
Expand Down
2 changes: 1 addition & 1 deletion src/common/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ interface HeapData {
identity?: string;
}

const HEAP_APPID_PROD = '4039408868'; // todo: Dev server, change it back to prod
const HEAP_APPID_PROD = '2388822444';
const HEAP_ENDPOINT = 'https://heapanalytics.com/api/track';
const HEAP_HEADERS = { 'Content-Type': 'application/json' };

Expand Down
4 changes: 4 additions & 0 deletions src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ export async function activate(context: vscode.ExtensionContext): Promise<void>
const CMD_SELECT_ENV = `${serverId}.selectEnvironment`;
const CMD_RUN_KEDRO_VIZ = `${serverId}.runKedroViz`;
const CMD_DEFINITION_REQUEST = `${serverId}.sendDefinitionRequest`;
const CMD_SHOW_OUTPUT_CHANNEL = `${serverId}.showOutputChannel`;

// Status Bar
const statusBarItem = await createStatusBar(CMD_SELECT_ENV, serverId);
Expand Down Expand Up @@ -149,6 +150,9 @@ export async function activate(context: vscode.ExtensionContext): Promise<void>
registerCommand(CMD_RUN_KEDRO_VIZ, async () => {
await handleKedroViz(context, lsClient);
}),
registerCommand(CMD_SHOW_OUTPUT_CHANNEL, () => {
outputChannel.show();
}),
);

setImmediate(async () => {
Expand Down
2 changes: 1 addition & 1 deletion src/webview/createOrShowKedroVizPanel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export async function handleKedroViz(
} else {
const header = 'Kedro-Viz Dependency Required';
const options: vscode.MessageOptions = {
detail: 'Kedro-Viz version 10.0.0 or later is required to visualize your project\'s data pipeline. It’s not installed in your virtual environment. Click "Install" to set it up.',
detail: 'Kedro-Viz version 10.0.0 or later is required to visualize your project\'s data pipeline. It’s not installed in your virtual environment. Click "Install" to set it up with pip.',
modal: true,
};

Expand Down
3 changes: 3 additions & 0 deletions src/webview/vizWebView.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ export default class KedroVizPanel {
await goToDefinition(message.node);
}
return;
case 'showOutput':
await vscode.commands.executeCommand('kedro.showOutputChannel');
return;
}
},
null,
Expand Down
45 changes: 37 additions & 8 deletions webview/src/App.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const vscodeApi = window.acquireVsCodeApi();

function App() {
const [data, setData] = React.useState({ nodes: [], edges: [] });
const [error, setError] = React.useState(false);
const [loading, setLoading] = React.useState(true);

useEffect(() => {
Expand All @@ -14,8 +15,13 @@ function App() {
const message = event.data;
switch (message.command) {
case "updateData":
setData(message.data);
setLoading(false);
if (message.data) {
setData(message.data);
setLoading(false);
} else {
setLoading(true);
setError(true);
}
break;
default:
break;
Expand All @@ -40,6 +46,33 @@ function App() {
}
};

const handleOutputClick = () => {
vscodeApi.postMessage({
command: "showOutput",
showOutput: true,
});

};

const showMessages = () => {
if (error) {
return (
<div style={{ display: "flex", justifyContent: "center", alignItems: "center", height: `100vh` }}>
<h2 style={{ textAlign: "center" }}>
{"Error: couldn't display Kedro Viz, check "}
<span style={{ textDecoration: "underline", cursor: "pointer", color: "#00bcff" }} onClick={handleOutputClick}> output</span>
{" logs for more information."}
</h2>
</div>
);
}
return (
<div style={{ display: "flex", justifyContent: "center", alignItems: "center", height: `100vh` }}>
<h2 style={{ textAlign: "center" }}>{'Loading Kedro Viz...'}</h2>
</div>
);
};

const handleActionCallback = (action) => {
if (action) {
switch (action.type) {
Expand All @@ -50,15 +83,11 @@ function App() {
break;
}
}
}
};

return (
<div style={{ height: `90vh`, width: `100%` }}>
{loading ? (
<div style={{ display: "flex", justifyContent: "center", alignItems: "center", height: `100vh` }}>
<h2>Loading Kedro Viz...</h2>
</div>
) : (<KedroViz
{loading ? showMessages() : (<KedroViz
data={data}
onActionCallback={handleActionCallback}
options={{
Expand Down