Skip to content

Commit

Permalink
Release 0.2.1 (official release) (#121)
Browse files Browse the repository at this point in the history
* Expand pipeline search folders

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* fix readme

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Update HEAP ID

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* update release note

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Modify the message with pip

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Server log and error messaged included

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* output link added

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* update release note and bump to 0.2.1

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Update CHANGELOG.md

Co-authored-by: Jitendra Gundaniya <38945204+jitu5@users.noreply.github.com>

* add public command

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* add command

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* update readme

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>
Co-authored-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>
Co-authored-by: Jitendra Gundaniya <38945204+jitu5@users.noreply.github.com>
  • Loading branch information
3 people authored Sep 23, 2024
1 parent 8ed1f30 commit b93c77c
Show file tree
Hide file tree
Showing 10 changed files with 137 additions and 56 deletions.
13 changes: 10 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
# Change Log

# 0.2.1
- Added a new command `Kedro: Show logs` for troubleshooting.

# 0.2.0
# 0.2.0 (pre-release)
## Major changes and new features
- Integrated Kedro-Viz flowchart into the extension with a new command `Kedro: Run Kedro Viz`.
- Modified the extension to search pipelines from all `<package_name>/pipelines` folder.

- Added new command`Kedro
## Bug fix
- `Select Environment` actions now update the run environment instead of base.
- Modified status bar to show both environments, i.e.`base + local`, and `base` is no longer selectable.
- Fixed a minor bug where status bar showing incorrect text, i.e. it shows `prod` insteadf of `base + prod`.
- Fixed a bug where "Find reference" return too many matches for catalog dataset, it returns only exact matches now.
- Fixed a bug where `Find reference` return too many matches for catalog dataset, it returns only exact matches now.
- Fixed a bug where namespace dataset navigation is not working properly
- Fixed a bug where navigating on nested parameters should go to the top level key.
- Modify the extension to search pipelines from all <package_name>/pipelines folder.
Expand Down
116 changes: 74 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,25 @@ The extension integrates [Kedro](https://github.com/kedro-org/kedro) projects wi

If you encounter issue, report it in [Github](https://github.com/kedro-org/vscode-kedro/issues) or [Slack](https://slack.kedro.org), we will try to fix ASAP.

- [vscode-kedro](#vscode-kedro)
- [Requirements](#requirements)
- [How to use this extension](#how-to-use-this-extension)
- [Feature](#feature)
- [Go to Definition from pipeline.py to configuration files](#go-to-definition-from-pipelinepy-to-configuration-files)
- [Go to Reference from configuration files to pipeline.py](#go-to-reference-from-configuration-files-to-pipelinepy)
- [Autocompletion in Python](#autocompletion-in-python)
- [Schema Validation](#schema-validation)
- [Hover](#hover)
- [To navigate between the flowchart and the code editor in Kedro-Viz:](#to-navigate-between-the-flowchart-and-the-code-editor-in-kedro-viz)
- [Settings](#settings)
- [Change Configuration Environment](#change-configuration-environment)
- [How to restart a server if there are error](#how-to-restart-a-server-if-there-are-error)
- [Assumptions](#assumptions)
- [Single Kedro Project](#single-kedro-project)
- [Configure Kedro Environment](#configure-kedro-environment)
- [Pipeline Discovery](#pipeline-discovery)
- [Visualisation with Kedro-Viz](#visualisation-with-kedro-viz)
- [Troubleshooting](#troubleshooting)

## Requirements
1. VS Code 1.64.0 or greater
Expand All @@ -12,11 +31,58 @@ If you encounter issue, report it in [Github](https://github.com/kedro-org/vscod
## How to use this extension
1. Install `Kedro` from the extension
2. Select the correct Python interpreter that you use to run the Kedro project with the `> Python: select interpreter` command
3. Open your VSCode workspace at the root of the Kedro project.

p.s. If you can `kedro run` with the environment, you are good to go.

The extension requires `bootstrap_project` in Kedro, you need to make sure you can do `kedro run` without getting any immediate error, otherwise you may get a server panic error.



# Feature
## Go to Definition from pipeline.py to configuration files
Use `Cmd` (Mac)/ `Ctrl` (Window) + `Click` or `F12` to trigger `Go to Definition`
![go to definition](assets/lsp-go-to-definition.gif)

## Go to Reference from configuration files to pipeline.py
- `Cmd` or `Ctrl` (Window) + `Click` on the definition.
- Use `Find Reference`
- Use the shortcut `Shift` + `F12`
![find reference](assets/lsp-find-reference.gif)

**Note:** You can find pipeline reference in all the python files under `<package_name>/pipelines`
```
- pipelines
- sub_pipeline
- pipeline_data_processing.py
- sub_pipeline_1
- pipeline_data_processing_1.py
```

## Autocompletion in Python
Type `"` in any `pipeline.py` and it should trigger the autocompletion list.
![autocompletion](assets/lsp-autocompletion.gif)

## Schema Validation
![schema validation](assets/lsp-schema-validation.gif)

## Hover
Just hover your mouse over any `params:`, datasets or hit the command `Show or Focus Hover`
![hover](assets/lsp-hover.gif)

## To navigate between the flowchart and the code editor in Kedro-Viz:

Navigate to Node Functions:
Click on a node in the Kedro-Viz flowchart, and it will automatically navigate to the corresponding node function in your code.
![navigation to node function](assets/viz-vsc-nav-function-node.gif)


Navigate to DataCatalog:
Clicking on a data node in the flowchart will open the corresponding dataset in the Data Catalog.
![navigation to dataset](assets/viz-vsc-nav-data-node.gif)



## Settings
### Change Configuration Environment
By default, the extension references the configuration loader's base_env (typically `base`). To change the directory where the extension looks for configurations, the extension provides 3 different ways to do this:
Expand All @@ -33,7 +99,10 @@ Click `Output` and select `Kedro` from the dropdown list. It may gives you some
Hit `Cmd` + `Shift` + `P` to open the VSCode command, look for `kedro: restart server` in case it's panic.

## Assumptions
### Configuration Source
### Single Kedro Project
The extension need to identify where is the Kedro project. It assumes the root of the workspace is a Kedro project, i.e. open the project where the `pyproject.toml` is.

### Configure Kedro Environment
Currently, the extension assume the source of configuration is in the `base_env` defined by the config loader (if you didn't speficy, [usually it is `conf/base`](https://docs.kedro.org/en/stable/configuration/configuration_basics.html#configuration-loading)).

This mean that if the configuration is overrided by the `default_run_env`(usually it is `local`), the extension may fails to resolve to the correct location.
Expand Down Expand Up @@ -68,45 +137,8 @@ This will launch Kedro-Viz and display your pipeline visually within the extensi

**Note:** To update the Kedro-Viz flowchart after making any changes to your Kedro project, please hit `Cmd` + `Shift` + `P` to open the VSCode command and look for `kedro: restart server`.

## Troubleshooting
In VSCode, you can check the `Output` tab next to your terminal and select `Kedro` to read the logs.

# Feature
## Go to Definition from pipeline.py to configuration files
Use `Cmd` (Mac)/ `Ctrl` (Window) + `Click` or `F12` to trigger `Go to Definition`
![go to definition](assets/lsp-go-to-definition.gif)

## Go to Reference from configuration files to pipeline.py
- `Cmd` or `Ctrl` (Window) + `Click` on the definition.
- Use `Find Reference`
- Use the shortcut `Shift` + `F12`
![find reference](assets/lsp-find-reference.gif)

**Note:** You can find pipeline reference in all the python files under `<package_name>/pipelines`
```
- pipelines
- sub_pipeline
- pipeline_data_processing.py
- sub_pipeline_1
- pipeline_data_processing_1.py
```

## Autocompletion in Python
Type `"` in any `pipeline.py` and it should trigger the autocompletion list.
![autocompletion](assets/lsp-autocompletion.gif)

## Schema Validation
![schema validation](assets/lsp-schema-validation.gif)

## Hover
Just hover your mouse over any `params:`, datasets or hit the command `Show or Focus Hover`
![hover](assets/lsp-hover.gif)

## To navigate between the flowchart and the code editor in Kedro-Viz:

Navigate to Node Functions:
Click on a node in the Kedro-Viz flowchart, and it will automatically navigate to the corresponding node function in your code.
![navigation to node function](assets/viz-vsc-nav-function-node.gif)


Navigate to DataCatalog:
Clicking on a data node in the flowchart will open the corresponding dataset in the Data Catalog.
![navigation to dataset](assets/viz-vsc-nav-data-node.gif)
You can either run the `Kedro: Show logs` command or click `Output` and select `Kedro` from the dropdown list. It may gives you some hints and report back if you think this is a bug.
![VSCode Output log](assets/troubleshooting.png)
Binary file added assets/troubleshooting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion bundled/tool/lsp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,8 @@ def get_project_data_from_viz(lsClient):
data = get_kedro_project_json_data()
return data
except Exception as e:
print(f"An error occurred: {e}")
print(f"Kedro-Viz: {e}")
log_error(f"Kedro-Viz: {e}")
finally:
print("Execution completed.")
return data
Expand Down
5 changes: 5 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,11 @@
"command": "kedro.runKedroViz",
"category": "kedro",
"title": "Run Kedro Viz"
},
{
"command": "kedro.showOutputChannel",
"category": "kedro",
"title": "Show logs"
}
]
},
Expand Down
2 changes: 1 addition & 1 deletion src/common/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ interface HeapData {
identity?: string;
}

const HEAP_APPID_PROD = '4039408868'; // todo: Dev server, change it back to prod
const HEAP_APPID_PROD = '2388822444';
const HEAP_ENDPOINT = 'https://heapanalytics.com/api/track';
const HEAP_HEADERS = { 'Content-Type': 'application/json' };

Expand Down
4 changes: 4 additions & 0 deletions src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ export async function activate(context: vscode.ExtensionContext): Promise<void>
const CMD_SELECT_ENV = `${serverId}.selectEnvironment`;
const CMD_RUN_KEDRO_VIZ = `${serverId}.runKedroViz`;
const CMD_DEFINITION_REQUEST = `${serverId}.sendDefinitionRequest`;
const CMD_SHOW_OUTPUT_CHANNEL = `${serverId}.showOutputChannel`;

// Status Bar
const statusBarItem = await createStatusBar(CMD_SELECT_ENV, serverId);
Expand Down Expand Up @@ -149,6 +150,9 @@ export async function activate(context: vscode.ExtensionContext): Promise<void>
registerCommand(CMD_RUN_KEDRO_VIZ, async () => {
await handleKedroViz(context, lsClient);
}),
registerCommand(CMD_SHOW_OUTPUT_CHANNEL, () => {
outputChannel.show();
}),
);

setImmediate(async () => {
Expand Down
2 changes: 1 addition & 1 deletion src/webview/createOrShowKedroVizPanel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export async function handleKedroViz(
} else {
const header = 'Kedro-Viz Dependency Required';
const options: vscode.MessageOptions = {
detail: 'Kedro-Viz version 10.0.0 or later is required to visualize your project\'s data pipeline. It’s not installed in your virtual environment. Click "Install" to set it up.',
detail: 'Kedro-Viz version 10.0.0 or later is required to visualize your project\'s data pipeline. It’s not installed in your virtual environment. Click "Install" to set it up with pip.',
modal: true,
};

Expand Down
3 changes: 3 additions & 0 deletions src/webview/vizWebView.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ export default class KedroVizPanel {
await goToDefinition(message.node);
}
return;
case 'showOutput':
await vscode.commands.executeCommand('kedro.showOutputChannel');
return;
}
},
null,
Expand Down
45 changes: 37 additions & 8 deletions webview/src/App.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const vscodeApi = window.acquireVsCodeApi();

function App() {
const [data, setData] = React.useState({ nodes: [], edges: [] });
const [error, setError] = React.useState(false);
const [loading, setLoading] = React.useState(true);

useEffect(() => {
Expand All @@ -14,8 +15,13 @@ function App() {
const message = event.data;
switch (message.command) {
case "updateData":
setData(message.data);
setLoading(false);
if (message.data) {
setData(message.data);
setLoading(false);
} else {
setLoading(true);
setError(true);
}
break;
default:
break;
Expand All @@ -40,6 +46,33 @@ function App() {
}
};

const handleOutputClick = () => {
vscodeApi.postMessage({
command: "showOutput",
showOutput: true,
});

};

const showMessages = () => {
if (error) {
return (
<div style={{ display: "flex", justifyContent: "center", alignItems: "center", height: `100vh` }}>
<h2 style={{ textAlign: "center" }}>
{"Error: couldn't display Kedro Viz, check "}
<span style={{ textDecoration: "underline", cursor: "pointer", color: "#00bcff" }} onClick={handleOutputClick}> output</span>
{" logs for more information."}
</h2>
</div>
);
}
return (
<div style={{ display: "flex", justifyContent: "center", alignItems: "center", height: `100vh` }}>
<h2 style={{ textAlign: "center" }}>{'Loading Kedro Viz...'}</h2>
</div>
);
};

const handleActionCallback = (action) => {
if (action) {
switch (action.type) {
Expand All @@ -50,15 +83,11 @@ function App() {
break;
}
}
}
};

return (
<div style={{ height: `90vh`, width: `100%` }}>
{loading ? (
<div style={{ display: "flex", justifyContent: "center", alignItems: "center", height: `100vh` }}>
<h2>Loading Kedro Viz...</h2>
</div>
) : (<KedroViz
{loading ? showMessages() : (<KedroViz
data={data}
onActionCallback={handleActionCallback}
options={{
Expand Down

0 comments on commit b93c77c

Please sign in to comment.