Deferred dependencies import into run_server

[jitu5]

Description

Resolves kedro-org/vscode-kedro#72

#2049 To get current Kedro project JSON data required by Kedro-Viz React component when running inside VSCode, We introduce get_kedro_project_json_data function in responses.py which returns current Kedro project JSON data. But before get_kedro_project_json_data we need to load and populate data with load_and_populate_data method from server.py

To avoid ImportError when importing load_and_populate_data from server.py in VSCode extension, Deferred dependencies like fsspec, uvicorn, multiprocessing and watchgod inside Run_server.

Development notes

• Moved dependencies like fsspec, uvicorn, multiprocessing and watchgod inside Run_server.

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added new entries to the RELEASE.md file
• Added tests to cover my changes

[noklam]

Is it possible to extract load and populate data instead? That doesn't sound like something belong to server.py to start with.

[jitu5]

Is it possible to extract load and populate data instead? That doesn't sound like something belong to server.py to start with.

@noklam I started with that moved populate_data and load_and_populate_data to separate file load_data.py but @ravi-kumar-pilla suggested to use deferred dependencies approach similar to what we did here #1920 (comment)

[ravi-kumar-pilla]

Is it possible to extract load and populate data instead? That doesn't sound like something belong to server.py to start with.

I think we can definitely move load data to a different file.

@noklam I started with that moved populate_data and load_and_populate_data to separate file load_data.py but @ravi-kumar-pilla suggested to use deferred dependencies approach similar to what we did here #1920 (comment)

I might be missing some context here. I was in an impression that you created the file just to resolve the import errors. So suggested to move the imports within the function. We can do the refactoring as well.

[noklam]

I started with that moved populate_data and load_and_populate_data to separate file load_data.p

What would that look likes?

@ravi-kumar-pilla suggested to use deferred dependencies approach similar to what we did here #1920 (comment)

What's the reason for that? I am fine with the deferred approach but I think load_data, load_and_populate_data does not belongs to server.py. There are some rooms for de-coupling as well, as

kedro-viz/package/kedro_viz/integrations/kedro/data_loader.py

Line 72 in 084d3e7

def load_data(

, include_hook also doesn't affect the data at all. This bundle the data for flowchart and the application itself (i.e. dataset preview).

[jitu5]

Is it possible to extract load and populate data instead? That doesn't sound like something belong to server.py to start with.

I think we can definitely move load data to a different file.

@noklam I started with that moved populate_data and load_and_populate_data to separate file load_data.py but @ravi-kumar-pilla suggested to use deferred dependencies approach similar to what we did here #1920 (comment)

I might be missing some context here. I was in an impression that you created the file just to resolve the import errors. So suggested to move the imports within the function. We can do the refactoring as well.

Yes, I created the file just to resolve the import errors.

[jitu5]

I started with that moved populate_data and load_and_populate_data to separate file load_data.p

What would that look likes?

80d35b6#diff-94c1c8a57783f10e120cb066fe8582a44e256679802723204e8b5b4d7fd8d6b4

[ravi-kumar-pilla]

What's the reason for that? I am fine with the deferred approach but I think load_data, load_and_populate_data does not belongs to server.py. There are some rooms for de-coupling as well, as

kedro-viz/package/kedro_viz/integrations/kedro/data_loader.py

Line 72 in 084d3e7

def load_data(

I agree with you, there is definitely a room to refactor the server.py file and keep it only to run server. We do have a refactoring ticket here - #2061 and we can consider some refactor of server.py in this.

, include_hook also doesn't affect the data at all. This bundle the data for flowchart and the application itself (i.e. dataset preview).

include_hook is used to create a NullPluginManager when we do KedroSession._hook_manager. Again, we can decouple and create more pure python functions.

We can go ahead with deferred imports for now if it makes sense and you can add suggestion to #2061 or we can create a different ticket to refactor these as it might involve file changes in tests as well and other references and keep PRs small. Let me know what do you think

Thank you

[noklam]

Perfect, since there is a refactor ticket going on let's do the proper refactoring there. For now keeping this import deferred is sufficient.

[noklam]

As discussed in the comment, let's do this as deferred import and address the refactoring in a separate PR.