Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor visualize dataset stats from DataNodeMetadata to DataNode #1499

Merged
merged 6 commits into from
Aug 24, 2023
Merged

Refactor visualize dataset stats from DataNodeMetadata to DataNode #1499

merged 6 commits into from
Aug 24, 2023

Conversation

ravi-kumar-pilla
Copy link
Contributor

@ravi-kumar-pilla ravi-kumar-pilla commented Aug 22, 2023 •
edited
Loading

Description

Resolves #1482

Development notes

  1. Remove stats field from DataNodeMetadata class
  2. Update the DataNodeMetadata call from router
  3. Add the stats field inside the data access manager (add_datasets method)
  4. Access the stats field via the DataNode object in DataNodeMetadata class
  5. Update the DataNodeAPI response to return stats field
  6. Modify the unit tests accordingly

QA notes

  1. Checkout the branch chore/refactor-stats
  2. In the metadata panel, find the Dataset Statistics row displaying stats for DataNode (no change from previous release)
  3. Open developer tools and find the /main network call response. The node object contains stats field

Note: This field is currently not used in the flowchart, but this ticket helps in getting closer to our goal of displaying stats in a debug view

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes

@ravi-kumar-pilla ravi-kumar-pilla marked this pull request as ready for review August 22, 2023 20:12
rashidakanchwala
rashidakanchwala approved these changes Aug 23, 2023
View reviewed changes
Copy link
Contributor

@rashidakanchwala rashidakanchwala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. thanks Ravi!!

@ravi-kumar-pilla ravi-kumar-pilla merged commit bc46f43 into main Aug 24, 2023
@ravi-kumar-pilla ravi-kumar-pilla deleted the chore/refactor-stats branch August 24, 2023 21:20
vladimir-mck pushed a commit that referenced this pull request Aug 29, 2023
@ravi-kumar-pilla @vladimir-mck
Refactor visualize dataset stats from DataNodeMetadata to DataNode (#…
fc865d1 
…1499)

* add stats to data node

* lint and format check fix

* fix pytests

* fix layout issue

* fix transcoded data stats

Signed-off-by: Vladimir <vladimir_nikolic@external.mckinsey.com>
vladimir-mck pushed a commit that referenced this pull request Aug 29, 2023
@ravi-kumar-pilla @vladimir-mck
Refactor visualize dataset stats from DataNodeMetadata to DataNode (#…
8140dca 
…1499)

* add stats to data node

* lint and format check fix

* fix pytests

* fix layout issue

* fix transcoded data stats

Signed-off-by: Vladimir <vladimir_nikolic@external.mckinsey.com>
vladimir-mck pushed a commit that referenced this pull request Aug 30, 2023
@ravi-kumar-pilla @vladimir-mck
Refactor visualize dataset stats from DataNodeMetadata to DataNode (#…
8a6e4ca 
…1499)

* add stats to data node

* lint and format check fix

* fix pytests

* fix layout issue

* fix transcoded data stats

Signed-off-by: Vladimir <vladimir_nikolic@external.mckinsey.com>
ravi-kumar-pilla added a commit that referenced this pull request Aug 31, 2023
@ravi-kumar-pilla
Refactor visualize dataset stats from DataNodeMetadata to DataNode (#…
79f05fd 
…1499)

* add stats to data node

* lint and format check fix

* fix pytests

* fix layout issue

* fix transcoded data stats

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
ravi-kumar-pilla added a commit that referenced this pull request Sep 11, 2023
@ravi-kumar-pilla @rashidakanchwala
@noklam @jitu5
Fix for Kedro Viz Connection Error (#1507)
335be7d 
* initial draft for resolving connection error

* refactor launchers and test code

* modify unit tests

* fix lint errors

* fix run_viz tests

* update unit test for coverage

* update unit tests

* Refactor visualize dataset stats from DataNodeMetadata to DataNode (#1499)

* add stats to data node

* lint and format check fix

* fix pytests

* fix layout issue

* fix transcoded data stats

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* initial draft for resolving connection error

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* Support for Python 3.11 (#1502)

* initial draft for python 3.11 support

* update release doc

* add python warnings for e2e tests

* modify e2e test

* modify e2e test

* test by removing lower req scenario

* skip e2e tests for lower bound requirement on python 3.11

* skip e2e tests for lower bound requirement on python 3.11

* remove print statements

---------

Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* Remove Python Upper Bound Requirements  (#1506)

* initial draft for python 3.11 support

* update release doc

* add python warnings for e2e tests

* modify e2e test

* modify e2e test

* test by removing lower req scenario

* skip e2e tests for lower bound requirement on python 3.11

* skip e2e tests for lower bound requirement on python 3.11

* remove python upperbounds initial draft

* fix lint and format errors

* test remove upperbound warning

* test lowerbound pandas install

* revert back pandas requirement

* bump lower requirements for pandas

* remove upper bound clean up

* update release notes

* fix PR comments

---------

Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* refactor launchers and test code

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* modify unit tests

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* fix lint errors

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* Fix: Adding favicon to Kedro Demo (#1509)

* Fix: Adding favicon to Kedro Demo

* Fix: Change in approach for serving favicon

* Lint error fix

* Lint error fix

* Favicon endpoint test added

* Favicon endpoint test added

* Lint error fixed

* Fix: Adding favicon to Kedro Demo

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Fix: Change in approach for serving favicon

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Lint error fix

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Lint error fix

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Favicon endpoint test added

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Favicon endpoint test added

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Lint error fixed

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Fixed favicon endpoint test

* Release doc updated

* Update RELEASE.md

Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>

* Removed pytest.fixture as per review comment

---------

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>
Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* fix run_viz tests

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* update unit test for coverage

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* Release v6.5.0 (#1513)

* v6.5.0

* release

* update-reminder-content

* update reminder

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* remove branch condition for automate release version check (#1514)

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* update unit tests

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* add release record

* modify comment

* fix PR comments

* DCO fix

* fixing dco

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

* update pytest

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>

---------

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>
Co-authored-by: Rashida Kanchwala <rashida.kanchwala@quantumblack.com>
Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com>
Co-authored-by: Jitendra Gundaniya <38945204+jitu5@users.noreply.github.com>
Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rashidakanchwala rashidakanchwala rashidakanchwala approved these changes

@NeroOkwa NeroOkwa NeroOkwa approved these changes

@tynandebold tynandebold Awaiting requested review from tynandebold

@jitu5 jitu5 Awaiting requested review from jitu5 jitu5 is a code owner

@vladimir-mck vladimir-mck Awaiting requested review from vladimir-mck

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Refactor visualize dataset stats from DataNodeMetadata to DataNode
3 participants
@ravi-kumar-pilla @rashidakanchwala @NeroOkwa