Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add concept of block events and some design notes #1059

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented Mar 4, 2025

This PR adds the concept of "events" that can be registered on block types; essentially, server-side functions that can be triggered separately from parsing/ingestion for things like setting parameters. It also introduces the handling of particular events raised by Bokeh plots in the base data block in Vue.

This is all described in a new design document for this current version of datablocks, also committed in this PR.

I think eventually we can add several lifecycle hooks to the base block for handling certain things (remake block from scratch etc) that can be reused by derived block types, though I haven't put any effort into doing this now. I'm already using this functionality in a custom block that sets some parameters based on user feedback that requires reanalysing all block data (essentially a slider setting some offset time parameter).

For review, I recommend reading the design doc in the rtd preview at https://the-datalab--1059.org.readthedocs.build/en/1059/design/blocks/.

There's still a few open questions about how errors in events should be handled, and what methods should be triggered afterwards (mostly due to the fact we always call the to_web() block method in the update-block route.

Copy link

codecov bot commented Mar 4, 2025

Codecov Report

Attention: Patch coverage is 90.90909% with 6 lines in your changes missing coverage. Please review.

Project coverage is 70.42%. Comparing base (da2b63f) to head (f7fc272).

Files with missing lines Patch % Lines
pydatalab/src/pydatalab/routes/v0_1/blocks.py 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1059      +/-   ##
==========================================
+ Coverage   70.14%   70.42%   +0.27%     
==========================================
  Files          63       63              
  Lines        4107     4166      +59     
==========================================
+ Hits         2881     2934      +53     
- Misses       1226     1232       +6     
Files with missing lines Coverage Δ
pydatalab/src/pydatalab/apps/xrd/blocks.py 67.64% <100.00%> (+2.75%) ⬆️
pydatalab/src/pydatalab/blocks/base.py 79.25% <100.00%> (+10.37%) ⬆️
pydatalab/src/pydatalab/routes/v0_1/blocks.py 42.26% <0.00%> (-2.79%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

cypress bot commented Mar 4, 2025

datalab    Run #2996

Run Properties:  status check passed Passed #2996  •  git commit be2a87110d ℹ️: Merge f7fc27209685f311c2fb7a5f9a36ae47068c83c5 into da2b63f230916093bc4be911664f...
Project datalab
Branch Review ml-evs/blocks
Run status status check passed Passed #2996
Run duration 07m 10s
Commit git commit be2a87110d ℹ️: Merge f7fc27209685f311c2fb7a5f9a36ae47068c83c5 into da2b63f230916093bc4be911664f...
Committer Matthew Evans
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 471
View all changes introduced in this branch ↗︎

@ml-evs ml-evs marked this pull request as ready for review March 10, 2025 22:06
@ml-evs ml-evs force-pushed the ml-evs/blocks branch 2 times, most recently from 2f892ca to f765a97 Compare March 19, 2025 18:56
@ml-evs
Copy link
Member Author

ml-evs commented Mar 19, 2025

Just pinging @BenjaminCharmes and @jdbocarsly for review on this one!

Copy link
Contributor

@BenjaminCharmes BenjaminCharmes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 very promising!
XRD Block works perfectly fine.

@@ -27,9 +27,19 @@ class XRDBlock(DataBlock):
def plot_functions(self):
return (self.generate_xrd_plot,)

@classmethod
@event
def set_wavelength(self, wavelength: float | None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you show how this gets implemented in the XRD block before we merge? That would be a good way to test that everything is working on the front end

@@ -207,6 +207,10 @@ export default {
this.contentMaxHeight = "none";
}
});
document.addEventListener("bokehStateUpdate", this.handleBokehEvent);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this adds a listener to the overall document, will we have multiple listeners if we have multiple blocks on a page? This could cause the handler to be called multiple times

you should be able to add this listener to a only the relevant node by assigning the outer div ref="thisBlock", then only add the listener to:
this.$refs.thisBlock.addEventListener("bokehStateUpdate", this.handleBokehEvent);

@ml-evs
Copy link
Member Author

ml-evs commented Mar 21, 2025

Made some fixes post-discussion with @jdbocarsly. Known limitations still:

  • The event handler currently triggers per block on the page, with the wrong data. We need to find a way to scope the handler, and then dispatch it correctly to the right block. Ideally this would be encapsulated in a function so that we don't have too much fiddly boilerplate in the Bokeh callbacks.
  • We should probably rename bokehStateUpdate
  • The whole block currently updates; it would be nice to be able to also do partial updates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants