Trame & PlotLy unwanted events #592

kerim371 · 2024-09-22T00:37:45Z

kerim371
Sep 22, 2024

Hi,

I explore RemoteSelection Trame & PlotLy example.

It works but there is some strange behaviour.

Lets suppose the browser is in the full screen mode.

In PlotLy we then select a number of points and these points appear in VTK (picture 1).
Then we click on the resize button of the browser and selected points disappear from VTK window (picture 2).
Finally we restore the full screen and the selection from PlotLy disappear as well.

Is it possible to fix that?

jourdain · 2024-09-22T16:12:12Z

jourdain
Sep 22, 2024
Maintainer

Yes, it could be a good exercise for you to get familiar with it.

0 replies

kerim371 · 2024-09-24T11:27:29Z

kerim371
Sep 24, 2024
Author

@jourdain thank you for support :)

I continue to work with this example.

Now I'm discovering js event and how to access them via python (if possible).

For example I modified the example by adding a table with dataframe where user can select rows to be displayed.
So now there are 3 ways to show selected points:

by selecting rectangle on PlotLy chart
by selecting rectangle in 3D scene
and by selecting checkboxes in the table

The code provided does work but it is hard to sinchronize selected rows (points) in the table with PlotLy selected points (colored with red).
I do that using state.selection = [itemgetter(*SELECTED_IDX)(state.table_rows)] in PlotLy on_chart_selection which updates table as well.
But now the PlotLy selection doesn't work. Most likely on_chart_selection excessively gets updated with empty selection so the selection_change function gets called twice per selection.

I think the only way to synchronize these ways of point selection is to recognize the events or caller objects. Of course I may mastake but can't find another ways of doing that synchronization.

"""
Version for trame 1.x - https://github.com/Kitware/trame/blob/release-v1/examples/VTK/Applications/RemoteSelection/app.py
Delta v1..v2          - https://github.com/Kitware/trame/commit/03f28bb0084490acabf218264b96a1dbb3a17f19
"""

from operator import itemgetter 
import pandas as pd

# Plotly/chart imports
import plotly.graph_objects as go
import plotly.express as px

# Trame imports
from trame.app import get_server
from trame.assets.remote import HttpFile
from trame.ui.vuetify import SinglePageLayout, SinglePageWithDrawerLayout
from trame.widgets import vuetify, plotly, vtk, trame

# VTK imports
from vtkmodules.vtkIOXML import vtkXMLUnstructuredGridReader
from vtkmodules.numpy_interface import dataset_adapter as dsa
from vtkmodules.vtkCommonDataModel import vtkSelection, vtkSelectionNode, vtkDataObject
from vtkmodules.vtkCommonCore import vtkIdTypeArray
from vtkmodules.vtkFiltersExtraction import vtkExtractSelection
from vtkmodules.vtkFiltersGeometry import vtkGeometryFilter
from vtkmodules.vtkRenderingCore import (
    vtkActor,
    vtkDataSetMapper,
    vtkRenderer,
    vtkRenderWindow,
    vtkRenderWindowInteractor,
    vtkHardwareSelector,
    vtkRenderedAreaPicker,
)

from vtkmodules.vtkInteractionStyle import (
    vtkInteractorStyleRubberBandPick,
    vtkInteractorStyleSwitch,
)  # noqa
import vtkmodules.vtkRenderingOpenGL2  # noqa

from vtkmodules.vtkInteractionStyle import vtkInteractorStyleRubberBandPick

# -----------------------------------------------------------------------------
# Data file information
# -----------------------------------------------------------------------------

dataset_file = HttpFile(
    "./data/disk_out_ref.vtu",
    "https://github.com/Kitware/trame/raw/master/examples/data/disk_out_ref.vtu",
    __file__,
)

# -----------------------------------------------------------------------------
# Trame setup
# -----------------------------------------------------------------------------

server = get_server(client_type="vue2")
state, ctrl = server.state, server.controller


# -----------------------------------------------------------------------------
# VTK
# -----------------------------------------------------------------------------

reader = vtkXMLUnstructuredGridReader()
reader.SetFileName(dataset_file.path)
reader.Update()
dataset = reader.GetOutput()

renderer = vtkRenderer()
renderer.SetBackground(1, 1, 1)
render_window = vtkRenderWindow()
render_window.AddRenderer(renderer)
render_window.GetInteractor()

rw_interactor = vtkRenderWindowInteractor()
rw_interactor.SetRenderWindow(render_window)
rw_interactor.GetInteractorStyle().SetCurrentStyleToTrackballCamera()

interactor_trackball = rw_interactor.GetInteractorStyle()
interactor_selection = vtkInteractorStyleRubberBandPick()
area_picker = vtkRenderedAreaPicker()
rw_interactor.SetPicker(area_picker)

surface_filter = vtkGeometryFilter()
surface_filter.SetInputConnection(reader.GetOutputPort())
surface_filter.SetPassThroughPointIds(True)

mapper = vtkDataSetMapper()
mapper.SetInputConnection(surface_filter.GetOutputPort())
actor = vtkActor()
actor.GetProperty().SetOpacity(0.5)
actor.SetMapper(mapper)

# Selection
selection_extract = vtkExtractSelection()
selection_mapper = vtkDataSetMapper()
selection_mapper.SetInputConnection(selection_extract.GetOutputPort())
selection_actor = vtkActor()
selection_actor.GetProperty().SetColor(1, 0, 1)
selection_actor.GetProperty().SetPointSize(5)
selection_actor.SetMapper(selection_mapper)
selection_actor.SetVisibility(0)

renderer.AddActor(actor)
renderer.AddActor(selection_actor)
renderer.ResetCamera()

selector = vtkHardwareSelector()
selector.SetRenderer(renderer)
selector.SetFieldAssociation(vtkDataObject.FIELD_ASSOCIATION_POINTS)

# vtkDataSet to DataFrame
py_ds = dsa.WrapDataObject(dataset)
pt_data = py_ds.PointData
cols = {}
for name in pt_data.keys():
    array = pt_data[name]
    shp = array.shape
    if len(shp) == 1:
        cols[name] = array
    else:
        for i in range(shp[1]):
            cols[name + "_%d" % i] = array[:, i]
DATAFRAME = pd.DataFrame(cols)
FIELD_NAMES = list(cols.keys())
SELECTED_IDX = []

DATAFRAME["id"] = DATAFRAME.index
DATAFRAME["show"] = False

DATAFRAME = DATAFRAME[['id', 'Temp', 'V_0', 'V_1', 'V_2', 'Pres', 'AsH3', 'GaMe3', 'CH4', 'H2']]


# -----------------------------------------------------------------------------
# Callbacks
# -----------------------------------------------------------------------------


# Selection Change from DataTable
@state.change("selection")
def selection_change(selection=[], **kwargs):
    print("selection_change, len(selection):", len(selection))

    global SELECTED_IDX
    if selection is None or len(selection) < 1:
        SELECTED_IDX = []
    else:
        if type(selection) is dict:
            selection_df = pd.DataFrame([selection])
        else:
            selection_df = pd.DataFrame(selection)
        SELECTED_IDX = selection_df['id'].values

    # SELECTED_IDX = selected_point_idxs if selected_point_idxs else []
    npts = len(SELECTED_IDX)

    ids = vtkIdTypeArray()
    ids.SetNumberOfTuples(npts)
    for idx, p_id in enumerate(SELECTED_IDX):
        ids.SetTuple1(idx, p_id)
        idx += 1

    sel_node = vtkSelectionNode()
    sel_node.GetProperties().Set(
        vtkSelectionNode.CONTENT_TYPE(), vtkSelectionNode.INDICES
    )
    sel_node.GetProperties().Set(vtkSelectionNode.FIELD_TYPE(), vtkSelectionNode.POINT)
    sel_node.SetSelectionList(ids)
    sel = vtkSelection()
    sel.AddNode(sel_node)

    selection_extract.SetInputDataObject(0, py_ds.VTKObject)
    selection_extract.SetInputDataObject(1, sel)
    selection_extract.Update()
    selection_actor.SetVisibility(1)

    # Update scatter plot with selection
    update_figure(**state.to_dict())

    # Update 3D view
    ctrl.view_update()


@state.change("figure_size", "scatter_x", "scatter_y")
def update_figure(figure_size, scatter_x, scatter_y, **kwargs):
    print("update_figure")
    if figure_size is None:
        return

    # Generate figure
    bounds = figure_size.get("size", {})
    fig = px.scatter(
        DATAFRAME,
        x=scatter_x,
        y=scatter_y,
        width=bounds.get("width", 200),
        height=bounds.get("height", 200),
    )

    # Update selection settings
    fig.data[0].update(
        selectedpoints=SELECTED_IDX,
        selected={"marker": {"color": "red"}},
        unselected={"marker": {"opacity": 0.5}},
    )

    # Update chart
    ctrl.update_figure(fig)


# -----------------------------------------------------------------------------


@state.change("vtk_selection")
def update_interactor(vtk_selection, **kwargs):
    if vtk_selection:
        rw_interactor.SetInteractorStyle(interactor_selection)
        interactor_selection.StartSelect()
    else:
        rw_interactor.SetInteractorStyle(interactor_trackball)


# -----------------------------------------------------------------------------


def on_chart_selection(selected_point_idxs):
    print("on_chart_selection")
    global SELECTED_IDX
    if selected_point_idxs is not None and len(selected_point_idxs) > 0:
        SELECTED_IDX = selected_point_idxs
    else:
        SELECTED_IDX = []
    # SELECTED_IDX = selected_point_idxs if selected_point_idxs else []
    npts = len(SELECTED_IDX)

    ids = vtkIdTypeArray()
    ids.SetNumberOfTuples(npts)
    for idx, p_id in enumerate(SELECTED_IDX):
        ids.SetTuple1(idx, p_id)
        idx += 1

    sel_node = vtkSelectionNode()
    sel_node.GetProperties().Set(
        vtkSelectionNode.CONTENT_TYPE(), vtkSelectionNode.INDICES
    )
    sel_node.GetProperties().Set(vtkSelectionNode.FIELD_TYPE(), vtkSelectionNode.POINT)
    sel_node.SetSelectionList(ids)
    sel = vtkSelection()
    sel.AddNode(sel_node)

    selection_extract.SetInputDataObject(0, py_ds.VTKObject)
    selection_extract.SetInputDataObject(1, sel)
    selection_extract.Update()
    selection_actor.SetVisibility(1)

    if len(SELECTED_IDX) < 1:
        state.selection = []
    elif len(SELECTED_IDX) == 1:
        state.selection = [itemgetter(*SELECTED_IDX)(state.table_rows)]
    else:
        state.selection = itemgetter(*SELECTED_IDX)(state.table_rows)

    # Update 3D view
    ctrl.view_update()


def on_box_selection_change(selection):
    print("on_box_selection_change")
    global SELECTED_IDX

    x0 = renderer.GetPickX1()
    y0 = renderer.GetPickY1()
    x1 = renderer.GetPickX2()
    y1 = renderer.GetPickY2()

    actor.GetProperty().SetOpacity(1)
    selector.SetArea(
        int(renderer.GetPickX1()),
        int(renderer.GetPickY1()),
        int(renderer.GetPickX2()),
        int(renderer.GetPickY2()),
    )
    s = selector.Select()
    n = s.GetNode(0)
    ids = dsa.vtkDataArrayToVTKArray(n.GetSelectionData().GetArray("SelectedIds"))
    surface = dsa.WrapDataObject(surface_filter.GetOutput())
    SELECTED_IDX = surface.PointData["vtkOriginalPointIds"][ids].tolist()

    selection_extract.SetInputConnection(surface_filter.GetOutputPort())
    selection_extract.SetInputDataObject(1, s)
    selection_extract.Update()
    selection_actor.SetVisibility(1)

    actor.GetProperty().SetOpacity(0.5)

    # Update scatter plot with selection
    update_figure(**state.to_dict())

    # Update 3D view
    ctrl.view_update()

    # disable selection mode
    state.vtk_selection = False


def generate_table(dataframe):
    # col_names = ['Temp', 'V_0', 'V_1', 'V_2', 'Pres', 'AsH3', 'GaMe3', 'CH4', 'H2']
    # fmt: off
    header_options = {
        "id":                   {"text": "Id", "align": "start", "sortable": True, "width": 80},
        "Temp":                 {"text": "Temp", "align": "start", "sortable": True, "width": 120},
        "V_0":                  {"text": "V_0", "align": "start", "sortable": True, "width": 120},
        "V_1":                  {"text": "V_1", "align": "start", "sortable": True, "width": 120},
        "V_2":                  {"text": "V_2", "align": "start", "sortable": True, "width": 120},
        "Pres":                 {"text": "Pres", "align": "start", "sortable": True, "width": 160},
        "AsH3":                 {"text": "AsH3", "align": "start", "sortable": True, "width": 170},
        "GaMe3":                {"text": "GaMe3", "align": "start", "sortable": True, "width": 120},
        "CH4":                  {"text": "CH4", "align": "start", "sortable": True, "width": 100},
        "H2":                   {"text": "H2", "align": "start", "sortable": True, "width": 100},
    }

    # fmt: on
    headers, rows = vuetify.dataframe_to_grid(dataframe, header_options)

    table = {
        "headers": ("table_headers", headers),
        "items": ("table_rows", rows),
        "v_model": ("selection", []),
        "show_select": True,
        "single_select": False,
        "search": ("query", ""),
        "classes": "elevation-1 ma-4",
        "multi_sort": True,
        "dense": True,
        "items_per_page": 10,   # -1
        "height": "200",
        "fixed_header": True,
        "item_key": "id",
    }

    return table


# -----------------------------------------------------------------------------
# Settings
# -----------------------------------------------------------------------------

DROPDOWN_STYLES = {
    "dense": True,
    "hide_details": True,
    "classes": "px-2",
    "style": "max-width: calc(25vw - 10px);",
}

CHART_STYLE = {
    "style": "position: absolute; left: 50%; transform: translateX(-50%);",
    "display_mode_bar": ("true",),
    "mode_bar_buttons_to_remove": (
        "chart_buttons",
        [
            "toImage",
            "resetScale2d",
            "zoomIn2d",
            "zoomOut2d",
            "toggleSpikelines",
            "hoverClosestCartesian",
            "hoverCompareCartesian",
        ],
    ),
    "display_logo": ("false",),
}

VTK_VIEW_SETTINGS = {
    "interactive_ratio": 1,
    "interactive_quality": 80,
}

# -----------------------------------------------------------------------------
# UI
# -----------------------------------------------------------------------------

state.trame__title = "VTK selection"
ctrl.on_server_ready.add(ctrl.view_update)

with SinglePageWithDrawerLayout(server) as layout:
    layout.title.set_text("VTK & plotly")
    layout.icon.click = ctrl.view_reset_camera

    with layout.toolbar as tb:
        tb.dense = True
        vuetify.VSpacer()
        vuetify.VSelect(
            v_model=("scatter_y", FIELD_NAMES[1]),
            items=("fields", FIELD_NAMES),
            **DROPDOWN_STYLES,
        )
        vuetify.VSelect(
            v_model=("scatter_x", FIELD_NAMES[0]),
            items=("fields", FIELD_NAMES),
            **DROPDOWN_STYLES,
        )

    with layout.drawer as drawer:
        # drawer components
        drawer.width = 600
        vuetify.VDivider(classes="mb-2")
        table_data = generate_table(DATAFRAME)
        vtable = vuetify.VDataTable(**table_data)

    with layout.content:
        with vuetify.VContainer(fluid=True, classes="fill-height pa-0 ma-0"):
            with vuetify.VRow(dense=True, style="height: 100%;"):
                with vuetify.VCol(
                    classes="pa-0",
                    style="border-right: 1px solid #ccc; position: relative;",
                ):
                    view = vtk.VtkRemoteView(
                        render_window,
                        box_selection=("vtk_selection",),
                        box_selection_change=(on_box_selection_change, "[$event]"),
                        **VTK_VIEW_SETTINGS,
                    )
                    # view = vtk.VtkLocalView(
                    #     render_window,
                    #     box_selection=("vtk_selection",),
                    #     box_selection_change=(on_box_selection_change, "[$event]"),
                    #     **VTK_VIEW_SETTINGS,
                    # )
                    ctrl.view_update = view.update
                    ctrl.view_reset_camera = view.reset_camera
                    vuetify.VCheckbox(
                        small=True,
                        on_icon="mdi-selection-drag",
                        off_icon="mdi-rotate-3d",
                        v_model=("vtk_selection", False),
                        style="position: absolute; top: 0; right: 0; z-index: 1;",
                        dense=True,
                        hide_details=True,
                    )
                with vuetify.VCol(classes="pa-0"):
                    with trame.SizeObserver("figure_size"):
                        html_plot = plotly.Figure(
                            selected=(
                                on_chart_selection,
                                "[$event?.points.map(({pointIndex}) => pointIndex)]",
                            ),
                            **CHART_STYLE,
                        )
                        ctrl.update_figure = html_plot.update

# -----------------------------------------------------------------------------
# Main
# -----------------------------------------------------------------------------

if __name__ == "__main__":
    server.start()

7 replies

jourdain Sep 24, 2024
Maintainer

you can use a state to allow or not allow that and you can do that test on the python side or pass it from the JS if that helps.

kerim371 Sep 24, 2024
Author

Alright, from the python side using state I do that using the following form (within on_chart_selection func):

state.do_update_figure = True


def on_chart_selection(selected_point_idxs):
    print("on_chart_selection")
    ...
    ...
    ...
    # Prevent from redrawing chart
    state.do_update_figure = False
    # This will invoke `selection_change` function
    if len(SELECTED_IDX) < 1:
        state.selection = []
    elif len(SELECTED_IDX) == 1:
        state.selection = [itemgetter(*SELECTED_IDX)(state.table_rows)]
    else:
        state.selection = itemgetter(*SELECTED_IDX)(state.table_rows)
    state.do_update_figure = True

    # Update 3D view
    ctrl.view_update()

and in the table updater I check that condition:

def selection_change(selection=[], **kwargs):
    print("selection_change, len(selection):", len(selection))
    print("state.do_update_figure:", state.do_update_figure)

    if state.do_update_figure == False:
        return
    ...
    ...
    ...
    # Update scatter plot with selection
    if state.do_update_figure:
        update_figure(**state.to_dict())

    # Update 3D view
    ctrl.view_update()

but I can see that in selection_change function state.do_update_figure is always True. I don't understand why but still.

And could you please show me the hints where to find how to do that using JS?

jourdain Sep 24, 2024
Maintainer

I see your issue. Indeed you can not do it like you did since the reactive aspect is handled out of order and in batch. So by surrounding your state.selection by a state change will do nothing...

It seems that you just need a variable to keep track of the origin of the event.

def on_chart_selection(...):
    self.state.selection_origin = "chart"

On the js side when selection change on the table, you need to add selection_origin = 'table'.
And you can do something similar on the VTK side.

kerim371 Sep 25, 2024
Author

Thank you for the hint.

I digged a little bit deeper with events in Trame and using your idea I did the following:

I stopped watching changes in the table using @state.change("selection") and added item_selected and toggle_select_all events to the datatable.
in the handle_item_selected and handle_toggle_select_all event handlers I call the function update_selection that updates the scene with the parameter update_selection(selection, is_table_origin=True) with is_table_origin=True argument
in on_chart_selection I also call update_selection(selection, is_table_origin=False) but with is_table_origin=False to not tupdate the chart.

It seems to be working, I'm happy :)

The code is like:

def on_chart_selection(selected_point_idxs):
    ...
       if len(SELECTED_IDX) < 1:
        state.selection = []
        update_selection([], is_table_origin=False)
    elif len(SELECTED_IDX) == 1:
        state.selection = [itemgetter(*SELECTED_IDX)(state.table_rows)]
        update_selection([itemgetter(*SELECTED_IDX)(state.table_rows)], is_table_origin=False)
    else:
        state.selection = itemgetter(*SELECTED_IDX)(state.table_rows)
        update_selection(itemgetter(*SELECTED_IDX)(state.table_rows), is_table_origin=False)

    # Update 3D view
    ctrl.view_update()


def update_selection(selection, is_table_origin=True):
    ...
    # Update scatter plot with selection
    if is_table_origin:
        update_figure(**state.to_dict())

    # Update 3D view
    ctrl.view_update()


def handle_item_selected(item, value, is_table_origin=True):
    update_selection(item['item'], True)

def handle_toggle_select_all(items, value, is_table_origin=True):
    update_selection(items['items'], True)

...


        vtable = vuetify.VDataTable(**table_data, page_count=(handle_page_count, "[$event]"),
                                    item_selected=(handle_item_selected, "[$event, 'value']"),
                                    toggle_select_all=(handle_toggle_select_all, "[$event, 'value']"))

Does that what you meant or there are other approaches?

jourdain Sep 25, 2024
Maintainer

Yes something along those lines. Good job for figuring it out.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trame & PlotLy unwanted events #592

{{title}}

Replies: 2 comments 7 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

Trame & PlotLy unwanted events #592

kerim371 Sep 22, 2024

Replies: 2 comments · 7 replies

jourdain Sep 22, 2024 Maintainer

kerim371 Sep 24, 2024 Author

jourdain Sep 24, 2024 Maintainer

kerim371 Sep 24, 2024 Author

jourdain Sep 24, 2024 Maintainer

kerim371 Sep 25, 2024 Author

jourdain Sep 25, 2024 Maintainer

kerim371
Sep 22, 2024

Replies: 2 comments 7 replies

jourdain
Sep 22, 2024
Maintainer

kerim371
Sep 24, 2024
Author

jourdain Sep 24, 2024
Maintainer

kerim371 Sep 24, 2024
Author

jourdain Sep 24, 2024
Maintainer

kerim371 Sep 25, 2024
Author

jourdain Sep 25, 2024
Maintainer