Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dashboard: when updating parameters, run only relevant queries #3804

Merged
merged 4 commits into from
Aug 30, 2019

Conversation

rauchy
Copy link
Contributor

@rauchy rauchy commented Aug 20, 2019

When dashboard-level parameters are updated, all widgets in that dashboard are updated, even if they are unaffected by the new parameter values.

For example - in this dashboard, the bottom widget is completely unrelated to param1, yet updating its value refreshes the widget.

dashboard

This PR changes this behavior by only refreshing the widgets that have at least parameter which maps to one of the dashboard-level parameter which were updated.

@arikfr arikfr added this to the Next milestone May 15, 2019
@@ -111,8 +111,9 @@ export class Parameters extends React.Component {
applyChanges = () => {
const { onValuesChange, disableUrlUpdate } = this.props;
this.setState(({ parameters }) => {
const parametersWithPendingValues = parameters.filter(p => p.hasPendingValue);
forEach(parameters, p => p.applyPendingValue());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now we can use parametersWithPendingValues here too :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand. Are you referring to using parametersWithPendingValues in the following line? As in

forEach(parametersWithPendingValues, p => p.applyPendingValue());

If that's what you mean, I deliberately avoided that in order to future-proof someone removing the hasPendingValue check in applyPendingValue without knowing of this external concern.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't be that concerned about it (doesn't seem a likely thing or something that would cause a big issue here). But not a big deal in any case :)

client/app/pages/dashboards/dashboard.js Outdated Show resolved Hide resolved
Copy link
Contributor

@ranbena ranbena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this correctly, it doesn't work as it should.

@rauchy your PR description is lacking - plz add a detailed problem->solution, a demo dashboard with STR so we can compare master to.

widget => widget.getQuery() && widget.getQuery().getParametersDefs().some(
({ name }) => updatedParameters.map(p => p.name).includes(name),
),
) : this.dashboard.widgets;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm uncertain Array.includes is supported in Edge 11. Better use lodash.

Copy link
Contributor

@ranbena ranbena Aug 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filtering by name won't do as param names are unique only within a widget scope.
The same widget can show up twice in a dashboard - one with a dashboard-level param, the other with a widget-level param, and with this implementation, both will get refreshed instead of only one. (Same goes for different widgets that have the same param names).

Here's a demo https://deploy-preview-3804--redash-preview.netlify.com/dashboard/param-update-tester. Change the dashboard parameter, and all widgets refresh even though the middle one shouldn't, since it only has a widget-level param.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I switched it to use the proper parameter mapping.

Copy link
Contributor

@ranbena ranbena Aug 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works perfectly 👍

I know you must have worked on making the filter logic readable but I think it's still a bit complex to comprehend. Here's my suggestion:

let affectedWidgets = dashboard.widgets;
if (updatedParameters.length) {
  const updateParamNames = updatedParameters.map(p => p.name); // do just once
  affectedWidgets = dashboard.widgets.filter(
    widget => _.chain(widget.getParameterMappings()) // I like chaining :P
      .filter({ type: 'dashboard-level' })
      .some(({ mapTo }) => _.includes(updateParamNames, mapTo))
      .value(),
  );
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chaining prevents webpack from including only the functions we use from lodash 😬

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ranbena thanks for introducing me to lodash chain sequences!

I don't think it would be significant to lose tree shaking on lodash, but I'm still not entirely convinced that the change from

filter({ type }) => type === 'dashboard-level')

to

filter({ type: 'dashboard-level' })

makes things simpler, especially given that some simpletons such as myself would possibly have to scratch their heads around _.chain and value(). The main disadvantage I see here is transforming this from an expression (const affectedWidgets = stuff) to a declarative block.

I personally find expressions a primary goal for achieving simplicity.

Copy link
Member

@gabrieldutra gabrieldutra Aug 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chaining prevents webpack from including only the functions we use from lodash

It does makes things a lot more readable, I wonder if it's really that costly when considering Lodash 😬

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forget it! Let's land this already!

@rauchy
Copy link
Contributor

rauchy commented Aug 27, 2019

your PR description is lacking - plz add a detailed problem->solution, a demo dashboard with STR so we can compare master to.

@ranbena hehe, I used hub pull-request so I guess the original issue was lacking that info. I can go ahead and update it, but it looks like you already created a demo dashboard :)

@rauchy rauchy requested a review from ranbena August 27, 2019 18:54
Copy link
Contributor

@ranbena ranbena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One code suggestion. Not a must.

widget => widget.getQuery() && widget.getQuery().getParametersDefs().some(
({ name }) => updatedParameters.map(p => p.name).includes(name),
),
) : this.dashboard.widgets;
Copy link
Contributor

@ranbena ranbena Aug 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works perfectly 👍

I know you must have worked on making the filter logic readable but I think it's still a bit complex to comprehend. Here's my suggestion:

let affectedWidgets = dashboard.widgets;
if (updatedParameters.length) {
  const updateParamNames = updatedParameters.map(p => p.name); // do just once
  affectedWidgets = dashboard.widgets.filter(
    widget => _.chain(widget.getParameterMappings()) // I like chaining :P
      .filter({ type: 'dashboard-level' })
      .some(({ mapTo }) => _.includes(updateParamNames, mapTo))
      .value(),
  );
}

@rauchy rauchy merged commit 31c888e into master Aug 30, 2019
arikfr added a commit that referenced this pull request Sep 18, 2019
* Migrate Parameters component to React (#4006)

* Start Parameters Migration

* Add dirtyCount

* Use workaround with setState

* Apply Changes

* Add EditSettingsDialog

* Add Cmd/Ctrl + Enter behavior

* Remove isApplying

* Delete Angular version of parameters

* Update tests

* Remove angular stuff

* Update jest

* Drag placeholder

* Update events

* Use old button styling and move css

* Reviewing code

* Add parameter rearrange test

* Add Parameter Settings title change test

* Update Parameter Settings button styling

* Move parameter url logic back to Parameters

* Disable url update when query is new

* Styling changes (#4019)

* Ran's title width styling

* Update drag test

* Improve sizing for Number inputs

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Fix issue with dragged parameter wrapping

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Don't reevaluate dirtyParamCount

* Allow multiple values :)

* Fix parameter alignments

* Fix Select width on search

* Update client/app/components/Parameters.less

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Humanize param.name

* Make sure angular updates Execute disabled status

* Add more flake8 tests and fail build if any test fails (#4055)

* Add more flake8 tests and fail build if any test fails

Run all flake8 E9xx + F63x + F7xx + F82x tests.

* long = long in Python 2

* Fix: MySQL connections without SSL are failing (#4090)

* Move connection logic into a single method & make sure not to pass ssl value if not used.

* Remove wildcard import and format file.

* Removed redash-newstyle.less (#4017)

* Make sure we always pass a list to _get_column_lists (#4095)

(some data sources might return None as the columns list)

* [Data Sources] Add: Azure Data Explorer (Kusto) query runner (#4091)

* [Data Sources] Add: Azure Data Explorer (Kusto) query runner

* CodeClimate fixes

* Remove TODO

* Fixed configuration properties names for Azure Kusto

* Azure Kusto: get_schema in one query

* azure-kusto-data update to 0.0.32

* Add Kusto to the default query runners list

* [Qubole] - Adding support to process Quantum query types. (#4066)

* [Qubole] - Adding support to process Quantum query types.

Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.

* -Making quantum as defult query.
-Dictionary safe access to connection parmeters

* keeping pep8 standards

* Maintainig pep8 std

* Use latest version of qds-sdk

* Use qds-sdk v1.13.0

* Use qds-sdk v1.12.0

* Use qds-sdk v1.13.0

* Updating SDK with verified version

* hive as default query type

* qds-sdk : Locking most recent release version

* qds-sdk : Locking recent release version

* falling back to original version of qds-sdk

* Dashboard: when updating parameters, run only relevant queries (#3804)

* refresh only affected queries in dashboard when parameters are changed

* rename pendingParameters to updatedParameters

* select which widgets to update according to their mapping as a dashboard-level parameter

* use lodash's include

* Migrate with SQL statements. (#4105)

* Update badge in README.md to link to CircleCI (#4104)

* Update README.md

* Update README.md

* Update README.md

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Update README.md

* Fix widget bottom element alignment (#4110)

* Fix: allow users with view only acces to use the queries in Query Results (#4112)

* Fix: allow users with view only acces to access the queries

* Add tests

* Update error message

* Update error message. Take 2

* Fix Dropdown parameter options appearing behind Dialog (#4109)

* Add ability to use Ant's Table loading property when using ItemsTable (#4117)

* Display data source icon in query editor (#4119)

* Move annotation logic into Query Runner (#4113)

* Code formatting

* Move annotation logic into query runner, so it can be overriden in the query runner.

* Add mixin to __all__

* Switch to flag instead of mixin

* Feature (Redshift): option to set query group for adhoc/scheduled queries  (#4114)

* Add scheduled status to query job metadata.

* Add: option to set query group for adhoc/scheduled Redshift queries

* Scheduled might not be set for already enqueued queries.

* Fix number param value normlization (#4116)

* Use ng-src for data source icons (#4123)

* hive_ds: show a user friendly error message when possible (#4121)

* Update botocore, to get pass pip warning (#4122)

* Widget table scroll-x visible (#4101)

* Table viz horizontal scroll made visible

* Added tests

* Fixed snapshot pre-condition

* Perhaps this would trigger percy

* Upgrade Sentry-SDK and enable additional integratoins (#4127)

* Update sentry-sdk version

* Add additional Sentry integrations

* Migrate Counter visualization to React (#4106)

* Migrate Counter to React: Renderer

* Migrate Counter to React: Editor

* Cleanup

* Review and fix rows indexing algorithm

* Counter not properly scaled in editor

* Fix wrong label for/input id pair

* Tests

* Tests

* Fix vendor prefixes

* Remove unnecessary useEffect dependencies

* Update tests

* Fix Percy snapshot names

* Sync botocor eversions across requirements files. (#4128)

* Decrease size of widget pagination (#4120)

* Added tests

* Perhaps this would trigger percy

* Decrease size of widget pagination

* Removed unused attr

* Updated tests

* Allow the user to decide how to handle null values in charts (#4071)

* #2629 Refactor Chart visualization, add option for handling NULL values (keep/convert to 0.0)

* Handle null values in line/area stacking code; some cleanup

* Handle edge case: line/area stacking when last value of one of series is missing

* Mjnor update to line/area stacking code

* Fix line/area normalize to percents feature

* Unit tests

* Refine tests; add tests for prepareLayout function

* Tests for prepareData (heatmap) function

* Tests for prepareData (pie) function

* Tests for prepareData (bar, line, area) function

* Tests for prepareData (scatter, bubble) function

* Tests for prepareData (box) function

* Remove unused file

* Alerts: Add more condition comparison options (#4134)

* #4132 Add more condition comparison options

* Add arguments to fallback lambda

* Remove duplicate messages method (#4131)

* Migrate Chart visualization to React Part 1: Renderer (#4130)

* Migrate Chart visualization: Renderer

* Refine PlotlyChart component; move stylesheets to visualization's folder

* Migrate Custom JS Chart to React

* Cleanup

* Add jsconfig settings with '@' webpack alias (#4135)

* Counter Editor: move components to own files (#4138)

* Allow users to share aggregated usage information with us (#4108)

* Initial commit of BeaconConsent component

* Add comment about being able to change setting

* Use <Text> correctly

* Final version of consent screen

* Show beacon consent message on homepage only if it wasn't enabled already.

* Add consent setting to organization settings screen.

* Add support for custom message in OrgSetting.save.

* Implmenet consent saving.

* If consent given, send extra data

* Add HelpTrigger

* Make CodeClimate happy

* Wrap everything with DynamicComponent

* CHANGELOG for V8-beta. (#4057)

* CHANGELOG for V8-beta.

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Bug fix: Query view doesn't sync parameters when selecting and deleting (#4146)

* Query Snippets: Use onClick instead of link for 'Click here' option (#4144)

* Snippets: Don't change url when not needed

* Revert "Snippets: Don't change url when not needed"

This reverts commit 2f346f3.

* Query Snippets: use onClick instead of link

* Color picker component (#4136)

* Widget filters overlapped by visualization (#4137)

* Fix: widget filters overlapped by visualization

* Fix tests

* Fix tests

* CHANGELOG for v8.0.0-beta.2 (#4145)

* Stop building tarballs.

* Update version reference.

* CHANGELOG for 8.0.0-beta.2
arikfr pushed a commit that referenced this pull request Oct 27, 2019
* refresh only affected queries in dashboard when parameters are changed

* rename pendingParameters to updatedParameters

* select which widgets to update according to their mapping as a dashboard-level parameter

* use lodash's include
harveyrendell pushed a commit to pushpay/redash that referenced this pull request Nov 14, 2019
…dash#3804)

* refresh only affected queries in dashboard when parameters are changed

* rename pendingParameters to updatedParameters

* select which widgets to update according to their mapping as a dashboard-level parameter

* use lodash's include
@guidopetri guidopetri deleted the refresh-only-affected-queries-when-applying-changes branch July 22, 2023 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants