Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Qubole] - Adding support to process Quantum query types. #4066

Merged
merged 14 commits into from
Aug 29, 2019

Conversation

sandeepV2
Copy link
Contributor

Quantum is a server-less interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept quantum query type from user which makes
Cluster Label as optional.

issue: #4065

Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.
@sandeepV2
Copy link
Contributor Author

Merging qubole_quantum to master

@sandeepV2 sandeepV2 closed this Aug 14, 2019
@sandeepV2 sandeepV2 reopened this Aug 14, 2019
@sandeepV2
Copy link
Contributor Author

@arikfr Can you please review this.

Copy link
Member

@arikfr arikfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @sandeepV2! Quantum sounds interesting and supporting it in Redash will be great.

See comments.

redash/query_runner/qubole.py Show resolved Hide resolved
redash/query_runner/qubole.py Outdated Show resolved Hide resolved
@sandeepV2
Copy link
Contributor Author

I have updated requirements_all_ds.txt with recent release version of SDK (v1.13.0). Checks with failure (build: importlib-metadata>=0.18 and percy) does not seem to relate with my changes.

@arikfr Can we merge it?

@sandeepV2
Copy link
Contributor Author

@arikfr can i get write access to re-run failed checks? and any hints to get over build issue?

Thanks in advance,
Sandeep

Copy link
Member

@arikfr arikfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The importlib-metadata is something we fixed. Did you rebase with latest master?

redash/query_runner/qubole.py Outdated Show resolved Hide resolved
redash/query_runner/qubole.py Outdated Show resolved Hide resolved
@sandeepV2
Copy link
Contributor Author

The importlib-metadata is something we fixed. Did you rebase with latest master?

Yes i did, It is up to date with master.

$git rebase master
Current branch qubole_quantum is up to date.

@arikfr
Copy link
Member

arikfr commented Aug 28, 2019

Did you update your local master branch with the remote one? Otherwise it's probably the same master you branched out from. Looking at your tree I can see that requirements_bundle.txt is not the same as the current master, which explains this issue.

While it's probably safe to merge regardless, it's better to be on the safe side.

@sandeepV2
Copy link
Contributor Author

Did you update your local master branch with the remote one? Otherwise it's probably the same master you branched out from. Looking at your tree I can see that requirements_bundle.txt is not the same as the current master, which explains this issue.

While it's probably safe to merge regardless, it's better to be on the safe side.

Done ! When can we expect it to be merged to master?

@arikfr arikfr merged commit 4430544 into getredash:master Aug 29, 2019
@arikfr
Copy link
Member

arikfr commented Aug 29, 2019

Merged. Thanks :)

arikfr added a commit that referenced this pull request Sep 18, 2019
* Migrate Parameters component to React (#4006)

* Start Parameters Migration

* Add dirtyCount

* Use workaround with setState

* Apply Changes

* Add EditSettingsDialog

* Add Cmd/Ctrl + Enter behavior

* Remove isApplying

* Delete Angular version of parameters

* Update tests

* Remove angular stuff

* Update jest

* Drag placeholder

* Update events

* Use old button styling and move css

* Reviewing code

* Add parameter rearrange test

* Add Parameter Settings title change test

* Update Parameter Settings button styling

* Move parameter url logic back to Parameters

* Disable url update when query is new

* Styling changes (#4019)

* Ran's title width styling

* Update drag test

* Improve sizing for Number inputs

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Fix issue with dragged parameter wrapping

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Don't reevaluate dirtyParamCount

* Allow multiple values :)

* Fix parameter alignments

* Fix Select width on search

* Update client/app/components/Parameters.less

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Humanize param.name

* Make sure angular updates Execute disabled status

* Add more flake8 tests and fail build if any test fails (#4055)

* Add more flake8 tests and fail build if any test fails

Run all flake8 E9xx + F63x + F7xx + F82x tests.

* long = long in Python 2

* Fix: MySQL connections without SSL are failing (#4090)

* Move connection logic into a single method & make sure not to pass ssl value if not used.

* Remove wildcard import and format file.

* Removed redash-newstyle.less (#4017)

* Make sure we always pass a list to _get_column_lists (#4095)

(some data sources might return None as the columns list)

* [Data Sources] Add: Azure Data Explorer (Kusto) query runner (#4091)

* [Data Sources] Add: Azure Data Explorer (Kusto) query runner

* CodeClimate fixes

* Remove TODO

* Fixed configuration properties names for Azure Kusto

* Azure Kusto: get_schema in one query

* azure-kusto-data update to 0.0.32

* Add Kusto to the default query runners list

* [Qubole] - Adding support to process Quantum query types. (#4066)

* [Qubole] - Adding support to process Quantum query types.

Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.

* -Making quantum as defult query.
-Dictionary safe access to connection parmeters

* keeping pep8 standards

* Maintainig pep8 std

* Use latest version of qds-sdk

* Use qds-sdk v1.13.0

* Use qds-sdk v1.12.0

* Use qds-sdk v1.13.0

* Updating SDK with verified version

* hive as default query type

* qds-sdk : Locking most recent release version

* qds-sdk : Locking recent release version

* falling back to original version of qds-sdk

* Dashboard: when updating parameters, run only relevant queries (#3804)

* refresh only affected queries in dashboard when parameters are changed

* rename pendingParameters to updatedParameters

* select which widgets to update according to their mapping as a dashboard-level parameter

* use lodash's include

* Migrate with SQL statements. (#4105)

* Update badge in README.md to link to CircleCI (#4104)

* Update README.md

* Update README.md

* Update README.md

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Update README.md

* Fix widget bottom element alignment (#4110)

* Fix: allow users with view only acces to use the queries in Query Results (#4112)

* Fix: allow users with view only acces to access the queries

* Add tests

* Update error message

* Update error message. Take 2

* Fix Dropdown parameter options appearing behind Dialog (#4109)

* Add ability to use Ant's Table loading property when using ItemsTable (#4117)

* Display data source icon in query editor (#4119)

* Move annotation logic into Query Runner (#4113)

* Code formatting

* Move annotation logic into query runner, so it can be overriden in the query runner.

* Add mixin to __all__

* Switch to flag instead of mixin

* Feature (Redshift): option to set query group for adhoc/scheduled queries  (#4114)

* Add scheduled status to query job metadata.

* Add: option to set query group for adhoc/scheduled Redshift queries

* Scheduled might not be set for already enqueued queries.

* Fix number param value normlization (#4116)

* Use ng-src for data source icons (#4123)

* hive_ds: show a user friendly error message when possible (#4121)

* Update botocore, to get pass pip warning (#4122)

* Widget table scroll-x visible (#4101)

* Table viz horizontal scroll made visible

* Added tests

* Fixed snapshot pre-condition

* Perhaps this would trigger percy

* Upgrade Sentry-SDK and enable additional integratoins (#4127)

* Update sentry-sdk version

* Add additional Sentry integrations

* Migrate Counter visualization to React (#4106)

* Migrate Counter to React: Renderer

* Migrate Counter to React: Editor

* Cleanup

* Review and fix rows indexing algorithm

* Counter not properly scaled in editor

* Fix wrong label for/input id pair

* Tests

* Tests

* Fix vendor prefixes

* Remove unnecessary useEffect dependencies

* Update tests

* Fix Percy snapshot names

* Sync botocor eversions across requirements files. (#4128)

* Decrease size of widget pagination (#4120)

* Added tests

* Perhaps this would trigger percy

* Decrease size of widget pagination

* Removed unused attr

* Updated tests

* Allow the user to decide how to handle null values in charts (#4071)

* #2629 Refactor Chart visualization, add option for handling NULL values (keep/convert to 0.0)

* Handle null values in line/area stacking code; some cleanup

* Handle edge case: line/area stacking when last value of one of series is missing

* Mjnor update to line/area stacking code

* Fix line/area normalize to percents feature

* Unit tests

* Refine tests; add tests for prepareLayout function

* Tests for prepareData (heatmap) function

* Tests for prepareData (pie) function

* Tests for prepareData (bar, line, area) function

* Tests for prepareData (scatter, bubble) function

* Tests for prepareData (box) function

* Remove unused file

* Alerts: Add more condition comparison options (#4134)

* #4132 Add more condition comparison options

* Add arguments to fallback lambda

* Remove duplicate messages method (#4131)

* Migrate Chart visualization to React Part 1: Renderer (#4130)

* Migrate Chart visualization: Renderer

* Refine PlotlyChart component; move stylesheets to visualization's folder

* Migrate Custom JS Chart to React

* Cleanup

* Add jsconfig settings with '@' webpack alias (#4135)

* Counter Editor: move components to own files (#4138)

* Allow users to share aggregated usage information with us (#4108)

* Initial commit of BeaconConsent component

* Add comment about being able to change setting

* Use <Text> correctly

* Final version of consent screen

* Show beacon consent message on homepage only if it wasn't enabled already.

* Add consent setting to organization settings screen.

* Add support for custom message in OrgSetting.save.

* Implmenet consent saving.

* If consent given, send extra data

* Add HelpTrigger

* Make CodeClimate happy

* Wrap everything with DynamicComponent

* CHANGELOG for V8-beta. (#4057)

* CHANGELOG for V8-beta.

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Bug fix: Query view doesn't sync parameters when selecting and deleting (#4146)

* Query Snippets: Use onClick instead of link for 'Click here' option (#4144)

* Snippets: Don't change url when not needed

* Revert "Snippets: Don't change url when not needed"

This reverts commit 2f346f3.

* Query Snippets: use onClick instead of link

* Color picker component (#4136)

* Widget filters overlapped by visualization (#4137)

* Fix: widget filters overlapped by visualization

* Fix tests

* Fix tests

* CHANGELOG for v8.0.0-beta.2 (#4145)

* Stop building tarballs.

* Update version reference.

* CHANGELOG for 8.0.0-beta.2
arikfr pushed a commit that referenced this pull request Oct 27, 2019
* [Qubole] - Adding support to process Quantum query types.

Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.

* -Making quantum as defult query.
-Dictionary safe access to connection parmeters

* keeping pep8 standards

* Maintainig pep8 std

* Use latest version of qds-sdk

* Use qds-sdk v1.13.0

* Use qds-sdk v1.12.0

* Use qds-sdk v1.13.0

* Updating SDK with verified version

* hive as default query type

* qds-sdk : Locking most recent release version

* qds-sdk : Locking recent release version

* falling back to original version of qds-sdk
harveyrendell pushed a commit to pushpay/redash that referenced this pull request Nov 14, 2019
…4066)

* [Qubole] - Adding support to process Quantum query types.

Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.

* -Making quantum as defult query.
-Dictionary safe access to connection parmeters

* keeping pep8 standards

* Maintainig pep8 std

* Use latest version of qds-sdk

* Use qds-sdk v1.13.0

* Use qds-sdk v1.12.0

* Use qds-sdk v1.13.0

* Updating SDK with verified version

* hive as default query type

* qds-sdk : Locking most recent release version

* qds-sdk : Locking recent release version

* falling back to original version of qds-sdk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants