Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: allow POST chart/data request without CSRF token #17429

Merged
merged 1 commit into from
Nov 12, 2021
Merged

Conversation

etr2460
Copy link
Member

@etr2460 etr2460 commented Nov 12, 2021

SUMMARY

#17400 broke requesting chart data without providing a CSRF token. This config change should fix it

TESTING INSTRUCTIONS

CI, I haven't tested yet, but will test with a testenv using fetch

Testing in the test env:

fetch("http://35.86.99.89:8080/api/v1/chart/data?form_data=%7B%22slice_id%22%3A132%7D", {
  "headers": {
    "accept": "*/*",
    "accept-language": "en-US,en;q=0.9",
    "content-type": "application/json",
    "x-internalauth-username": "erik_ritter"
  },
  "referrer": "http://35.86.99.89:8080/superset/explore/?form_data=%7B%22viz_type%22%3A%22table%22%2C%22datasource%22%3A%2212__table%22%2C%22slice_id%22%3A132%2C%22url_params%22%3A%7B%7D%2C%22time_range_endpoints%22%3A%5B%22inclusive%22%2C%22exclusive%22%5D%2C%22granularity_sqla%22%3A%22year%22%2C%22time_grain_sqla%22%3A%22P1D%22%2C%22time_range%22%3A%22No+filter%22%2C%22query_mode%22%3A%22raw%22%2C%22groupby%22%3A%5B%22name%22%5D%2C%22metrics%22%3A%5B%7B%22aggregate%22%3A%22SUM%22%2C%22column%22%3A%7B%22column_name%22%3A%22global_sales%22%2C%22description%22%3Anull%2C%22expression%22%3Anull%2C%22filterable%22%3Atrue%2C%22groupby%22%3Atrue%2C%22id%22%3A887%2C%22is_dttm%22%3Afalse%2C%22optionName%22%3A%22_col_Global_Sales%22%2C%22python_date_format%22%3Anull%2C%22type%22%3A%22DOUBLE+PRECISION%22%2C%22verbose_name%22%3Anull%7D%2C%22expressionType%22%3A%22SIMPLE%22%2C%22hasCustomLabel%22%3Afalse%2C%22isNew%22%3Afalse%2C%22label%22%3A%22SUM%28Global_Sales%29%22%2C%22optionName%22%3A%22metric_pkpvgdsf70d_pnqv77v0x2p%22%2C%22sqlExpression%22%3Anull%7D%5D%2C%22all_columns%22%3A%5B%22name%22%2C%22global_sales%22%2C%22platform%22%2C%22genre%22%2C%22publisher%22%2C%22year%22%5D%2C%22percent_metrics%22%3A%5B%5D%2C%22order_by_cols%22%3A%5B%22%5B%5C%22global_sales%5C%22%2C+false%5D%22%5D%2C%22row_limit%22%3Anull%2C%22server_page_length%22%3A10%2C%22order_desc%22%3Atrue%2C%22adhoc_filters%22%3A%5B%5D%2C%22table_timestamp_format%22%3A%22smart_date%22%2C%22page_length%22%3A%2215%22%2C%22include_search%22%3Atrue%2C%22show_cell_bars%22%3Afalse%2C%22color_pn%22%3Afalse%2C%22extra_form_data%22%3A%7B%7D%7D",
  "referrerPolicy": "strict-origin-when-cross-origin",
  "body": "{\"datasource\":{\"id\":12,\"type\":\"table\"},\"force\":false,\"queries\":[{\"time_range\":\"No filter\",\"granularity\":\"year\",\"filters\":[],\"extras\":{\"time_grain_sqla\":\"P1D\",\"time_range_endpoints\":[\"inclusive\",\"exclusive\"],\"having\":\"\",\"having_druid\":[],\"where\":\"\"},\"applied_time_extras\":{},\"columns\":[\"name\",\"global_sales\",\"platform\",\"genre\",\"publisher\",\"year\"],\"orderby\":[[\"global_sales\",false]],\"annotation_layers\":[],\"timeseries_limit\":0,\"order_desc\":true,\"url_params\":{},\"custom_params\":{},\"custom_form_data\":{},\"post_processing\":[]}],\"result_format\":\"json\",\"result_type\":\"full\"}",
  "method": "POST",
  "mode": "cors",
  "credentials": "include"
}).then(res => res.json());

The above request returns properly without the csrf token

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

to: @ofekisr @amitmiran137 @serenajiang @john-bodley @villebro

@codecov
Copy link

codecov bot commented Nov 12, 2021

Codecov Report

Merging #17429 (94f8b84) into master (f10bc6d) will decrease coverage by 0.07%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17429      +/-   ##
==========================================
- Coverage   77.04%   76.96%   -0.08%     
==========================================
  Files        1041     1041              
  Lines       56073    56073              
  Branches     7738     7738              
==========================================
- Hits        43201    43157      -44     
- Misses      12614    12658      +44     
  Partials      258      258              
Flag Coverage Δ
hive 81.51% <100.00%> (ø)
javascript 71.22% <ø> (ø)
mysql 81.94% <100.00%> (ø)
postgres 81.95% <100.00%> (ø)
presto ?
python 82.30% <100.00%> (-0.16%) ⬇️
sqlite 81.62% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/config.py 91.50% <100.00%> (ø)
superset/db_engine_specs/presto.py 84.30% <0.00%> (-6.07%) ⬇️
superset/connectors/sqla/models.py 86.58% <0.00%> (-1.39%) ⬇️
superset/models/core.py 89.26% <0.00%> (-0.74%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f10bc6d...94f8b84. Read the comment docs.

@ofekisr
Copy link
Contributor

ofekisr commented Nov 12, 2021

OK, but where is the new test case to prevent bugs like these In the future? If I missed it how the next developer who try to improve our product will not missed it?

@etr2460
Copy link
Member Author

etr2460 commented Nov 12, 2021

If you'd rather we fix by reverting the breaking PR, i'm happy to do that too. Ideally, tests should be added prior to refactors so that the refactors don't cause issues. In this case, all I honestly have time to do is fix the bug (I'm on PTO today). We can either fix forward (this PR) or revert back to the functional state. Happy to do either, but regardless it needs to be fixed.

@etr2460
Copy link
Member Author

etr2460 commented Nov 12, 2021

/testenv up

@github-actions
Copy link
Contributor

@etr2460 Ephemeral environment spinning up at http://35.86.99.89:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

@ofekisr
Copy link
Contributor

ofekisr commented Nov 12, 2021

If you'd rather we fix by reverting the breaking PR, i'm happy to do that too. Ideally, tests should be added prior to refactors so that the refactors don't cause issues. In this case, all I honestly have time to do is fix the bug (I'm on PTO today). We can either fix forward (this PR) or revert back to the functional state. Happy to do either, but regardless it needs to be fixed.

So revert the code and keep the bad state of the code that no one can understand and no one has the courage to improve it... Please don't take it personally but when you fix a bug without adding a test is the same as adding a new feature without any tests.
When someone wants to clean the code without adding new logic it must rely on the current tests cases, and if you say the current test cases are lame and don't cover anything, why do you keep them at all?
Everybody chose the take the easy solution, I could add a small hack solution and prevent the code cleaning, but when you prefer the easy solution you don't promote the value of the product so use it anyway?

@etr2460
Copy link
Member Author

etr2460 commented Nov 12, 2021

Unfortunately #17400 doesn't revert cleanly (probably because of other PRs stacked on top of it). As you say, reverting is probably the best way to resolve the issue, but as i'm both unfamiliar with the code and the changes stacked on top of it, it's not really feasible for me to dig in today. If we feel reverting is the right option going forward, feel free to stack this change as part of the revert.

To unbreak master for now, I'll make this fix, and hopefully will be able to follow up with a test (although I can't guarantee anything, as this would essentially be testing that Flask-WTForms does what it's supposed to and i'm not really sure how best to test the functionality of a dependency).

when you fix a bug without adding a test is the same as adding a new feature without any tests

Personally I disagree. This PR fixes a recent breakage on the master branch that would block any future releases of Superset. Obviously having tests for the code is better than not, but I'd say having a functional product without tests is better than a non-functional one (also without tests).

@etr2460 etr2460 merged commit aa8040e into master Nov 12, 2021
@github-actions
Copy link
Contributor

Ephemeral environment shutdown and build artifacts deleted.

@villebro
Copy link
Member

villebro commented Nov 15, 2021

Thanks for the fix @etr2460 . We've been seeing similar regressions in other PRs lately, many of which I've unfortunately been party to, either as an author or reviewer. I think it's important for everyone to accept that the state of test coverage is what it is, and we need to do our best to

  1. make sure we keep Superset as stable as possible
  2. do our best to encourage adding more tests to existing critical functionality going foward

I agree with @ofekisr that it's a tall order to expect every developer to have full understanding of what side-effects a code change can have. If this is an implicit requirement (=having full understanding of what breakage may occur despite CI being green), then it will become increasingly difficult for new community contributions to get through the review pipeline.

Regarding this regression, optimally #10397 that originally introduced the functionality would have added an integration test that made sure the endpoint works without CSRF tokens (in hindsight, as a reviewer, I should have pushed for that). But in the meantime, whenever we do refactors to code that may be dangerous, it's probably a good idea to request reviews from additional people who may have more context, along with a very detailed PR description and targeted questions (e.g. "are there any known consequences of moving the x endpoint from palce A to B?") to make it easier for reviewers to jump in and not have to spend considerable time parsing the intent of the PR.

Having said that, I'm happy to start coordinating an effort to add test coverage to code that has either 1) been subject to a regression 2) is known to have a high risk of regressions due to lacking test coverage.

Ping @john-bodley @junlincc

@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
@mistercrunch mistercrunch deleted the etr2460-patch-4 branch March 26, 2024 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XS 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants