Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Reporting] fix too_long_frame_exception by passing scroll_id in the request body #102972

Merged
merged 5 commits into from
Jun 30, 2021

Conversation

tsullivan
Copy link
Member

@tsullivan tsullivan commented Jun 22, 2021

Summary

Fixes #102971

An HTTP line can not be longer than 4096 bytes. This becomes a problem when using scan and scroll with Elasticsearch, and having the scroll ID placed in the path of the URL.

Bad:

GET /_search/scroll/really_really_long_scroll_id?scroll=30s
DELETE /_search/scroll/really_really_long_scroll_id

When the URL is too long, Elasticsearch may respond with an error:

[too_long_frame_exception]: An HTTP line is larger than 4096 bytes. 

The scroll ID should be in the POST payload body.

Good:

POST /_search/scroll
{"scroll":"30s","scroll_id":["really_really_long_scroll_id"]}
DELETE /_search/scroll
{"scroll_id":["really_really_long_scroll_id"]}

Checklist

Delete any items that are not applicable to this PR.

Release note

Fixed an issue with CSV export that could limit the size of the export to 500 lines.

@tsullivan tsullivan added bug Fixes for quality problems that affect the customer experience Team:AppServices v7.13.3 v7.14.0 v8.0.0 labels Jun 22, 2021
@tsullivan tsullivan requested a review from a team June 22, 2021 18:12
@tsullivan tsullivan force-pushed the reporting/csv-scroll-id-fixes branch from b21ed81 to 618a760 Compare June 22, 2021 18:32
@tsullivan
Copy link
Member Author

@elasticmachine merge upstream

@tsullivan tsullivan force-pushed the reporting/csv-scroll-id-fixes branch from fd8f58c to 7a35363 Compare June 29, 2021 23:01
@tsullivan tsullivan marked this pull request as ready for review June 29, 2021 23:02
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServices)

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

@spalger
Copy link
Contributor

spalger commented Jun 30, 2021

@elasticmachine merge upstream

Sorry @tsullivan, the cloud instance behind ci-stats had a little fit so that build wasn't going to succeed due to metrics at least, so I started it over.

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@jloleysens jloleysens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job finding this @tsullivan ! I had no idea that scroll ids were that big potentially.

@spalger spalger added v7.15.0 and removed v7.14.0 labels Jun 30, 2021
@tsullivan tsullivan merged commit 50c755d into elastic:master Jun 30, 2021
@tsullivan tsullivan deleted the reporting/csv-scroll-id-fixes branch June 30, 2021 19:20
tsullivan added a commit to tsullivan/kibana that referenced this pull request Jun 30, 2021
…request body (elastic#102972)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
tsullivan added a commit to tsullivan/kibana that referenced this pull request Jun 30, 2021
…request body (elastic#102972)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
tsullivan added a commit to tsullivan/kibana that referenced this pull request Jun 30, 2021
…request body (elastic#102972)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
tsullivan added a commit that referenced this pull request Jun 30, 2021
…request body (#102972) (#103986)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
tsullivan added a commit that referenced this pull request Jun 30, 2021
…request body (#102972) (#103984)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
tsullivan added a commit that referenced this pull request Jun 30, 2021
…request body (#102972) (#103985)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
madirey pushed a commit to madirey/kibana that referenced this pull request Jul 6, 2021
…request body (elastic#102972)

* pass scroll_id in the request body not a param

* update test to match

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@sophiec20 sophiec20 added the (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead label Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead release_note:fix v7.13.3 v7.15.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Reporting/CSV Export] Encountering "too_long_frame_exception" error when using scroll to fetch data
6 participants