Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout option to SQL CLI tool. #1076

Merged
merged 4 commits into from
Nov 23, 2022

Conversation

Yury-Fridlyand
Copy link
Collaborator

Signed-off-by: Yury-Fridlyand yuryf@bitquilltech.com

Description

Add response timeout option to SQL CLI tool. This is very handy for SQL plugin debugging purposes.
Imagine you are debugging a query and placed a breakpoint into the plugin code. While you are going step-by-step debugging, CLI times out.

See team review discussion in Bit-Quill#153.

New command line optional argument added:

-t <value in seconds>

If not given, SQL CLI works as usual. Default timeout is 10 sec.

Before

$ opensearchsql
...
opensearchsql> select 1;
fetched rows / total rows = 1/1      <--- response in 0-10 sec
+-----+
| 1   |
|-----|
| 1   |
+-----+
opensearchsql> select 1;
Reconnecting...
Reconnected! Please run query again      <--- response in 10-19 sec
opensearchsql> select 1;
Reconnecting...
Connection Failed. Check your OpenSearch is running and then come back     <---- 20+ sec
ConnectionTimeout('TIMEOUT', "HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)", ReadTimeoutError("HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)"))
opensearchsql>

After

$ opensearchsql -t 10000
...
opensearchsql> select 1;
fetched rows / total rows = 1/1          <---- 30 min of intensive debugging with breaks - no timeout!
+-----+
| 1   |
|-----|
| 1   |
+-----+

Parameter request_timeout is processed by underlying library:
https://github.com/opensearch-project/opensearch-py/blob/8879644ec404b3ab15b739b97775d7bd7e51018b/opensearchpy/transport.py#L457

How to test

Ref: https://github.com/opensearch-project/sql/blob/2.x/sql-cli/development_guide.md#development-environment-set-up

Do once:

cd sql-cli
pip install virtualenv
virtualenv venv
pip install --editable .

Then every time you want to debug SQL CLI:

<checkout>
cd sql-cli
source ./venv/bin/activate

To exit from venv:

deactivate

You can run SQL CLI and switch branch. It will keep working with given timeout until closed.

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Add timeout option to SQL CLI tool.
@codecov-commenter
Copy link

codecov-commenter commented Nov 15, 2022

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.71%. Comparing base (21373b9) to head (dad0f8f).
Report is 483 commits behind head on 2.x.

Additional details and impacted files
@@             Coverage Diff              @@
##                2.x    #1076      +/-   ##
============================================
- Coverage     98.22%   95.71%   -2.52%     
- Complexity     3272     3351      +79     
============================================
  Files           314      337      +23     
  Lines          8205     9115     +910     
  Branches        535      672     +137     
============================================
+ Hits           8059     8724     +665     
- Misses          142      334     +192     
- Partials          4       57      +53     
Flag Coverage Δ
query-workbench 62.76% <ø> (?)
sql-engine 98.27% <ø> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

"response_timeout",
type=click.INT,
default=10,
help="Timeout to await a response from the server"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the help message indicate it's in seconds? and is the default 10 seconds too short? I think frontend request times out 30 seconds by default

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 sec is default timeout in PY client.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Help message updated in dad0f8f.

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Copy link
Collaborator

@derek-ho derek-ho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dai-chen dai-chen added the CLI label Nov 16, 2022
Copy link
Collaborator

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@dai-chen dai-chen merged commit cfb2650 into opensearch-project:2.x Nov 23, 2022
@Yury-Fridlyand Yury-Fridlyand deleted the integ-timeout-cli branch December 19, 2022 21:37
@Swiddis Swiddis added v2.18.0 Issues targeting release v2.18.0 backport 2.x labels Oct 16, 2024
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.x
# Create a new branch
git switch --create backport/backport-1076-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 cfb26508acc3b200f77f9059955bcdf92815b9ec
# Push it to GitHub
git push --set-upstream origin backport/backport-1076-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1076-to-2.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants