-
Notifications
You must be signed in to change notification settings - Fork 115
Enhance SEA HTTP Client #618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
varun-edachali-dbx
wants to merge
447
commits into
main
Choose a base branch
from
sea-http-client
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e, reversing changes made to 37813ba. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add more unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * streamline unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove parsing in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
…nd forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Co-authored-by: jayant <167047871+jayantsing-db@users.noreply.github.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
…ions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
This reverts commit a1f9b9c.
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
This reverts commit 0e87374.
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Description
Enhance the current, simplified SEA Http Client by introducing retries. Most of the auth related functionality is pulled from the Thrift backend. All of the retry-related tests are expected to pass since they represent user-facing functionality (in terms of exceptions to be thrown in particular cases).
We continue using a low-level connection pool instead of using the higher level
requests
module because the latter does not have first-class support for passing a password to a client certificate, which we want to support. There are workarounds involving subclassing theHttpAdapter
and invokinginit_poolmanager
, but the docstring of the method states that it should not be invoked from client code, so we do not consider it a long term solution.The exceptions thrown by the SEA and Thrift backends are not completely the same (yet), because this PR passes
None
to the context and session id of theRequestError
object. This misparity will be logged and addressed later.How is this tested?
Related Tickets & Documents
Design Doc