Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-502598: Add async query execution #672

Merged
merged 20 commits into from
Nov 6, 2023

Conversation

sfc-gh-ext-simba-lf
Copy link
Collaborator

@sfc-gh-ext-simba-lf sfc-gh-ext-simba-lf commented Oct 20, 2023

Description

  • Adds the option to execute queries in async mode by setting the asyncExec flag when calling execute
  • Adds the capability to asynchronously wait for the query to finish and get the results through query ID with getResultsFromQueryId()
  • Adds asynchronously checking the query status with getQueryStatus() or getQueryStatusThrowIfError()
  • Adds checking if query status indicates if it's still running or an error with isStillRunning() and isAnError()

Checklist

  • Format code according to the existing code style (run npm run lint:check -- CHANGED_FILES and fix problems in changed code)
  • Create tests which fail without the change (if possible)
  • Make all tests (unit and integration) pass (npm run test:unit and npm run test:integration)
  • Extend the README / documentation and ensure is properly displayed (if necessary)
  • Provide JIRA issue id (if possible) or GitHub issue id in commit message

@sfc-gh-ext-simba-lf sfc-gh-ext-simba-lf marked this pull request as ready for review October 21, 2023 03:34
@sfc-gh-ext-simba-lf sfc-gh-ext-simba-lf requested a review from a team as a code owner October 21, 2023 03:34
lib/connection/connection.js Show resolved Hide resolved
lib/connection/connection.js Outdated Show resolved Hide resolved
lib/connection/connection.js Outdated Show resolved Hide resolved
lib/http/base.js Outdated Show resolved Hide resolved
@sfc-gh-pmotacki
Copy link
Collaborator

sfc-gh-pmotacki commented Oct 23, 2023

It is to confirm that issue #208 is a duplicate?

@sfc-gh-ext-simba-lf
Copy link
Collaborator Author

It is to confirm that issue #208 is a duplicate?

That seems to be what issue 19 from sdks-drivers is based off

@codecov
Copy link

codecov bot commented Oct 23, 2023

Codecov Report

Merging #672 (6944e99) into master (6cc2fda) will increase coverage by 0.03%.
The diff coverage is 90.90%.

@@            Coverage Diff             @@
##           master     #672      +/-   ##
==========================================
+ Coverage   87.77%   87.80%   +0.03%     
==========================================
  Files          60       61       +1     
  Lines        5628     5725      +97     
==========================================
+ Hits         4940     5027      +87     
- Misses        688      698      +10     
Files Coverage Δ
lib/connection/statement.js 93.34% <100.00%> (+0.21%) ⬆️
lib/constants/error_messages.js 100.00% <100.00%> (ø)
lib/constants/query_status.js 100.00% <100.00%> (ø)
lib/errors.js 96.63% <100.00%> (+0.06%) ⬆️
lib/http/base.js 88.60% <100.00%> (+3.08%) ⬆️
lib/services/sf.js 86.49% <100.00%> (ø)
lib/connection/connection.js 86.78% <81.13%> (-2.48%) ⬇️

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

test/integration/testExecuteAsync.js Outdated Show resolved Hide resolved
test/integration/testExecuteAsync.js Outdated Show resolved Hide resolved
lib/connection/connection.js Outdated Show resolved Hide resolved
@sfc-gh-pmotacki
Copy link
Collaborator

Please rebase and check all test pass.

@sfc-gh-ext-simba-lf
Copy link
Collaborator Author

Please rebase and check all test pass.

Ok

@sfc-gh-ext-simba-lf sfc-gh-ext-simba-lf merged commit d7c33ea into master Nov 6, 2023
43 of 44 checks passed
@sfc-gh-ext-simba-lf sfc-gh-ext-simba-lf deleted the SNOW-502598-async-query-exec branch November 6, 2023 19:27
@github-actions github-actions bot locked and limited conversation to collaborators Nov 6, 2023
return axios.request(requestOptions);
let response = await axios.request(requestOptions);

if (Util.isString(response['data']) &&
Copy link
Contributor

@sfc-gh-fpawlowski sfc-gh-fpawlowski Dec 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sfc-gh-ext-simba-lf - we have been refactoring and some questions arose from this part of code.
Specifically - did You ever noticed such case, that response['data'] is string, but 'content-type' headers are set to 'application/json' at the same time?

We expected axios to parse such data automatically (ad. e.g. https://masteringjs.io/tutorials/axios/json).
Could You provide some reproduction for such scenario?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, the tests added with the PR can help repro that scenario. Or a simpler repro would be something like:

connection.execute({
	sqlText: "select 1;",
	asyncExec: true,
	complete: async function (err, stmt) {
		queryId = stmt.getQueryId();
		await connection.getQueryStatus(queryId);
	}
});

The getQueryStatus function receives a response in string that is parsed into JSON

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants