[ResponseOps] Fix Azure Connector Streaming Validation #191552
Merged
+9
−28
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stephmilovic
added
release_note:fix
Team: SecuritySolution
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
stephmilovic
changed the title
💚 Build Succeeded
Metrics [docs]
To update your PR or re-run it, just comment with: |
sorenlouv
approved these changes
Aug 27, 2024
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Aug 28, 2024
(cherry picked from commit 97b5479)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Aug 28, 2024
… (#191639) # Backport This will backport the following commits from `main` to `8.15`: - [[ResponseOps] Fix Azure Connector Streaming Validation (#191552)](#191552) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Steph Milovic","email":"stephanie.milovic@elastic.co"},"sourceCommit":{"committedDate":"2024-08-28T14:04:37Z","message":"[ResponseOps] Fix Azure Connector Streaming Validation (#191552)","sha":"97b54795f45d470faa7b0aa7aa14fd4b2277d9d5","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team: SecuritySolution","Team:Security Generative AI","v8.16.0","v8.15.1"],"title":"[ResponseOps] Fix Azure Connector Streaming Validation","number":191552,"url":"https://github.com/elastic/kibana/pull/191552","mergeCommit":{"message":"[ResponseOps] Fix Azure Connector Streaming Validation (#191552)","sha":"97b54795f45d470faa7b0aa7aa14fd4b2277d9d5"}},"sourceBranch":"main","suggestedTargetBranches":["8.15"],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191552","number":191552,"mergeCommit":{"message":"[ResponseOps] Fix Azure Connector Streaming Validation (#191552)","sha":"97b54795f45d470faa7b0aa7aa14fd4b2277d9d5"}},{"branch":"8.15","label":"v8.15.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
A Security SDH reported errors in the assistant when using Azure with streaming. I discovered a function
getRequestWithStreamOptionthat intercepts the Azure Open AI request body to set the stream parameter to ensure it is only sent on the Chat API, the Completion API and the Completions Extensions API. However, the function unnecessarily compared the connector URL to both the root url and the route, which led to a bug for the customer. Their URL used a custom domain (not{your-resource}.openai.azure.com) along with being a.ausite. As a result,getRequestWithStreamOptiondeemed the URL invalid, omitted the stream tag from the request, and returned a non-streaming response when a streaming response was expected, causing the error.By matching only the route instead of the full url, the bug is eliminated.
Note
This bug was reported on 8.14.3. In 8.15, Security solution stopped using the
invokeStreamsubaction and instead uses theinvokeAsyncIteratorsubaction so the bug does not exist for us anymore. However, I still think the fix should go through as other teams may run into the issue if they try to use theinvokeStreamorstreamApisubactions