Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CustomQuestionAnswering answers return empty metadata #4288

Closed
jesus-diaz-rivero opened this issue Jul 19, 2022 · 3 comments · Fixed by #4311
Closed

CustomQuestionAnswering answers return empty metadata #4288

jesus-diaz-rivero opened this issue Jul 19, 2022 · 3 comments · Fixed by #4311
Assignees
Labels
Bot Services Required for internal Azure reporting. Do not delete. Do not change color. bug Indicates an unexpected problem or an unintended behavior. customer-replied-to Indicates that the team has replied to the issue reported by the customer. Do not delete. customer-reported Issue is created by anyone that is not a collaborator in the repository.

Comments

@jesus-diaz-rivero
Copy link

jesus-diaz-rivero commented Jul 19, 2022

Versions

SDK: 4.16.0
NodeJS: 16.15.1
Browser: Chrome 103
OS: macOS 11.2.3

Describe the bug

When using the CustomQuestionAnswering class to query against a Language project, the answers returned contain empty arrays in place of the actual metadata key/value pairs configured in Language Studio.

To Reproduce

  1. Go to Language Studio
  2. Create a dummy project
  3. Create random Question/Answer pairs with some metadata tags
  4. Initialize an instance of the CustomQuestionAnswering class in a node project.
  5. Call getAnswers or generateAnswers
  6. Inspect result
  7. Compare result vs. a call against the REST API as documented here (https://docs.microsoft.com/en-us/rest/api/cognitiveservices/questionanswering/question-answering/get-answers?tabs=HTTP#knowledgebaseanswercontext)

Expected behavior

Output of the get/generateAnswer methods should include all metadata tags

Screenshots

N/A

Additional context

At a first glance it seems the issue is that, unlike the QnA Maker API, CustomQuestionAnswering stores its metadata in an object rather than an array. However, the function languageServiceUtils.formatQnaResult tries to map the metadata object from the response as if it were an array.

@jesus-diaz-rivero jesus-diaz-rivero added bug Indicates an unexpected problem or an unintended behavior. needs-triage The issue has just been created and it has not been reviewed by the team. labels Jul 19, 2022
@dmvtech dmvtech added customer-reported Issue is created by anyone that is not a collaborator in the repository. Bot Services Required for internal Azure reporting. Do not delete. Do not change color. labels Jul 19, 2022
@anishprasad01 anishprasad01 self-assigned this Jul 19, 2022
@anishprasad01
Copy link
Contributor

Hi @jesus-diaz-rivero,

I'm able to reproduce this behavior. Looking into next steps.

@anishprasad01 anishprasad01 added customer-replied-to Indicates that the team has replied to the issue reported by the customer. Do not delete. and removed needs-triage The issue has just been created and it has not been reviewed by the team. labels Jul 21, 2022
@anishprasad01
Copy link
Contributor

anishprasad01 commented Jul 22, 2022

This can be reproduced with the following steps:

  1. Create a Language resource
  2. Add some question and answer pairs with metadata
  3. Save and deploy the resource
  4. Clone a fresh sample 12.customQABot and put the connection details for the language resource in the env
  5. In the sample, go to line 53 in CustomQABot.js and add console.log(response); to print out the response received from the Language service.
  6. Run the bot and ask a question, then observe the result.
  7. Optionally, also ask the question externally (for example by curl or Postman), and observe the metadata being received.

@johnataylor
Copy link
Member

@jesus-diaz-rivero we have a fix in the works for this, thanks.

tracyboehrer pushed a commit that referenced this issue Aug 26, 2022
…4311)

* Fix bad formatting of metadata object

* Add unit test for metadata
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bot Services Required for internal Azure reporting. Do not delete. Do not change color. bug Indicates an unexpected problem or an unintended behavior. customer-replied-to Indicates that the team has replied to the issue reported by the customer. Do not delete. customer-reported Issue is created by anyone that is not a collaborator in the repository.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants