Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes issue #3960 [QnAMaker] [DotNet] Implementing QnAMaker's precise answering capability as an additional feature for BotFramework users #3935

Merged
merged 63 commits into from
Jun 18, 2020
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
a82c9fa
AnswerSpanRequest property in Request.
vipeketi May 14, 2020
abf369a
Answer span changes for request and response data sstructures
vipeketi May 14, 2020
f4c8c4a
Request and Result object conversion
vipeketi May 14, 2020
96df775
datatype definition change from boolean to bool for enable AnswerSpan
vipeketi May 14, 2020
9a37d25
Text
vipeketi May 15, 2020
c157ca3
spaces
vipeketi May 15, 2020
b058ddd
MRCEnable instead of AnswerSpanRequest
vipeketi May 18, 2020
202db55
Answerspan text set to answer for MRC
vipeketi May 20, 2020
e131a74
Mock changes
vipeketi May 20, 2020
57a5c53
Review comments changes
vipeketi May 21, 2020
a28e2c1
Tests for both and precise
vipeketi May 22, 2020
c244d07
Update GetQnAPromptsCard invocation in tests
vipeketi May 22, 2020
9241f3d
GetQnAPromptsContentCard method
vipeketi May 22, 2020
761ddd2
Review comments
vipeketi May 22, 2020
75c01c9
Request changes
vipeketi May 22, 2020
7bbf06a
refactoring
vipeketi May 23, 2020
7d27031
Refactoring Tests
vipeketi May 24, 2020
d1a4fb4
Tests refactoring for comments
vipeketi May 25, 2020
84233ed
Added Copyright headers
vipeketi May 26, 2020
55f3e2a
Modified comment text for review comment.
vipeketi May 26, 2020
c556fc1
Update QnAMakerDialog.cs
vipeketi May 26, 2020
2074646
document text modified.
vipeketi May 26, 2020
e97cd51
Copy right header formatting
vipeketi May 27, 2020
a5cfa1c
Remove unused import
vipeketi May 27, 2020
09c5372
Revert "Added Copyright headers"
vipeketi May 28, 2020
a19c8d4
Copyright header
vipeketi May 28, 2020
d0f51b5
Comment modified for grammar and clarity
vipeketi May 28, 2020
8b7aa4a
Review comments
vipeketi Jun 2, 2020
b04b049
For fixing assembly difference issue
vipeketi Jun 2, 2020
1e5640c
Revert "For fixing assembly difference issue"
vipeketi Jun 2, 2020
ffd0bd8
For assemblies issue
vipeketi Jun 2, 2020
a17d450
Added purpose of property in summary.
vipeketi Jun 2, 2020
9ce95ce
Comments improved
vipeketi Jun 3, 2020
b56b246
For Review comments
vipeketi Jun 3, 2020
22a3f57
Merge branch 'vipeketi/mrcsupport' of https://github.com/microsoft/bo…
vipeketi Jun 3, 2020
e651a14
For review comments
vipeketi Jun 3, 2020
7b35226
Spellings Syntax
vipeketi Jun 6, 2020
8827223
Content corrections
vipeketi Jun 8, 2020
07ee9e5
Content comments changes
vipeketi Jun 8, 2020
d9d42bb
content request
vipeketi Jun 8, 2020
be73b1e
content changes
vipeketi Jun 8, 2020
3f57d1f
Content changes for comments
vipeketi Jun 8, 2020
1a06a92
Content review updates
vipeketi Jun 8, 2020
918f816
Content changes
vipeketi Jun 8, 2020
2152ae5
Content changes
vipeketi Jun 8, 2020
d6cf4c6
content chagnes
vipeketi Jun 8, 2020
1eb0c10
Revert enablePreciseAnswer to bool
vipeketi Jun 9, 2020
7b94291
removing json property for enable preccise
vipeketi Jun 9, 2020
60d5fda
Reverting Recognizer changes
vipeketi Jun 9, 2020
757ee08
schema change and BoolExpression
vipeketi Jun 10, 2020
cb8cfcf
Review comments
vipeketi Jun 10, 2020
11c9fa1
initialization of EnablePreciseAnswer
vipeketi Jun 10, 2020
7b0c684
schema changes for Display Precise AnswerOnly
vipeketi Jun 10, 2020
d144de8
BoolExpression for DisplayPreciseAnswerOnly
vipeketi Jun 11, 2020
bf2bdc1
Merge branch 'vipeketi/mrcsupport' of https://github.com/microsoft/bo…
vipeketi Jun 11, 2020
331bfc7
bool property for displayPreciseAnswer
vipeketi Jun 16, 2020
27fe97b
Updated comments
vipeketi Jun 16, 2020
41708f8
Null checks and CardEqualityComparer comments
vipeketi Jun 16, 2020
00e5d69
Undo contenttype change.
vipeketi Jun 16, 2020
946c4b7
format
vipeketi Jun 16, 2020
b98c43c
Final Commit
vipeketi Jun 17, 2020
b23c293
Spaces formating.
vipeketi Jun 17, 2020
c99e793
Indentation for schema files.
vipeketi Jun 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions libraries/Microsoft.Bot.Builder.AI.QnA/AnswerSpanRequest.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
using System;
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
using System.Collections.Generic;
using System.Text;
using Newtonsoft.Json;

namespace Microsoft.Bot.Builder.AI.QnA
{
public class AnswerSpanRequest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class does not have a corresponding Unit Test class. I looked for AnswerSpanRequestTest.cs and cannot find it.

{
/// <summary>
/// Gets or sets a value indicating whether gets or sets the enablet.
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

@mrivera-ms mrivera-ms Jun 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// Gets or sets a value indicating whether gets or sets the enablet. [](start = 7, length = 70)

This summary is not clear. Please improve. #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated as per comment.


In reply to: 433508388 [](ancestors = 433508388)

/// </summary>
/// <value>
/// The answer text.
/// </value>
[JsonProperty("enable")]
public bool Enable { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -504,6 +504,11 @@ private async Task<DialogTurnResult> CallGenerateAnswerAsync(WaterfallStepContex
var result = new List<QueryResult>();
if (response.Answers.Any())
{
if (response.Answers.First() != null && response.Answers.First().AnswerSpan != null && !string.IsNullOrEmpty(response.Answers.First().AnswerSpan.Text))
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
{
response.Answers.First().Answer = response.Answers.First().AnswerSpan.Text;
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
}

result.Add(response.Answers.First());
}

Expand Down
46 changes: 46 additions & 0 deletions libraries/Microsoft.Bot.Builder.AI.QnA/Models/AnswerSpan.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

using Newtonsoft.Json;

namespace Microsoft.Bot.Builder.AI.QnA
{
public class AnswerSpan
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
{
/// <summary>
/// Gets or sets the answer text.
/// </summary>
/// <value>
/// The answer text.
/// </value>
[JsonProperty("text")]
public string Text { get; set; }

/// <summary>
/// Gets or sets the answer score.
/// </summary>
/// <value>
/// The answer score.
/// </value>
[JsonProperty("score")]
public float Score { get; set; }

/// <summary>
/// Gets or sets the answer startIndex.
/// </summary>
/// <value>
/// The answer startIndex.
/// </value>
[JsonProperty("startIndex")]
public int StartIndex { get; set; }

/// <summary>
/// Gets or sets the answer endIndex.
/// </summary>
/// <value>
/// The answer endIndex.
/// </value>
[JsonProperty("endIndex")]
public int EndIndex { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,18 @@ public class QnAMakerTraceInfo
/// </value>
[JsonProperty("rankerType")]
public string RankerType { get; set; }

[Obsolete("This property is no longer used and will be ignored")]
[JsonIgnore]
public Metadata[] MetadataBoost { get; set; }

/// <summary>
/// Gets or sets AnswerSpanRequest of the previous turn.
/// </summary>
/// <value>
/// The AnswerSpanRequest.
/// </value>
[JsonProperty("answerSpanRequest")]
public AnswerSpanRequest AnswerSpanRequest { get; set; }
}
}
9 changes: 9 additions & 0 deletions libraries/Microsoft.Bot.Builder.AI.QnA/Models/QueryResult.cs
Original file line number Diff line number Diff line change
Expand Up @@ -76,5 +76,14 @@ public class QueryResult
/// </value>
[JsonProperty(PropertyName = "context")]
public QnAResponseContext Context { get; set; }

/// <summary>
/// Gets or sets AnswerSpan of the previous turn.
/// </summary>
/// <value>
/// The answerspan value.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Summary "AnswerSpan". In the value here, "answerspan". Please use consistant capitalization.

Consider elaborating and adding more detail.

/// </value>
[JsonProperty("answerSpan")]
public AnswerSpan AnswerSpan { get; set; }
}
}
9 changes: 9 additions & 0 deletions libraries/Microsoft.Bot.Builder.AI.QnA/QnAMakerOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -92,5 +92,14 @@ public QnAMakerOptions()
/// <seealso cref="RankerTypes"/>
[JsonProperty("rankerType")]
public string RankerType { get; set; }

/// <summary>
/// Gets or sets a value indicating whether MRCEnable of the previous turn.
/// </summary>
/// <value>
/// To enable MRC.
/// </value>
[JsonProperty("mRCEnable")]
public bool MRCEnable { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -169,12 +169,13 @@ private QnAMakerOptions HydrateOptions(QnAMakerOptions queryOptions)
if (queryOptions.StrictFilters?.Length > 0)
{
hydratedOptions.StrictFilters = queryOptions.StrictFilters;
}
}

hydratedOptions.Context = queryOptions.Context;
hydratedOptions.QnAId = queryOptions.QnAId;
hydratedOptions.IsTest = queryOptions.IsTest;
hydratedOptions.RankerType = queryOptions.RankerType != null ? queryOptions.RankerType : RankerTypes.DefaultRankerType;
hydratedOptions.MRCEnable = queryOptions.MRCEnable;
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
}

return hydratedOptions;
Expand All @@ -183,17 +184,20 @@ private QnAMakerOptions HydrateOptions(QnAMakerOptions queryOptions)
private async Task<QueryResults> QueryQnaServiceAsync(Activity messageActivity, QnAMakerOptions options)
{
var requestUrl = $"{_endpoint.Host}/knowledgebases/{_endpoint.KnowledgeBaseId}/generateanswer";
var answerSpanRequest = new AnswerSpanRequest();
answerSpanRequest.Enable = options.MRCEnable;
var jsonRequest = JsonConvert.SerializeObject(
new
{
question = messageActivity.Text,
top = options.Top,
strictFilters = options.StrictFilters,
strictFilters = options.StrictFilters,
scoreThreshold = options.ScoreThreshold,
context = options.Context,
qnaId = options.QnAId,
isTest = options.IsTest,
rankerType = options.RankerType
rankerType = options.RankerType,
answerSpanRequest = answerSpanRequest
}, Formatting.None);

var httpRequestHelper = new HttpRequestUtils(httpClient);
Expand All @@ -206,18 +210,21 @@ private async Task<QueryResults> QueryQnaServiceAsync(Activity messageActivity,

private async Task EmitTraceInfoAsync(ITurnContext turnContext, Activity messageActivity, QueryResult[] result, QnAMakerOptions options)
{
var answerSpanRequest = new AnswerSpanRequest();
answerSpanRequest.Enable = options.MRCEnable;
var traceInfo = new QnAMakerTraceInfo
{
Message = (Activity)messageActivity,
QueryResults = result,
KnowledgeBaseId = _endpoint.KnowledgeBaseId,
ScoreThreshold = options.ScoreThreshold,
Top = options.Top,
StrictFilters = options.StrictFilters,
StrictFilters = options.StrictFilters,
Context = options.Context,
QnAId = options.QnAId,
IsTest = options.IsTest,
RankerType = options.RankerType
RankerType = options.RankerType,
AnswerSpanRequest = answerSpanRequest
};
var traceActivity = Activity.CreateTraceActivity(QnAMaker.QnAMakerName, QnAMaker.QnAMakerTraceType, traceInfo, QnAMaker.QnAMakerTraceLabel);
await turnContext.SendActivityAsync(traceActivity).ConfigureAwait(false);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,16 @@ public AdaptiveDialog QnAMakerRecognizer_DialogBase()
{
var mockHttp = new MockHttpMessageHandler();
mockHttp.When(HttpMethod.Post, GetRequestUrl())
.WithContent("{\"question\":\"QnaMaker_ReturnsAnswer\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
.WithContent("{\"question\":\"QnaMaker_ReturnsAnswer\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_ReturnsAnswer.json"));
mockHttp.When(HttpMethod.Post, GetRequestUrl())
.WithContent("{\"question\":\"QnaMaker_ReturnsNoAnswer\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
.WithContent("{\"question\":\"QnaMaker_ReturnsNoAnswer\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_ReturnsNoAnswer.json"));
mockHttp.When(HttpMethod.Post, GetRequestUrl())
.WithContent("{\"question\":\"QnaMaker_TopNAnswer\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
.WithContent("{\"question\":\"QnaMaker_TopNAnswer\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_TopNAnswer.json"));
mockHttp.When(HttpMethod.Post, GetRequestUrl())
.WithContent("{\"question\":\"QnaMaker_ReturnsAnswerWithIntent\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
.WithContent("{\"question\":\"QnaMaker_ReturnsAnswerWithIntent\",\"top\":3,\"strictFilters\":[{\"name\":\"dialogName\",\"value\":\"outer\"}],\"scoreThreshold\":0.3,\"context\":null,\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_ReturnsAnswerWithIntent.json"));

return CreateQnAMakerActionDialog(mockHttp);
Expand Down
8 changes: 4 additions & 4 deletions tests/Microsoft.Bot.Builder.AI.QnA.Tests/QnAMakerTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,11 @@ public class QnAMakerTests
public AdaptiveDialog QnAMakerAction_ActiveLearningDialogBase()
{
var mockHttp = new MockHttpMessageHandler();
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"Q11\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":0,\"previousUserQuery\":\"\"},\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"Q11\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":0,\"previousUserQuery\":\"\"},\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_TopNAnswer.json"));
mockHttp.When(HttpMethod.Post, GetTrainRequestUrl())
.Respond(HttpStatusCode.NoContent, "application/json", "{ }");
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"Q12\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":0,\"previousUserQuery\":\"\"},\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"Q12\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":0,\"previousUserQuery\":\"\"},\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
vipeketi marked this conversation as resolved.
Show resolved Hide resolved
.Respond("application/json", GetResponse("QnaMaker_ReturnsAnswer_WhenNoAnswerFoundInKb.json"));

vipeketi marked this conversation as resolved.
Show resolved Hide resolved
return CreateQnAMakerActionDialog(mockHttp);
Expand Down Expand Up @@ -106,9 +106,9 @@ await CreateFlow(rootDialog)
public AdaptiveDialog QnAMakerAction_MultiTurnDialogBase()
{
var mockHttp = new MockHttpMessageHandler();
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"I have issues related to KB\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":0,\"previousUserQuery\":\"\"},\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\"}")
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"I have issues related to KB\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":0,\"previousUserQuery\":\"\"},\"qnaId\":0,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_ReturnAnswer_withPrompts.json"));
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"Accidently deleted KB\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":27,\"previousUserQuery\":\"\"},\"qnaId\":1,\"isTest\":false,\"rankerType\":\"Default\"}")
mockHttp.When(HttpMethod.Post, GetRequestUrl()).WithContent("{\"question\":\"Accidently deleted KB\",\"top\":3,\"strictFilters\":[],\"scoreThreshold\":0.3,\"context\":{\"previousQnAId\":27,\"previousUserQuery\":\"\"},\"qnaId\":1,\"isTest\":false,\"rankerType\":\"Default\",\"answerSpanRequest\":{\"enable\":false}}")
.Respond("application/json", GetResponse("QnaMaker_ReturnAnswer_MultiTurnLevel1.json"));

return CreateQnAMakerActionDialog(mockHttp);
Expand Down