[Feature Request]: Add 'url' field to citations constructed during sayCommand function execution

[indrek-l]

Scenario

When I use azure_search data source which adds citations to the bot's response in Teams, the added citations do not come with links to the sources. When the citation objects are initially returned by gpt-4o, they do include a url field, but that field is not carried over to the bot's response that is constructed in the sayCommand function (as seen in actions/SayCommand.js, lines 32-41). For this reason, the user cannot go back to the sources that the bot bases its answer on, and the bot therefore cannot double as a smart search engine. This is a pity because it otherwise has the capacity for it.

Solution

The solution would be to add the url field to the clientCitation object construction in lib/actions/SayCommand.js, lines 32-41, i.e. turn block A into block B:

Block A

const clientCitation = {
    '@type': 'Claim',
    position: `${i + 1}`,
    appearance: {
        '@type': 'DigitalDocument',
        name: citation.title || `Document #${i + 1}`,
        abstract: Utilities_1.Utilities.snippet(citation.content, 500)
    }
};

Block B

const clientCitation = {
    '@type': 'Claim',
    position: `${i + 1}`,
    appearance: {
        '@type': 'DigitalDocument',
        name: citation.title || `Document #${i + 1}`,
        abstract: Utilities_1.Utilities.snippet(citation.content, 500),
        url: citation.url
    }
};

I made the change temporarily on my local machine to see what would happen, and surprisingly, I did not even need to make changes to the bot's code to fetch the urls from the citations and add them to the message, the urls appeared in the bot's messages by themselves, once the url field was added to the clientCitation objects in the sayCommand function, as seen in Image A which is taken prior to adding the url field and Image B which is taken after:

Image A: Without url

Image B: With url

I haven't found the block of code in the library that is responsible for making the links appear in the bot's message autmatically.

Additional Context

No response

[indrek-l]

PS. Note that in the screenshots, the markdown also starts working once the url field is added. In Image A, the markdown title symbols "###" are simply shown in the bot's response, but in Image B, they are used to convert the text into a title. This looks like a bug in the original code that somehow gets fixed once the url field is added. I have no idea how this happens but it does.

[corinagum]

@indrek-l this was a deliberate decision made at the time of implementation because the url links provided from Azure were not helpful in that they inevitably pointed to a 404 that even the owner of the blob storage would hit when clicking the link.

When you add the url to citations now, does the link now lead to the actual document and/or something that is helpful?

FYI, it is the Teams client that handles adding that url to the file title, not the Teams AI library. We simply pass the data on.

[indrek-l]

@indrek-l this was a deliberate decision made at the time of implementation because the url links provided from Azure were not helpful in that they inevitably pointed to a 404 that even the owner of the blob storage would hit when clicking the link.

When you add the url to citations now, does the link now lead to the actual document and/or something that is helpful?

FYI, it is the Teams client that handles adding that url to the file title, not the Teams AI library. We simply pass the data on.

Thank you for the reply. No, the url still points to a 404. I assumed that this was due to how access rights are configured for the specific storage account that I'm using and figured I would deal with that issue later. But I understand now that this cannot be fixed on my end. Is there any change that this will get fixed in the near future? At the moment, I'm unable to add the citation links to the bot message.

Initially, before I found that the links automatically appear in the bot message, I planned to add a middleware to the bot that would 1) intercept the activity object of the bot's response prior to the response being shown to the user, 2) get the urls from the response object, and 3) add the urls to the response that then gets shown to the user. Since the url field is not present in the citations, I cannot access the field in the middleware, either. At least I haven't found a way to do it. Is there any other way to get the urls to appear in the bot's response?

[corinagum]

@indrek-l If you were to write your own version of SayCommand.ts, you could potentially modify the url that it sent from instead of being citations.url to something that is hosted that the user can click on and view at their leisure. The only real change you would need to make is the url string on the citations.

This unfortunately is an Azure decision - the url they send us is exactly what citations.url is. We've brought it up to the Azure On Your Data team but it sounds like there hasn't been a change made so far.

We can keep this feature request open for now as informational to other developers, but to really utilize the url property well, we would need Azure to change the link they are sending us. Until then I would recommend what I wrote above as a workaround.

cc @Carter425

[singhk97]

Removing blocked tag since there is a workaround for this.