Typing middleware not working as expected in teams as well as web chat channel

[deeptishashidharaiah]

Github issues should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.

Version

What package version of the SDK are you using.
->4.21.0

Describe the bug

Give a clear and concise description of what the bug is.
->The latest version 'typing' middle ware is only able to send "typing" for first response and its missing in all other subsequent responses for a single question in bot framework.
Earlier we were using 4.18.0, and it was working as expected as in typing was thrown before all responses.

To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

Give a clear and concise description of what you expected to happen.
->It should be able to send "typing" for first response and as well all other subsequent responses

Screenshots

If applicable, add screenshots to help explain your problem.
-> attached video example

Additional context

Add any other context about the problem here.
Recording 2023-10-09 161456.zip

[ramfattah]

Thanks @deeptishashidharaiah, I'm investigating this issue.

[ramfattah]

Hi @deeptishashidharaiah,

I'm not able to reproduce this issue.

Environment:

Sample: 57.teams-conversation-bot
BotBuilder v4.21.0
Channel: Teams

TypingActivityMiddlewareRecording.mov

If the bot answers very fast, the "typing..." activity might show only for a very short time. Sometimes, it might be so quick that you don't see it.

The 'typing' middleware in my testing bot works like this:

1. When the bot gets a message from a user, I force the bot to wait for short time.
2. While waiting, it shows a "typing..." message to the user.
3. It keeps showing "typing..." until the bot gives its answer.

Sharing my code below.

TypingMiddleware.cs:

This code listens for user messages. When it gets a message, it shows the "typing..." message.

using System.Threading;
using System.Threading.Tasks;
using Microsoft.Bot.Builder;
using Microsoft.Bot.Schema;

namespace Microsoft.BotBuilderSamples.Middlewares
{
    public class TypingMiddleware : IMiddleware
    {
        public async Task OnTurnAsync(ITurnContext turnContext, NextDelegate next, CancellationToken cancellationToken = default)
        {
            if (turnContext.Activity.Type == ActivityTypes.Message)
            {
                // send typing activity
                var typingActivity = Activity.CreateTypingActivity();
                await turnContext.SendActivityAsync(typingActivity, cancellationToken);
            }

            await next(cancellationToken);
        }
    }
}

AdapterWithErrorHandler.cs:

This is where I add the 'typing' middleware to the bot. So the bot knows it should show "typing" activity when it gets a message.

using Microsoft.Bot.Builder.Integration.AspNet.Core;
using Microsoft.Bot.Builder.TraceExtensions;
using Microsoft.Bot.Connector.Authentication;
using Microsoft.BotBuilderSamples.Middlewares;
using Microsoft.Extensions.Logging;

namespace Microsoft.BotBuilderSamples
{
    public class AdapterWithErrorHandler : CloudAdapter
    {
        public AdapterWithErrorHandler(BotFrameworkAuthentication auth, ILogger<IBotFrameworkHttpAdapter> logger)
            : base(auth, logger)
        {
            Use(new TypingMiddleware());
            OnTurnError = async (turnContext, exception) =>
            {
                // Log any leaked exception from the application.
                // NOTE: In production environment, you should consider logging this to
                // Azure Application Insights. Visit https://aka.ms/bottelemetry to see how
                // to add telemetry capture to your bot.
                logger.LogError($"Exception caught : {exception.Message}");

                // Send a catch-all apology to the user.
                await turnContext.SendActivityAsync("Sorry, it looks like something went wrong.");

                // Send a trace activity, which will be displayed in the Bot Framework Emulator
                await turnContext.TraceActivityAsync("OnTurnError Trace", exception.Message, "https://www.botframework.com/schemas/error", "TurnError");
            };
        }
    }
}

TeamsActivityHandler:

This is bot's main code. When it gets a message, it waits for 1 second (await Task.Delay(1000);) and then sends back an answer.

        protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
        {
            await Task.Delay(1000);
            var userMessage = turnContext.Activity.Text;
            var echo = $"You said: {userMessage}";
            await turnContext.SendActivityAsync(MessageFactory.Text(echo, echo), cancellationToken);
        }

[deeptishashidharaiah]

@ramfattah
The issue is not happening with first reply in a series of replies. It happened when suppose user asks a question and we have multiple responses for that question, bot will try to send them all one by one and there could be delay between each response and we wanted typing.. to work there before each response is sent to user. Is this scenario working at your end and not just for a single response for a question. Also, I can see you are using TypingMiddleware class and we are trying to use ShowTypingMiddleware class. Is ShowTypinMiddleware no more a viable class to use then?

[ramfattah]

Hi @deeptishashidharaiah, thanks for the update.

1. Clarification on Middleware:

The "TypingMiddleware" is a custom middleware class I created for demonstration purposes. It's not part of the official Bot Framework SDK. The SDK provides a built-in middleware called "ShowTypingMiddleware" that sends repeated "typing..." indicators at regular intervals until the bot sends its actual response.

2. Handling Multiple Responses:

For your specific use case where you want to send multiple bot responses with "typing..." activities in between, it's more effective to manually send the "typing..." activity within the OnMessageActivityAsync handler. This approach gives you more control and flexibility over when and how the "typing..." indicators appear.

TypingActivity.mov

Here's a code snippet for reference:

protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
{
    await Task.Delay(1000);
    await turnContext.SendActivityAsync(Activity.CreateTypingActivity(), cancellationToken);
    await Task.Delay(2000);
    var userMessage = turnContext.Activity.Text;
    var echo = $"You said: {userMessage}";
    await turnContext.SendActivityAsync(MessageFactory.Text(echo, echo), cancellationToken);

    await Task.Delay(3000);
    await turnContext.SendActivityAsync(Activity.CreateTypingActivity(), cancellationToken);
    await Task.Delay(2000);
    var secondEcho = $"Second Echo: You said: {userMessage}";
    await turnContext.SendActivityAsync(MessageFactory.Text(secondEcho, secondEcho), cancellationToken);
}

With this approach, you don't need to use the custom TypingMiddleware I shared earlier.

I hope this clears things up. Please let me know if you have further questions or if there's anything else I can assist with.

[deeptishashidharaiah]

Thanks for your response and suggestions.

1. ShowTypingMiddleware worked for us in version 4.18.0. it's the latest version that has the problem. so just want to clear it off if its regression in the latest version.
2. The suggested approach will still send typing only once but we would like to send it repeatedly till next response is sent and delay between each response is not fixed in our bot. So we want existing middleware to work as it did before. this will helps other SDK users as well and avoid redundant work.

Hope to know what's the change that introduced between these versions.

[ramfattah]

Thanks @deeptishashidharaiah, Interesting, from my end I used version 4.21.0 to do the tests.

are you able to share a minimal reproducible .zip bot?

[deeptishashidharaiah]

please give it a try here: https://polarisbot-test.microsoft.com/

[ramfattah]

Thanks @deeptishashidharaiah, I was referring to the bot code.

are you able to share a zipped file of your bot that I could test from my end? If so, please be sure to remove any secrets/keys beforehand.

[deeptishashidharaiah]

Cant share entire code here but please reach out to me on teams/outlook, will arrange a call
Deeptish@microsoft.com

sharing small snippet hete

[deeptishashidharaiah]

@ramfattah any update?

[deeptishashidharaiah]

@ramfattah gentle ping! please update

[ramfattah]

CC @stevkan @dmvtech

[deeptishashidharaiah]

@stevkan and @dmvtech any update?

[tracyboehrer]

@ceciliaavila Can your team help with this? Related to the "New Teams" issue?