Telegram integration error "Flood control exceeded"

[aksenk]

Hi!

I have a problem with configuring telegram integration.

I have deployed grafana oncall via helm chart and installed telegram token and telegram webhookUrl (like https://oncall.mydomain.com/).

But when I open "ChatOps -> Telegram" page I see message: To connect channel setup Telegram environment first, which includes connection to your bot and host URL.

If I go to the page "Env Variables" I see that my variables are filled in, but in column Status for variable TELEGRAM_WEBHOOK_HOST I see error: Telegram error: Flood control exceeded. Retry in 1.0 seconds.

I don't understand what's wrong, because I created a new bot for a new installation (which doesn't even have a single alert yet).

Helm chart version: 1.1.0
Oncall app version: 1.1.5
Grafana oncall plugin version: 1.1.14
Grafana version: 9.2.4

Please help.

UPD1: I tried to install a compatible versions of oncall app and oncall plugin (both versions 1.1.5) and it didn't help.

UPD2: I rolled back both versions to 1.1.0 and have the same problem

UPD3: It works correctly when I rolled back to version 1.0.51
it looks like this functionality has been broken since v1.1.0

[Sammyant]

Hi, we have the same issue with the new version (v1.1.19) in docker compose

[joanyko]

Hi, same issue in helm chart (v1.1.0)

[Acetolyne]

please try removing the trailing / from the url I noticed when setting mine up that i could not have the trailing / in the url else I got the same message. If this fixes the issue for you please let us know and I can put in a PR to automatically remove the trailing slash so that the plugin handles these URLs in a better way.

Note that this can also be caused by the url not being publicly accessible by telegram please confirm that the url you are putting in is accessible in your web browser and returns Ok on the page that is served

[kirpozh]

The problem still persists. I tried to proxy traffic through a separate endpoint, I also tried through ngrok, as described in the tutorial video - it does not work.
Removing trailing from the message above didn't help either.
As described in the first message, that only downgrade to a version lower than 1.0.51 works, then I can't do it because of the inability to put this version through compose.
Please help

[vitvord]

We have the same problem. Removing the database and completely reinstalling redis/rabbitmq/oncall does not help.
webhookUrl is publicly available.

oncall app and plugin: v1.1.29

[kuzaxak]

Version: 1.2.22

Same behaviour without tailing /

[alexintech]

I had the same problem. I suppose if for some period TELEGRAM_WEBHOOK_HOST was wrong or publicly unavailable, such error may occur and oncall saves it into database.

When I change TELEGRAM_WEBHOOK_HOST in UI, it successfully registers webhook. It can be verified by using:

curl --location --request POST 'https://api.telegram.org/bot{{token}}/getWebhookInfo'

You should see "url": "{{TELEGRAM_WEBHOOK_HOST}}/telegram/", and if you type some message to your bot, in oncall-engine's logs you will see that message is received by oncall.

The problem is when you change TELEGRAM_WEBHOOK_HOST in UI, it does not clear previous error in database. And other telegram UI functionality looks at the TELEGRAM* variables without errors:

https://github.com/grafana/oncall/blob/f18858882eb8239ec6cfa844c7d676b4125a1211/engine/apps/api/serializers/organization.py#LL117C19-L117C19

I have cleaned error value in base_livesetting table, and after that all is working

update base_livesetting set error = null where name = 'TELEGRAM_WEBHOOK_HOST';

Hope it will help to fix an issue.