It should be possible to respond in one thread, we should demonstrate this: https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/#threads
Additionally, we can enable it as a config option
Which will send one thread per pipeline for the automated messages.