FLB_THROTTLE return for outputs

[PettitWesley]

Fluent Bit's behavior when an output is throttled is not ideal.

If one of my outputs gets a throttling error, I only have 3 choices for what to tell the core pipeline:

• FLB_OK: the entire chunk was successfully sent/handled
• FLB_RETRY: retry the entire chunk up to the max user configured retries
• FLB_ERROR: something went very wrong, don't retry

If I return FLB_RETRY, then that specific chunk will be retried with exponential backoff. However, the pipeline will continue to send my output new chunks. Thus, from the POV of my destination, I am not slowing down and backing off at all, I am continuing to send new data.

Thus, I think we need an FLB_THROTTLE return which would be like a special case for FLB_RETRY. It would tell the core to freeze the output for some period of time, and not send any new chunks. Basically, the core should backoff the entire output instance.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[PettitWesley]

I still want this.

[wchaws]

related issue: #6369