From e71cf4333987e4bef94fb5d7f55f9162a365bed4 Mon Sep 17 00:00:00 2001 From: Tsubasa Miyamoto Date: Wed, 16 Jul 2025 13:29:38 +0900 Subject: [PATCH 1/2] out_bigquery: add documentation for configurable buffer_size option Signed-off-by: Tsubasa Miyamoto --- pipeline/outputs/bigquery.md | 1 + 1 file changed, 1 insertion(+) diff --git a/pipeline/outputs/bigquery.md b/pipeline/outputs/bigquery.md index e8eb68f71..4aeaece6d 100644 --- a/pipeline/outputs/bigquery.md +++ b/pipeline/outputs/bigquery.md @@ -45,6 +45,7 @@ Fluent Bit streams data into an existing BigQuery table using a service account | `provider_id` | GCP workload identity provider. Used to construct the full resource name of the identity provider. Currently only AWS accounts are supported. | _none_ | | `google_service_account` | The email address of the Google service account to impersonate. The workload identity provider must have permissions to impersonate this service account, and the service account must have permissions to access Google BigQuery resources ( `write` access to tables) | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | +| buffer_size | Sets the upper limit for the HTTP client buffer size (in bytes) when reading API responses from BigQuery. The buffer grows dynamically as needed until it reaches this limit; if a response exceeds this size, any additional data is discarded and a warning is logged. | `4192` | See Google's [official documentation](https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details. From b83c3cafe5c46956ad82eab087ad4013d99ef409 Mon Sep 17 00:00:00 2001 From: Lynette Miles <6818907+esmerel@users.noreply.github.com> Date: Mon, 20 Oct 2025 14:13:46 -0700 Subject: [PATCH 2/2] Update pipeline/outputs/bigquery.md Signed-off-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> --- pipeline/outputs/bigquery.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/bigquery.md b/pipeline/outputs/bigquery.md index 4aeaece6d..f27603b96 100644 --- a/pipeline/outputs/bigquery.md +++ b/pipeline/outputs/bigquery.md @@ -45,7 +45,7 @@ Fluent Bit streams data into an existing BigQuery table using a service account | `provider_id` | GCP workload identity provider. Used to construct the full resource name of the identity provider. Currently only AWS accounts are supported. | _none_ | | `google_service_account` | The email address of the Google service account to impersonate. The workload identity provider must have permissions to impersonate this service account, and the service account must have permissions to access Google BigQuery resources ( `write` access to tables) | _none_ | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | -| buffer_size | Sets the upper limit for the HTTP client buffer size (in bytes) when reading API responses from BigQuery. The buffer grows dynamically as needed until it reaches this limit; if a response exceeds this size, any additional data is discarded and a warning is logged. | `4192` | +| `buffer_size` | Sets the upper limit for the HTTP client buffer size (in bytes) when reading API responses from BigQuery. The buffer grows dynamically as needed until it reaches this limit. If a response exceeds this size, additional data is discarded and a warning is logged. | `4192` | See Google's [official documentation](https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details.