[Feature Request]: Add support for BigQuery Storge Write API in Python

[esciara]

What would you like to happen?

Much like #20687 to read to BigQuery, The Java SDK contains an implementation which uses the BigQuery Storage API to write to BigQuery. We should implement the same for Python.

In our case, we would like to use it instead of using the FILE_LOADS ingestion method in streaming mode, where errors during the loading process are not propagated to the pipeline. We are currently using it as we do not need STREAMING_INSERT features, and hence want to save on cost.

Issue Priority

Priority: 3

Issue Component

Component: io-py-gcp

[rizenfrmtheashes]

I took a look at this, and it seems like the bigquery storage api has a straightforward jsonstreamwriter but for java. for python, it only exists as protobufs. Hence why the bigquery storage write api for dataflow in python is missing, but exists in java. That api will likely need to be made before the dataflow functions can be made by the core contributors group. Sad that this blog post made it seem like it was in GA for everything.

[kennknowles]

@chamikaramj FYI for if/when this can be done via xlang