Skip to content

[Bug]: BigQueryIO streaming read issue #36911

@stankiewicz

Description

@stankiewicz

What happened?

Probably due to redistributing, it tries to write to already materialized table

INFO 2025-11-26T17:31:50.349845Z Caused by: java.lang.IllegalArgumentException: Refusing to write on existing table temp_table_beam_bq_job_QUERY_batchwithimpulsenostate_a9ef0206b6f049a0992428af8bf9488b in the specified temp dataset xxxxx_beam_jobs
INFO 2025-11-26T17:31:50.349861Z at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:445)
INFO 2025-11-26T17:31:50.349878Z at org.apache.beam.sdk.io.gcp.bigquery.BigQueryQueryHelper.executeQuery(BigQueryQueryHelper.java:157)
INFO 2025-11-26T17:31:50.349893Z at org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageQuerySource.getTargetTable(BigQueryStorageQuerySource.java:169)
INFO 2025-11-26T17:31:50.349908Z at org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO$DynamicRead$CreateBoundedSourceForTable.processElement(BigQueryIO.java:982)

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions