Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Microbatch error #1376

Open
2 tasks done
jschintz-nytimes opened this issue Oct 18, 2024 · 0 comments
Open
2 tasks done

[Bug] Microbatch error #1376

jschintz-nytimes opened this issue Oct 18, 2024 · 0 comments
Labels
bug Something isn't working triage

Comments

@jschintz-nytimes
Copy link

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When running microbatch incremental models, receive the following error

Encountered an error while running operation: Compilation Error argument of type 'ModelNode' is not iterable

Expected Behavior

Model should run

Steps To Reproduce

with this model config

{{ config(
    materialized='incremental',
    incremental_strategy='microbatch',
    partition_by={
      'field': 'date_field',
      'granularity': 'day'
    },
    event_time='date_field',
    batch_size='day',
    begin=modules.datetime.datetime(2024, 10, 16, 0, 0, 0)
    )
}}
select 
    DATE(2024, 10, 16) as date_field

dbt run

Relevant log output

20:20:06 Re-using an available connection from the pool ()
20:20:06 Encountered an error while running operation: Compilation Error
  argument of type 'ModelNode' is not iterable
  
  > in macro bigquery__create_table_as (macros/adapters.sql)
  > called by macro create_table_as (macros/relations/table/create.sql)
  > called by macro bq_create_table_as (macros/materializations/incremental.sql)
  > called by macro statement (macros/etc/statement.sql)
  > called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
  > called by 
20:20:06 Encountered an error while running operation: Compilation Error
  argument of type 'ModelNode' is not iterable
  
  > in macro bigquery__create_table_as (macros/adapters.sql)
  > called by macro create_table_as (macros/relations/table/create.sql)
  > called by macro bq_create_table_as (macros/materializations/incremental.sql)
  > called by macro statement (macros/etc/statement.sql)
  > called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
  > called by 
20:20:06 Encountered an error while running operation: Compilation Error
  argument of type 'ModelNode' is not iterable
  
  > in macro bigquery__create_table_as (macros/adapters.sql)
  > called by macro create_table_as (macros/relations/table/create.sql)
  > called by macro bq_create_table_as (macros/materializations/incremental.sql)
  > called by macro statement (macros/etc/statement.sql)
  > called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
  > called by 
20:20:06 Opening a new connection, currently in state closed
20:20:06 Connection 'master' was properly closed.
20:20:06 Finished running 1 incremental model in 0 hours 0 minutes and 1.09 seconds (1.09s).
20:20:06 Command end result
20:20:06 Completed with 1 error, 0 partial successs, and 0 warnings:
20:20:06 Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
20:20:06 Resource report: {"command_name": "build", "command_wall_clock_time": 1.4175539, "process_user_time": 8.919335, "process_kernel_time": 3.221199, "process_mem_max_rss": "323776", "process_out_blocks": "106816", "command_success": false, "process_in_blocks": "0"}
20:20:06 Observability Metric: command_success=0.0
20:20:06 Observability Metric: command_wall_clock_time=1.4175539016723633
20:20:06 Observability Metric: process_user_time=8.91933536529541
20:20:06 Observability Metric: process_kernel_time=3.2211990356445312
20:20:06 Observability Metric: process_mem_max_rss=323776.0
20:20:06 Command `cli build` failed at 20:20:06.992467 after 1.42 seconds
Running dbt...

Environment

dbt-bigquery: 1.9.0

Additional Context

No response

@jschintz-nytimes jschintz-nytimes added bug Something isn't working triage labels Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant