Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: model execution failed to include multiline message for BigQuery #361

Closed
SPTKL opened this issue Jun 20, 2023 · 0 comments
Closed
Labels
bug Something isn't working

Comments

@SPTKL
Copy link
Contributor

SPTKL commented Jun 20, 2023

Overview

When there's a model execution error in BigQuery, dbt artifacts fails to complete operations because of unenclosed string literals.
BigQuery has multiline messages, currently we are not escaping newline when parsing messages

'{{ model.message | replace("\\", "\\\\") | replace("'", "\\'") | replace('"', '\\"') }}', {# message #}

How to reproduce

  1. create a fake model called fake.sql with the following sql SELECT AS fake (this command is meant to fail
  2. execute the model dbt run -s fake
  3. observe how error is handled by dbt artifacts

Expected behaviour

dbt artifacts should complete upload even when there's a model failure, and dbt artifacts should capture the error in bigquery

Screenshots

generated SQL example:

        insert into `xxxx-xxx-xxx.src__dbt_artifacts.model_executions`
        VALUES
        
        
        
        (
            'xxxxxxx-xxxxx-xxx-xxx-xxxxxxxxxxx', 
            'model.xxx.xxx', 
            '2023-06-20 19:28:54.182664+00:00', 

            
            
                
            
            False, 

            'Thread-1', 
            'error', 

            
                null, 
                null, 
            

            0.8062765598297119, 
            safe_cast('' as int64),
            safe_cast('' as int64),
            'view', 
            'xxxx', 
            'xxxx', 
            'xxxx', 
            'Database Error in model test (models/xxx.sql)
  Syntax error: Unexpected end of script at [10:1]
  compiled Code at target/run/xxx/models/xxx.sql' 
            )
image

console output

19:41:16  Running 1 on-run-end hook
19:41:16  Uploading model executions
19:41:17  BigQuery adapter: https://console.cloud.google.com/bigquery?project=xxxxxxxxxx&j=bq:US:xxxxxx-xxxx-xxxx-xxxx-xxxxxxx&page=queryresults
19:41:17  Database error while running on-run-end
19:41:17  Encountered an error:
Database Error
  Syntax error: Unclosed string literal at [35:13]

Environment

Results of running dbt --version:

dbt=1.3.4

Please paste the contents of your packages.yml file here:

packages:
  - package: dbt-labs/codegen
    version: 0.9.0
  - package: calogica/dbt_expectations
    version: [">=0.8.0", "<0.9.0"]
  - package: dbt-labs/dbt_utils
    version: 1.1.0
  - package: brooklyn-data/dbt_artifacts
    version: 2.4.2
@SPTKL SPTKL added the bug Something isn't working label Jun 20, 2023
@SPTKL SPTKL closed this as completed Aug 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant