[CT-552] [Bug] Rows_affected showing as 1 in run_results.json and in UI

[crystalro0]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

User upgraded to v1.0 and is not seeing the same number of rows_affected in run logging.

Seen in 0.18.1 logging:

00:00:01 | 1 of 1 OK created incremental model GEN.foobar [ [32mSUCCESS 130692981 [0m in 175.96s]

in v1.0 logging:

00:00:01 | 1 of 1 OK created incremental model GEN.foobar [ [32mSUCCESS 1 [0m in 188.32s]

Screenshot of user's snowflake history shows ~130M rows.

Expected Behavior

The rows_affected number should be displaying the actual number (unless bound by limit).

Steps To Reproduce

1. Using 0.18.1 and snowflake: create job and run.
2. Upgrade to 1.0.
3. Re-run job.

Relevant log output

Please ping in slack for additional information if needed. ZD ticket and logging available.

Environment

- dbt: 1.0.4

What database are you using dbt with?

snowflake

Additional Context

No response

[jtcohen6]

Thanks for opening @crystalro0!

I realized this is due to dbt-labs/dbt-core#3510, which was included in v0.21. Previously, the last statement run during incremental transformations was merge or insert, and it would include the number of rows_affected. Now, it's an explicit commit statement (per the advice of Snowflake), which only ever returns 1:

08:21:48.768927 [debug] [Thread-1  ]: On model.testy.model_a: insert into ad_hoc.dbt_jcohen.model_a ("ID")
        (
            select "ID"
            from ad_hoc.dbt_jcohen.model_a__dbt_tmp
        );
08:21:49.480126 [debug] [Thread-1  ]: SQL status: SUCCESS 123 in 0.71 seconds
08:21:49.480654 [debug] [Thread-1  ]: Using snowflake connection "model.testy.model_a"
08:21:49.480949 [debug] [Thread-1  ]: On model.testy.model_a: commit;
08:21:49.844254 [debug] [Thread-1  ]: SQL status: SUCCESS 1 in 0.36 seconds

It's that final SUCCESS 1 which is now reported as the result for the incremental model, rather than SUCCESS 123.

We'd need to see if there's a way we can return the rows_affected from the pre-commit statement(s). It may be tricky, given how we're (naively) wrapping each statement in naive begin + commit to guarantee transaction success and cost-effectiveness. I'm going to transfer this issue to dbt-snowflake, since that's where we'd need to make the change.