Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bytes_billed to BigQueryAdapterResponse #570

Conversation

bruno-szdl
Copy link
Contributor

@bruno-szdl bruno-szdl commented Feb 28, 2023

resolves #560

Description

This PR adds the information of bytes_billed to the adapter response. This was made to have better monitoring of BQ's costs. (more detailed explanation of bytes_billed in #560).

Tests

I tested it in a local project. The bytes_billed can be seen in the response of the adapter, and they are different from bytes_processed in this case because the minimum bytes billed by BQ is 10MB. I just removed some information like model name and profiles directory.

{
    "metadata": {
        "dbt_schema_version": "https://schemas.getdbt.com/dbt/run-results/v4.json",
        "dbt_version": "1.2.4",
        "generated_at": "2023-02-28T01:39:04.120437Z",
        "invocation_id": "a33bc4ad-581e-445e-bc41-64471d9b92ea",
        "env": {}
    },
    "results": [
        {
            "status": "success",
            "timing": [
                {
                    "name": "compile",
                    "started_at": "2023-02-28T01:39:00.186375Z",
                    "completed_at": "2023-02-28T01:39:00.205345Z"
                },
                {
                    "name": "execute",
                    "started_at": "2023-02-28T01:39:00.215123Z",
                    "completed_at": "2023-02-28T01:39:04.114279Z"
                }
            ],
            "thread_id": "Thread-1",
            "execution_time": 3.929593086242676,
            "adapter_response": {
                "_message": "CREATE TABLE (1.3k rows, 173.7 KB processed)",
                "code": "CREATE TABLE",
                "rows_affected": 1267,
                "bytes_processed": 177910,
                "bytes_billed": 10485760
            },
            "message": "CREATE TABLE (1.3k rows, 173.7 KB processed)",
            "failures": null,
            "unique_id": ""
        }
    ],
    "elapsed_time": 7.412775993347168,
    "args": {
        "write_json": true,
        "use_colors": true,
        "printer_width": 80,
        "version_check": true,
        "partial_parse": true,
        "static_parser": true,
        "profiles_dir": ""
        "send_anonymous_usage_stats": true,
        "event_buffer_size": 100000,
        "quiet": false,
        "no_print": false,
        "cache_selected_only": false,
        "select": [
            ""
        ],
        "which": "run",
        "rpc_method": "run",
        "indirect_selection": "eager"
    }
}

Checklist

@bruno-szdl bruno-szdl requested a review from a team as a code owner February 28, 2023 12:41
@cla-bot cla-bot bot added the cla:yes label Feb 28, 2023
@Fleid Fleid added ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering pr_tracked labels Mar 7, 2023
@Fleid Fleid linked an issue Mar 7, 2023 that may be closed by this pull request
@McKnight-42 McKnight-42 self-requested a review March 14, 2023 20:45
@McKnight-42
Copy link
Contributor

@cla-bot check

@cla-bot
Copy link

cla-bot bot commented Mar 20, 2023

The cla-bot has been summoned, and re-checked this pull request!

Copy link
Contributor

@McKnight-42 McKnight-42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like agreat addition not seeing any problems while doing local testing

@McKnight-42 McKnight-42 merged commit 1cb3c7c into dbt-labs:main Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering
Projects
None yet
3 participants