Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] fivetran_log_json_parse.string_path arg should be an array in fivetran_platform__audit_user_activity model #124

Closed
2 of 4 tasks
fivetran-jamie opened this issue May 6, 2024 · 0 comments · Fixed by #126
Assignees
Labels
error:unforced priority:p3 Affects many users; can wait status:scoping Currently being scoped type:bug Something is broken or incorrect update_type:models Primary focus requires model updates

Comments

@fivetran-jamie
Copy link
Contributor

fivetran-jamie commented May 6, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Describe the issue

On line 5 we have
{{ fivetran_log.fivetran_log_json_parse(string='message_data', string_path='actor') }} as actor_email

But it should be
{{ fivetran_log.fivetran_log_json_parse(string='message_data', string_path=['actor']) }} as actor_email

This is how the macro is used everywhere else.

This causes the json parsing to not output anything and therefore limit the output of the fivetran_platform__audit_user_activity model

Relevant error log or model output

Compiled fivetran_platform__audit_user_activity code for Snowflake:
try_parse_json(message_data)['a']['c']['t']['o']['r'] as actor_email

For Postgres:
  case when message_data ~ '^\s*[\{].*[\}]?\s*$' -- Postgres has no native json check, so this will check the string for indicators of a JSON object
    then message_data::json #>> '{a,c,t,o,r}'
    else null end

For BQ:
json_extract_scalar(message_data, '$.a.c.t.o.r')

etc.

Expected behavior

I would expect the macro to compile to:

Snowflake
try_parse_json(message_data)['actor'] as actor_email

Postgres
case when message_data ~ '^\s*[\{].*[\}]?\s*$' -- Postgres has no native json check, so this will check the string for indicators of a JSON object then message_data::json #>> '{actor}' else null end

BQ
json_extract_scalar(message_data, '$.actor')

dbt Project configurations

na

Package versions

na

What database are you using dbt with?

snowflake

dbt Version

na

Additional Context

https://fivetran.slack.com/archives/C02919TN9AT/p1714746313138259

Are you willing to open a PR to help address this issue?

  • Yes.
  • Yes, but I will need assistance and will schedule time during our office hours for guidance
  • No.
@fivetran-jamie fivetran-jamie added type:bug Something is broken or incorrect priority:p3 Affects many users; can wait status:scoping Currently being scoped update_type:models Primary focus requires model updates error:unforced labels May 6, 2024
@fivetran-catfritz fivetran-catfritz self-assigned this May 13, 2024
@fivetran-catfritz fivetran-catfritz linked a pull request May 13, 2024 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error:unforced priority:p3 Affects many users; can wait status:scoping Currently being scoped type:bug Something is broken or incorrect update_type:models Primary focus requires model updates
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants