-
Notifications
You must be signed in to change notification settings - Fork 196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Indirect write drops policy tags #1043
Labels
enhancement
New feature or request
Comments
khaledh
changed the title
Indirect write drops column nullability and policy tags
Indirect write drops ~column nullability~ and policy tags
Aug 21, 2023
khaledh
changed the title
Indirect write drops ~column nullability~ and policy tags
Indirect write drops policy tags
Aug 21, 2023
I updated the title/description to remove the issue about nullability. It turns out to be an issue on our side. The policy tags, however, are still not populated as expected with indirect writes. |
vishalkarve15
added a commit
to vishalkarve15/spark-bigquery-connector
that referenced
this issue
Jun 25, 2024
vishalkarve15
added a commit
to vishalkarve15/spark-bigquery-connector
that referenced
this issue
Jun 25, 2024
vishalkarve15
added a commit
to vishalkarve15/spark-bigquery-connector
that referenced
this issue
Jun 25, 2024
isha97
added a commit
that referenced
this issue
Jul 3, 2024
The fix will be available in the next release. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We noticed that while this issue is resolved for direct writes (#1028), it still happens when we use indirect writes; the policy tags are dropped after the indirect write.
We also noticed that non-nullable columns are created as nullable when using indirect write.We're trying to work around this issue, but we're hoping it would be fixed in the connector itself.We tried both parquet and avro intermediate formats.
(Original comment: #1028 (comment))
The text was updated successfully, but these errors were encountered: