Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow partial schemas in to_gbq() table_schema #218

Closed
melissachang opened this issue Sep 17, 2018 · 3 comments
Closed

Allow partial schemas in to_gbq() table_schema #218

melissachang opened this issue Sep 17, 2018 · 3 comments
Labels
accepting pull requests type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@melissachang
Copy link
Contributor

I am using pandas with BigQuery tables. Because pandas doesn't support boolean missing values, BigQuery BOOLEAN columns become dataframe object columns. I have to use to_gbq() table_schema to convert force those columns back to BOOLEAN.

My dataframe has 62 columns. table_schema has to include every column. It would be nice if I could only pass in the boolean columns in table_schema; for the other columns, to_gbq should default to dataframe column type.
@tswast tswast added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. accepting pull requests labels Sep 17, 2018
@tswast
Copy link
Collaborator

tswast commented Sep 17, 2018

Currently, if not all columns are specified in the table_schema, you get an error, correct? I think it's reasonable to allow specifying a subset and treating them as overrides, leaving the others unchanged.

@melissachang
Copy link
Contributor Author

Yes, if not all columns are specified, you get an error.

For completeness, I want column order to be preserved. Eg say my df has columns A, B, C, D, E. I pass in table schema with C. I want output BigQuery table to have column order A, B, C, D, E.

@melissachang melissachang mentioned this issue Sep 18, 2018
Closed
@JohnPaton JohnPaton mentioned this issue Mar 12, 2019
Merged
max-sixty pushed a commit that referenced this issue Mar 12, 2019
@JohnPaton @max-sixty
ENH: Allow partial table schema in to_gbq() table_schema (#218) (#257)
547812e 
* ENH: Allow partial table schema in to_gbq

* CLN: applied black

* BUG: make update_schema python 2.7 compatible

* DOC: update docs to allow for a subset of columns in to_gbq table_schema

* DOC: what's new

* DOC: close parens around issue in changelog
@tswast
Copy link
Collaborator

tswast commented May 24, 2019

Closed by #257

@tswast tswast closed this as completed May 24, 2019
@tswast tswast mentioned this issue May 24, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
accepting pull requests type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tswast @melissachang