Skip to content

API: ensure read_json closes file handle #49921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 2, 2022

Conversation

topper-123
Copy link
Contributor

@topper-123 topper-123 commented Nov 26, 2022

Use context manager instead of closing manually with self.close to ensure file always closes.

Not sure if this should have a whatsnew entry and where.

Copy link
Member

@MarcoGorelli MarcoGorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me (easier to review with the "hide whitespace" option)

@MarcoGorelli MarcoGorelli added the IO JSON read_json, to_json, json_normalize label Nov 27, 2022
@MarcoGorelli MarcoGorelli added this to the 2.0 milestone Nov 27, 2022
@topper-123
Copy link
Contributor Author

easier to review with the "hide whitespace" option

Nice one, I didn't know about that one. It does indeed makes changes that are mostly changing the indention level easier to review.

@MarcoGorelli
Copy link
Member

Yeah I don't think this needs a whatsnew entry - leaving open for a bit in case others have comments / disagree

@mroeschke
Copy link
Member

Could note in the Other API changes that read_json will always close the file if there was an error in reading the file?

@topper-123
Copy link
Contributor Author

Thanks for the comments.

I've added a whatsnew entry + updated the PR to also improve the handling of closing files in JsonReader.__next__ also.

Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be easy to test that the files handles are closed when an exception is encountered?

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>
@topper-123
Copy link
Contributor Author

Would it be easy to test that the files handles are closed when an exception is encountered?

not sure, maybe serving a malformed file (though haven't checked if that is tested already or maybe caught in the init)?

@topper-123
Copy link
Contributor Author

error looks unrelated ("ERROR: Could not find a version that satisfies the requirement setuptools==59.2.0")

@mroeschke mroeschke merged commit e5fb655 into pandas-dev:main Dec 2, 2022
@mroeschke
Copy link
Member

Thanks @topper-123. If you have any inspirations on how to test that files are closed on exceptions happy to have a follow up

@topper-123 topper-123 deleted the json_reader_read branch December 5, 2022 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants