Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-1635741: pyexpat error handling #22489

Merged
merged 8 commits into from
Nov 4, 2020

Conversation

koubaa
Copy link
Contributor

@koubaa koubaa commented Oct 2, 2020

Preparation for multi-phase init. See #22222 for origin of most of these changes

https://bugs.python.org/issue1635741

@koubaa
Copy link
Contributor Author

koubaa commented Oct 2, 2020

@vstinner please review

Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
@vstinner
Copy link
Member

vstinner commented Oct 2, 2020

Since you plan to convert pyexpat to multiphase init, could you already add an "exec" function, and modify the PyInit to call it? So later you will have no replace all "goto error" with "return -1".

@koubaa
Copy link
Contributor Author

koubaa commented Oct 12, 2020

@vstinner this is ready now

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, add_submodule() is nice. But my idea was to go further and put all code using errors_module in a sub-function and call it from the init function. Same for model_module.

@koubaa
Copy link
Contributor Author

koubaa commented Oct 14, 2020

Thanks, add_submodule() is nice. But my idea was to go further and put all code using errors_module in a sub-function and call it from the init function. Same for model_module.

Oh - sure I can do that.

@koubaa
Copy link
Contributor Author

koubaa commented Oct 19, 2020

@vstinner done

Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it's way better! Variables have a shorter scopes, error handling is easier to review. More comments.

Modules/pyexpat.c Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Show resolved Hide resolved
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but please address the few minor remaining issues before I can merge your change.

Oh wow, I didn't expect a so large PR to add error handling to PyInit_pyexpat()! Maybe there was a reason to ignore errors :-D It's complex to handle errors in C :-(

I like the new design of the code with many smaller functions and more narrow scope for variables. It's way simpler to track the lifetime of each variable, and so check manually for reference leaks!

Note: I still hate the INCREF/DECREF required by PyModule_AddObject() (that's unrelated to your PR, I just hate this function.)

Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
Modules/pyexpat.c Outdated Show resolved Hide resolved
@koubaa
Copy link
Contributor Author

koubaa commented Nov 4, 2020

@vstinner done

@vstinner vstinner merged commit 7184218 into python:master Nov 4, 2020
@vstinner
Copy link
Member

vstinner commented Nov 4, 2020

Merged. I rewrote your PR title and description.

shihai1991 added a commit to shihai1991/cpython that referenced this pull request Nov 5, 2020
* master:
  bpo-42260: Add _PyInterpreterState_SetConfig() (pythonGH-23158)
  Disable peg generator tests when building with PGO (pythonGH-23141)
  bpo-1635741: _sqlite3 uses PyModule_AddObjectRef() (pythonGH-23148)
  bpo-1635741: Fix PyInit_pyexpat() error handling (pythonGH-22489)
  bpo-42260: Main init modify sys.flags in-place (pythonGH-23150)
  bpo-1635741: Fix ref leak in _PyWarnings_Init() error path (pythonGH-23151)
  bpo-1635741: _ast uses PyModule_AddObjectRef() (pythonGH-23146)
  bpo-1635741: _contextvars uses PyModule_AddType() (pythonGH-23147)
  bpo-42260: Reorganize PyConfig (pythonGH-23149)
  bpo-1635741: Add PyModule_AddObjectRef() function (pythonGH-23122)
  bpo-42236: os.device_encoding() respects UTF-8 Mode (pythonGH-23119)
  bpo-42251: Add gettrace and getprofile to threading (pythonGH-23125)
  Enable signing of nuget.org packages and update to supported timestamp server (pythonGH-23132)
  Fix incorrect links in ast docs (pythonGH-23017)
  Add _PyType_GetModuleByDef (pythonGH-22835)
  Post 3.10.0a2
  bpo-41796: Call _PyAST_Fini() earlier to fix a leak (pythonGH-23131)
  bpo-42249: Fix writing binary Plist files larger than 4 GiB. (pythonGH-23121)
  bpo-40077: Convert mmap.mmap static type to a heap type (pythonGH-23108)
  Python 3.10.0a2
shihai1991 added a commit to shihai1991/cpython that referenced this pull request Nov 7, 2020
…lots1

* origin/master: (80 commits)
  bpo-42282: Fold constants inside named expressions (pythonGH-23190)
  bpo-41028: Doc: Move switchers to docsbuild-scripts. (pythonGH-20969)
  bpo-42133: update parts of the stdlib to fall back to `__spec__.loader` when `__loader__` is missing (python#22929)
  Remove outdated reference to pywin32 from platform module (pythonGH-22005)
  bpo-41832: PyType_FromModuleAndSpec() now accepts NULL tp_doc (pythonGH-23123)
  Minor grammar edits for the descriptor howto guide (GH-python#23175)
  bpo-42179: Doc/tutorial: Remove mention of __cause__ (pythonGH-23162)
  bpo-26389: Allow passing an exception object in the traceback module (pythonGH-22610)
  bpo-42260: PyConfig_Read() only parses argv once (pythonGH-23168)
  bpo-42260: Add _PyConfig_FromDict() (pythonGH-23167)
  bpo-41877 Check for asert, aseert, assrt in mocks (pythonGH-23165)
  [docs] fix wrongly named AsyncContextDecorator (pythonGH-23164)
  bpo-42262: Add Py_NewRef() and Py_XNewRef() (pythonGH-23152)
  bpo-42266: Handle monkey-patching descriptors in LOAD_ATTR cache (pythonGH-23157)
  bpo-40816 Add AsyncContextDecorator class (pythonGH-20516)
  bpo-42260: Add _PyInterpreterState_SetConfig() (pythonGH-23158)
  Disable peg generator tests when building with PGO (pythonGH-23141)
  bpo-1635741: _sqlite3 uses PyModule_AddObjectRef() (pythonGH-23148)
  bpo-1635741: Fix PyInit_pyexpat() error handling (pythonGH-22489)
  bpo-42260: Main init modify sys.flags in-place (pythonGH-23150)
  ...
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
Split PyInit_pyexpat() into sub-functions and fix reference leaks
on error paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants