Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a MRO walker for slot methods and Convert _csv to use PEP 573 #4

Closed
wants to merge 120 commits into from

Conversation

encukou
Copy link
Owner

@encukou encukou commented May 13, 2020

No description provided.

scoder and others added 30 commits May 12, 2020 07:12
…ythonGH-20024)

… now that we allow subtypes of PyCFunction.

Also add PyCMethod_CheckExact() and PyCMethod_Check() for checks against the PyCMethod subtype.
Add a new _Py_HashPointerRaw() function which avoids replacing -1
with -2 to micro-optimize hash table using pointer keys: using
_Py_hashtable_hash_ptr() hash function.
Rewrite how the _tracemalloc module stores traces of other domains.
Rather than storing the domain inside the key, it now uses a new hash
table with the domain as the key, and the data is a per-domain traces
hash table.

* Add tracemalloc_domain hash table.
* Remove _Py_tracemalloc_config.use_domain.
* Remove pointer_t and related functions.
Rewrite _Py_hashtable_t type to always store the key as
a "const void *" pointer. Add an explicit "key" member to
_Py_hashtable_entry_t.

Remove _Py_hashtable_t.key_size member.

hash and compare functions drop their hash table parameter, and their
'key' parameter type becomes "const void *".
Add key_destroy_func and value_destroy_func parameters to
_Py_hashtable_new_full().

marshal.c and _tracemalloc.c use these destroy functions.
Rewrite _tracemalloc to store "trace_t*" rather than directly
"trace_t" in traces hash tables. Traces are now allocated on the heap
memory, outside the hash table.

Add tracemalloc_copy_traces() and tracemalloc_copy_domains() helper
functions.

Remove _Py_hashtable_copy() function since there is no API to copy a
key or a value.

Remove also _Py_hashtable_delete() function which was commented.
_Py_hashtable_t values become regular "void *" pointers.

* Add _Py_hashtable_entry_t.data member
* Remove _Py_hashtable_t.data_size member
* Remove _Py_hashtable_t.get_func member. It is no longer needed
  to specialize _Py_hashtable_get() for a specific value size, since
  all entries now have the same size (void*).
* Remove the following macros:

  * _Py_HASHTABLE_GET()
  * _Py_HASHTABLE_SET()
  * _Py_HASHTABLE_SET_NODATA()
  * _Py_HASHTABLE_POP()

* Rename _Py_hashtable_pop() to _Py_hashtable_steal()
* _Py_hashtable_foreach() callback now gets key and value rather than
  entry.
* Remove _Py_hashtable_value_destroy_func type. value_destroy_func
  callback now only has a single parameter: data (void*).
_Py_hashtable_get_entry_ptr() avoids comparing the entry hash:
compare directly keys.

Move _Py_hashtable_get_entry_ptr() just after
_Py_hashtable_get_entry_generic().
This commit fixes SyntaxError locations when the caret is not displayed,
by doing the following:

- `col_number` always gets set to the location of the offending
  node/expr. When no caret is to be displayed, this gets achieved
  by setting the object holding the error line to None.

- Introduce a new function `_PyPegen_raise_error_known_location`,
  which can be called, when an arbitrary `lineno`/`col_offset`
  needs to be passed. This function then gets used in the grammar
  (through some new macros and inline functions) so that SyntaxError
  locations of the new parser match that of the old.
Don't hardcode defining_class parameter name to "cls":

* Define CConverter.set_template_dict(): do nothing by default
* CLanguage.render_function() now calls set_template_dict() on all
  converters.
…onGH-18007)

The crash that this fixes occurs when using traceback and other modules from multiple threads; 
del cache[filename] can raise a KeyError.
…H-20078)

Remove the _PyUnicode_ClearStaticStrings() function from the C API.
Make the function fully private (declare it with "static").
…H-19858)

The previous commits on bpo-29587 got exception chaining working
with gen.throw() in the `yield` case. This patch also gets the
`yield from` case working.

As a consequence, implicit exception chaining now also works in
the asyncio scenario of awaiting on a task when an exception is
already active.

Tests are included for both the asyncio case and the pure
generator-only case.
Move PyInterpreterState.fs_codec into a new
PyInterpreterState.unicode structure.

Give a name to the fs_codec structure and use this structure in
unicodeobject.c.
…nsistently (pythongh-20038)

raw_data_manager (default for EmailPolicy, EmailMessage)
does correct wrapping of 'text' parts as long as the message contains
characters outside of 7bit US-ASCII set: base64 or qp
Content-Transfer-Encoding is applied if the lines would be too long
without it.  It did not, however, do this for ascii-only text,
which could result in lines that were longer than
policy.max_line_length or even the rfc 998  maximum.

This changeset fixes the heuristic so that if lines are longer than
policy.max_line_length, it will always apply a
content-transfer-encoding so that the lines are wrapped correctly.
…H-19901)

compileall is now able to use hardlinks to prevent duplicates in a
case when .pyc files for different optimization levels have the same content.

Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Victor Stinner <vstinner@python.org>
Pass PEP 573 defining_class to os.DirEntry methods. The module state
is now retrieve from defining_class rather than Py_TYPE(self), to
support subclasses (even if DirEntry doesn't support subclasses yet).

* Pass the module rather than defining_class to DirEntry_fetch_stat().
* Only get the module state once in _posix_clear(),
  _posix_traverse() and _posixmodule_exec().
…GH-19768)" (pythonGH-20089)

* Revert "bpo-40613: Remove compiler warning from _xxsubinterpretersmodule (pythonGH-20069)"

This reverts commit fa0a66e.

* Revert "bpo-32604: [_xxsubinterpreters] Propagate exceptions. (pythonGH-19768)"

This reverts commit a1d9e0a.
Cleanup also hashtable.c.
Rename _Py_hashtable_t members:

* Rename entries to nentries
* Rename num_buckets to nbuckets
…ythonGH-20083)

This commit fixes the new parser to disallow invalid targets in the
following scenarios:
- Augmented assignments must only accept a single target (Name,
  Attribute or Subscript), but no tuples or lists.
- `except` clauses should only accept a single `Name` as a target.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
If _Py_hashtable_set() fails to grow the hash table (rehash), it now
fails rather than ignoring the error.
serhiy-storchaka and others added 26 commits May 18, 2020 20:25
Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\n𗶽`.

Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
…ythonGH-19968)

- Switch from getopt to argparse.
- Removed the limitation of not being able to produce both C and H simultaneously.

This will make it run faster since it parses the asdl definition once and uses the generated tree to generate both the header and the C source.
…plete location information (pythonGH-20157)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
…20183)

Create a `make venv` target, that creates a virtual environment
and installs the dependency in that venv. `make time` and all
the related targets are changed to use the virtual environment
python.

Automerge-Triggered-By: @pablogsal
…thonGH-20166)

Unprintable characters such as `\x00` weren't correctly roundtripped
due to not using default string repr when generating docstrings. This
patch correctly encodes all unprintable characters (except `\n` and `\t`, which
are commonly used for formatting, and found unescaped).

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
When unparsing a non-empty tuple, the parentheses can be safely
omitted if there aren't any elements that explicitly require them (such as starred expressions).
Make the the following imports lazy in test.support:

* bz2
* gzip
* lzma
* resource
* zlib

The following test.support decorators now need to be called
with parenthesis:

* @support.requires_bz2
* @support.requires_gzip
* @support.requires_lzma
* @support.requires_zlib

For example, "@requires_zlib" becomes "@requires_zlib()".
_PyErr_ChainStackItem was just added in pythonGH-19951 (for bpo-31033).
```
Warning, treated as error:
/tmp/code/Doc/library/zoneinfo.rst:303:Error in "code-block" directive:
1 argument(s) required, 0 supplied.

.. code-block::

    >>> a = ZoneInfo("Europe/Berlin")
    >>> b = pickle.loads(europe_berlin_pkl)
    >>> a is b
    True
```
Implements `asyncio.to_thread`, a coroutine for asynchronously running IO-bound functions in a separate thread without blocking the event loop. See the discussion starting from [here](python#18410 (comment)) in pythonGH-18410 for context.

Automerge-Triggered-By: @aeros
…ythonGH-20205)

Use text/x-python instead of text/plain to avoid issues with tools assuming that "ShellExecute(script)" is a non-executable operation.
This allows building with older versions of the Windows SDK where the value is not defined.
Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
See: python#16078
This is slightly nicer.

Co-authored-by: Eddie Elizondo <eduardo.elizondorueda@gmail.com>
Modules/_csv.c Show resolved Hide resolved
Modules/_csv.c Show resolved Hide resolved
@encukou encukou closed this Oct 20, 2020
@encukou
Copy link
Owner Author

encukou commented Oct 21, 2020

Thanks, Shi Hai!
I've made the change and posted this as python#22835

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.