Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[mypyc] Switch to table-driven imports for smaller IR (python#14917)
Add CPyImport_ImportFromMany() which imports a module and a tuple of
names, placing them in the globals dict in one go.
Previously, each name would imported and placed in globals manually
in IR, leading to some pretty verbose code.
The other option to collect all from imports and perform them all at
once in the helper would remove even more ops, however, it has some
major downsides:
- It wouldn't be able to be done in IRBuild directly, instead being
handled in the prebuild visitor and codegen... which all sounds
really involved.
- It would cause from imports to be performed eagerly, potentially
causing circular imports (especially in functions whose imports are
probably there to avoid a circular import!).
The latter is the nail in the coffin for this idea.
---
Change how imports (not from imports!) are processed so they can be
table-driven (tuple-driven, really) and compact. Here's how it works:
Import nodes are divided in groups (in the prebuild visitor). Each group
consists of consecutive Import nodes:
import mod <| group 1
import mod2 |
def foo() -> None:
import mod3 <- group 2 (*)
import mod4 <- group 3
Every time we encounter the first import of a group, build IR to call
CPyImport_ImportMany() that will perform all of the group's imports in
one go.
(*) Imports in functions or classes are still transformed into the
original,
verbose IR as speed is more important than codesize.
Previously, each module would imported and placed in globals manually
in IR, leading to some pretty verbose code.
The other option to collect all imports and perform them all at once in
the helper would remove even more ops, however, it's problematic for
the same reasons from the previous commit (spoiler: it's not safe).
Implementation notes:
- I had to add support for loading the address of a static directly,
so I shoehorned in LoadStatic support for LoadAddress.
- Unfortunately by replacing multiple import nodes with a single
function call at the IR level, if any import within a group fails,
the traceback line number is static and will be probably wrong
(pointing to the first import node in my original impl.).
To fix this, I had to make CPyImport_ImportMany() add the traceback
entry itself on failure (instead of letting codegen handle it
automatically). This is admittedly ugly.
Overall, this doesn't speed up initialization. The only real speed
impact is that back to back imports in a tight loop seems to be 10-20%
slower. I believe that's acceptable given the code size reduction.
---
**Other changes:**
- Don't declare internal static for non-compiled modules
It won't be read anywhere as the internal statics are only used to
avoid runaway recursion with import cycles in our module init
functions.
- Wrap long RArray initializers and long annotations in codegen
Table-driven imports can load some rather large RArrays and tuple
literals so this was needed to keep the generated C readable.
- Add LLBuilder helper for setting up a RArray
Resolves mypyc/mypyc#591.- Loading branch information
