Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[draft] register machine #100276

Closed
wants to merge 77 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
5e0ef80
add oparg2, oparg3 to the bytecode
iritkatriel Dec 15, 2022
8f30b8a
Add consts with borrowed refs to the end of the localsplus array.
iritkatriel Nov 25, 2022
e7bc5f2
add register versions of the UNARY_OPS
iritkatriel Dec 13, 2022
e9c6474
fix error check
iritkatriel Dec 16, 2022
12a37f5
remove i_stackdepth and STACK_REG, we won't use it
iritkatriel Dec 16, 2022
68b9b4b
add oparg2/oparg3 option to the _Py_CODEUNIT union
iritkatriel Dec 16, 2022
2686f86
add LOAD/STORE_FAST_R to help debugging. Fix bug in write_instr
iritkatriel Dec 16, 2022
bb7467b
sanitizer warnings
iritkatriel Dec 16, 2022
50dfc14
int--> Py_ssize_t
iritkatriel Dec 16, 2022
9dbb32e
fix error check
iritkatriel Dec 16, 2022
32b35d0
put the tmp registers after the stack
iritkatriel Dec 17, 2022
1e8531b
Merge remote-tracking branch 'upstream/main' into regmachine
iritkatriel Dec 17, 2022
e6461f1
fix bug in deopt_code
iritkatriel Dec 17, 2022
2439dff
temps are locals named (still need to revert the co_ntmps business, …
iritkatriel Dec 18, 2022
92e04e9
remove ntmps from code object and frame
iritkatriel Dec 18, 2022
9f57f0e
fix test_peepholer
iritkatriel Dec 18, 2022
ed7b38f
fix dis._get_co_positions
iritkatriel Dec 18, 2022
3ee9a3c
fix test_datetime
iritkatriel Dec 18, 2022
1461ebc
Add LOAD_CONST_R . Fix two bugs: _PyFrame_PushUnchecked does not copy…
iritkatriel Dec 20, 2022
1e83cb2
Enhance generator for register instructions (#48)
gvanrossum Dec 20, 2022
f100337
always use register versions of unary ops
iritkatriel Dec 20, 2022
ecc8a73
teach dis about LOAD_CONST_R
iritkatriel Dec 20, 2022
b86a450
wipe out oparg1,2,3 in INSTR_SET_OP0/1
iritkatriel Dec 20, 2022
d90af0c
handle EXTENDED_ARGS
iritkatriel Dec 21, 2022
44a9f4b
enable LOAD_CONST_R always
iritkatriel Dec 21, 2022
a1ebba7
skip hacky test
iritkatriel Dec 21, 2022
313b6e7
regen-all
iritkatriel Dec 21, 2022
ac0bb65
rewrite other unary-ops as register insts
iritkatriel Dec 21, 2022
603b265
add BINARY_OP_R (not specialized yet)
iritkatriel Dec 21, 2022
53facae
Revert "add BINARY_OP_R (not specialized yet)"
iritkatriel Dec 22, 2022
8e37f3d
OPSIZE parameterized by the opcode
iritkatriel Dec 22, 2022
7bb8104
move next_instr updates to where so that we always know which opcode'…
iritkatriel Dec 22, 2022
107c557
set VERBOSE to 0
iritkatriel Dec 22, 2022
cf7fc4b
regen-cases
iritkatriel Dec 23, 2022
a7efdd7
Revert "regen-cases"
iritkatriel Dec 23, 2022
dea531c
Revert "set VERBOSE to 0"
iritkatriel Dec 23, 2022
c437a64
Revert "move next_instr updates to where so that we always know which…
iritkatriel Dec 23, 2022
453bbbe
Disable specialization
iritkatriel Dec 23, 2022
ff80053
freeze only importlib
iritkatriel Dec 23, 2022
7fbeb6a
move next_instr updates to where we always know which opcode's OPSIZE…
iritkatriel Dec 24, 2022
1378f80
fix extended_arg
iritkatriel Dec 26, 2022
f785019
indent of #ifdefs
iritkatriel Dec 26, 2022
1ee1006
remove the ENABLE_SPECIALIZATION from generate_cases.py
iritkatriel Dec 26, 2022
fdc73cb
Merged main into regmachine branch (#50)
gvanrossum Dec 28, 2022
96aaf43
Revert " Merged main into regmachine branch (#50)"
iritkatriel Dec 28, 2022
2753cb4
Merge remote-tracking branch 'upstream/main' into regmachine
iritkatriel Dec 28, 2022
8f945db
skip tests that require frozen module (which we disabled in this branch)
iritkatriel Dec 28, 2022
d294fbe
test_specialized_static_code_gets_unspecialized_at_Py_FINALIZE requir…
iritkatriel Dec 28, 2022
e464a49
make test_asyncio fail instead of hang
iritkatriel Dec 28, 2022
0f8c4d3
add back BINARY_OP_R as a 3-word instruction
iritkatriel Dec 28, 2022
4f51677
update dis.py to use the parameterized _oparg
iritkatriel Dec 28, 2022
d8be0f9
fix _get_co_positions
iritkatriel Dec 28, 2022
b88cc45
fix a couple of issues in dis. Fix test_dis (all but two tests passing)
iritkatriel Dec 29, 2022
02dd61b
add COMPARE_OP_R
iritkatriel Dec 29, 2022
1a61812
add JUMP_IF_FALSE_R/JUMP_IF_TRUE_R
iritkatriel Dec 29, 2022
2a8e12a
make test_patma work (might revert this if we get rid of the tmp vars)
iritkatriel Dec 29, 2022
8d8772f
used 'unused' to prevent lookup generation for first arg of JUMP_IF_F…
iritkatriel Dec 29, 2022
121ccfb
added COPY_R and a simple peephole optimization
iritkatriel Dec 29, 2022
f33d835
Revert "make test_patma work (might revert this if we get rid of the …
iritkatriel Dec 29, 2022
bade862
filter 569XZilmstmps out of locals()
iritkatriel Dec 29, 2022
7776088
remove 569XZilmstmps from f_locals in test
iritkatriel Dec 29, 2022
6a72d8a
filter from f_locals in inspect
iritkatriel Dec 29, 2022
172d12d
filter from f_locals in test_frame
iritkatriel Dec 29, 2022
f864be7
Revert "filter 569XZilmstmps out of locals()"
iritkatriel Dec 29, 2022
9e0d60e
filter locals in test_patma
iritkatriel Dec 29, 2022
d2aceb1
update a few more tests
iritkatriel Dec 29, 2022
df7ffdf
update test_dis. dis code_info now filters out $ vars
iritkatriel Dec 30, 2022
905eac2
merge LOAD_*_R with STORE_FAST_R across COPY_Rs
iritkatriel Dec 30, 2022
7399686
improve register peepholer: squash load+store across COPY_R and NOPs.…
iritkatriel Dec 30, 2022
0f31185
add RETURN_VALUE_R
iritkatriel Dec 30, 2022
dc9b211
update some tests
iritkatriel Dec 31, 2022
a2d7122
added CHECK_FAST_R and fixed bug with unbounded locals access
iritkatriel Dec 31, 2022
d5830af
Merge remote-tracking branch 'upstream/main' into regmachine
iritkatriel Jan 1, 2023
9ff6911
STORE_FAST_R instead of more STORE_FASTs. Fixed peepholer to not opti…
iritkatriel Jan 2, 2023
599de74
typo
iritkatriel Jan 2, 2023
b9af7ac
Revert "typo"
iritkatriel Jan 2, 2023
a623a06
Revert "STORE_FAST_R instead of more STORE_FASTs. Fixed peepholer to …
iritkatriel Jan 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 20 additions & 20 deletions Lib/test/test_patma.py
Original file line number Diff line number Diff line change
Expand Up @@ -2172,7 +2172,7 @@ def test_patma_204(self):
def f(w):
match w:
case 42:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
Copy link
Member

@gvanrossum gvanrossum Dec 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the exception for $ should be implemented in locals() rather than in its callers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t know if we want to keep the $ vars. we did this because I couldn’t get it to work with a dedicated range for temps, but now we fixed it for consts so it should work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want these for debugging purposes. Perhaps locals/frame.f_locals should omit them, and we add frame._f_registers (or something) which is a list of temporaries?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe relevant: the unnamed iterator used in comprehensions is currently visible:

>>> [locals() for _ in [None]]
[{'.0': <tuple_iterator object at 0x7f1a741370a0>, '_': None}]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The $ registers will be visible in a lot more cases than the .0 iterator, and they are a detail of the implementation that will mostly just confuse users. If we're going to pull the consistency card I'd vote for not revealing that iterator either.

I could live with the registers showing in f_locals but not in locals() -- frame attributes are more of an implementation detail anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the change to filter locals(), but that's no good it breaks locals()["x"] = 43.

(see test.test_scope.ScopeTests.testClassNamespaceOverridesClosure)

Copy link
Member

@markshannon markshannon Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we adding all these "$" variables to the locals?
We don't expose the stack, why expose the temporaries?

It looks like a considerable amount of the changes to both code and tests are supporting/working around these additional, artificial local variables.

del out["w"]
return out
self.assertEqual(f(42), {})
Expand All @@ -2184,7 +2184,7 @@ def test_patma_205(self):
def f(w):
match w:
case 42.0:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f(42.0), {})
Expand All @@ -2196,7 +2196,7 @@ def test_patma_206(self):
def f(w):
match w:
case 1 | 2 | 3:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f(1), {})
Expand All @@ -2211,7 +2211,7 @@ def test_patma_207(self):
def f(w):
match w:
case [1, 2] | [3, 4]:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f([1, 2]), {})
Expand All @@ -2225,7 +2225,7 @@ def test_patma_208(self):
def f(w):
match w:
case x:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f(42), {"x": 42})
Expand All @@ -2236,7 +2236,7 @@ def test_patma_209(self):
def f(w):
match w:
case _:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f(42), {})
Expand All @@ -2247,7 +2247,7 @@ def test_patma_210(self):
def f(w):
match w:
case (x, y, z):
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f((1, 2, 3)), {"x": 1, "y": 2, "z": 3})
Expand All @@ -2264,7 +2264,7 @@ def test_patma_211(self):
def f(w):
match w:
case {"x": x, "y": "y", "z": z}:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f({"x": "x", "y": "y", "z": "z"}), {"x": "x", "z": "z"})
Expand All @@ -2276,7 +2276,7 @@ def test_patma_212(self):
def f(w):
match w:
case Point(int(xx), y="hello"):
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f(Point(42, "hello")), {"xx": 42})
Expand All @@ -2285,7 +2285,7 @@ def test_patma_213(self):
def f(w):
match w:
case (p, q) as x:
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["w"]
return out
self.assertEqual(f((1, 2)), {"p": 1, "q": 2, "x": (1, 2)})
Expand All @@ -2297,56 +2297,56 @@ def test_patma_214(self):
def f():
match 42:
case 42:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), set())

def test_patma_215(self):
def f():
match 1:
case 1 | 2 | 3:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), set())

def test_patma_216(self):
def f():
match ...:
case _:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), set())

def test_patma_217(self):
def f():
match ...:
case abc:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), {"abc"})

def test_patma_218(self):
def f():
match ..., ...:
case a, b:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), {"a", "b"})

def test_patma_219(self):
def f():
match {"k": ..., "l": ...}:
case {"k": a, "l": b}:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), {"a", "b"})

def test_patma_220(self):
def f():
match Point(..., ...):
case Point(x, y=y):
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), {"x", "y"})

def test_patma_221(self):
def f():
match ...:
case b as a:
return locals()
return [l for l in locals() if not l.startswith('$')]
self.assertEqual(set(f()), {"a", "b"})

def test_patma_222(self):
Expand Down Expand Up @@ -2601,7 +2601,7 @@ def f(x):
(g, b, a, c, d, -5, e, h, i, f) |
(-1, d, f, b, g, e, i, a, h, c)):
w = 0
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["x"]
return out
alts = [
Expand All @@ -2625,7 +2625,7 @@ def f(x):
(g, b, a, c, d, -5, e, h, i, f) |
(-1, d, f, b, g, e, i, a, h, c), z]:
w = 0
out = locals()
out = {k : v for k,v in locals().items() if not k.startswith('$')}
del out["x"]
return out
alts = [
Expand Down
4 changes: 2 additions & 2 deletions Python/bytecodes.c
Original file line number Diff line number Diff line change
Expand Up @@ -2211,7 +2211,7 @@ dummy_func(
}
}

register inst(JUMP_IF_FALSE_R, (__0, cond -- )) {
register inst(JUMP_IF_FALSE_R, (unused, cond -- )) {
int offset = oparg1;
if (Py_IsTrue(cond)) {
}
Expand Down Expand Up @@ -2253,7 +2253,7 @@ dummy_func(
}
}

register inst(JUMP_IF_TRUE_R, (__0, cond -- )) {
register inst(JUMP_IF_TRUE_R, (unused, cond -- )) {
int offset = oparg1;
if (Py_IsFalse(cond)) {
}
Expand Down
2 changes: 0 additions & 2 deletions Python/generated_cases.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.