Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JIT Compilation #113464

Closed
brandtbucher opened this issue Dec 25, 2023 · 4 comments
Closed

JIT Compilation #113464

brandtbucher opened this issue Dec 25, 2023 · 4 comments
Labels
3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage topic-JIT type-feature A feature request or enhancement

Comments

@brandtbucher
Copy link
Member

brandtbucher commented Dec 25, 2023 •
edited by bedevere-app bot
Loading

It's probably about time to start thinking seriously about just-in-time compilation. To kick things off, here's my talk from this year's sprint about "copy-and-patch", which I think is a really promising path forward: https://youtu.be/HxSHIpEQRjs

I'll also be opening a draft PR for discussion soon.

Linked PRs
@brandtbucher brandtbucher added performance Performance or resource usage interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.13 bugs and security fixes labels Dec 25, 2023
@bedevere-app bedevere-app bot mentioned this issue Dec 25, 2023
Closed
@Eclips4 Eclips4 added the type-feature A feature request or enhancement label Dec 25, 2023
@cosmicexplorer
Copy link

Do you have a writeup or other non-video explainer available, or is the PR itself the best reference for this technique?

@MarshalX
Copy link

Do you have a writeup or other non-video explainer available, or is the PR itself the best reference for this technique?

Maybe this one will be helpful to you https://aha.stanford.edu/sites/g/files/sbiybj20066/files/media/file/aha_050621_xu_copy-and-patch.pdf

@brandtbucher
Copy link
Member Author

Do you have a writeup or other non-video explainer available, or is the PR itself the best reference for this technique?

There four main resources that I've been directing people towards... they each explain different things in different levels of detail:

  • The white paper: high-level overview of copy-and-patch.
  • The blog post: low-level implementation details for copy-and-patch.
  • The YouTube video: high-level overview of CPython's approach to copy-and-patch.
  • The pull request: low-level implementation details for CPython's approach to copy-and-patch.

z764969689 pushed a commit to z764969689/cpython that referenced this issue Jan 29, 2024
@brandtbucher
pythonGH-113464: Add a JIT backend for tier 2 (pythonGH-113465)
f6d9e59 
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information on how to install the required build-time tooling.
pull bot pushed a commit to roacn/cpython that referenced this issue Jan 29, 2024
@brandtbucher
pythonGH-113464: A JIT backend for tier 2 (pythonGH-113465)
a16a9f9 
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch
(https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information, including how to install the required build-time tooling. Merry JIT-mas! ;)
@brandtbucher
Copy link
Member Author

Done! I'll be opening follow-up issues for further improvements to both the build process and the generated code.

@bedevere-app bedevere-app bot mentioned this issue Jan 29, 2024
Merged
brandtbucher added a commit that referenced this issue Jan 30, 2024
@brandtbucher
GH-113464: Make Brandt a codeowner for JIT stuff (GH-114739)
742ba60
@bedevere-app bedevere-app bot mentioned this issue Jan 30, 2024
Merged
brandtbucher added a commit that referenced this issue Jan 30, 2024
@brandtbucher
GH-113464: Add aarch64-apple-darwin/clang to JIT CI (GH-114759)
348a72c
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@brandtbucher @aisk
pythonGH-113464: Add a JIT backend for tier 2 (pythonGH-113465)
a4c3b8f 
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information on how to install the required build-time tooling.
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@brandtbucher @aisk
pythonGH-113464: A JIT backend for tier 2 (pythonGH-113465)
534119a 
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch
(https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information, including how to install the required build-time tooling. Merry JIT-mas! ;)
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@brandtbucher @aisk
pythonGH-113464: Make Brandt a codeowner for JIT stuff (pythonGH-114739)
ea3a744
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
@brandtbucher @aisk
pythonGH-113464: Add aarch64-apple-darwin/clang to JIT CI (pythonGH-1…
5f0dfd7 
…14759)
@bedevere-app bedevere-app bot mentioned this issue Feb 22, 2024
Merged
brandtbucher added a commit that referenced this issue Feb 22, 2024
@brandtbucher
GH-113464: Clean up JIT stencil generation (GH-115800)
17dab2e
WillChilds-Klein pushed a commit to WillChilds-Klein/cpython that referenced this issue Mar 4, 2024
@brandtbucher @WillChilds-Klein
pythonGH-113464: A JIT backend for tier 2 (pythonGH-113465)
d6f355e 
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch
(https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information, including how to install the required build-time tooling. Merry JIT-mas! ;)
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this issue Mar 4, 2024
@brandtbucher @woodruffw
pythonGH-113464: Clean up JIT stencil generation (pythonGH-115800)
3fefe21
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@brandtbucher @diegorusso
pythonGH-113464: Clean up JIT stencil generation (pythonGH-115800)
004e818
@bedevere-app bedevere-app bot mentioned this issue May 1, 2024
Merged
brandtbucher added a commit that referenced this issue May 1, 2024
@brandtbucher
GH-113464: Display a warning when building the JIT (GH-118481)
424438b
This was referenced May 2, 2024
Merged
Merged
brandtbucher added a commit that referenced this issue May 3, 2024
@brandtbucher
GH-113464: Generate a more efficient JIT (GH-118512)
1b7e5e6
markshannon added a commit that referenced this issue May 4, 2024
@markshannon
GH-113464: Remove the extra jump via _SIDE_EXIT in _EXIT_TRACE (G…
da2cfc4 
…H-118545)
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@brandtbucher @SonicField
pythonGH-113464: Display a warning when building the JIT (pythonGH-11…
8d6eeda 
…8481)
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@brandtbucher @SonicField
pythonGH-113464: Generate a more efficient JIT (pythonGH-118512)
1493daa
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@markshannon @SonicField
pythonGH-113464: Remove the extra jump via _SIDE_EXIT in `_EXIT_TRA…
1e4281a 
…CE` (pythonGH-118545)
@bedevere-app bedevere-app bot mentioned this issue May 23, 2024
Merged
brandtbucher added a commit that referenced this issue May 24, 2024
@brandtbucher
GH-113464: Run the JIT interpreter before any other JIT CI (GH-119466)
b48a3db
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 24, 2024
@brandtbucher @miss-islington
pythonGH-113464: Run the JIT interpreter before any other JIT CI (pyt…
175d857 
…honGH-119466)

(cherry picked from commit b48a3db)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
@bedevere-app bedevere-app bot mentioned this issue May 24, 2024
Merged
brandtbucher pushed a commit that referenced this issue May 24, 2024
@miss-islington
[3.13] GH-113464: Run the JIT interpreter before any other JIT CI (GH…
e27e369 
…-119490)

(cherry picked from commit b48a3db)
@JelleZijlstra JelleZijlstra mentioned this issue May 28, 2024
Closed
estyxx pushed a commit to estyxx/cpython that referenced this issue Jul 17, 2024
@brandtbucher @estyxx
pythonGH-113464: Run the JIT interpreter before any other JIT CI (pyt…
d196f49 
…honGH-119466)
@bedevere-app bedevere-app bot mentioned this issue Aug 8, 2024
Merged
brandtbucher added a commit that referenced this issue Aug 14, 2024
@brandtbucher
GH-113464: Speed up JIT builds (GH-122839)
5118592
blhsing pushed a commit to blhsing/cpython that referenced this issue Aug 22, 2024
@brandtbucher @blhsing
pythonGH-113464: Speed up JIT builds (pythonGH-122839)
a0edf4c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage topic-JIT type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@cosmicexplorer @MarshalX @brandtbucher @Eclips4