Unfortunate. I've been following the work on it occasionally.
On one hand, it seemed like JIT work was really progressing in high gear until SC requested a new PEP. Also unfortunate, as it seemed like part of the hope for the JIT was that it'd counter the performance penalty from free threading.
On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
> It said that the PEP was "overly optimistic" about user-visible differences with the JIT enabled.
The JIT is technically generated from the same source as the interpreter loop itself, so originally the differences were extremely unlikely, but I imagine the risk just kept going up as new JIT-only paths and optimizations were introduced.
And politically, I feel like SC’s reasoning was just “this side project has landed in main branch almost by accident and has been stewing without oversight ever since, with no stabilization in sight”; and the latest PEP was probably still too vague, with a lot of “here’s more things we can add to the stew” - so I can understand if SC wasn’t satisfied.
I think this is a major issue with requiring a PEP for what's a performance optimization.
JITs are big nasty things that by their nature have to do a lot of things. You are basically saying "Give us a PEP for a new python interpreter and compiler".
What they do is fuzzy by their nature because optimizations are very rarely universally beneficial. It's a game of measuring, looking for regressions, and tackling the biggest problem next for whatever sample applications you have available. You always run into the risk of overfitting on top of all that (You may have made django faster, but you've made flask slower).
This is the sort of thing that, IMO, should not go through the PEP process. It's an implementation detail for the runtime.
Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements. They'd never get done. Or if any javascript engine needed ECMA acceptance before they could do a new JIT optimization.
Does not have the same level of requirements being asked of the Python JIT. Further, it's well understood that this JEP will "live on main" for quite some time in preview before it's accepted and merged.
What the JEP does, instead, is simply describe the feature and potential risks.
The vast majority of JVM optimizations do not get a JEP.
Fair, I don’t argue with that, I’m not knowledgeable about Java’s processes. Just one thing:
> The vast majority of JVM optimizations do not get a JEP.
In Python too. Still, the JIT really is just a really big thing. I don’t know if there could be a better way to do it, as either 1. You develop it off-branch and… make an enormous PR when you’re done, with risks of getting rejected anyway? Or 2, you allow more experiments on main like Rust, but then you end up with hundreds of minor half-maintained nightly-only features, possibly conflicting with each other, and eventually go through bureaucracy to remove them anyway.
So, I'd grant that it'd probably be a JEP for java (the initial JIT) but not that every change to the JIT would result in a new JEP. Java's JIT, hotspot, was integrated right before they started the JEP process. The JEP for the JIT would likely be pretty boring like "It's a JIT, it does this, in some applications we see this sort of speed up". Much like the AOT JEP https://openjdk.org/jeps/514
The Java governance allows for a lot more open ended projects with follow ups. Not that it's reckless, Valhalla is proof of that, but rather that once something is good enough they are pretty open to just shipping it as a preview feature and refining until it is worthy of being stable.
That's why, for example, we saw ZGC get merged, and then JDK after JDK seeing continuing development and improvement on ZGC (Or G1 for that matter). These GCs were fairly rough on their first release and now they are pretty much best in class.
I see a Python JIT as the same thing. Delivering the infrastructure for a JIT would need a PIP, but I wouldn't put in things like "acceptance criteria" beyond "doesn't break older code". I would expect such a project to get continual updates without PEPs.
> I would expect such a project to get continual updates without PEPs.
Just for completeness: the original JIT PR was merged quite enthusiastically
and only after the fact it was pointed out that it should have went through a PEP process [1]. The following PEP 744 was of the “informational” type, and the just-withdrawn PEP 836 was the first “standards track” PEP, which means that (in my understanding) it’s technically the first actual PEP that the SC would formally comment on. So in a way it’s not “can we update the JIT” PEP, but the first real “does SC want this JIT as proposed” PEP.
That said, I do agree with most of what you said, especially the acceptance criteria part.
The issue I have and find it hard to resolve is that since the code was already on main, the feature was already in “sunk cost” mode. It’s hard when the code is already on main and another 3 optimization PRs are always there behind the corner, but it needs to be possible for SC to say “we decided it’s not actually worth all this complexity for <some small number>% speed improvement” (I’m not commenting whether in this particular case it’d have been a right call), and the PEP (if not withdrawn) was the way to exercise that control.
Aren’t you confusing internals and API? I don’t think nobody cares about HOW the optimisation is done, but SC wanted API with the rest of the interpreter be defined and stable. Two different things. I think.
Oh, it's not just a performance optimization. But you've said it yourself: "a new python interpreter and compiler". A lot of Python code out there lives with assumptions of CPython interpreter built into it. Pretty sure pytest would've been affected, should JIT replace the existing interpreter, for example.
Anyways. I don't believe that Python needs a JIT compiler. I don't believe that CPython can realistically replace its interpreter with JIT compiler without causing major problems with the community packages.
The way I see it: Python users today desperately want Java. They want it called Python because there's some vague memory still around of Python being cool and Java not being cool.
In Russian there's a children tale about a hungry soldier returning from a war and a greedy old woman who offered him a bed for the night. The old woman then pretends to have no food for dinner, and a soldier promises her a magic trick of making a porridge out of a hatchet. And so he tells the woman to put a pot on a stove with a hatchet in it, and step by step he makes her add all the ingredients necessary to make a porridge to the pot. The story gave birth to the idiom "hatchet porridge", which is used to describe anything that was created in a similar way: by deception, when another goal was allegedly being pursued.
So, to me, the current state of Python is this "hatchet porridge". None of its users want the actual Python (the hatchet is still sticking out of the pot), and they keep adding Java ingredients to the pot in hopes to cook up Java. And so was JIT. It just turned out to be too difficult to throw away the hatchet. An the porridge still has some chipped wood in it as well as a metallic taste.
I am confused. Why can't they have an experimental, optionally-enabled feature for a few years while they iron out all the issues ? What is wrong with that ? The JDK keeps doing this all the while. They write a new JEP if there are changes across previews.
> On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
Zero chance of shoehorning that into a standards-compliant Python runtime. Some things are too fundamental to change without affecting the user-visible language: JITting, lazy imports, threading, copying gc, ref counting, etc.
I mean, I have had great success with prototyping stuff in python, then letting an llm translate it to C. My 3090 with Gemma4:31b does this very well with a few guidelines.
Dunno how much investment is needed into language optimization anymore.
That works well for greenfield code, but if working on compiler optimisations has taught me anything, it's that there is a lot of legacy code out there in the world that would benefit from re-compilation with optimisations - but would be very difficult to re-write for speed (regardless of whether the work is done by a human, AI, or other machine tool). JIT compilation fits very nicely in this niche, as it means that existing Python projects can get (theoretically) better performance without re-writing.
You can use static type analysers and runtime type profilers to add type annotations to a legacy codebase, but the interesting bit would be if you're built on top of a large untyped codebase you don't control like, say, Django, how far it can go.
In my experiments with this, maybe, sorta, it'll do some AST stuff then it'll go "dang this shit is dynamic AF" then suggest a static type inference tool, then a runtime type profiler.
People have been calling Python a "prototyping" language for decades. I'd be interested to see how many Python projects ever actually get rewritten in a more performant language. My guess is, up until the last five years that was almost never done. Even with LLMs to help, is that the best use of your tokens and humans' time? It doesn't add a feature. It could conceivably cut costs after enough time, but only if done correctly.
> My guess is, up until the last five years that was almost never done.
Because having just the hot loop in the performant language is often deemed good enough. And there's tons of Python code out there where the "hot loop" is already automatically in C (the various operations implemented in C by the runtime) or where you can get close enough e.g. with Numpy. Consider that things like Pygame can exist; for a lot of the games people want to make, the only thing that really needs to happen anywhere near "bare-metal" speed is blitting.
Doing it currently, Python to Go. Early indications are a 50-98% increase in speed (e.g. one testcase went from 26hrs to 30mins) for network/CPU heavy workload. Conversion via Qwem, lots of testing needed but compute savings will be worth it.
Where you using standard optimized libraries (e.g., NumPy for numerical computation) or did you do everything in Python? This kind of increase of speed seems to indicate extremely sloppy Python programming.
Did you mean “50-98% decrease in running time”? Just calculating from your 26hrs to 30 mins seems to imply so. In terms of speed that translates to 100% to 5100% increase in speed.
It's the same level of testing for us for either Python2->3 or Go (it's a legacy code base, low automated coverage).
Conversion is now (somewhat) easier, the testing will be the slow part. For the speed improvements, which we also need for a migration from on-prem to cloud, it's a no brainer vs the other options at our disposal (like GraalPy).
Here’s an idea instead (maybe it already exists). Allow a decorator for writing inline Cython. It solves, I think, 90% of the problem. On first run, or eagerly, compile the decorated functions. Basically, Cython is the JIT, and user enables it explicitly.
This way I can easily write inlined functions with proper C speed, in a Python esque language.
Reasons why I rarely use Cython without this:
- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.
- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.
- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.
Such a decorator would give you a low friction way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics, in particular "development semantics".
I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.
I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)
> I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
Python already compiles to bytecode (the .pyc files are a cache thereof). It's just that interpreting even the simple bytecode instructions can take dozens of clock cycles each. I imagine that a lot of this is not in the interpreter loop (which has seen quite a bit of optimization, IIRC up to the level of "predictive" gotos within a big switch statement within a loop) but just unwrapping objects and moving values to and from registers, dealing with the fact that there is basically never any cache coherency for anything etc.
Basically I'm not convinced that writing out the set of machine-code instructions needed to interpret the bytecodes would be a big win, because it takes too many instructions for simple tasks.
And? That doesn't change the fact it was compiled into machine code before execution, nor that with exception of 8 bit home computers, there were plenty of compiled BASICs in mainframes and later 16 bit home computers.
Since we have to prove everything on Internet forums,
However because of this, Julia has has persistent problems with "time to first plot", which has gotten better, but they cannot fully ameliorate due to the way it fundamentally works. So there are tradeoffs -- the performance gets better, but the interpreted REPL experience is markedly worse.
It's not unavoidable, it just hasn't yet been avoided. It could be avoided the same way as other JITs: by interpreting until a compiled version of the code is available. That's actually being worked on, but it adds considerable complexity. Among other things, it requires being able to jump out of an executing interpreter loop into an optimized, compiled version of the same function. Doable, but annoying to implement and requires compiling two versions of the function instead of one. It can also cause performance instability — see https://tratt.net/laurie/blog/2022/more_evidence_for_problem... and the older paper referenced there.
It is overly exaggerated, especially when taken into account how long it takes to compile native libraries that people keep calling "Python" libraries, if there isn't a suitable wheel for the current environment.
A tradeoff that doesn't make sense to complain about, when the alternative is waiting for native libraries being compiled, in a mix of C, C++, Fortran and more recently Rust.
A pain point that is big enough that at least two programming languages, Julia and Mojo, were created to move beyond it.
Which would not even be an issue, if Lisp, Scheme and Prolog had not lost their aura among AI devs.
Julia and Mojo are attempts at solving the "two language problem", which not every Python user actually has -- they're fine doing work exclusively in Python, not even using any of the libraries you're talking about.
Mojo in particular recognized to compete with Python they'd have to be a superset, then they backed of that goal when they realized it would be too hard.
So at the end of the day, Python has a value proposition and it cannot be entirely subsumed by other languages, because those languages make tradeoffs that move away from Python's value.
Python's value is being a portable scripting language, a better Perl, unfortunately some try to do full blown applications with an interpreted language.
Perhaps it's worse that some even succeed in building very complex systems that can handle a lot of load too (see eg. Instagram)? ;-)
Python has a known set of trade-offs, mostly related to loop performance. But if you understand the trade-offs and available mitigations, it should not stop you from building good software. You know, just like any other ecosystem.
Julia gets a lot more benefit out of it because it's compiling type-specialized version of the function all the way down to machine code. So for example, a function that's being called with a `Vector{Int64}` ends up dealing with an array of unboxed ints, not going through dynamic dispatch on a generic array for every operation.
I have no doubt Julia's far better with this. I just wanted to point out Python does have a similar precedence for ahead-of-time compilation, albeit with byte-code only. If the Python steering council ever gets its act together, maybe Python could have real AOT and JIT.
This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.
I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!
The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems
If you want to use the same code unmodified *and* your code uses type hints, then you should use mypyc [1].
There are times that it generates faster code than Cython! Of course with Cython you can tweak it and push its limits with various optimizations and other techniques, but for using your existing code as it is and gain massive performance improvements, mypyc it's a must!
Code sample:
import time
def fib(n: int) -> int:
if n <= 1:
return n
else:
return fib(n-2) + fib(n-1)
startTime = time.time()
fib(35)
duration = time.time() - startTime
msduration = duration * 1000.0
uduration = msduration * 1000.0
print(f'{duration:.6f} seconds, {msduration:.2f} ms, {uduration:.2f} μs')
$ python3 mypyfib.py
1.452477 seconds, 1452.48 ms, 1452477.22 μs
$ # the same code compiled with `mypyc mypyfib.py` produces the following:
$ python3 -c "import mypyfib"
0.063701 seconds, 63.70 ms, 63700.91 μs
This is huge, if you think about it!
95.61% reduction in time
2180.15% increase in performance
22.80x faster
This benchmark only measures function call overhead, which is important but not the whole story. What kind of figures do you get with a more realistic benchmark?
I think the idea is correct, but hard to retrofit, you sort of have to have it there from the start.
Objective-Smalltalk runs on the Objective-C runtime, so Objective-Smalltalk methods look like Objective-C methods accessed via objc_msgSend() to the entire rest of the system, including to other Objective-Smalltalk code.
That means it doesn't really matter to the rest of the system if any particular method is interpreted, AOT-compiled or JIT-compiled. They can't tell, except for execution speed.
And of course the execution model includes machine primitives, which the interpreter also deals with but doesn't make faster, but the native compilers do.
Yeah, this is numba. I find it really frustrating the extent to which people don't know about it.
Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably already have and understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.
With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).
[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.
[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.
That's a really intense restriction. Sounds like the name fits; it's not general purpose, it's jit for number crunching and little else.
I'm not sure exactly what you mean by "bound by the interpreter" but there's nothing about touching those data structures that requires the interpreter to be involved in most cases. The problem of slow APIs can be solved by the JIT having intimate knowledge of python's internals, can't it?
Probably "bound by the data model" is a better way to put it. The interpreter might not be required to touch the data structure, but you have to do a lot of touching of the data structure to get the work done (plus, its mere existence ruins any dream of cache coherency).
For most objects, you access it directly and nothing weird happens, right?
Can you not mostly cache whether anything weird needs to happen and do the direct access with minimal overhead?
You said elsewhere "But that doesn't help much if/when you're doing two hash lookups per iteration just to find two small integers to add together." but looking up two fields isn't inherently a very big overhead. And you can cache the actual hash, and if you want to you can make sure the fields on similar objects have a consistent offset and cache that offset to make it even faster. This code using lots of objects is presumably not the core number crunching on arrays; we're doing more logic per access and we can handle a few instructions to get a value.
Is it? An object field access in python is a dynamic hash lookup followed by a method call as a fallback (itself involving hashed lookups up the class hierarchy). That's not an "interpreter vs. native" kind of thing, that's just the semantics of the language that have to be preserved. And that's just not going to be "fast" in the way that the user of a JIT would want. Code dominated by Python datastructures is going to be "slow" in ways that JITs can't fix, ever.
Which is why Python appears to have given up on the effort in this space. PyPy too[1] really never delivered on the desired performance gains (like 5x or so on typical code) without forcing the user into careful optimization.
Numba, to its credit, doesn't even try. It falls back to python if it must for python stuff, but is quite clear that it's providing no benefit if that's what you're doing. And the stuff it does do is obvious and clean and extremely useful.
[1] PyPy is also looking pretty moribund recently.
A hash lookup can be very fast. There's no reason for it to be slow when it doesn't need to invoke the fallback. JITs can't remove the slowest case but they can make the normal case very fast. Saying it can't be fast ever is a huge cop-out.
Maybe, but the world is still waiting for the counterexample. Python is simply not a runtime well-suited to optimization.
In the meantime, those of us who actually enjoy real HPC problems are using numba and rolling our eyes about all the confuffery around JITs. Maybe one will arrive that proves us wrong, but numba is fast today.
I have moved my more complex scripting from Perl/PHP/Python to Lua for a lot of reasons. One being that Lua (even PUC Lua) is a good deal faster than Python (or /bin/sh); another is that, if I want even more speed, I can use LuaJIT (which keeps me stuck at Lua 5.1, but Lua 5.1 is perfect for my needs).
I wish Lua 5.1 could become part of POSIX; there are a lot of implementations of it out there.
Yeah, this is sort of the disconnect. Lua is a fast dynamic interpreter, sitting well above Python but rather below e.g. V8 in its ability to run scalar code over "interpreter types" like doubles and strings and vectors and hash tables.
The upthread comment was about an idea for "inline Cython", which was then hijacked because that's largely what numba is. These are technologies aimed at optimized native code running over native types to do stuff like numerics and ML and other HPC tasks. Lua isn't any more responsive to this domain than Javascript has been, it's not really the subject under discussion.
LuaJIT handles double, strings, and dictionaries at near C level performance. Also, arrays, but Lua’s data model has dictionaries and arrays be the same data structure. Lua 5.1 (i.e. LuaJIT) only supports double numeric types,[1] and its strings are immutable.
Lua 5.1’s edge over Javascript and Python is that it’s a lot smaller and simpler.
[1] It’s possible to recompile Lua 5.1 to have the numeric data type be an int, but the standard compiles of Lua5.1 only have double precision floats for numbers. Bitwise operators are technically not part of Lua 5.1, but a library to add support is easy and is usually done: Most Lua5.1 forks including LuaJIT and MediaWiki’s version of Lua5.1 have such a library available. Later versions of Lua have both int and float support.
If you're saying V8 is quite fast why did you say the world is "waiting for a counterexample" to JITs on very dynamic objects being slow?
When OP wrote "inline Cython" they didn't seem focused on numerics, they talked about "proper C speed", and they mentioned modifying the language. I don't think it's getting off-topic to suggest a deeper integration that lets the JIT be very intimate with the interpreter and truly inline things in a way that bypasses language-crossing and type-dispatching overhead. As long as you're in the hot path, let it look up object fields in a couple instructions.
> When OP wrote "inline Cython" they didn't seem focused on numerics, they talked about "proper C speed"
Isn't that basically what I said? Lua isn't "proper C speed", nor is PyPy, the cancelled Python JIT in question, nor even V8. That's a different problem area with different technologies. The fact that you like coding backend stuff in Lua or whatever isn't really saying much about whether numba is great.
I agree Numba is great. The reason I stay clear of Python is because the Python2 → Python3 transition burned me; the bureaucrats at Debian decided that, since my open source project had one Python2 script in it I wrote many years ago, I had to update my code to make that script a Python3 script. A very toxic and arrogant attitude; old versions of languages should not be killed like that when there is so much code written in the language, and when it can be very difficult [1] to convert a script from Python2 to Python3.
Lua5.1 is much more stable by the virtue that the only updates it code gets are very minor changes to fix any lingering security issues it has (the last Lua5.1 CVE is from 2014).
[1] I guess these days a LLM can do the grunt work of converting a Python2 script to Python3.
V8 gets a hell of a lot closer than this experimental JIT. Numba speed for number crunching and V8 speed for non-number-crunching would satisfy OP, I think.
Numba is great at its niche. But OP directly said Numba is "not really" what they were looking for because it's so narrow.
Last I tried numba, admittedly a long time ago, it was a pain due to extensive system requirements or alternatively requiring use of conda, which I dislike (for many reasons, but it's besides the point - I want to use standard python tools). Did they start to bundle their dependencies or..?
PyPI integration is clean; "pip install numba" does what you expect, uv works fine, yada yada.
The one aesthetic gripe I have is that the LLVM included is a fork. It won't even try to work with the system LLVM and insists on installing a huge wheel with custom binaries. But on a modern system an extra LLVM is noise, so I guess I don't care that much.
> It won't even try to work with the system LLVM and insists on installing a huge wheel with custom binaries.
Not only does LLVM not have a stable ABI, it doesn't even have a stable API. It's difficult to write code that works reliably against a non-trivial number of LLVM major versions.
It's unfortunate that a technically exciting new feature is being killed partly because of what is essentially bureaucracy (and the lack of the developer's personal bandwidth). On the other hand, it's pretty safe to say that the strict adherence to the PEP policy and all of the bureaucracy that comes with it is what makes Python a great language, for better or worse.
Not to say that the technical objections were without merit, they likely very much weren't, and the debugger and instrumentation support and maintenance work sounds fairly painful on its own.
I think it would be a mistake to conclude this has anything at all to do with bureaucracy as a matter of inanimate policy that the SC was helplessly in thrall to. Bureaucracy was the tool the SC used to demonstrate that this feature lacked many important properties that they cared about, as mentioned: "maintenance, compatibility, and ... clear success metrics". For whatever detailed reasons, the PEP was withdrawn after it couldn't convincingly demonstrate those things to the satisfaction of the critics on the SC.
Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
If having to put all that in writing and get a bunch of people to agree feels too burdensome to you, maybe that’s a sign that you would be happier working on something other than core changes to the world’s second most popular programming language.
(edit: That wasn’t meant to be a jab at the core developers working on the JIT. Based on what’s in the public discussion threads, they have been handling things very well throughout the process. It’s unfortunate that the SC let the project get this far before realizing the project had grown well beyond its original PIP’s mandate, and I’m sure that is deeply frustrating to the people leading the project. I know I would not be handling the setback nearly as gracefully as they are.)
The problem is that clearly documenting all details as required in the PEP is at odds with complicated new features like a JIT where all the details can’t be known and have to be decided on as they are realized. Process / bureaucracy is at direct odds with experimentation.
But they weren’t asking for all the implementation details. They were asking for a clearly stated scope, success criteria, support plan, etc.
As they themselves pointed out, the problem isn’t that there’s anything wrong with an open-ended experiment. It’s just that open-ended experimentation doesn’t belong in main, of all places. They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
The scope of a JIT is the entire python runtime. Not at the beginning, it can be more narrow, but certainly it can evolve to cover everything. JITs touch everything.
> success criteria
Impossible to measure. JITs don't universally make things faster and unless the SC was willing to accept "some programs will see around x% improvement" then it would be impossible to actually give them the criteria they want. There's a reason C++ compilers and their optimizations aren't defined by the ISO committee.
> support plan
Like most JITs, support for a JIT is an ongoing and ever evolving process. What's needed depends entirely on what's needed today. You usually need more than 1 or 2 dev working on it, which may be the issue here.
> It’s just that open-ended experimentation doesn’t belong in main, of all places.
Very strong disagree. That's simply the nature of performance optimizations for JITs. They belong on main because new optimizations and heuristics will always be generated. The role of doing a compiler is a long term experiment in performance, seeing what your runtime is currently doing and how it's being used and tweaking it to work better.
pypy isn't in a worse place because it's been running a longterm performance experiment.
> They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
None of the things mentioned actually do that, other than perhaps a support plan. There are plenty of critical infrastructure projects who all can manage to develop JITs. Python is the special snowflake that has decided to do JIT development incorrectly.
This is a runtime optimization/implementation detail. It's foolish to send it through a committee approval process. That's why you won't be able to find a single language with a JIT or advanced optimizations that has do it by committee.
In that case I can understand why it's not a fit for Python. The risks you're describing are more what a brand new language would face: universal scope, impossible to measure success, no support plan... sounds like a nightmare in a mature product. If that's actually what it means to add a JIT to Python, the SC is doing its job by steering Python around this quixotic task.
> This is a runtime optimization/implementation detail.
I view JITs as a critical architectural component. Like you said, it impacts the whole system. Optimization and implementation details don't usually do that.
The problem is that “Python” the language includes native poorly isolated FFI interfaces exposed by CPython. So the language isn’t “Python” but CPython. That’s why alternate implementations like PyPy haven’t seen adoption despite having a very mature JIT while languages like JavaScript have JITs that put CPython’s performance to shame.
It’s also part of why Python has been so successful in many domains. I was there for Python’s takeover of the data space, and I got quite used to porting things from Java (static, JIT, decent at multithreading) to Python (dynamic, interpreted, GIL) and realizing substantial performance improvements in the process.
It’s kind of wild how nice Python’s C extension system, which enables basically zero cost interop with native code, is for certain purposes. It supports a situation that’s comparable to the games industry’s tradition of putting a Lua or Lisp tip on top of a C++ iceberg.
I think there is enough evidence to conclude that most language ecosystems cannot support multiple implementations that are fit for general purpose usage. There are almost no successful examples, and there are many, many failures.
The most successful things I can think of:
C & C++ - often does require at least some pre-processor logic or different build system settings.
Java - I'm not 100% sure - I think there are credible implementations of both the JVM and compiler though of course almost everyone is building from the openjdk or the commercial fork.
Nodejs & Bun - Most of NPM just works.
...
Thats it. Honorable mentions like C#/Mono exist, but if you look at that ecosystem a majority of applications only ever built with one of those, not both.
There are niches maybe like some subsets of scheme or common lisp - but I think most apps have a single implementation creating the builds that they ship. There are re-implementations perhaps that would count that I don't know enough about like Delphi/Lazarus, but I truly don't think there is another mainstream example.
In Haskell, SPJ noted years ago that GHC was really the reference implementation and he no longer considered standards work to be worthwhile (though it has continued past that).
Others I think have reach the same conclusions, such as Ruby. I could be missing something but I don't think most platforms should even aspire to support this.
And yet, not introducing unnecessary chaos into the CPython project remains important.
Perhaps these priorities were always irreconcilable.
Perhaps that’s why the existence of competing Python JIT projects that weren’t trying to commit directly into CPython’s main branch kept coming up in the conversation.
> Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
Bureaucracy doesn't definitionally mean Kafkaesque. It's popular to hate on it, but good or neutral bureaucracy does exist.
It doesn’t have to mean kafkaesque, but it does mean governance through layered hierarchies and fixed procedures.
In this case a single committee directly engaged with both core developers and the community, through a fairly open process, explicitly bent several of their own official rules to suit the occasion, and indicated cases where further flexibility might be needed depending on how things develop.
I would call that a pretty stellar example of non-bureaucratic governance.
I have a hard time seeing it as organizational hell, either? From the outside, it looks like the process went smoothly and reached a decision in about half the time they originally estimated.
It’s true that that decision didn’t involve everybody getting their way. But that’s not organizational hell, that’s just organization. If you don’t want to contend with other people’s competing interests, the solution to that is solo development.
Large codebases are bureaucracies of their own, that is, the complexity of extra layers starts to bog things down at some point.
In this case, it seems like the human bureaucracy made wise and measured decisions to contain the software bureaucracy. They made reasonable requests to clarify the scope and impact of a major, major core feature. The proposers didn't feel like that was doable, so a complexity nuke was avoided.
How many Python JIT experiments have failed? I find it interesting because Ruby is probably harder to optimize than Python but Ruby has had a JIT for a long time. Maybe base python is more optimized than base Ruby so the JIT is incrementally not changing much? Also, I'd expect javascript to be harder to optimize but it has JITs. That could just be a funding issue, though. There's probably much more money and incentive to optimize Javascript.
The type of dynamic language gymnastics done with Python is huge: any sufficiently well known library basically employs some sort of meta-programming of the language:
* Typing is a non-obligatory add-on to the syntax
* SQLAlchemy and other ORMs override basic operators like "=" and employ metaclass programming heavily to create Python-like SQL query builders
* Pytest rewrites all the assertions as it side-steps bytecode generation (so your .pyc files are different than what they would be without pytest)
* Many libraries depend on monkey-patching live library calls, including ones to external dependencies
* The way async was added in is another bolt-on where the same code behaves differently in different contexts
* Decorators can introduce any side-effect you imagine, and nobody (including core libraries) follow the golden rule of "don't do, only decorate"
* Many debugging and profiling mechanisms (including REPL) depend on these properties to function — including observability tooling like Sentry, Datadog etc.
...
JavaScript never had that level of flexibility (for better and for worse), and by extension, it's easier to build an alternative implementation and keep it compatible with the rest of the world that claims to be JavaScript.
Ruby is essentially Smalltalk (see also Objective-C).
I don't know Python's innards well enough to understand why it's so especially difficult in this regard, but I gather around the edges that its object model may not be as well founded.
JIT compiling Ruby or Python in a way that speeds up code using existing libraries and frameworks is harder. We built up decades of code that does stuff that it would be a lot easier for JIT performance if we didn't do, like using C extensions instead of defined FFI.
The common excuse is indeed wrong, as pypy proved. IMO the real issue is the python C API.
The python C API is the "killer app" of python, and what gave its edge against otherwise similar languages (ruby/js). That's makes it unique (the whole scientific stack). But a lot of early choices leaked into this API, and this (+ preference from guido for simple implementation) is at odds with doing JIT: lots of global state (instead of explicitly passing interpreter state), ref counting, no tagged pointers meaning boxing/unboxing everywhere, etc. Some of those are getting gradually fixed, but that's a slog, though maybe AI can accelerate a lot.
I had high hopes for HPy but that unfortunately fizzled out.
Obviously, I am not saying making a JIT for Python is impossible: both PyPy and GraalPy prove otherwise. We are talking about full ecosystem support, not just the base language and stdlib.
Can you just replace CPython with PyPy or GraalPy in every project? And why not? :)
Because the Python community writes libraries in C, C++, Fortran and Rust, to make up for the CPython slowness, using CPython C API, calls them Python libraries, and they naturally only work in CPython.
PyPy and GraalPy then never get to use those pseudo "Python" libraries, because their API for C based extensions differs enough that you cannot take a random shared library and load them into either.
They came up with a common C API proposal, but has hardly taken that much adoption.
Indeed, mixing all the language features and anti-features with non-native Python libraries is a big reason too — both because it mitigates many of the performance issues a JIT would solve, but mostly because it complicates or makes the implementation unfeasible altogether.
Operator overloading is a non-issue, it's just another form of method calls.
JavaScript also has monkey-patching, and it's also very common. E.g. SolidJS replaces objects with dependency-tracking proxies.
There are really no fundamental language reasons why Python should be more complex. From what I heard from a JIT developer, the main issue is not the code itself but the module system and code loading.
JavaScript was designed from the start to operate inside a sealed environment with only few external handles, with the code loaded upfront or added on-demand via "eval". Not so for Python, where native code modules can (and will) result in code loaded dynamically and it's hard to track what depends on what.
> Python is harder to JIT because of the way __dict__ works.
Heh, how about how `__class__` works? Not that I recommend actually doing this, but you can literally change the type of an already instantiated object by reassigning `__class__`:
That's also possible in JS with __proto__. The prototypes of the classes in the chain can be mutated at runtime too. How to handle these cases is well understood for JITs though and python could do the same.
The thing that actually makes it harder in Python is threading, because it breaks all ability to reason locally about what might happen and optimize. In Python you have to assume that another thread may do these things at any point mid-execution.
Modifying a variable that is read by another thread needs special care in all multithreaded languages I know. If you don't use a synchronization primitive to tell the other thread "hey, this thing here may have changed, read it again from memory"; the other thread is free to use its old value for as long as it whishes.
Lua has the same property with metatables. LuaJIT, which is very fast, has a solution: if you change a metatable after assigning it, or reassign a metatable, Mike Pall turns his back on you. Your code dies slowly, in the outer darkness of the interpreter loop, never to be JIT’ted upon again.
I've never bought this argument. Underneath all of the magic inheritance in Python is just the MRO (method resolution order). That's not characteristically any different from JavaScript's prototype chain. There's no reason a Python JIT couldn't optimize everything JavaScript does, and if you do specifically access `self.__dict__`, that can be a cold, unoptimizable path.
That works fine as long as nobody actually uses the unhappy path. If major libraries and frameworks do it all the time, you lose. So there is a certain aspect of culture outside what the language itself is capable of doing.
I'd say that's about right. The way people usually code in Python, mro is quite stable and most lookups would be cache hits. You can still get more weird invalidation in Python cuz they let you change stuff like `__bases__`, or you get some weird `__getattribute__` change in some place that would force you to invalidate more than you'd initially think should be necessary.
Same as for the common lisp or raku mop. You check for the default, and use the fast path if so. And if you can guarantee that nobody changed those keys or values, lift that check to the front.
Wait, how many? I'd say just this one. The other that I know of, pypy, was massively successful for a single maintainer project. And definitely showed the possibility/potential of JIT python.
I wouldn't call it a failure for not over taking mainline CPython.
There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
>There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
And all of them are lacking in different ways, we ALSO, want faster pure Python.
Onlything is a lot of cextensions are super slow there. that is only bottleneck left which could be solved with resources that those guys spent on experimental jit.
The problem with PyPy, which I'm a huge fan of BTW, is it's on life-support. The only reason the past couple major versions were released was due to a few small companies or individuals directly implementing the updates themselves. That's my understanding at least. It's disheartening because I've been primarily using PyPy for 5 years, advocated for it, and was looking into contributing more to it before finding out about its state.
Perhaps if the money which was spent on the built-in JIT had been spent on PyPy we'd be somewhere.
PyPy wasn't great for me because its startup time interfered with the main usecase I wanted it for in one instance and in another there bugs that made it not quite compatible enough. Money could solve the bug issue.
On the other hand I did get pygame working with it at one point and that seemed like a nice scenario.
PyPy's problem with slow startup is, in principle, easily fixable.
When it was created, PyPy did not seek to optimise startup performance. Instead, its aim was steady-state performance of JITed code, and it succeeded handsomely. A conceptually simple fix would be to optimise traces in one processor core, while the interpreter runs in another. Once when the trace is finalised, does interpretation stop and switch to native execution. But PyPy is currently a sequential system and making it parallel, while conceptually easy, is a lot of work and needs a rewrite of the whole system. But, for better or worse, the PyPy team does not have the resources to do this.
The other of PyPy's big performance issues (Python code using a lot of FFI calls via a suboptimal FFI) is also, in principle, easily fixable. But this is a social problem: the Python community would have to move towards better FFIs.
In principle, it would also be easy to fix PyPy's resource problem. I am not sure about the social problem ...
That's why I never want to work in CPython core. The 2 jit PEP's were totally fine, the implementation solid and stable, and even easy to understand for the SC manager types (unlike all other JITs which are much more convoluted), the future impact promising (already faster than Luajit in the Lua port). But then they got on their power trips.
profiling frames? Jit templates are jit templates for micro ops, what's not to understand? The very same architecture and ops they use in core already.
As we see over and over again in python core development. It's the worst.
A sad day for CPython but it doesn't have to be a sad day for Python. There is a fully maintained implementation of Python 3.13 that has an advanced JIT compiler. Once your Python warms up it can be 4x faster or higher (depending on what your program is doing). That implementation is called GraalPy and it builds on the JVM:
A bit like Chrome there's a fully open source version and a proprietary version with a few extra bells and whistles, but the latter is entirely free for both open and commercial usage (you can't redistribute it for a fee). So you may as well use it.
It has other interesting features
• Scalable pauseless GC
• Single binary distribution of Python programs (like Go)
• Fancy and optimized interop with other languages including Java, JavaScript, Kotlin, Ruby and any other Truffle language.
In recent weeks there has been a release of a new web framework for Python that only runs on GraalPy, called Pyronaut (https://pyronaut.io/). It takes advantage of JVM interop to expose the Micronaut framework, which is similar to Spring for those who know it. Thanks to the JIT and well optimized base libraries it runs extremely fast, being about 3x faster than FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many more features.
(that was presented at Devoxx but the Pyronaut devs will start touring Python specific conferences soon)
So although CPython may not have made it past the JIT milestone, that doesn't mean Python as a language or community has to fail it. It does require openness to alternative implementations, but historically Pythonistas have been fairly good about that (e.g. Pypy was embraced back when it was more active).
This stuff does require you to accept that it's made by Oracle, but as it's open source, that should be OK.
Disclosure: I sit next to some of the people who work on this stuff.
GraalPy and PyPy are great technology achievements, the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.
>the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.
They are Python libraries. Don't make the mistake of screwing up the cause and effect here. C, Rust etc - those are the replaceable parts of the chain. There is an official C++ pytorch library and virtually nobody uses it for ML Research. People want to code in python and build those libraries so they can.
I know what bindings are. A front-end to C++ for pytorch was only released from 1.0, literally 2 years later. The relationship is very clear here. There was no native C++ library. It was an afterthought. Even in instances where there was, like Tensorflow, functionally it was still an afterthought.
They are python libraries because people reached for those languages so they could continue work in python. They are there to serve python and nothing else. It's the same with Rust. It's by no means a C++ thing.
EDIT: Not saying this is always the case. Qt>Pyside is an example where it's both technically and functionally bindings, but the vast majority of the time they are serving python. Some don't even have a native library to speak of.
Are you arguing that JIT Python or GraalPy will be as performant in a reasonable time as Python on top of those C++/Rust libraries? If not then I don't think it matters since unless you are claiming that all the existing C++/Rust libraries can be re-written in reasonable time and with the same performance. Keep in mind that many of these libraries are using low level tricks to optimize performance. Python is the UI and while the UI is important in the end it's just the UI.
Yes, well, even with fancy JITs Python will never be a speed monster. Interop with libraries written in Java can be very high performance though as Pyronaut shows. And the heavy lifters that are hard to replace like NumPy and PyTorch do work with GraalPy.
I think if the Python community wants an implementation that has a proven JIT and where the remaining work is easily scaled compatibility testing, then there's one available and ready. Whether they're calling into C or Rust or FORTRAN or Java shouldn't make a big difference in principle.
Now, I've met lots of people who think that binding to Rust will always be the fastest thing to do but that underestimates Java significantly, especially once you factor in the fact that GraalPy can optimize out interop overheads.
Consider LLM tokenization. It's one of the most heavily optimized pieces of code on the planet. Yet, toknroll written in Java can beat tiktoken which is written in Rust:
Moreover the Rust code has some unsafe{} blocks as Rust often does, whereas toknroll has no unsafe code anywhere.
So the scripting language communities need to think hard about where they go next, IMHO. DHH already got up and announced at a Rails conference that Ruby is over, you should use Rust for everything. I don't think that's quite true, but it's going to be a meme that with agents you should just pick the fastest language and go. If Python/Ruby/etc want to stay relevant then they need to care a lot more about their performance story. Just being more pleasant to read and write for humans doesn't cut it anymore. CPython is clearly not going to happen in this regard, so alternative implementations need to start being taken a lot more seriously than they were in the past.
> Latency lines come from the load ramp (+25% per 15-second step); throughput from 45-second validation steps (+2% each) after a 180-second warm-up. Each framework ran twice and we show its best run. No profilers were attached.
So that's a "realistic" web serving benchmark if you're willing to grant the use of a GraalPy-optimized framework rather than Flask/FastAPI.
Warmup times for other benchmarks are currently quite unstable because there's a big focus on interpreter performance at the moment, which is one of the main influences on warmup. There's a blog post here from August about how the compiler is being taught to recognize and optimize bytecode interpreter loops, to get results that normally require handcrafted assembly:
So as you know already, warmup was historically a weakness of PE based stuff - although in this case the fair comparison is with CPython which never warms up at all - but a fast interpreter will make a big difference. In some benchmarks the GraalPy interpreter is already faster than the CPython interpreter, which is heavily optimized.
Python is a mature production language with a lot of users. Even before it was huge - like 20 years ago when I was in college - getting a small backwards compatible API change into a python extension module (gnu readline in my case) required convincing GvR personally that the API change was necessary.
Every JIT attempt I've seen for python just tries to cut too deep in the first pass, and scope themselves too large.
I believe any successful attempt at optimizing python performance from JIT will FIRST start with a 100% semantically compatible hidden type model for optimizing object property access and method dispatch.
The single biggest performance delta in most dynamic language jits come from that, not from optimizing guards off of bytecode operations or from compiling to machine code. It's from converting hashtable lookups into constant-time guard + offset lookups.
The hidden type model is also a hard nut to crack. It needs to perserve dict semantics which have very dynamic behaviour via the various `__*__` methods, and handle exotics scope objects since scoped name lookups are the most common ones which have their own exotic behaviours (I remember python providing some ability for callee frames to inspect caller scopes in some limited ways).
I haven't kept up with python development lately, so thanks for pointing that out. I looked this up and it does seem like some sort of property access is being optimized. What's not clear is what the hidden runtime type model is being used for this, or if it's a lightweight tracking that captures some subset of property accesses.
Python's object semantics are broadly similar to Javascript, bar the prototypical inheritance chain, but with the extra complexity of potentially needing to handle dynamic `__*attr__` overrides. Optimizing for this would require a tree of hidden map objects much like JS JIT engines use. I dug around the codebase a while back looking for something like that but didn't find it. `OrderedDict` seemed promising but after more investigation that turned out to just be adding property ordering to dictionaries.
The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement. Once you have the hidden type model, you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
> Optimizing for this would require a tree of hidden map objects much like JS JIT engines use.
I don’t really think that’s true or even that relevant , at least in context of classes (especially with slots - the class, by definition, _is_ the “hidden map”) and not generic dicts.
> The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement.
Pretty sure the motivation was exactly the opposite - that a simple “if a LOAD_ATTR encounters the same class with slots N times in a row, swap it out for LOAD_ATTR_SLOT with inline cached slot index, and swap back to generic one if something changes” is way easier (both to implement and reason about) than any alternative that’d interact with the object model.
(Also it means you skip having to branch between a slow and fast path at all)
> you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
Sorry, that sounds a bit vague and not too different from what CPython already is doing.
Ah, the `__slots__` feature makes it make a lot more sense.
> Sorry, that sounds a bit vague and not too different from what CPython already is doing.
With slots being available, I can see where I was misled. I went looking for a type structure that modeled the dynamic properties on an object which would be used as a typeguard, but the class itself functions as that if you're restricting the optimization to slotted properties.
(Note that this optimization can also trigger for plain classes (that’s LOAD_ATTR_INSTANCE_VALUE), it just needs a bit more guards and is only useful if you don’t abuse instances being dynamic objects. I just used slots as the most obvious example, but there are lots of different op specializations since like 3.11)
This has been very informative. With object property/method lookup optimized the next big leap in JIT-style perf is actually a hard pull. The next big perf win lies behind local type inference, and a method-jit with your standard bevy of SSA optimizations to hoist the typeguards.
I suppose the various JIT projects are trying to come at that from different angles incrementally, but that's a genuinely hard problem as you don't see real benefits until multiple complex pieces of infrastructure are in place and with good coverage over the language.
It still blows my mind that the big "we're breaking backwards compatibility Python 3000" release didn't do anything to lay the groundwork for this, or any of Python's other structural problems.
https://peps.python.org/pep-3000/ - PEP 3000 says it will be a derived from the Python 2 codebase and not a complete rewrite (referencing Joel Spolsky's views on complete rewrites and Netscape Navigator).
https://peps.python.org/pep-3099/ - PEP 3099 is changes Guido van Rossum rejected from Python 3 (including := as a binding operator), parallel iteration, programmable syntax, parser going beyond LL(1), changing the for/else construct.
https://peps.python.org/pep-3100/ - PEP 3100 is things he set as goals for it, things like all strings are Unicode and byte arrays are a separate type, optional type annotations, 'exec' and 'print' change from statements to functions, None/True/False become keywords, old-style classes are gone, changing inequality comparison from <> to != , require C99, drop BeOS support, drop old standard library modules, and more.
Last release on Oct 14. Beta support for 3.12. It's not that bad really and 1/10 of the people who complain about it did something then the situation would be better.
The problem was he exposed too much of the implementation into the language itself, destroying its ability to be a simple reference implementation that others could re-implement.
That's at best a partial fix for the API side. Python side CPython sill exposes a LOT of implementation details, and as a result occasionally projects will choose to rely on some of these details. Then other implementations need to decide whether to also pick the same implementations details and expose them, or use different details, and lack compatibility with any library that depended upon them.
The python documentation has historically also not been especially clear at distinguishing between what the language guarantees, vs what is just implementation details of the cPython implementation. So even if libraries wanted to try to avoid depending on implementation specifics, it isn't always clear what those are.
The net result is that for the end user python the language is effectively: whatever cpython does. This really, really discourages creating alternate implementations, as the only way to be an alternative implementation with good compatibility is to be a living fork of cpython, constantly merging in all the changes from it, while also maintaining your improvements.
At the time, Python had a clear purpose as an easy to learn "glue" language between other, perhaps more performant languages. There was no reason to make Python as fast because that would muddy its purpose by introducing compromises and complexity. For better or worse Python sat pretty well in its niche.
Today there are far more people using it, and everyone has a different perspective on what it is. No longer is it a niche glue language, now it's a backbone of AI technology at major tech firms, so it's understandable they'd like it to be faster on its own.
This is what happens what the defacto implementation, CPython and its external API, is effectively the specification. That includes all the incidental side effects that were not explicitly designed, but inherited as a quirk of historical implementation. It's next to impossible to retrofit a JIT given these constraints. ECMAScript has a proper specification and several implementations.
I mean, if your multi-year-old project is ever asked to just take a moment to write out a quick formal roadmap and justification ... it is doomed, 100% of the time. The actual content of the request doesn't really matter, it's dead.
To be clear, such a request doesn't even mean that someone is consciously attempting to kill the project (although in the corporate world, ime it probably does). It just means that something went critically wrong. In this case, it appears like python's development process really required a PEP for this sort of work, especially since it was happening on the main branch.
The sus bit is that the PEP they requested greatly increased the scope of the feature.
>> For example, rather than proposing one single concrete JIT implementation, it may make more sense for the PEP to describe a JIT infrastructure that can support multiple implementation strategies.
> we don't feel we have the personal bandwidth to carry the current version forward
Are the JIT developers the same people who were working on Faster CPython at Microsoft? No surprise that they don't have the same amount of energy to dedicate to the JIT now that it's gone from a full-time job to a hobby project.
Out of the box it's comparable to the current JIT, but yes, major gains are only possible with at least type annotations. But that may be the only real way forward for a JIT implementation for CPython with the ability to gradually speed up slow parts of large programs.
I did some experiments translating a typed subset of Python to C for small system use, certain features like lists of primitive types, generators and comprehensions, frozendict, simple classes, generic functions, etc, map with little or no cost but are so much cleaner than the equivalent code in plain C. You also get to write test harnesses using all the tools available to full Python. The main issue is unsurprisingly memory management, Obj-C / Swift style automatic reference counting works but has drawbacks, and it gets much more complicated if you want to mix the static and CPython code.
The static + AOT approach has a lot of promise now that type annotations are widely adopted, predictable behavior and good performance on code that is mostly idiomatic Python seem like a good alternative in many situations to the weight of Rust.
Yes, exactly. If you can get a 2-3x speedup in CPython just with type annotations, and then a possible additional 2-3x by dropping down to "Static Python," that may be an overall near optimal solution for the ecosystem, all things considered.
I didn't touch JIT (complexity and footprint way beyond what I can use or manage myself in any case) but the AOT path is pretty much what you'd get if you wrote naive C by hand. The emitted code is readable, in some cases you probably do better because more efficient algorithms are easier to code or reuse, but with the current refcounting that will have some overhead. I do want to try some ideas like supporting an @soa decorator for a List of DataClass type that keeps the visible interface but changes the memory layout. If someone wanted to sponsor tokens I could probably pure robot through a stage of fleshing out enough to port a few representative libraries and tune against the results.
Surprising, even though (IIRC) the performance of the JIT was always fairly underwhelming.
What about the other major change to Python's implementation: free threading? Has it lived up to expectations? I assume it's not also in doubt, despite Guido recently saying "Personally I wish we’d stuck with the GIL"? [0]
Getting a JIT from underwhelming to actually good is always going to be a whole lot of iterative work to remove blocking parts of the existing implementation. In Ruby-land we have had a good JIT for a while now, but it took many release cycles and a ton of resources before it was able to outperform the non JITed version when using Rails. Don't expect quick fixes that make great strides and any investments in it needs to be persistent and apply over time.
It boggles my mind that it apparently is the case that such a popular language as Python isn't able to get the resources to do this.
It is more than that. CRuby have had multiple JIT attempt before YJIT landed. From what I can tell CPython didn't have that. And luckily enough we now have ZJIT replacing YJIT.
> We want the PEP to be clear that it’s ultimately the JIT team’s responsibility to find solutions for these packages and tools, and how to deal with situations where a tool can’t be supported or made to work
Is it just me or is this absurdly impossible, to the point of willfully killing the project? Committing a team of brilliant compiler engineers to doing free consulting ad infinitum for every bit of tooling in the world, however hacked-together?
The alternative would be to have the bravery to say: this is the way forward to keep this language relevant into the future, and as a steering committee we collectively own that some tooling may be out of date. Instead, this is a signal that Python is going to rest on its laurels for a very long time.
Python use is widely diverse. Anyone wishing to contribute to that which is published to such a diverse community, all of whom expect Python to continue working for them without any regressions, has to be ready to cater to that diversity in its entirety.
Given that the speedup is modest (~10%) and the Python 3 migration caused enough pain for users, I get it though. If it was a 2x speedup it would be a different story IMO
Even with 2x speed, I don’t think it would be worth it. Do we really want a repeat of that with years of packages not working with each other and a bunch of smug people telling others to update their code or whatever.
Sad to read this. I had hoped that with the advance of AI capabilities, features like this could be prototyped, tested, implemented and shipped even faster. And I was really looking forward to see performance gains for my favourite language.
Where do we go from here? Microsoft did not continue to finance the Faster CPython project and the current speed gains are good, but not what we hoped.
A biased opinion as I dev in F#, but it strikes me that AI is going to play best with functionally pure languages (strongly typed, no side effects, whatever we call that these days)?
F# is on the softest side of that, but letting the compiler tell the AI "nope, here's why" seems like a huge win in saving cycles and time.
The downside at first was simply, "how many examples of F# are there to learn from", and early on it showed, but I think that's been long overcome.
And F# is obviously NOT the best example since it's both "soft" on functional purity and niche. I imagine something like haskell is probably doing even better, although I've yet to check that at all (not that I can code haskell but I haven't looked at anecdotes or anything).
I realize this doesn't really help anyone who's grown to love python, and there's surely no perfect language, but just like how the web catapulted javascript, I see a future where the answer isn't "use AI to make python better" so much as "what already works super well with AI, and does it ease the user adoption"
But I want to continue to understand the code. I dont know Rust (maybe AI can teach me?) and if I let Claude rewrite my current 50k LOC project in Rust, how can I understand what is happening and how can I work on it and think it further?
Rust is not your only option. Nor is it automatically the best option, though it is certainly a solid option for a certain set of requirements. Go is a pretty decent match for a Python rewrite; superficially it seems like you're giving up a lot but under the hood they're philosophically more compatible than meets the eye in a lot of ways. But any static language that you have some other local reasons to prefer is going to blow its socks off in terms of performance. And "blow its socks off" is just for the naive translation of a conventional Python code base over to a new language. If your codebase is amenable to threading and can take advantage of multiple CPUs somehow, the result may blow Python's socks off twice over.
The problem is that Python just isn't going to go fast. There are some languages that may have had more resources poured into them to make them go faster in absolute terms, like perhaps Java. But I don't think there's a single language that has had so many ways of trying to make it go faster given so much effort over so many decades. It's really time to call it... if you have Python code and it's a performance problem, if one of the many many many different solutions aren't adequate for some reason, Python is not going to get any faster than it is today and you're going to have to move off of it.
If you need high performance code, Python is simply not the right answer and you're going to have to skill up. Or accept what you have, which you'll have to decide if that's a viable option. If it's fast enough I don't advocate switching casually. But if it isn't, there isn't a realistic prospect that you're going to wake up in a year or two to a 30x-faster Python. I wasn't terribly convinced by that claim back in the early 2000s, but it hadn't really been tested yet. It has been tested now.
First, "if one of the many many many different solutions aren't adequate for some reason" was there for a reason.
Second, no, it isn't. Pypy is really fast compared to Python. "Compared to Python" is just about the lowest bar there is, though. In absolute terms it is still only a medium-fast language runtime, and it comes with a lot of costs and tradeoffs.
That's the problem with Python. Lots of ways to take a very slow language and make it a medium-fast language with substantial tradeoffs, or make it a fast language with enormous tradeoffs. If you can handle the tradeoffs, fine. But if you can't you just need to switch to a language that is fast without anywhere near as many tradeoffs.
> RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
I mean, it'd be pretty easy to speed it up by 10x — just need to drop backwards compat and do away with the ridiculous runtime dynamicism that is only truly useful for <%1 use cases.
But yeah, clearly nobody's willing to rip off the bandaid, so that's not happening.
Have the AI find the slow parts of your code. Then have it find libraries that do that slow part fast (they're almost certainly in Rust/C++) since thats low hanging fruit. Then have it re-write any remaining slow parts in Rust with Py03. Eventually that hits a limit but hopefully by then you understand Rust enough or it's fast enough to not matter.
There are already many solutions to the performance issue from which you can pick. You don't need to understand the C in numpy to use it for example. And I'd say it's highly unlikely that you have a use case that, if it's applicable, isn't already addressed. Like it's generally pointless to optimize language performance for code that's doing a lot of network operations, because network lag will always dwarf such optimizations.
There's so much Python code that uses numpy etc but still ends up having some hot part in Python that's nontrivial to translate to something faster. Same reason people get frustrated with the GIL.
You don't have to throw away the original Python codebase.
You can have:
1. An implementation in slow language (i. e. Python) that you maintain manually or semi-manually (with minimal AI assistance) and understand well.
2. A fast implementation that is fully generated and maintained by AI, using slow implementation (1) as a reference.
You can, for example, run two "adversarial" agents, one writes tests that compare output of (1) and (2) and tries to find a discrepancy, the other tries to fix (2) so that its output never differs from (1).
This way, you can be reasonably confident that (2) does exactly the same thing as (1), despite being a blackbox that you don't understand. While (1) remains a reference implementation that you understand well and can work with.
It's already typical to call into a native library with Python bindings for performance critical work in Python. Even Pydantic's validation runs in Rust code.
This works well if there are obvious hot spots that can be easily isolated, but sometimes the perf issues are more uniformly distributed throughout a codebase.
Mature projects simply don't like large unexpected code drops if it is not made clear beforehand what the blast radius is and how it is going to be maintained. On top of that it seems that the performance gains weren't as expected and require further invasive changes.
Implementation effort has never been what has held these efforts back. LLM helps with squirting out code, but the hard tasks are benchmarking, profiling, community engagement, knowledge management, and dealing with the hard questions such as whether the change is actually worth the additional complexity.
These concerns won't go away even if we give up on human developers being able to grok the code and instead completely relying on LLMs for coding.
We pick languages that embrace machine code like Julia, Mojo, Common Lisp, Scheme, and stop calling libraries that are written in C, C++, Fortran, Rust, "Python Libraries", when they are thin bindings.
Clearly the community doesn't want PyPy, GraalPy, or even a CPython JIT.
Sad news, I put a lot of hope in this project. At least we keep the base interpreter improvements they brought along, and we still have PyPy. CPython JIT is dead. Long live PyPy!
It clearly isn't hurting Python's popularity, but man the performance of the reference implementation is embarrassing.
V8 made JavaScript fast in 2008, soon that will be 20 years ago.
Perhaps I live in a bubble, but so much of the production Python code I see is fully type-hinted and makes used of an aggressive linter & type checker to the point where it might as well be written in Go/C#/Java/TypeScript so that it can actually take advantage of the performance gains instead of being loaded into a runtime that turns everything into heap-allocated PyObject pointers.
I work in Python every day and think about this a lot - people really don't seem worried about its performance. Sure people want it to be faster, but I don't see very many people changing their language choice for a faster language, despite Python being a lot slower than most alternatives.
If I think about where I see python being used though, this kind of makes sense, the two big use cases are:
- Web APIs and backends, where for the most part either the network or the database is the bottleneck
- Data science work, where for the most part, work exists in libraries like polars, pytorch etc that ship the main work out to other languages.
So from that perspective, although we'd all like python do be faster, it's speed isn't the bottleneck in the use cases where its popular (that's probably also the reason why we don't see Python getting used in other speed critical places).
That's a really different story to JavaScript, which doesn't have a handy CFFI or otherwise you can just reach out to. There are a lot of cases where JavaScript speed improving would speed up the reactivity of a web app, even now.
I'm not arguing against a faster CPython - I'd love one! But I think there's a clear reason why it hasn't been prioritised historically in the way JavaScript's performance has.
This is exactly how I write my Python. I'm currently debating between going in deep and transitioning to either TypeScript or Rust. Go's object system and error handling is too weak. Java is too verbose, but I have a soft spot for it.
If you want V8 like optimizations, you have to introduce a strict mode.
Python interpreter is known to leak its implementation (__class_ modification after creation, monkey patching with __getattr__, sys._getframe, eval/exec, deterministic call to __del__, no proper python API for libraries, etc.)
You have to plug all of those holes before having a chance to get a decent JIT.
Otherwise it will be just another JIT attempt, like Psyco, Unladen Swallow, Pyston.
Python is not uniquely dynamic to a degree that would prevent an efficient implementation.
V8 predates strict mode, and handles all the exec/eval, monkey patching and object system abuse you can do in JavaScript just fine. Common Lisp is one of the most dynamic languages in the world and allows you to do ridiculous things in terms of meta-programming and essentially building domain-specific languages into your program, yet it too manages to have ahead-of-time compiled implementations that are very performant. Also PyPy exists.
There are definitely some technical aspects that make it hard to make Python fast, but they are not insurmountable. The reason CPython is slow today is simply because performance has not historically been prioritized by its developers.
It seems as if today, Python is in most cases a very well designed frontend for accelerators and their proprietary, vectorized, massively parallel IRs. Those libraries (Jax, MLX, etc) already take care of JIT code optimizations - Polyhedral et al - and therefore it makes sense for the frontend to not worry about runtime performance.
ffffffff.... damnit I love python but why is it so difficult like this. JIT would have been so perfect so the language, increasingly overtime with AI code coming through I am not sure not focusing on making the performance better is going to be better for longer term. Even though a lot of the other operations done in python are done faster in libraries I would still like faster startup times, or faster loops.
This makes me think that mojo could actually become a python 4 of sorts meaning a python with more emphasis en performance and support for native gpu programming
Being slower than a compiled language is not saying much for an interpreted language. But it's slower than PHP which is a closer comparison, and PHP has had JIT for six years now.
Ruby has the reputation of being a very slow language, but Python has been slower still since Ruby jumped ahead with version 1.9, almost 20 years ago. Ruby also has had far more success with JIT compilers than Python has.
>Ruby also has had far more success with JIT compilers than Python has.
Ruby is on its third (I think?) JIT now? I'm a little surprised, given Ruby's reputation as a wacky "anything goes" language with a very complex grammar, and it makes me wonder why that is? Does it have to do with the design of the runtime itself more than the language semantics? Or is it more incidental, e.g. many popular ibs are C wrappers, already generates bytecode files, and so on.
The difference is that Ruby follow the BDFL model, and Ruby's BDFL was willing to allow people to experiment with in-tree development of JITs.
If the Python steering committee isn't willing to do the same, they'll probably won't get a working JIT any time soon. Developing a JIT externally, or even from a branch, is just too hard.
Getting a JIT right is hard work because time spent by the JIT is time not spent doing actual work, and finding the correct tradeoffs is very much an empirical problem.
And PHP is quite the hot mess. Always has been, really. See the security issues over the years, including the most recent for example. Meanwhile I've never seen any serious issues for Python that's triggered mass required fixing. There's something to be said for moving carefully and ensuring things aren't broken to the best of one's ability, which is the SC's purpose.
Are you helping me to prove my point? I see 156 results, with 1 in 2023 and the rest before that. Meanwhile there are 801 results with at least a dozen in 2026 for PHP[0], and we still have 2 more months to go.
Not at all. I specifically stated "serious issues for Python that's triggered mass required fixing", ie. it was so bad that there were public notices for users to update ASAP. The closest thing I can find is CVE-2022-34265, and that's in Django, not Python itself.
Agreed. And Python usage is far more diversified. But I fail to see the relevance of usage to a language being stable vs ridden with issues. Well, beyond that one would expect the one being so many things to so many different communities would have more issues than the other used by a single community for a single purpose.
Why had Python to succeed to Lisp in the AI wave? (rhetoric question)
Since PyPy and GraalPy will never get the love they deserve, I guess there is hope that Julia, Mojo, and the whole Rewrite in Go/Rust keep pushing along then.
I don't know why they don't just adopt Pypy as a faster Python implementation. Just let CPython develop slowly as the reference and give support to Pypy. It is already much faster than CPython.
The Rust based Python3 port can keep it and it can still be far more secure than CPython. Not everything it's JIT'ted on RustPython, it's ondemand per function, you have to enable it explicitly.
Python is a poor target for compilation, even the best JIT is never gonna make fundamental design flaws (that some may consider beloved features) disappear. It's never going to give "good" performance.
In my view, the right course of action is to specify an official strictly typed subset of the language with compilation and static analysis as the justification, then implement both JIT and AOT on top of that. Legacy code can be rewritten or thrown out in phases, C extensions can be ported.
Of course, any such proposal would invite monumental amounts of bikeshedding, which is why it wouldn't get far, much like the JIT proposal.
> strictly typed subset of the language with compilation and static analysis
This is a great idea for popular dynamic languages with existing users and codebases, particularly the focus on "subset" to limit the scope and emphasizing that full backward compatibility is not a goal; and allowing for gradual, incremental adoption. That makes it a lot more practical to implement and adopt.
I think we'll see more of this. Recent examples include strictly typed TypeScript subset designed for ahead-of-time compilation.
Python is no doubt a successful language. But, boy - this level of bureaucracy is annoying to no ends. Do people feel eager to propose a PEP? There are so many more enjoyable things to do in one's short life ...
Assuming that the biggest bang-for-buck of JIT compilation is optimizing hot loops (which is also the biggest bang-for-buck of optimization in every other context as well), and assuming that, per the historical best practices, all hot loops in Python are already written in C (or Rust, these days), this outcome is somewhat what I would expect.
The performance of python indicates otherwise. There are some performance critical libraries written in C or C++ but a lot of python is written in python.
Python in itself is poorly suited to ever become performant as it lacks the types required. I imagine that causes some issues for effective jit-ing too.
They had a chance to fix this when adding type annotations, but instead of making it "optional/gradual typing" they opted to make it "documentation or linting support", which is obviously the wrong choice, but there were cultural reasons
If you know the type you need fewer checks in the jitted code, and you can optimise to native types.
If it helps python specifically, given that the types themselves are pretty dynamic, who knows. A python integer doesn't map well to a native integer etc. Python really isn't designed for performance, and retrofitting it is hard.
Makes sense. And that's rough, even the integers need dynamic memory. Maybe they can just use 64-bit static ints and throw an exception (or even crash) if you go oob. Probably quite rare that anything would rely on huge ints.
Interesting, makes sense. JIT in CPython main is a big deal, glad they’re proceeding with caution. Overall looks like they added it as an experimental feature for two years, then decided to back it out. Perfectly reasonable outcome.
It looks like it very much should never have happened like this. The actually hard and awkward questions have not been asked beforehand, and the SC doesn't like their hands being forced, and rightly so.
what features would need to be removed to help python hit a baseline level of performance? adding typing? Some of the “functional” stuff. Is the performance a legacy issue or essential to the language’s design?
From my limited understanding, the C interop is the biggest obstacle to both JIT and GIL. It's also the feature that makes Python fast enough for so many use cases. And the dynamic typing is the reason it even has to be JIT instead of just pre-compiled.
To me, goals without plans behind them aren't really goals at all
Ok but that's not how specs and plans work -- they are very intentionally two distinct artifacts.
As usual, though, I might just be biased by default towards defending the Python aristocracy; they seem to pretty much always be on the right side of history, even in the little ways!
Python is my favorite language, but I'm honestly tired of having to rewrite performance bottlenecks in Cython or Rust all the time. When the Faster CPython project started with Microsoft's backing, I was really excited. Mark Shannon had that plan promising huge speedups across a few cycles, and while that clearly turned out to be much harder than anticipated, the team still kept pushing forward to deliver a real JIT.
Seeing all of that get bogged down in committee bureaucracy now is just frustrating.
I have a lot of respect for the core devs and I know maintaining CPython, and it is a time donation job. It isn't easy. But I feel that over-caution and red tape ends up discouraging serious contributions. They are setting an unreasonably high bar for something that would obviously help the entire ecosystem.
Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Python really deserves a proper JIT. I genuinely hope they reconsider this.
Unladen Swallow died for lack of upstream interest.
PyPy lacks upstream interest, looked like it could have a shot after many years of development but recently NumPy removed support because they just couldn’t fathom how a non-dead project could be a couple of versions behind CPython.
Now the new JIT is dying because of what seems to be a lack of upstream interest. (The things TFA describes as being required are not unreasonable, but not realistic for a necessarily speculative long-running project without wild amounts of committed funding.)
> Unladen Swallow died for lack of upstream interest
IIRC Unladen Swallow was, despite the extreme hype, nothing more than the internship project of a couple of students. It died because their internship was over.
Otherwise you're right, though. Lots of CPython alternatives suffer from lack of upstream interest - the Python language designers don't seem very interested in anything other than CPython. In particular, they appear to only consider CPython when adding features to the language. Two examples:
- Requiring `dict`s to be ordered because that's how they work in CPython, without considering how they work in other implementations
- Adding pattern matching syntax that's suitable for CPython's PEG parser, even though it's not LL(1) so can't be parsed by, say, MicroPython
> Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Every single time I remember the first AI wave was done with Lisp and Prolog, both with tooling that supports machine code generation since about 50 years.
That's what's kept the language and ecosystem improving and from becoming a hot mess. If something isn't properly specced then it really shouldn't be in prod. They allowed the experiment in the first place so it'd be easier to gather sufficient data to guide said speccing, but the ball was dropped by the proposers and so the experiment was an ultimate failure (unlike free threading) and prod is now being appropriately cleaned up.
It's not red tape, it's about making sure that the effort actually delivers what it promises and can continue to so in the future. From the state of the draft PEP it seems it doesn't and it won't ever. The SC merely put the nail into the coffin.
I know not everyone loves them; but `Claude rewrite the performance bottlenecks in Rust` works so well it is hard to see the security surface area and harder reason about performance as something that should be paid in core.
It's strange that typescript has 5 different LLM written competing static translators to compiled languages and python is killing JIT because the PEP was not accepted.
I would not like a JIT to be embedded in python for one reason. The sole reason why Python didn't have all the security related bugs of other languages (just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities) is sorely because it's an interpreted language, that is python is compiled in a bytecode that runs in a VM.
Under these premises is very difficult that something not implemented correctly is really exploitable, and moreover it's simple to verify the security of the whole thing.
Opening a JIT, that would mean mapping memory pages as RWX, would open the door to all soft of exploits that now are not really possible.
While I don't disagree with you in principle, the context for JS JIT vulns and potential Python JIT vulns is pretty different and worth taking into consideration.
For JS, your machine running untrusted JS code is the intended use case. In this scenario the JS is sandboxed in whatever JS engine (V8 etc) and does not have access to "dangerous" methods, and the main use case for a JIT vuln is to escape this sandbox to execute code on the host. This applies to regular internet browsing (and visiting potentially malicious sites) and also, e.g., using the built-in browser in a locked down device (such as game consoles) to gain arbitrary code execution.
For Python, this security boundary doesn't exist. I can't really think of a scenario where you would be running potentially malicious Python code in CPython that would need a JIT vuln to escape whatever sandbox it is in instead of, say, calling the functions in Python directly (or using the C FFI to do whatever). I'm sure that in some niche use cases, this scenario might exist, but to me, it seems like the potential attack surface is fairly minimal.
EDIT: Also, I'm pretty sure that Python's copy-and-patch JIT writes code to a writable mapping and then mprotects it to W^X? I don't think it leaves RWX pages lying around at all.
I would want a python implementation with a JIT and the ability to choose JIT or no-JIT when defining a python environment. And I want all my libraries to work either way.
I'll get what I want eventually but I might be retired by then.
Unfortunate. I've been following the work on it occasionally.
On one hand, it seemed like JIT work was really progressing in high gear until SC requested a new PEP. Also unfortunate, as it seemed like part of the hope for the JIT was that it'd counter the performance penalty from free threading.
On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
> It said that the PEP was "overly optimistic" about user-visible differences with the JIT enabled.
The JIT is technically generated from the same source as the interpreter loop itself, so originally the differences were extremely unlikely, but I imagine the risk just kept going up as new JIT-only paths and optimizations were introduced.
And politically, I feel like SC’s reasoning was just “this side project has landed in main branch almost by accident and has been stewing without oversight ever since, with no stabilization in sight”; and the latest PEP was probably still too vague, with a lot of “here’s more things we can add to the stew” - so I can understand if SC wasn’t satisfied.
I think this is a major issue with requiring a PEP for what's a performance optimization.
JITs are big nasty things that by their nature have to do a lot of things. You are basically saying "Give us a PEP for a new python interpreter and compiler".
What they do is fuzzy by their nature because optimizations are very rarely universally beneficial. It's a game of measuring, looking for regressions, and tackling the biggest problem next for whatever sample applications you have available. You always run into the risk of overfitting on top of all that (You may have made django faster, but you've made flask slower).
This is the sort of thing that, IMO, should not go through the PEP process. It's an implementation detail for the runtime.
Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements. They'd never get done. Or if any javascript engine needed ECMA acceptance before they could do a new JIT optimization.
> Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements.
They do:
https://openjdk.org/jeps/197
https://openjdk.org/jeps/515
Not every optimization requires a JEP, but certainly something as big as "add or remove a JIT" would 100% require a JEP.
> Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements.
FWIW, I definitely can find a bunch of both PEPs and JEPs for internal optimizations, random examples:
https://peps.python.org/pep-0509/
https://openjdk.org/jeps/147
You can find those JEPs, but frankly they are there for 1 of 2 reasons.
1. An outsider to the regular OpenJDK dev team proposing an optimization.
2. Advertising for a cool new performance tweak with potentially large impact.
You'll also notice that the JEP descriptions are very different from PEP descriptions.
Probably the largest JEP for the JVM, Valhalla
https://openjdk.org/jeps/401
Does not have the same level of requirements being asked of the Python JIT. Further, it's well understood that this JEP will "live on main" for quite some time in preview before it's accepted and merged.
What the JEP does, instead, is simply describe the feature and potential risks.
The vast majority of JVM optimizations do not get a JEP.
Fair, I don’t argue with that, I’m not knowledgeable about Java’s processes. Just one thing:
> The vast majority of JVM optimizations do not get a JEP.
In Python too. Still, the JIT really is just a really big thing. I don’t know if there could be a better way to do it, as either 1. You develop it off-branch and… make an enormous PR when you’re done, with risks of getting rejected anyway? Or 2, you allow more experiments on main like Rust, but then you end up with hundreds of minor half-maintained nightly-only features, possibly conflicting with each other, and eventually go through bureaucracy to remove them anyway.
So, I'd grant that it'd probably be a JEP for java (the initial JIT) but not that every change to the JIT would result in a new JEP. Java's JIT, hotspot, was integrated right before they started the JEP process. The JEP for the JIT would likely be pretty boring like "It's a JIT, it does this, in some applications we see this sort of speed up". Much like the AOT JEP https://openjdk.org/jeps/514
The Java governance allows for a lot more open ended projects with follow ups. Not that it's reckless, Valhalla is proof of that, but rather that once something is good enough they are pretty open to just shipping it as a preview feature and refining until it is worthy of being stable.
That's why, for example, we saw ZGC get merged, and then JDK after JDK seeing continuing development and improvement on ZGC (Or G1 for that matter). These GCs were fairly rough on their first release and now they are pretty much best in class.
I see a Python JIT as the same thing. Delivering the infrastructure for a JIT would need a PIP, but I wouldn't put in things like "acceptance criteria" beyond "doesn't break older code". I would expect such a project to get continual updates without PEPs.
> I would expect such a project to get continual updates without PEPs.
Just for completeness: the original JIT PR was merged quite enthusiastically and only after the fact it was pointed out that it should have went through a PEP process [1]. The following PEP 744 was of the “informational” type, and the just-withdrawn PEP 836 was the first “standards track” PEP, which means that (in my understanding) it’s technically the first actual PEP that the SC would formally comment on. So in a way it’s not “can we update the JIT” PEP, but the first real “does SC want this JIT as proposed” PEP.
That said, I do agree with most of what you said, especially the acceptance criteria part.
The issue I have and find it hard to resolve is that since the code was already on main, the feature was already in “sunk cost” mode. It’s hard when the code is already on main and another 3 optimization PRs are always there behind the corner, but it needs to be possible for SC to say “we decided it’s not actually worth all this complexity for <some small number>% speed improvement” (I’m not commenting whether in this particular case it’d have been a right call), and the PEP (if not withdrawn) was the way to exercise that control.
[1] https://github.com/python/cpython/pull/113465#issuecomment-1...
The original JIT certainly did not get a JEP, because JEPs replaced the previous JSR process.
And even then, Hotspot did not got a JSR, because all that work was done before Sun started collaborating with industry partners for Java evolution.
They "simply" ported the JIT compiler from the Strongtalk acquisition into Java.
Aren’t you confusing internals and API? I don’t think nobody cares about HOW the optimisation is done, but SC wanted API with the rest of the interpreter be defined and stable. Two different things. I think.
Oh, it's not just a performance optimization. But you've said it yourself: "a new python interpreter and compiler". A lot of Python code out there lives with assumptions of CPython interpreter built into it. Pretty sure pytest would've been affected, should JIT replace the existing interpreter, for example.
Anyways. I don't believe that Python needs a JIT compiler. I don't believe that CPython can realistically replace its interpreter with JIT compiler without causing major problems with the community packages.
The way I see it: Python users today desperately want Java. They want it called Python because there's some vague memory still around of Python being cool and Java not being cool.
In Russian there's a children tale about a hungry soldier returning from a war and a greedy old woman who offered him a bed for the night. The old woman then pretends to have no food for dinner, and a soldier promises her a magic trick of making a porridge out of a hatchet. And so he tells the woman to put a pot on a stove with a hatchet in it, and step by step he makes her add all the ingredients necessary to make a porridge to the pot. The story gave birth to the idiom "hatchet porridge", which is used to describe anything that was created in a similar way: by deception, when another goal was allegedly being pursued.
So, to me, the current state of Python is this "hatchet porridge". None of its users want the actual Python (the hatchet is still sticking out of the pot), and they keep adding Java ingredients to the pot in hopes to cook up Java. And so was JIT. It just turned out to be too difficult to throw away the hatchet. An the porridge still has some chipped wood in it as well as a metallic taste.
Just FYI, basically the same story is common in European folklore, and there's a somewhat idiomatic "stone soup" concept in English
https://en.wikipedia.org/wiki/Stone_Soup
I am confused. Why can't they have an experimental, optionally-enabled feature for a few years while they iron out all the issues ? What is wrong with that ? The JDK keeps doing this all the while. They write a new JEP if there are changes across previews.
I also dont get it
> On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
Zero chance of shoehorning that into a standards-compliant Python runtime. Some things are too fundamental to change without affecting the user-visible language: JITting, lazy imports, threading, copying gc, ref counting, etc.
I mean, I have had great success with prototyping stuff in python, then letting an llm translate it to C. My 3090 with Gemma4:31b does this very well with a few guidelines.
Dunno how much investment is needed into language optimization anymore.
That works well for greenfield code, but if working on compiler optimisations has taught me anything, it's that there is a lot of legacy code out there in the world that would benefit from re-compilation with optimisations - but would be very difficult to re-write for speed (regardless of whether the work is done by a human, AI, or other machine tool). JIT compilation fits very nicely in this niche, as it means that existing Python projects can get (theoretically) better performance without re-writing.
I've had good success with just using mypyc (https://mypyc.readthedocs.io/en/latest/introduction.html) on typed python code.
> on typed python code.
A fairly massive undertaking for existing code, which jit was supposed to be useful for.
You can use static type analysers and runtime type profilers to add type annotations to a legacy codebase, but the interesting bit would be if you're built on top of a large untyped codebase you don't control like, say, Django, how far it can go.
I would imagine that the LLM has a much easier time adding correct type annotations to old Python code than it does translating it to C.
In my experiments with this, maybe, sorta, it'll do some AST stuff then it'll go "dang this shit is dynamic AF" then suggest a static type inference tool, then a runtime type profiler.
Which honestly, is the right approach.
Agree with the general workflow: prototype with an easy-to-read language then port to lower-level, faster language
People have been calling Python a "prototyping" language for decades. I'd be interested to see how many Python projects ever actually get rewritten in a more performant language. My guess is, up until the last five years that was almost never done. Even with LLMs to help, is that the best use of your tokens and humans' time? It doesn't add a feature. It could conceivably cut costs after enough time, but only if done correctly.
> My guess is, up until the last five years that was almost never done.
Because having just the hot loop in the performant language is often deemed good enough. And there's tons of Python code out there where the "hot loop" is already automatically in C (the various operations implemented in C by the runtime) or where you can get close enough e.g. with Numpy. Consider that things like Pygame can exist; for a lot of the games people want to make, the only thing that really needs to happen anywhere near "bare-metal" speed is blitting.
Doing it currently, Python to Go. Early indications are a 50-98% increase in speed (e.g. one testcase went from 26hrs to 30mins) for network/CPU heavy workload. Conversion via Qwem, lots of testing needed but compute savings will be worth it.
Where you using standard optimized libraries (e.g., NumPy for numerical computation) or did you do everything in Python? This kind of increase of speed seems to indicate extremely sloppy Python programming.
Did you mean “50-98% decrease in running time”? Just calculating from your 26hrs to 30 mins seems to imply so. In terms of speed that translates to 100% to 5100% increase in speed.
Yes, sorry, typing fast - we knew Go would be faster but not quite on that scale.
Why not go with something like graalpy first? That's much less risk in terms of "we end up slightly changing behavior"
It's the same level of testing for us for either Python2->3 or Go (it's a legacy code base, low automated coverage).
Conversion is now (somewhat) easier, the testing will be the slow part. For the speed improvements, which we also need for a migration from on-prem to cloud, it's a no brainer vs the other options at our disposal (like GraalPy).
> My guess is, up until the last five years that was almost never done.
It was done plenty. Just with extensions to python like numpy and opencv. There is no reason to translate everything versus just the critical pieces.
Here’s an idea instead (maybe it already exists). Allow a decorator for writing inline Cython. It solves, I think, 90% of the problem. On first run, or eagerly, compile the decorated functions. Basically, Cython is the JIT, and user enables it explicitly.
This way I can easily write inlined functions with proper C speed, in a Python esque language.
Reasons why I rarely use Cython without this:
- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.
- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.
- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.
Such a decorator would give you a low friction way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics, in particular "development semantics".
I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.
I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)
> I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
Python already compiles to bytecode (the .pyc files are a cache thereof). It's just that interpreting even the simple bytecode instructions can take dozens of clock cycles each. I imagine that a lot of this is not in the interpreter loop (which has seen quite a bit of optimization, IIRC up to the level of "predictive" gotos within a big switch statement within a loop) but just unwrapping objects and moving values to and from registers, dealing with the fact that there is basically never any cache coherency for anything etc.
Basically I'm not convinced that writing out the set of machine-code instructions needed to interpret the bytecodes would be a big win, because it takes too many instructions for simple tasks.
Python doesn’t really compile to *.pyc. These are just parsed tokens and AST, really not proper machine code.
The Julia approach is the same as BASIC (as originally designed), Lisp, Prolog, as added info.
I don't recall being able to inspect compiled machine code in basic like I can in Julia.
[EDIT - I assume you are commenting on Dartmouth BASIC.]
And? That doesn't change the fact it was compiled into machine code before execution, nor that with exception of 8 bit home computers, there were plenty of compiled BASICs in mainframes and later 16 bit home computers.
Since we have to prove everything on Internet forums,
https://archive.org/details/bitsavers_dartmouthB_3679804/mod...
=> Saving of Compiled Programs
The language forum is the place to start- you should already have a thread there gathering feedback before writing a pep?
Thank-you, it's a longtime idea, but I was finally able to implement it, but now I'm trying to learn the procedure
However because of this, Julia has has persistent problems with "time to first plot", which has gotten better, but they cannot fully ameliorate due to the way it fundamentally works. So there are tradeoffs -- the performance gets better, but the interpreted REPL experience is markedly worse.
It's not unavoidable, it just hasn't yet been avoided. It could be avoided the same way as other JITs: by interpreting until a compiled version of the code is available. That's actually being worked on, but it adds considerable complexity. Among other things, it requires being able to jump out of an executing interpreter loop into an optimized, compiled version of the same function. Doable, but annoying to implement and requires compiling two versions of the function instead of one. It can also cause performance instability — see https://tratt.net/laurie/blog/2022/more_evidence_for_problem... and the older paper referenced there.
By the way, the current nightly (especially if considering the optimized build) has a very substantial TTFX drop compared to the current release [1]
[1] https://perf.julialang.org/?tab=overview
It is overly exaggerated, especially when taken into account how long it takes to compile native libraries that people keep calling "Python" libraries, if there isn't a suitable wheel for the current environment.
I think it's true that it's exaggerated, but that doesn't mean the tradeoff doesn't exist.
A tradeoff that doesn't make sense to complain about, when the alternative is waiting for native libraries being compiled, in a mix of C, C++, Fortran and more recently Rust.
A pain point that is big enough that at least two programming languages, Julia and Mojo, were created to move beyond it.
Which would not even be an issue, if Lisp, Scheme and Prolog had not lost their aura among AI devs.
Julia and Mojo are attempts at solving the "two language problem", which not every Python user actually has -- they're fine doing work exclusively in Python, not even using any of the libraries you're talking about.
Mojo in particular recognized to compete with Python they'd have to be a superset, then they backed of that goal when they realized it would be too hard.
So at the end of the day, Python has a value proposition and it cannot be entirely subsumed by other languages, because those languages make tradeoffs that move away from Python's value.
Python's value is being a portable scripting language, a better Perl, unfortunately some try to do full blown applications with an interpreted language.
Perhaps it's worse that some even succeed in building very complex systems that can handle a lot of load too (see eg. Instagram)? ;-)
Python has a known set of trade-offs, mostly related to loop performance. But if you understand the trade-offs and available mitigations, it should not stop you from building good software. You know, just like any other ecosystem.
There’s an #internals-and-peps channel in the Python discord where some folks hang out.
That's basically how Python already compiles source to byte-code.
Julia gets a lot more benefit out of it because it's compiling type-specialized version of the function all the way down to machine code. So for example, a function that's being called with a `Vector{Int64}` ends up dealing with an array of unboxed ints, not going through dynamic dispatch on a generic array for every operation.
I have no doubt Julia's far better with this. I just wanted to point out Python does have a similar precedence for ahead-of-time compilation, albeit with byte-code only. If the Python steering council ever gets its act together, maybe Python could have real AOT and JIT.
This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.
Good point.
Maybe also then, importing module as Cython?
compiles on first run?I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!
The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems
If you want to use the same code unmodified *and* your code uses type hints, then you should use mypyc [1].
There are times that it generates faster code than Cython! Of course with Cython you can tweak it and push its limits with various optimizations and other techniques, but for using your existing code as it is and gain massive performance improvements, mypyc it's a must!
Code sample:
This is huge, if you think about it! [1] https://github.com/mypyc/mypycThis benchmark only measures function call overhead, which is important but not the whole story. What kind of figures do you get with a more realistic benchmark?
This is largely the idea behind py2many + fquery, but we want to get rid of C-API as well.
I think the idea is correct, but hard to retrofit, you sort of have to have it there from the start.
Objective-Smalltalk runs on the Objective-C runtime, so Objective-Smalltalk methods look like Objective-C methods accessed via objc_msgSend() to the entire rest of the system, including to other Objective-Smalltalk code.
That means it doesn't really matter to the rest of the system if any particular method is interpreted, AOT-compiled or JIT-compiled. They can't tell, except for execution speed.
And of course the execution model includes machine primitives, which the interpreter also deals with but doesn't make faster, but the native compilers do.
This is numba?
Yeah, this is numba. I find it really frustrating the extent to which people don't know about it.
Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably already have and understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.
With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).
[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.
[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.
> No dicts or objects or lists or strings, etc.
That's a really intense restriction. Sounds like the name fits; it's not general purpose, it's jit for number crunching and little else.
I'm not sure exactly what you mean by "bound by the interpreter" but there's nothing about touching those data structures that requires the interpreter to be involved in most cases. The problem of slow APIs can be solved by the JIT having intimate knowledge of python's internals, can't it?
Probably "bound by the data model" is a better way to put it. The interpreter might not be required to touch the data structure, but you have to do a lot of touching of the data structure to get the work done (plus, its mere existence ruins any dream of cache coherency).
For most objects, you access it directly and nothing weird happens, right?
Can you not mostly cache whether anything weird needs to happen and do the direct access with minimal overhead?
You said elsewhere "But that doesn't help much if/when you're doing two hash lookups per iteration just to find two small integers to add together." but looking up two fields isn't inherently a very big overhead. And you can cache the actual hash, and if you want to you can make sure the fields on similar objects have a consistent offset and cache that offset to make it even faster. This code using lots of objects is presumably not the core number crunching on arrays; we're doing more logic per access and we can handle a few instructions to get a value.
> That's a really intense restriction.
Is it? An object field access in python is a dynamic hash lookup followed by a method call as a fallback (itself involving hashed lookups up the class hierarchy). That's not an "interpreter vs. native" kind of thing, that's just the semantics of the language that have to be preserved. And that's just not going to be "fast" in the way that the user of a JIT would want. Code dominated by Python datastructures is going to be "slow" in ways that JITs can't fix, ever.
Which is why Python appears to have given up on the effort in this space. PyPy too[1] really never delivered on the desired performance gains (like 5x or so on typical code) without forcing the user into careful optimization.
Numba, to its credit, doesn't even try. It falls back to python if it must for python stuff, but is quite clear that it's providing no benefit if that's what you're doing. And the stuff it does do is obvious and clean and extremely useful.
[1] PyPy is also looking pretty moribund recently.
A hash lookup can be very fast. There's no reason for it to be slow when it doesn't need to invoke the fallback. JITs can't remove the slowest case but they can make the normal case very fast. Saying it can't be fast ever is a huge cop-out.
Yes, Python's hash lookups are actually pretty good. The comments in the implementation are an interesting read.
But that doesn't help much if/when you're doing two hash lookups per iteration just to find two small integers to add together.
> Saying it can't be fast ever is a huge cop-out.
Maybe, but the world is still waiting for the counterexample. Python is simply not a runtime well-suited to optimization.
In the meantime, those of us who actually enjoy real HPC problems are using numba and rolling our eyes about all the confuffery around JITs. Maybe one will arrive that proves us wrong, but numba is fast today.
LuaJIT has the same dynamism with dictionaries and it's very fast at accessing them in JIT code.
I don't have enough experience with PyPy to say much about it.
I have moved my more complex scripting from Perl/PHP/Python to Lua for a lot of reasons. One being that Lua (even PUC Lua) is a good deal faster than Python (or /bin/sh); another is that, if I want even more speed, I can use LuaJIT (which keeps me stuck at Lua 5.1, but Lua 5.1 is perfect for my needs).
I wish Lua 5.1 could become part of POSIX; there are a lot of implementations of it out there.
Yeah, this is sort of the disconnect. Lua is a fast dynamic interpreter, sitting well above Python but rather below e.g. V8 in its ability to run scalar code over "interpreter types" like doubles and strings and vectors and hash tables.
The upthread comment was about an idea for "inline Cython", which was then hijacked because that's largely what numba is. These are technologies aimed at optimized native code running over native types to do stuff like numerics and ML and other HPC tasks. Lua isn't any more responsive to this domain than Javascript has been, it's not really the subject under discussion.
LuaJIT handles double, strings, and dictionaries at near C level performance. Also, arrays, but Lua’s data model has dictionaries and arrays be the same data structure. Lua 5.1 (i.e. LuaJIT) only supports double numeric types,[1] and its strings are immutable.
Lua 5.1’s edge over Javascript and Python is that it’s a lot smaller and simpler.
[1] It’s possible to recompile Lua 5.1 to have the numeric data type be an int, but the standard compiles of Lua5.1 only have double precision floats for numbers. Bitwise operators are technically not part of Lua 5.1, but a library to add support is easy and is usually done: Most Lua5.1 forks including LuaJIT and MediaWiki’s version of Lua5.1 have such a library available. Later versions of Lua have both int and float support.
If you're saying V8 is quite fast why did you say the world is "waiting for a counterexample" to JITs on very dynamic objects being slow?
When OP wrote "inline Cython" they didn't seem focused on numerics, they talked about "proper C speed", and they mentioned modifying the language. I don't think it's getting off-topic to suggest a deeper integration that lets the JIT be very intimate with the interpreter and truly inline things in a way that bypasses language-crossing and type-dispatching overhead. As long as you're in the hot path, let it look up object fields in a couple instructions.
> When OP wrote "inline Cython" they didn't seem focused on numerics, they talked about "proper C speed"
Isn't that basically what I said? Lua isn't "proper C speed", nor is PyPy, the cancelled Python JIT in question, nor even V8. That's a different problem area with different technologies. The fact that you like coding backend stuff in Lua or whatever isn't really saying much about whether numba is great.
But it is. Numba is great.
I agree Numba is great. The reason I stay clear of Python is because the Python2 → Python3 transition burned me; the bureaucrats at Debian decided that, since my open source project had one Python2 script in it I wrote many years ago, I had to update my code to make that script a Python3 script. A very toxic and arrogant attitude; old versions of languages should not be killed like that when there is so much code written in the language, and when it can be very difficult [1] to convert a script from Python2 to Python3.
Lua5.1 is much more stable by the virtue that the only updates it code gets are very minor changes to fix any lingering security issues it has (the last Lua5.1 CVE is from 2014).
[1] I guess these days a LLM can do the grunt work of converting a Python2 script to Python3.
[2] There is a project which converts Lua in to native code: https://github.com/samyeyo/clx
V8 gets a hell of a lot closer than this experimental JIT. Numba speed for number crunching and V8 speed for non-number-crunching would satisfy OP, I think.
Numba is great at its niche. But OP directly said Numba is "not really" what they were looking for because it's so narrow.
I posted it as well before scrolling down and seeing yours. Deleted mine, as your post goes into much more detail.
I don’t know how people who’ve worked with Python and been frustrated with its performance don’t know about Numba.
Last I tried numba, admittedly a long time ago, it was a pain due to extensive system requirements or alternatively requiring use of conda, which I dislike (for many reasons, but it's besides the point - I want to use standard python tools). Did they start to bundle their dependencies or..?
PyPI integration is clean; "pip install numba" does what you expect, uv works fine, yada yada.
The one aesthetic gripe I have is that the LLVM included is a fork. It won't even try to work with the system LLVM and insists on installing a huge wheel with custom binaries. But on a modern system an extra LLVM is noise, so I guess I don't care that much.
> The one aesthetic gripe I have is that the LLVM included is a fork.
It's not technically a fork, it's a standalone build of vanilla LLVM (ok, there are a couple patches: https://github.com/numba/llvmlite/tree/main/conda-recipes).
> It won't even try to work with the system LLVM and insists on installing a huge wheel with custom binaries.
Not only does LLVM not have a stable ABI, it doesn't even have a stable API. It's difficult to write code that works reliably against a non-trivial number of LLVM major versions.
For the record, here is the LLVM compatibility table for llvmlite versions (llvmlite is the Python-C++ shim that Numba uses to talk to LLVM): https://llvmlite.readthedocs.io/en/latest/#llvm-compatibilit...
If you want a Numba linked against your system LLVM, the solution would be for your distribution to provide a system package for Numba (and llvmlite).
I don't understand why your comment was dead; I've vouched for it.
if you want GPU-backed/compiled numpy code then JAX is also a good option.
Not really. Numba is great at optimizing numpy code but not necessarily other general Python code.
We just need Mojo to have 99% Python friendly syntax and we can forget about CPython.
Numba does this. Generally as fast as Cython, but much easier to write.
Numba is basically this.
It's unfortunate that a technically exciting new feature is being killed partly because of what is essentially bureaucracy (and the lack of the developer's personal bandwidth). On the other hand, it's pretty safe to say that the strict adherence to the PEP policy and all of the bureaucracy that comes with it is what makes Python a great language, for better or worse.
Not to say that the technical objections were without merit, they likely very much weren't, and the debugger and instrumentation support and maintenance work sounds fairly painful on its own.
I think it would be a mistake to conclude this has anything at all to do with bureaucracy as a matter of inanimate policy that the SC was helplessly in thrall to. Bureaucracy was the tool the SC used to demonstrate that this feature lacked many important properties that they cared about, as mentioned: "maintenance, compatibility, and ... clear success metrics". For whatever detailed reasons, the PEP was withdrawn after it couldn't convincingly demonstrate those things to the satisfaction of the critics on the SC.
Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
If having to put all that in writing and get a bunch of people to agree feels too burdensome to you, maybe that’s a sign that you would be happier working on something other than core changes to the world’s second most popular programming language.
(edit: That wasn’t meant to be a jab at the core developers working on the JIT. Based on what’s in the public discussion threads, they have been handling things very well throughout the process. It’s unfortunate that the SC let the project get this far before realizing the project had grown well beyond its original PIP’s mandate, and I’m sure that is deeply frustrating to the people leading the project. I know I would not be handling the setback nearly as gracefully as they are.)
The problem is that clearly documenting all details as required in the PEP is at odds with complicated new features like a JIT where all the details can’t be known and have to be decided on as they are realized. Process / bureaucracy is at direct odds with experimentation.
But they weren’t asking for all the implementation details. They were asking for a clearly stated scope, success criteria, support plan, etc.
As they themselves pointed out, the problem isn’t that there’s anything wrong with an open-ended experiment. It’s just that open-ended experimentation doesn’t belong in main, of all places. They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
> clearly stated scope
The scope of a JIT is the entire python runtime. Not at the beginning, it can be more narrow, but certainly it can evolve to cover everything. JITs touch everything.
> success criteria
Impossible to measure. JITs don't universally make things faster and unless the SC was willing to accept "some programs will see around x% improvement" then it would be impossible to actually give them the criteria they want. There's a reason C++ compilers and their optimizations aren't defined by the ISO committee.
> support plan
Like most JITs, support for a JIT is an ongoing and ever evolving process. What's needed depends entirely on what's needed today. You usually need more than 1 or 2 dev working on it, which may be the issue here.
> It’s just that open-ended experimentation doesn’t belong in main, of all places.
Very strong disagree. That's simply the nature of performance optimizations for JITs. They belong on main because new optimizations and heuristics will always be generated. The role of doing a compiler is a long term experiment in performance, seeing what your runtime is currently doing and how it's being used and tweaking it to work better.
pypy isn't in a worse place because it's been running a longterm performance experiment.
> They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
None of the things mentioned actually do that, other than perhaps a support plan. There are plenty of critical infrastructure projects who all can manage to develop JITs. Python is the special snowflake that has decided to do JIT development incorrectly.
This is a runtime optimization/implementation detail. It's foolish to send it through a committee approval process. That's why you won't be able to find a single language with a JIT or advanced optimizations that has do it by committee.
In that case I can understand why it's not a fit for Python. The risks you're describing are more what a brand new language would face: universal scope, impossible to measure success, no support plan... sounds like a nightmare in a mature product. If that's actually what it means to add a JIT to Python, the SC is doing its job by steering Python around this quixotic task.
> This is a runtime optimization/implementation detail.
I view JITs as a critical architectural component. Like you said, it impacts the whole system. Optimization and implementation details don't usually do that.
The problem is that “Python” the language includes native poorly isolated FFI interfaces exposed by CPython. So the language isn’t “Python” but CPython. That’s why alternate implementations like PyPy haven’t seen adoption despite having a very mature JIT while languages like JavaScript have JITs that put CPython’s performance to shame.
It’s also part of why Python has been so successful in many domains. I was there for Python’s takeover of the data space, and I got quite used to porting things from Java (static, JIT, decent at multithreading) to Python (dynamic, interpreted, GIL) and realizing substantial performance improvements in the process.
It’s kind of wild how nice Python’s C extension system, which enables basically zero cost interop with native code, is for certain purposes. It supports a situation that’s comparable to the games industry’s tradition of putting a Lua or Lisp tip on top of a C++ iceberg.
> The problem
Who is this a problem for?
I think there is enough evidence to conclude that most language ecosystems cannot support multiple implementations that are fit for general purpose usage. There are almost no successful examples, and there are many, many failures.
The most successful things I can think of:
C & C++ - often does require at least some pre-processor logic or different build system settings.
Java - I'm not 100% sure - I think there are credible implementations of both the JVM and compiler though of course almost everyone is building from the openjdk or the commercial fork.
Nodejs & Bun - Most of NPM just works.
...
Thats it. Honorable mentions like C#/Mono exist, but if you look at that ecosystem a majority of applications only ever built with one of those, not both.
There are niches maybe like some subsets of scheme or common lisp - but I think most apps have a single implementation creating the builds that they ship. There are re-implementations perhaps that would count that I don't know enough about like Delphi/Lazarus, but I truly don't think there is another mainstream example.
In Haskell, SPJ noted years ago that GHC was really the reference implementation and he no longer considered standards work to be worthwhile (though it has continued past that).
Others I think have reach the same conclusions, such as Ruby. I could be missing something but I don't think most platforms should even aspire to support this.
And yet, not introducing unnecessary chaos into the CPython project remains important.
Perhaps these priorities were always irreconcilable.
Perhaps that’s why the existence of competing Python JIT projects that weren’t trying to commit directly into CPython’s main branch kept coming up in the conversation.
What are you basing this statement on?
The experiments had long been done. The topic at hand was including it as a core feature.
> Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
Bureaucracy doesn't definitionally mean Kafkaesque. It's popular to hate on it, but good or neutral bureaucracy does exist.
It doesn’t have to mean kafkaesque, but it does mean governance through layered hierarchies and fixed procedures.
In this case a single committee directly engaged with both core developers and the community, through a fairly open process, explicitly bent several of their own official rules to suit the occasion, and indicated cases where further flexibility might be needed depending on how things develop.
I would call that a pretty stellar example of non-bureaucratic governance.
Fair enough. I was reacting more to the "organizational hell" part and didn't focus as much on the "layers" part.
I have a hard time seeing it as organizational hell, either? From the outside, it looks like the process went smoothly and reached a decision in about half the time they originally estimated.
It’s true that that decision didn’t involve everybody getting their way. But that’s not organizational hell, that’s just organization. If you don’t want to contend with other people’s competing interests, the solution to that is solo development.
HN when AI is used to bomb a school: technology is neutral.
HN when an organization achieves its intended goals: this can't be bureaucracy because it's good.
Looking at amount of JIT related (for example type confusion) security bugs in V8 I am not surprised other developers are vary of its inclusion.
Large codebases are bureaucracies of their own, that is, the complexity of extra layers starts to bog things down at some point.
In this case, it seems like the human bureaucracy made wise and measured decisions to contain the software bureaucracy. They made reasonable requests to clarify the scope and impact of a major, major core feature. The proposers didn't feel like that was doable, so a complexity nuke was avoided.
How many Python JIT experiments have failed? I find it interesting because Ruby is probably harder to optimize than Python but Ruby has had a JIT for a long time. Maybe base python is more optimized than base Ruby so the JIT is incrementally not changing much? Also, I'd expect javascript to be harder to optimize but it has JITs. That could just be a funding issue, though. There's probably much more money and incentive to optimize Javascript.
The type of dynamic language gymnastics done with Python is huge: any sufficiently well known library basically employs some sort of meta-programming of the language:
JavaScript never had that level of flexibility (for better and for worse), and by extension, it's easier to build an alternative implementation and keep it compatible with the rest of the world that claims to be JavaScript.> The type of dynamic language gymnastics done with Python is huge
Have you seen Ruby?
Ruby is essentially Smalltalk (see also Objective-C).
I don't know Python's innards well enough to understand why it's so especially difficult in this regard, but I gather around the edges that its object model may not be as well founded.
JIT compiling Ruby or Python in a way that speeds up code using existing libraries and frameworks is harder. We built up decades of code that does stuff that it would be a lot easier for JIT performance if we didn't do, like using C extensions instead of defined FFI.
https://arxiv.org/html/2609.01502v1
The common excuse that ignores Smalltalk, Self, Lisp Machines like Interlisp-D, and more recently Ruby.
All just as dynamic as Python, and with quite usable JIT compilers.
Or for that matter, PyPy and GraalPy do exist.
The common excuse is indeed wrong, as pypy proved. IMO the real issue is the python C API.
The python C API is the "killer app" of python, and what gave its edge against otherwise similar languages (ruby/js). That's makes it unique (the whole scientific stack). But a lot of early choices leaked into this API, and this (+ preference from guido for simple implementation) is at odds with doing JIT: lots of global state (instead of explicitly passing interpreter state), ref counting, no tagged pointers meaning boxing/unboxing everywhere, etc. Some of those are getting gradually fixed, but that's a slog, though maybe AI can accelerate a lot.
I had high hopes for HPy but that unfortunately fizzled out.
Obviously, I am not saying making a JIT for Python is impossible: both PyPy and GraalPy prove otherwise. We are talking about full ecosystem support, not just the base language and stdlib.
Can you just replace CPython with PyPy or GraalPy in every project? And why not? :)
Because the Python community writes libraries in C, C++, Fortran and Rust, to make up for the CPython slowness, using CPython C API, calls them Python libraries, and they naturally only work in CPython.
PyPy and GraalPy then never get to use those pseudo "Python" libraries, because their API for C based extensions differs enough that you cannot take a random shared library and load them into either.
They came up with a common C API proposal, but has hardly taken that much adoption.
Indeed, mixing all the language features and anti-features with non-native Python libraries is a big reason too — both because it mitigates many of the performance issues a JIT would solve, but mostly because it complicates or makes the implementation unfeasible altogether.
Operator overloading is a non-issue, it's just another form of method calls.
JavaScript also has monkey-patching, and it's also very common. E.g. SolidJS replaces objects with dependency-tracking proxies.
There are really no fundamental language reasons why Python should be more complex. From what I heard from a JIT developer, the main issue is not the code itself but the module system and code loading.
JavaScript was designed from the start to operate inside a sealed environment with only few external handles, with the code loaded upfront or added on-demand via "eval". Not so for Python, where native code modules can (and will) result in code loaded dynamically and it's hard to track what depends on what.
See Self and Smalltalk for such kind of highly dynamic environment with code being loaded, and completely changing the running image on the fly.
Yet both languages are on the genesis of modern JIT compilers, going beyond what was already being done for Lisps, Prolog, BASIC.
> JavaScript also has monkey-patching, and it's also very common. E.g. SolidJS replaces objects with dependency-tracking proxies.
And the proxies make it slow as Python. Plain objects cannot suddenly become proxies, so the runtime can treat them as a special case.
> Many libraries depend on monkey-patching live library calls, including ones to external dependencies
someone please think of the monkey patchers!
Python is harder to JIT because of the way __dict__ works.
If all python had was __slots__ then we would not be struggling to JIT python.
Tight numerical loops is where the JIT adds performance.
Most performance sensitive features are already available in python between numpy, numba and Cython without really leaving the language core.
You'll mostly be speeding up the slow parts of python, not supercharging the fast parts.
> Python is harder to JIT because of the way __dict__ works.
Heh, how about how `__class__` works? Not that I recommend actually doing this, but you can literally change the type of an already instantiated object by reassigning `__class__`:
I've been doing things in Python for over 20 years. Only recently I learn that I can do this.
Smalltalk, Self, Common Lisp, Ruby as well, no big deal in regards to JIT code.
This is wild. Is it part of Python's specification, or an implementation detail of CPython?
As far as I understand, CPython implementation is THE specification. Some billions dollar ERP system firmly rely on this behavior, btw.
Yeah, I was afraid you would say that.
Just check with pypy and Graalpy. Both support this as well.
Is the question whether multiple base classes are part of Python’s spec, or if the ability to assign to `__class__` is part of the spec?
That's also possible in JS with __proto__. The prototypes of the classes in the chain can be mutated at runtime too. How to handle these cases is well understood for JITs though and python could do the same.
See also the warning here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
And this related article:
https://mathiasbynens.be/notes/prototypes
The thing that actually makes it harder in Python is threading, because it breaks all ability to reason locally about what might happen and optimize. In Python you have to assume that another thread may do these things at any point mid-execution.
I see no issue here.
Modifying a variable that is read by another thread needs special care in all multithreaded languages I know. If you don't use a synchronization primitive to tell the other thread "hey, this thing here may have changed, read it again from memory"; the other thread is free to use its old value for as long as it whishes.
Smalltalk, Self and Lisp Machines enter the bar, with live image changes.
Lua has the same property with metatables. LuaJIT, which is very fast, has a solution: if you change a metatable after assigning it, or reassign a metatable, Mike Pall turns his back on you. Your code dies slowly, in the outer darkness of the interpreter loop, never to be JIT’ted upon again.
I've never bought this argument. Underneath all of the magic inheritance in Python is just the MRO (method resolution order). That's not characteristically any different from JavaScript's prototype chain. There's no reason a Python JIT couldn't optimize everything JavaScript does, and if you do specifically access `self.__dict__`, that can be a cold, unoptimizable path.
One of the tricks here is to treat the happy path as a quasi-invariant and leave a guard that branches to slow code if the invariant didn’t hold.
I saw this presented as new tech at either OOPSLA or OSCon almost exactly 20 years ago.
CPU branch prediction is still pretty good even with the loss of function in recent years.
That works fine as long as nobody actually uses the unhappy path. If major libraries and frameworks do it all the time, you lose. So there is a certain aspect of culture outside what the language itself is capable of doing.
If there's common real world paths that need to be supported, you can tailor optimizations to target those uses cases.
I'd say that's about right. The way people usually code in Python, mro is quite stable and most lookups would be cache hits. You can still get more weird invalidation in Python cuz they let you change stuff like `__bases__`, or you get some weird `__getattribute__` change in some place that would force you to invalidate more than you'd initially think should be necessary.
Same as for the common lisp or raku mop. You check for the default, and use the fast path if so. And if you can guarantee that nobody changed those keys or values, lift that check to the front.
See Self.
> How many Python JIT experiments have failed?
Wait, how many? I'd say just this one. The other that I know of, pypy, was massively successful for a single maintainer project. And definitely showed the possibility/potential of JIT python.
I wouldn't call it a failure for not over taking mainline CPython.
>> How many Python JIT experiments have failed?
> Wait, how many? I'd say just this one.
I can name at least a couple others: Unladen Swallow, Pyston.
> The other that I know of, pypy, was massively successful for a single maintainer project.
PyPy had multiple active core developers during the 2000s.
> Maybe base python is more optimized than base Ruby
Not since before Ruby 1.9, no.
If you look at the reasons for this it seems like a good decision: https://discuss.python.org/t/pep-836-jit-go-brrr-the-path-to...
There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
>There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
And all of them are lacking in different ways, we ALSO, want faster pure Python.
They all have drawbacks, but they also reduce the case for needing to speed up CPython.
Pypy does provide faster pure Python. Its a compatible implementation with JIT.
The newest version of python that PyPy supports is 3.12. CPython is on 3.15.
And from what I understand there are limitations with using libraries that use the CPython API to interface with native code.
3.12 vs 3.15 would be fine, it's not like most depend on cutting edge features.
But the libraries limitations, and that it's not the "real" Python project, do get you.
Onlything is a lot of cextensions are super slow there. that is only bottleneck left which could be solved with resources that those guys spent on experimental jit.
Different guys. The CPython guys are not those behind Pypy. And they also don't want to see Pypy replace their project.
The problem with PyPy, which I'm a huge fan of BTW, is it's on life-support. The only reason the past couple major versions were released was due to a few small companies or individuals directly implementing the updates themselves. That's my understanding at least. It's disheartening because I've been primarily using PyPy for 5 years, advocated for it, and was looking into contributing more to it before finding out about its state.
Perhaps if the money which was spent on the built-in JIT had been spent on PyPy we'd be somewhere.
PyPy wasn't great for me because its startup time interfered with the main usecase I wanted it for in one instance and in another there bugs that made it not quite compatible enough. Money could solve the bug issue.
On the other hand I did get pygame working with it at one point and that seemed like a nice scenario.
PyPy's problem with slow startup is, in principle, easily fixable.
When it was created, PyPy did not seek to optimise startup performance. Instead, its aim was steady-state performance of JITed code, and it succeeded handsomely. A conceptually simple fix would be to optimise traces in one processor core, while the interpreter runs in another. Once when the trace is finalised, does interpretation stop and switch to native execution. But PyPy is currently a sequential system and making it parallel, while conceptually easy, is a lot of work and needs a rewrite of the whole system. But, for better or worse, the PyPy team does not have the resources to do this.
The other of PyPy's big performance issues (Python code using a lot of FFI calls via a suboptimal FFI) is also, in principle, easily fixable. But this is a social problem: the Python community would have to move towards better FFIs.
In principle, it would also be easy to fix PyPy's resource problem. I am not sure about the social problem ...
At least CPython could agree to fix its FFI so that PyPy and others would have a chance.
CPython could agree, and they have been asked to do so. But they refuse, because it would be a lot of work.
What would have been really nice is if the $1M/y in funding the PSF rejected due to ideological purity could have been reallocated to PyPy.
That's why I never want to work in CPython core. The 2 jit PEP's were totally fine, the implementation solid and stable, and even easy to understand for the SC manager types (unlike all other JITs which are much more convoluted), the future impact promising (already faster than Luajit in the Lua port). But then they got on their power trips. profiling frames? Jit templates are jit templates for micro ops, what's not to understand? The very same architecture and ops they use in core already.
As we see over and over again in python core development. It's the worst.
A sad day for CPython but it doesn't have to be a sad day for Python. There is a fully maintained implementation of Python 3.13 that has an advanced JIT compiler. Once your Python warms up it can be 4x faster or higher (depending on what your program is doing). That implementation is called GraalPy and it builds on the JVM:
https://graalpy.org/
A bit like Chrome there's a fully open source version and a proprietary version with a few extra bells and whistles, but the latter is entirely free for both open and commercial usage (you can't redistribute it for a fee). So you may as well use it.
It has other interesting features
• Scalable pauseless GC
• Single binary distribution of Python programs (like Go)
• Fancy and optimized interop with other languages including Java, JavaScript, Kotlin, Ruby and any other Truffle language.
• Supports native extensions. Tested and works with common libs like SciPy, NumPy, PyTorch, Pandas etc: https://graalpy.org/python-developers/compatibility/
• They're working on a GIL-free mode
In recent weeks there has been a release of a new web framework for Python that only runs on GraalPy, called Pyronaut (https://pyronaut.io/). It takes advantage of JVM interop to expose the Micronaut framework, which is similar to Spring for those who know it. Thanks to the JIT and well optimized base libraries it runs extremely fast, being about 3x faster than FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many more features.
https://www.youtube.com/watch?v=zo6L7bxn4OY
(that was presented at Devoxx but the Pyronaut devs will start touring Python specific conferences soon)
So although CPython may not have made it past the JIT milestone, that doesn't mean Python as a language or community has to fail it. It does require openness to alternative implementations, but historically Pythonistas have been fairly good about that (e.g. Pypy was embraced back when it was more active).
This stuff does require you to accept that it's made by Oracle, but as it's open source, that should be OK.
Disclosure: I sit next to some of the people who work on this stuff.
GraalPy and PyPy are great technology achievements, the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.
>the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.
They are Python libraries. Don't make the mistake of screwing up the cause and effect here. C, Rust etc - those are the replaceable parts of the chain. There is an official C++ pytorch library and virtually nobody uses it for ML Research. People want to code in python and build those libraries so they can.
Nope, they are Python bindings to native libraries.
PyTorch in C++ naturally gets little use, because most people don't want to use C++, and although possible, e.g. CINT, very few care about C++ REPLs.
Also these libraries only exist because Python cannot deliver on its own, like Lisp or Scheme, and needs such libraries in first place.
Even BASIC has better options to do machine code and system programming than Python, starting with the original Dartmouth BASIC.
I know what bindings are. A front-end to C++ for pytorch was only released from 1.0, literally 2 years later. The relationship is very clear here. There was no native C++ library. It was an afterthought. Even in instances where there was, like Tensorflow, functionally it was still an afterthought.
They are python libraries because people reached for those languages so they could continue work in python. They are there to serve python and nothing else. It's the same with Rust. It's by no means a C++ thing.
EDIT: Not saying this is always the case. Qt>Pyside is an example where it's both technically and functionally bindings, but the vast majority of the time they are serving python. Some don't even have a native library to speak of.
For Python libraries they would be written in Python, eventually ctypes for syscalls.
Otherwise they are bindings to native libraries.
We can keep discussing in circles.
>For Python libraries they would be written in Python, eventually ctypes for syscalls.
So if you can't call/use the 'library' from the 'native' language, it's still a library? Whatever floats your boat. Lol.
I think it’s actually reasonable to say that a Blub library should run on every implementation of Blub. So, GraalBlub and BlubBlub, not just CBlub.
Otherwise it’s just a sparkling FFI.
Are you arguing that JIT Python or GraalPy will be as performant in a reasonable time as Python on top of those C++/Rust libraries? If not then I don't think it matters since unless you are claiming that all the existing C++/Rust libraries can be re-written in reasonable time and with the same performance. Keep in mind that many of these libraries are using low level tricks to optimize performance. Python is the UI and while the UI is important in the end it's just the UI.
Yes, well, even with fancy JITs Python will never be a speed monster. Interop with libraries written in Java can be very high performance though as Pyronaut shows. And the heavy lifters that are hard to replace like NumPy and PyTorch do work with GraalPy.
I think if the Python community wants an implementation that has a proven JIT and where the remaining work is easily scaled compatibility testing, then there's one available and ready. Whether they're calling into C or Rust or FORTRAN or Java shouldn't make a big difference in principle.
Now, I've met lots of people who think that binding to Rust will always be the fastest thing to do but that underestimates Java significantly, especially once you factor in the fact that GraalPy can optimize out interop overheads.
Consider LLM tokenization. It's one of the most heavily optimized pieces of code on the planet. Yet, toknroll written in Java can beat tiktoken which is written in Rust:
https://github.com/qxoticai/qxotic/tree/main/toknroll
Moreover the Rust code has some unsafe{} blocks as Rust often does, whereas toknroll has no unsafe code anywhere.
So the scripting language communities need to think hard about where they go next, IMHO. DHH already got up and announced at a Rails conference that Ruby is over, you should use Rust for everything. I don't think that's quite true, but it's going to be a meme that with agents you should just pick the fastest language and go. If Python/Ruby/etc want to stay relevant then they need to care a lot more about their performance story. Just being more pleasant to read and write for humans doesn't cut it anymore. CPython is clearly not going to happen in this regard, so alternative implementations need to start being taken a lot more seriously than they were in the past.
> Once your Python warms up
How long does that take?
The Pyronaut performance page has some graphs with the warmup parameters specified precisely:
https://pyronaut.io/performance/
> Latency lines come from the load ramp (+25% per 15-second step); throughput from 45-second validation steps (+2% each) after a 180-second warm-up. Each framework ran twice and we show its best run. No profilers were attached.
So that's a "realistic" web serving benchmark if you're willing to grant the use of a GraalPy-optimized framework rather than Flask/FastAPI.
Warmup times for other benchmarks are currently quite unstable because there's a big focus on interpreter performance at the moment, which is one of the main influences on warmup. There's a blog post here from August about how the compiler is being taught to recognize and optimize bytecode interpreter loops, to get results that normally require handcrafted assembly:
https://medium.com/graalvm/escaping-the-giant-switch-dec20b5...
So as you know already, warmup was historically a weakness of PE based stuff - although in this case the fair comparison is with CPython which never warms up at all - but a fast interpreter will make a big difference. In some benchmarks the GraalPy interpreter is already faster than the CPython interpreter, which is heavily optimized.
Gonna dump a technical hot take here.
Python is a mature production language with a lot of users. Even before it was huge - like 20 years ago when I was in college - getting a small backwards compatible API change into a python extension module (gnu readline in my case) required convincing GvR personally that the API change was necessary.
Every JIT attempt I've seen for python just tries to cut too deep in the first pass, and scope themselves too large.
I believe any successful attempt at optimizing python performance from JIT will FIRST start with a 100% semantically compatible hidden type model for optimizing object property access and method dispatch.
The single biggest performance delta in most dynamic language jits come from that, not from optimizing guards off of bytecode operations or from compiling to machine code. It's from converting hashtable lookups into constant-time guard + offset lookups.
The hidden type model is also a hard nut to crack. It needs to perserve dict semantics which have very dynamic behaviour via the various `__*__` methods, and handle exotics scope objects since scoped name lookups are the most common ones which have their own exotic behaviours (I remember python providing some ability for callee frames to inspect caller scopes in some limited ways).
> It's from converting hashtable lookups into constant-time guard + offset lookups.
Python already does it in a lot of cases, see opcode specializations like LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_SLOT.
I haven't kept up with python development lately, so thanks for pointing that out. I looked this up and it does seem like some sort of property access is being optimized. What's not clear is what the hidden runtime type model is being used for this, or if it's a lightweight tracking that captures some subset of property accesses.
More digging led to this: https://peps.python.org/pep-0659/ which seems to be what the effort is organized around?
Python's object semantics are broadly similar to Javascript, bar the prototypical inheritance chain, but with the extra complexity of potentially needing to handle dynamic `__*attr__` overrides. Optimizing for this would require a tree of hidden map objects much like JS JIT engines use. I dug around the codebase a while back looking for something like that but didn't find it. `OrderedDict` seemed promising but after more investigation that turned out to just be adding property ordering to dictionaries.
The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement. Once you have the hidden type model, you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
> Optimizing for this would require a tree of hidden map objects much like JS JIT engines use.
I don’t really think that’s true or even that relevant , at least in context of classes (especially with slots - the class, by definition, _is_ the “hidden map”) and not generic dicts.
> The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement.
Pretty sure the motivation was exactly the opposite - that a simple “if a LOAD_ATTR encounters the same class with slots N times in a row, swap it out for LOAD_ATTR_SLOT with inline cached slot index, and swap back to generic one if something changes” is way easier (both to implement and reason about) than any alternative that’d interact with the object model. (Also it means you skip having to branch between a slow and fast path at all)
> you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
Sorry, that sounds a bit vague and not too different from what CPython already is doing.
Ah, the `__slots__` feature makes it make a lot more sense.
> Sorry, that sounds a bit vague and not too different from what CPython already is doing.
With slots being available, I can see where I was misled. I went looking for a type structure that modeled the dynamic properties on an object which would be used as a typeguard, but the class itself functions as that if you're restricting the optimization to slotted properties.
Appreciate the clarfication!
(Note that this optimization can also trigger for plain classes (that’s LOAD_ATTR_INSTANCE_VALUE), it just needs a bit more guards and is only useful if you don’t abuse instances being dynamic objects. I just used slots as the most obvious example, but there are lots of different op specializations since like 3.11)
And we note that these specializations are part of production Python and will not be removed when the JIT is removed.
This has been very informative. With object property/method lookup optimized the next big leap in JIT-style perf is actually a hard pull. The next big perf win lies behind local type inference, and a method-jit with your standard bevy of SSA optimizations to hoist the typeguards.
I suppose the various JIT projects are trying to come at that from different angles incrementally, but that's a genuinely hard problem as you don't see real benefits until multiple complex pieces of infrastructure are in place and with good coverage over the language.
Anyway, my hot take was misplaced.
It still blows my mind that the big "we're breaking backwards compatibility Python 3000" release didn't do anything to lay the groundwork for this, or any of Python's other structural problems.
They weren't rebuilding the whole language.
https://peps.python.org/pep-3000/ - PEP 3000 says it will be a derived from the Python 2 codebase and not a complete rewrite (referencing Joel Spolsky's views on complete rewrites and Netscape Navigator).
https://peps.python.org/pep-3099/ - PEP 3099 is changes Guido van Rossum rejected from Python 3 (including := as a binding operator), parallel iteration, programmable syntax, parser going beyond LL(1), changing the for/else construct.
https://peps.python.org/pep-3100/ - PEP 3100 is things he set as goals for it, things like all strings are Unicode and byte arrays are a separate type, optional type annotations, 'exec' and 'print' change from statements to functions, None/True/False become keywords, old-style classes are gone, changing inequality comparison from <> to != , require C99, drop BeOS support, drop old standard library modules, and more.
actually, more changes than I remember.
For a long time Guido wanted CPython to remain simple, as the reference implementation.
Probably also during the 2 to 3 migration, but that goal appears to have now been abandoned.
I wonder how the authors of other implementations like pypy will be able to keep up as the reference implementation becomes more and more complex.
That question is already answered. pypy only supports 3.11 today and is largely unmaintained.
https://pypy.org/posts/2026/10/pypy-v801-release.html
Last release on Oct 14. Beta support for 3.12. It's not that bad really and 1/10 of the people who complain about it did something then the situation would be better.
> the reference implementation.
The problem was he exposed too much of the implementation into the language itself, destroying its ability to be a simple reference implementation that others could re-implement.
This is slowly being addressed now, each version of Python sees the c API become more opaque.
That's at best a partial fix for the API side. Python side CPython sill exposes a LOT of implementation details, and as a result occasionally projects will choose to rely on some of these details. Then other implementations need to decide whether to also pick the same implementations details and expose them, or use different details, and lack compatibility with any library that depended upon them.
The python documentation has historically also not been especially clear at distinguishing between what the language guarantees, vs what is just implementation details of the cPython implementation. So even if libraries wanted to try to avoid depending on implementation specifics, it isn't always clear what those are.
The net result is that for the end user python the language is effectively: whatever cpython does. This really, really discourages creating alternate implementations, as the only way to be an alternative implementation with good compatibility is to be a living fork of cpython, constantly merging in all the changes from it, while also maintaining your improvements.
This is also slowly being addressed
Are there any PEPs that describe this work?
They reference this one in the article https://peps.python.org/pep-0836/
I meant the trend of making the C interop more opaque.
See the stable ABI for Python 3.15 and PEP 793, 803, 820 for recent movements in this general direction and the following draft: https://github.com/py-ni/peps/blob/main/peps/pep-08xx.rst (the py-ni project is pushing for this)
At the time, Python had a clear purpose as an easy to learn "glue" language between other, perhaps more performant languages. There was no reason to make Python as fast because that would muddy its purpose by introducing compromises and complexity. For better or worse Python sat pretty well in its niche.
Today there are far more people using it, and everyone has a different perspective on what it is. No longer is it a niche glue language, now it's a backbone of AI technology at major tech firms, so it's understandable they'd like it to be faster on its own.
Its possible this could have been the concrete pillar that broke the camel's back and set Python 3 on the path of Perl 6.
This is what happens what the defacto implementation, CPython and its external API, is effectively the specification. That includes all the incidental side effects that were not explicitly designed, but inherited as a quirk of historical implementation. It's next to impossible to retrofit a JIT given these constraints. ECMAScript has a proper specification and several implementations.
The poison-pill requirements had their intended effect: https://discuss.python.org/t/an-announcement-from-the-steeri...
It would have been more honest to just cut the feature instead of haranguing a developer until they withdrew it.
I mean, if your multi-year-old project is ever asked to just take a moment to write out a quick formal roadmap and justification ... it is doomed, 100% of the time. The actual content of the request doesn't really matter, it's dead.
To be clear, such a request doesn't even mean that someone is consciously attempting to kill the project (although in the corporate world, ime it probably does). It just means that something went critically wrong. In this case, it appears like python's development process really required a PEP for this sort of work, especially since it was happening on the main branch.
The sus bit is that the PEP they requested greatly increased the scope of the feature.
>> For example, rather than proposing one single concrete JIT implementation, it may make more sense for the PEP to describe a JIT infrastructure that can support multiple implementation strategies.
> we don't feel we have the personal bandwidth to carry the current version forward
Are the JIT developers the same people who were working on Faster CPython at Microsoft? No surprise that they don't have the same amount of energy to dedicate to the JIT now that it's gone from a full-time job to a hobby project.
That team was mostly hired away by NVIDIA.
What's the status of 'Faster CPython'?
I wouldn't be surprised if they all went nodejs instead :/
While JIT is being dropped, the original proposer and leader of the initiative was hired by ARM and continues work on speeding up Python.
His current focus seems to be speeding up GC and free threading.
Microsoft is not financing it any more and laid off most of the team working on it.
https://github.com/facebookincubator/cinderx might be a better approach.
CinderX isn't generally faster unless you use "Static Python" which is a major change in how programs are written compared to typical python code.
Out of the box it's comparable to the current JIT, but yes, major gains are only possible with at least type annotations. But that may be the only real way forward for a JIT implementation for CPython with the ability to gradually speed up slow parts of large programs.
https://facebookincubator.github.io/cinderx/#performance
I did some experiments translating a typed subset of Python to C for small system use, certain features like lists of primitive types, generators and comprehensions, frozendict, simple classes, generic functions, etc, map with little or no cost but are so much cleaner than the equivalent code in plain C. You also get to write test harnesses using all the tools available to full Python. The main issue is unsurprisingly memory management, Obj-C / Swift style automatic reference counting works but has drawbacks, and it gets much more complicated if you want to mix the static and CPython code.
The static + AOT approach has a lot of promise now that type annotations are widely adopted, predictable behavior and good performance on code that is mostly idiomatic Python seem like a good alternative in many situations to the weight of Rust.
Yes, exactly. If you can get a 2-3x speedup in CPython just with type annotations, and then a possible additional 2-3x by dropping down to "Static Python," that may be an overall near optimal solution for the ecosystem, all things considered.
I didn't touch JIT (complexity and footprint way beyond what I can use or manage myself in any case) but the AOT path is pretty much what you'd get if you wrote naive C by hand. The emitted code is readable, in some cases you probably do better because more efficient algorithms are easier to code or reuse, but with the current refcounting that will have some overhead. I do want to try some ideas like supporting an @soa decorator for a List of DataClass type that keeps the visible interface but changes the memory layout. If someone wanted to sponsor tokens I could probably pure robot through a stage of fleshing out enough to port a few representative libraries and tune against the results.
1. Python is slow, therefor we use C extensions. 2. We cannot make Python faster because of C extensions. 3. Go to 1.
Surprising, even though (IIRC) the performance of the JIT was always fairly underwhelming.
What about the other major change to Python's implementation: free threading? Has it lived up to expectations? I assume it's not also in doubt, despite Guido recently saying "Personally I wish we’d stuck with the GIL"? [0]
[0] https://discuss.python.org/t/pep-805-safe-parallel-python/10...
Getting a JIT from underwhelming to actually good is always going to be a whole lot of iterative work to remove blocking parts of the existing implementation. In Ruby-land we have had a good JIT for a while now, but it took many release cycles and a ton of resources before it was able to outperform the non JITed version when using Rails. Don't expect quick fixes that make great strides and any investments in it needs to be persistent and apply over time.
It boggles my mind that it apparently is the case that such a popular language as Python isn't able to get the resources to do this.
It is more than that. CRuby have had multiple JIT attempt before YJIT landed. From what I can tell CPython didn't have that. And luckily enough we now have ZJIT replacing YJIT.
From https://discuss.python.org/t/pep-836-jit-go-brrr-the-path-to... :
> We want the PEP to be clear that it’s ultimately the JIT team’s responsibility to find solutions for these packages and tools, and how to deal with situations where a tool can’t be supported or made to work
Is it just me or is this absurdly impossible, to the point of willfully killing the project? Committing a team of brilliant compiler engineers to doing free consulting ad infinitum for every bit of tooling in the world, however hacked-together?
The alternative would be to have the bravery to say: this is the way forward to keep this language relevant into the future, and as a steering committee we collectively own that some tooling may be out of date. Instead, this is a signal that Python is going to rest on its laurels for a very long time.
Python use is widely diverse. Anyone wishing to contribute to that which is published to such a diverse community, all of whom expect Python to continue working for them without any regressions, has to be ready to cater to that diversity in its entirety.
Given that the speedup is modest (~10%) and the Python 3 migration caused enough pain for users, I get it though. If it was a 2x speedup it would be a different story IMO
Even with 2x speed, I don’t think it would be worth it. Do we really want a repeat of that with years of packages not working with each other and a bunch of smug people telling others to update their code or whatever.
Sad to read this. I had hoped that with the advance of AI capabilities, features like this could be prototyped, tested, implemented and shipped even faster. And I was really looking forward to see performance gains for my favourite language.
Where do we go from here? Microsoft did not continue to finance the Faster CPython project and the current speed gains are good, but not what we hoped.
A biased opinion as I dev in F#, but it strikes me that AI is going to play best with functionally pure languages (strongly typed, no side effects, whatever we call that these days)?
F# is on the softest side of that, but letting the compiler tell the AI "nope, here's why" seems like a huge win in saving cycles and time.
The downside at first was simply, "how many examples of F# are there to learn from", and early on it showed, but I think that's been long overcome.
And F# is obviously NOT the best example since it's both "soft" on functional purity and niche. I imagine something like haskell is probably doing even better, although I've yet to check that at all (not that I can code haskell but I haven't looked at anecdotes or anything).
I realize this doesn't really help anyone who's grown to love python, and there's surely no perfect language, but just like how the web catapulted javascript, I see a future where the answer isn't "use AI to make python better" so much as "what already works super well with AI, and does it ease the user adoption"
The biggest winner from AI indeed appears to be Rust.
3GL code generation is only a transition step until models get more mature for direct machine code generation.
AI needs formal verification, not a borrow checker.
At this point if you have a python codebase where perf is an issue, the "AI-pilled" answer is to rewrite it in a faster language.
But I want to continue to understand the code. I dont know Rust (maybe AI can teach me?) and if I let Claude rewrite my current 50k LOC project in Rust, how can I understand what is happening and how can I work on it and think it further?
Rust is not your only option. Nor is it automatically the best option, though it is certainly a solid option for a certain set of requirements. Go is a pretty decent match for a Python rewrite; superficially it seems like you're giving up a lot but under the hood they're philosophically more compatible than meets the eye in a lot of ways. But any static language that you have some other local reasons to prefer is going to blow its socks off in terms of performance. And "blow its socks off" is just for the naive translation of a conventional Python code base over to a new language. If your codebase is amenable to threading and can take advantage of multiple CPUs somehow, the result may blow Python's socks off twice over.
The problem is that Python just isn't going to go fast. There are some languages that may have had more resources poured into them to make them go faster in absolute terms, like perhaps Java. But I don't think there's a single language that has had so many ways of trying to make it go faster given so much effort over so many decades. It's really time to call it... if you have Python code and it's a performance problem, if one of the many many many different solutions aren't adequate for some reason, Python is not going to get any faster than it is today and you're going to have to move off of it.
If you need high performance code, Python is simply not the right answer and you're going to have to skill up. Or accept what you have, which you'll have to decide if that's a viable option. If it's fast enough I don't advocate switching casually. But if it isn't, there isn't a realistic prospect that you're going to wake up in a year or two to a 30x-faster Python. I wasn't terribly convinced by that claim back in the early 2000s, but it hadn't really been tested yet. It has been tested now.
Pypy is really fast. Have you tried it?
First, "if one of the many many many different solutions aren't adequate for some reason" was there for a reason.
Second, no, it isn't. Pypy is really fast compared to Python. "Compared to Python" is just about the lowest bar there is, though. In absolute terms it is still only a medium-fast language runtime, and it comes with a lot of costs and tradeoffs.
That's the problem with Python. Lots of ways to take a very slow language and make it a medium-fast language with substantial tradeoffs, or make it a fast language with enormous tradeoffs. If you can handle the tradeoffs, fine. But if you can't you just need to switch to a language that is fast without anywhere near as many tradeoffs.
RustPython it's pretty mature.
Not at all.
> RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
- https://github.com/RustPython/RustPython
I've seen worse reimplementations. For what it does it's pretty close.
I mean, it'd be pretty easy to speed it up by 10x — just need to drop backwards compat and do away with the ridiculous runtime dynamicism that is only truly useful for <%1 use cases.
But yeah, clearly nobody's willing to rip off the bandaid, so that's not happening.
You could try having it rewrite parts of the hot path into Rust with PyO3.
Have the AI find the slow parts of your code. Then have it find libraries that do that slow part fast (they're almost certainly in Rust/C++) since thats low hanging fruit. Then have it re-write any remaining slow parts in Rust with Py03. Eventually that hits a limit but hopefully by then you understand Rust enough or it's fast enough to not matter.
There are already many solutions to the performance issue from which you can pick. You don't need to understand the C in numpy to use it for example. And I'd say it's highly unlikely that you have a use case that, if it's applicable, isn't already addressed. Like it's generally pointless to optimize language performance for code that's doing a lot of network operations, because network lag will always dwarf such optimizations.
There's so much Python code that uses numpy etc but still ends up having some hot part in Python that's nontrivial to translate to something faster. Same reason people get frustrated with the GIL.
Those would be pretty niche applications, as again it's basically all solved in some way. More solutions out there than needed IMO.
It's non-niche enough that they made multiprocessing and then put special focus on removing the GIL.
> But I want to continue to understand the code.
You don't have to throw away the original Python codebase.
You can have:
1. An implementation in slow language (i. e. Python) that you maintain manually or semi-manually (with minimal AI assistance) and understand well.
2. A fast implementation that is fully generated and maintained by AI, using slow implementation (1) as a reference.
You can, for example, run two "adversarial" agents, one writes tests that compare output of (1) and (2) and tries to find a discrepancy, the other tries to fix (2) so that its output never differs from (1).
This way, you can be reasonably confident that (2) does exactly the same thing as (1), despite being a blackbox that you don't understand. While (1) remains a reference implementation that you understand well and can work with.
It's already typical to call into a native library with Python bindings for performance critical work in Python. Even Pydantic's validation runs in Rust code.
This works well if there are obvious hot spots that can be easily isolated, but sometimes the perf issues are more uniformly distributed throughout a codebase.
Mature projects simply don't like large unexpected code drops if it is not made clear beforehand what the blast radius is and how it is going to be maintained. On top of that it seems that the performance gains weren't as expected and require further invasive changes.
Implementation effort has never been what has held these efforts back. LLM helps with squirting out code, but the hard tasks are benchmarking, profiling, community engagement, knowledge management, and dealing with the hard questions such as whether the change is actually worth the additional complexity.
These concerns won't go away even if we give up on human developers being able to grok the code and instead completely relying on LLMs for coding.
We pick languages that embrace machine code like Julia, Mojo, Common Lisp, Scheme, and stop calling libraries that are written in C, C++, Fortran, Rust, "Python Libraries", when they are thin bindings.
Clearly the community doesn't want PyPy, GraalPy, or even a CPython JIT.
Sad news, I put a lot of hope in this project. At least we keep the base interpreter improvements they brought along, and we still have PyPy. CPython JIT is dead. Long live PyPy!
It clearly isn't hurting Python's popularity, but man the performance of the reference implementation is embarrassing.
V8 made JavaScript fast in 2008, soon that will be 20 years ago.
Perhaps I live in a bubble, but so much of the production Python code I see is fully type-hinted and makes used of an aggressive linter & type checker to the point where it might as well be written in Go/C#/Java/TypeScript so that it can actually take advantage of the performance gains instead of being loaded into a runtime that turns everything into heap-allocated PyObject pointers.
I work in Python every day and think about this a lot - people really don't seem worried about its performance. Sure people want it to be faster, but I don't see very many people changing their language choice for a faster language, despite Python being a lot slower than most alternatives.
If I think about where I see python being used though, this kind of makes sense, the two big use cases are:
- Web APIs and backends, where for the most part either the network or the database is the bottleneck
- Data science work, where for the most part, work exists in libraries like polars, pytorch etc that ship the main work out to other languages.
So from that perspective, although we'd all like python do be faster, it's speed isn't the bottleneck in the use cases where its popular (that's probably also the reason why we don't see Python getting used in other speed critical places).
That's a really different story to JavaScript, which doesn't have a handy CFFI or otherwise you can just reach out to. There are a lot of cases where JavaScript speed improving would speed up the reactivity of a web app, even now.
I'm not arguing against a faster CPython - I'd love one! But I think there's a clear reason why it hasn't been prioritised historically in the way JavaScript's performance has.
This is exactly how I write my Python. I'm currently debating between going in deep and transitioning to either TypeScript or Rust. Go's object system and error handling is too weak. Java is too verbose, but I have a soft spot for it.
C# is worth a look if you want less verbose Java with async/await like you have in other languages, and better performance than Typescript.
Modern Java is not that verbose, plus you can use Scala, Kotlin, Clojure instead.
New java is not very verbose.
If you want V8 like optimizations, you have to introduce a strict mode.
Python interpreter is known to leak its implementation (__class_ modification after creation, monkey patching with __getattr__, sys._getframe, eval/exec, deterministic call to __del__, no proper python API for libraries, etc.)
You have to plug all of those holes before having a chance to get a decent JIT. Otherwise it will be just another JIT attempt, like Psyco, Unladen Swallow, Pyston.
Python is not uniquely dynamic to a degree that would prevent an efficient implementation.
V8 predates strict mode, and handles all the exec/eval, monkey patching and object system abuse you can do in JavaScript just fine. Common Lisp is one of the most dynamic languages in the world and allows you to do ridiculous things in terms of meta-programming and essentially building domain-specific languages into your program, yet it too manages to have ahead-of-time compiled implementations that are very performant. Also PyPy exists.
There are definitely some technical aspects that make it hard to make Python fast, but they are not insurmountable. The reason CPython is slow today is simply because performance has not historically been prioritized by its developers.
I only just saw it mentioned in this thread, but CinderX might suit you well.
They should had just port everything to pyPy.
It seems as if today, Python is in most cases a very well designed frontend for accelerators and their proprietary, vectorized, massively parallel IRs. Those libraries (Jax, MLX, etc) already take care of JIT code optimizations - Polyhedral et al - and therefore it makes sense for the frontend to not worry about runtime performance.
What’s the latest on getting rid of the GIL? Did that work out?
It did. It's slower though.
And you get it back with threads, which can now scale.
I didn't need compute threads. Just I/O threads. So now it's just slower.
Python needs less features, not more.
You work within the confines of the process and the existing workflow.
If you can't do that, then, forking is the right strategy.
It does not suggest one strategy is better than the other.
Plenty of `grown ups` disagree; It's natural.
ffffffff.... damnit I love python but why is it so difficult like this. JIT would have been so perfect so the language, increasingly overtime with AI code coming through I am not sure not focusing on making the performance better is going to be better for longer term. Even though a lot of the other operations done in python are done faster in libraries I would still like faster startup times, or faster loops.
This makes me think that mojo could actually become a python 4 of sorts meaning a python with more emphasis en performance and support for native gpu programming
Hope the copy-and-patch work gets a proper write-up before it’s removed... The technique is interesting well beyond CPython.
Have people been finding bugs? Having trouble with it? It does work, right? Surely, this is just bureaucratic nonsense.
This seems to be all about drift from the main implementation of Cpython.
If JIT is different enough in maintenance, security best practices, and consumer use, we don’t have two compilers, we have two different languages.
And that’s the crux of it.
Maybe Python isn’t a language for when performance is a big concern (at least in the user code) and that’s OK?
What a shame. The alternative is CinderX? It's slower unless you stick to "Static Python" which is not most python code in the wild.
I hope it does not lead to performance issues later. Python is already famous for being slower than compiled languages.
Being slower than a compiled language is not saying much for an interpreted language. But it's slower than PHP which is a closer comparison, and PHP has had JIT for six years now.
Ruby has the reputation of being a very slow language, but Python has been slower still since Ruby jumped ahead with version 1.9, almost 20 years ago. Ruby also has had far more success with JIT compilers than Python has.
>Ruby also has had far more success with JIT compilers than Python has.
Ruby is on its third (I think?) JIT now? I'm a little surprised, given Ruby's reputation as a wacky "anything goes" language with a very complex grammar, and it makes me wonder why that is? Does it have to do with the design of the runtime itself more than the language semantics? Or is it more incidental, e.g. many popular ibs are C wrappers, already generates bytecode files, and so on.
The difference is that Ruby follow the BDFL model, and Ruby's BDFL was willing to allow people to experiment with in-tree development of JITs.
If the Python steering committee isn't willing to do the same, they'll probably won't get a working JIT any time soon. Developing a JIT externally, or even from a branch, is just too hard.
Getting a JIT right is hard work because time spent by the JIT is time not spent doing actual work, and finding the correct tradeoffs is very much an empirical problem.
And PHP is quite the hot mess. Always has been, really. See the security issues over the years, including the most recent for example. Meanwhile I've never seen any serious issues for Python that's triggered mass required fixing. There's something to be said for moving carefully and ensuring things aren't broken to the best of one's ability, which is the SC's purpose.
https://app.opencve.io/cve/?product=python&vendor=python
Are you helping me to prove my point? I see 156 results, with 1 in 2023 and the rest before that. Meanwhile there are 801 results with at least a dozen in 2026 for PHP[0], and we still have 2 more months to go.
[0] https://app.opencve.io/cve/?vendor=php
Hardly, one CVE is enough.
Not at all. I specifically stated "serious issues for Python that's triggered mass required fixing", ie. it was so bad that there were public notices for users to update ASAP. The closest thing I can find is CVE-2022-34265, and that's in Django, not Python itself.
Python is hardly used at PHP scale for Web development.
Agreed. And Python usage is far more diversified. But I fail to see the relevance of usage to a language being stable vs ridden with issues. Well, beyond that one would expect the one being so many things to so many different communities would have more issues than the other used by a single community for a single purpose.
and the biggest one, JS
It is also slower than any other interpreted dynamic language in wide use.
This is why we cannot have nice things.
Why had Python to succeed to Lisp in the AI wave? (rhetoric question)
Since PyPy and GraalPy will never get the love they deserve, I guess there is hope that Julia, Mojo, and the whole Rewrite in Go/Rust keep pushing along then.
I don't know why they don't just adopt Pypy as a faster Python implementation. Just let CPython develop slowly as the reference and give support to Pypy. It is already much faster than CPython.
The Rust based Python3 port can keep it and it can still be far more secure than CPython. Not everything it's JIT'ted on RustPython, it's ondemand per function, you have to enable it explicitly.
Python is a poor target for compilation, even the best JIT is never gonna make fundamental design flaws (that some may consider beloved features) disappear. It's never going to give "good" performance.
In my view, the right course of action is to specify an official strictly typed subset of the language with compilation and static analysis as the justification, then implement both JIT and AOT on top of that. Legacy code can be rewritten or thrown out in phases, C extensions can be ported.
Of course, any such proposal would invite monumental amounts of bikeshedding, which is why it wouldn't get far, much like the JIT proposal.
> strictly typed subset of the language with compilation and static analysis
This is a great idea for popular dynamic languages with existing users and codebases, particularly the focus on "subset" to limit the scope and emphasizing that full backward compatibility is not a goal; and allowing for gradual, incremental adoption. That makes it a lot more practical to implement and adopt.
I think we'll see more of this. Recent examples include strictly typed TypeScript subset designed for ahead-of-time compilation.
That’s effectively what Mojo is. Hardly catching the world on fire but pretty good for what it does.
Python is no doubt a successful language. But, boy - this level of bureaucracy is annoying to no ends. Do people feel eager to propose a PEP? There are so many more enjoyable things to do in one's short life ...
Assuming that the biggest bang-for-buck of JIT compilation is optimizing hot loops (which is also the biggest bang-for-buck of optimization in every other context as well), and assuming that, per the historical best practices, all hot loops in Python are already written in C (or Rust, these days), this outcome is somewhat what I would expect.
The performance of python indicates otherwise. There are some performance critical libraries written in C or C++ but a lot of python is written in python.
Python in itself is poorly suited to ever become performant as it lacks the types required. I imagine that causes some issues for effective jit-ing too.
They had a chance to fix this when adding type annotations, but instead of making it "optional/gradual typing" they opted to make it "documentation or linting support", which is obviously the wrong choice, but there were cultural reasons
Does optional/gradual typing even help with JIT?
If you know the type you need fewer checks in the jitted code, and you can optimise to native types.
If it helps python specifically, given that the types themselves are pretty dynamic, who knows. A python integer doesn't map well to a native integer etc. Python really isn't designed for performance, and retrofitting it is hard.
Makes sense. And that's rough, even the integers need dynamic memory. Maybe they can just use 64-bit static ints and throw an exception (or even crash) if you go oob. Probably quite rare that anything would rely on huge ints.
Interesting, makes sense. JIT in CPython main is a big deal, glad they’re proceeding with caution. Overall looks like they added it as an experimental feature for two years, then decided to back it out. Perfectly reasonable outcome.
It looks like it very much should never have happened like this. The actually hard and awkward questions have not been asked beforehand, and the SC doesn't like their hands being forced, and rightly so.
what features would need to be removed to help python hit a baseline level of performance? adding typing? Some of the “functional” stuff. Is the performance a legacy issue or essential to the language’s design?
From my limited understanding, the C interop is the biggest obstacle to both JIT and GIL. It's also the feature that makes Python fast enough for so many use cases. And the dynamic typing is the reason it even has to be JIT instead of just pre-compiled.
bureaucracy and politics kills another innovation, fork found in kitchen
As usual, though, I might just be biased by default towards defending the Python aristocracy; they seem to pretty much always be on the right side of history, even in the little ways!
This is pretty bittersweet to see.
Python is my favorite language, but I'm honestly tired of having to rewrite performance bottlenecks in Cython or Rust all the time. When the Faster CPython project started with Microsoft's backing, I was really excited. Mark Shannon had that plan promising huge speedups across a few cycles, and while that clearly turned out to be much harder than anticipated, the team still kept pushing forward to deliver a real JIT.
Seeing all of that get bogged down in committee bureaucracy now is just frustrating.
I have a lot of respect for the core devs and I know maintaining CPython, and it is a time donation job. It isn't easy. But I feel that over-caution and red tape ends up discouraging serious contributions. They are setting an unreasonably high bar for something that would obviously help the entire ecosystem.
Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Python really deserves a proper JIT. I genuinely hope they reconsider this.
Unladen Swallow died for lack of upstream interest.
PyPy lacks upstream interest, looked like it could have a shot after many years of development but recently NumPy removed support because they just couldn’t fathom how a non-dead project could be a couple of versions behind CPython.
Now the new JIT is dying because of what seems to be a lack of upstream interest. (The things TFA describes as being required are not unreasonable, but not realistic for a necessarily speculative long-running project without wild amounts of committed funding.)
I’m not optimistic.
> Unladen Swallow died for lack of upstream interest
IIRC Unladen Swallow was, despite the extreme hype, nothing more than the internship project of a couple of students. It died because their internship was over.
Otherwise you're right, though. Lots of CPython alternatives suffer from lack of upstream interest - the Python language designers don't seem very interested in anything other than CPython. In particular, they appear to only consider CPython when adding features to the language. Two examples:
- Requiring `dict`s to be ordered because that's how they work in CPython, without considering how they work in other implementations
- Adding pattern matching syntax that's suitable for CPython's PEG parser, even though it's not LL(1) so can't be parsed by, say, MicroPython
Core Python is antagonistic to all other Python implementations. There is only one Python.
> Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Every single time I remember the first AI wave was done with Lisp and Prolog, both with tooling that supports machine code generation since about 50 years.
> committee bureaucracy
That's what's kept the language and ecosystem improving and from becoming a hot mess. If something isn't properly specced then it really shouldn't be in prod. They allowed the experiment in the first place so it'd be easier to gather sufficient data to guide said speccing, but the ball was dropped by the proposers and so the experiment was an ultimate failure (unlike free threading) and prod is now being appropriately cleaned up.
It's not red tape, it's about making sure that the effort actually delivers what it promises and can continue to so in the future. From the state of the draft PEP it seems it doesn't and it won't ever. The SC merely put the nail into the coffin.
I know not everyone loves them; but `Claude rewrite the performance bottlenecks in Rust` works so well it is hard to see the security surface area and harder reason about performance as something that should be paid in core.
Done https://github.com/can1357/pon
Seems unmaintained from July.
I think its a meme but an interesting proof of concept nevertheless
This would never have happened with a BDFL
Lol, it seems then they want to have a crippled ecosystem on purpose.
I actually expected something good to come out of this, my mistake for trusting Python "engineers". Never again.
It's strange that typescript has 5 different LLM written competing static translators to compiled languages and python is killing JIT because the PEP was not accepted.
cPython is killing it. You are very free to vibe code a new python interpreter that does JIT.
In the remote eventuality that it should actually work and be maintained up to date and secure, I'm sure it will acquire users.
More interested in creating a variant of python that's practical.
Wrote a parser with a rust compatible pattern matching syntax here:
https://github.com/py2many/spy-ast
Should help both JIT and static translation.
It doesn’t matter. Just write it in Rust. AI will make Python less relevant in the future.
Other than the trolling, you couldn't be more wrong.
AI literally turned Python from this silly esolang that PhDs seem to love into the Lingua Franca of Computing.
All LLM tools are built Python-first and all LLMs are trained Python-first.
If there is anything you can believe in beyond gravity and taxes is that an LLM's first tokens on a new shell will be py thon 3.
It seems like such a failing and waste of human life that we spend all this time waiting around for python when the JS engines are so cutting edge.
Why is this controversial? We spend hours waiting for python code when it could be much faster if it was treated like a serious programming language
I think it's mainly because of how easy C interop is in Python. Same reason no other interpreter replaced CPython.
> Same reason no other interpreter replaced CPython.
Source?
I just use Go and Rust for everything; most of my mobile apps are in Flutter. I don't see any point in using anything else.
for me it's over for python
I would not like a JIT to be embedded in python for one reason. The sole reason why Python didn't have all the security related bugs of other languages (just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities) is sorely because it's an interpreted language, that is python is compiled in a bytecode that runs in a VM.
Under these premises is very difficult that something not implemented correctly is really exploitable, and moreover it's simple to verify the security of the whole thing.
Opening a JIT, that would mean mapping memory pages as RWX, would open the door to all soft of exploits that now are not really possible.
While I don't disagree with you in principle, the context for JS JIT vulns and potential Python JIT vulns is pretty different and worth taking into consideration.
For JS, your machine running untrusted JS code is the intended use case. In this scenario the JS is sandboxed in whatever JS engine (V8 etc) and does not have access to "dangerous" methods, and the main use case for a JIT vuln is to escape this sandbox to execute code on the host. This applies to regular internet browsing (and visiting potentially malicious sites) and also, e.g., using the built-in browser in a locked down device (such as game consoles) to gain arbitrary code execution.
For Python, this security boundary doesn't exist. I can't really think of a scenario where you would be running potentially malicious Python code in CPython that would need a JIT vuln to escape whatever sandbox it is in instead of, say, calling the functions in Python directly (or using the C FFI to do whatever). I'm sure that in some niche use cases, this scenario might exist, but to me, it seems like the potential attack surface is fairly minimal.
EDIT: Also, I'm pretty sure that Python's copy-and-patch JIT writes code to a writable mapping and then mprotects it to W^X? I don't think it leaves RWX pages lying around at all.
Who is running sandboxed untrusted Python code?
JS has a completely different threat model. And PHP has just been a basketcase from inception.
> just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities
Can you name some of these JS engine vulnerabilities? And the PHP one had nothing to do with JIT.
Not OP, but JS engine JIT bugs are pretty common.
https://phrack.org/issues/70/exploiting-logic-bugs-in-javasc...
https://www.sentinelone.com/vulnerability-database/cve-2026-...
https://projectzero.google/2020/09/jitsploitation-one.html
https://www.zerodayinitiative.com/blog/2021/12/6/two-birds-w...
This argument would be more convincing if, due to performance reasons, most Python code didn't use C extensions, where exploits are highly prevalent.
I would want a python implementation with a JIT and the ability to choose JIT or no-JIT when defining a python environment. And I want all my libraries to work either way.
I'll get what I want eventually but I might be retired by then.