The inliner is yielding benefits for ZJIT
Originally published on Rails At Scale . We recently enabled a really cool feature in ZJIT that makes it feel like a Real Compiler™: the inliner! We’ll write more about it soon. In this post, we’ll talk about one excellent concrete benefit we are already seeing and how it optimizes blocks in pretty much every Ruby program. I’ll start off with a refresher on how blocks work in the Ruby interpreter, then show you how ZJIT understands and optimizes that bytecode, and then show you the impact of the inliner. In the beginning, there were loops. People used them to navigate and manipulate variable-length structures, like arrays and strings. This was fine. Then, in the 1970s, a small group of computer scientists at Palo Alto Research Center invented a programming language called Smalltalk. One of the core features of Smalltalk was that everything was an object and computation was done by sending messages to objects. This meant that iteration wouldn’t do at all. Instead, we would have to send the message to the array object and pass it a block object. Then, in the 1990s, Matz, inspired by Smalltalk and Perl, created Ruby. We still have “normal” loops but we also have a very Smalltalk-y way of doing it, too: When this program gets compiled to Ruby bytecode, it ends up looking like a mostly normal method call to except that we pass a special kind of argument to it: a block argument. To see how this works inside CRuby, we’re going to look at a listing of YARV bytecode—CRuby bytecode. For more on YARV, I recommend Kevin Newton’s excellent Advent of YARV . Ignore most of the bytecode dump below except for the instruction at , the instruction. We are send -ing (see? a message!) with the block argument (passed a different way than “normal” arguments, hence the ). This from the bytecode listing above is the generated name of the instruction sequence (bytecode) corresponding to the block we passed to . Its code, shown below, is the next thing in the bytecode dump. You can see the usage of local variables and via and variants and also , which represents addition ( ). Again, the details are not terribly important: To make this work, has an method. This method takes in its optional block argument and calls it once for every element in the array. As with most of the core data structures, ’s methods tend to be written in C and is no different. Here is its nice and short definition with comments added by me: You can’t really see the block parameter to the C code because it’s passed in a special location on the Ruby VM’s own stack called the block handler . All you need to know is that knows where to find that and how to call it. There’s just one more thing, which is… hang on, weren’t we building a JIT to optimize Ruby code? How are we going to optimize this C code? Rewriting code from C to Ruby means that the JIT compiler gets a chance to introspect the run-time behavior and code. This means that, over time, as the compiler and the runtime system grow together, more and more code might get rewritten in Ruby. This started happening a couple of years ago with YJIT. YJIT precipitated some interesting changes to Ruby VM internals. For example, in 2022, being written in C started to hurt: it was an opaque blob that the JIT couldn’t reason about. So Kokubun submitted a PR to rewrite it in Ruby. After some back and forth, in 2024, Kokubun landed a different PR that everyone was happy with. Ah, finally. A version that JITs can reason about. I keep saying “reason about” and what that means concretely here is a) that it’s written in a format that the JIT can ingest and optimize: Ruby, and b) mostly a brief rehashing of the key lessons in the venerable Smalltalk (!) paper Efficient Implementation of the Smalltalk-80 System (PDF): (And if you don’t believe me that the lessons still apply, check out the excellent paper Who You Gonna Call (PDF) by Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr.) So JITs like to watch what types of objects flow through methods before compiling them. And JITs like to, since they know the types of objects, cache method lookups and specialize method invocations on those objects. For example, take a look at this code: could be anything. There is no way of knowing its type by looking at the code. This means that the method could be anything. Furthermore, there is no way of knowing what the return type of is, so we can’t specialize the method lookups or invocations of or either. But! Per our lessons above, likely only a few types flow through this code. Say the JIT’s profiler notices that has historically been an . Then we might reasonably assume that it will continue to be an , so when we compile the method, we add a run-time type check: if the type is no longer an , jump back into the interpreter. Let’s see what optimized code ZJIT can construct by combining profiling information with the above bytecode. ZJIT operates on its own high-level intermediate representation called, uncreatively, HIR. In the following HIR snippet, we can see this very run-time type check (“guard”) for the class: (with real pointers replaced by fake ones for readability) Because classes are, among other things, collections of methods, this type information tells us what the call target of is: it’s ! We have a special fast code snippet to read an array’s length so we do that instead of a method call. And in case the methods ever get changed out from underneath us, we leave behind these markers called that invalidate the code. Finally, because we know that the result of is always a small integer ( ), we can special case the method lookups for and as well. There you have it. This is how JITs work: observe, assume, specialize. So why am I telling you all this? How does this circle back to blocks? Well, blocks work not quite the same way, but similarly. Instead of having an object that we call a method on, we just have the target instruction sequence 1 . But if we observe that the block argument, in its special location, has been consistently one object, we can specialize the call to it. This is more or less fine for some cases of code. For example, in the following code snippet, we only have one caller to , so its profiled block will be monomorphic (one observed shape). This reinforces what we know and love about the Smalltalk-80 paper: code locality wins! Yes! But unfortunately this falls apart when we start thinking about all of the core library methods (and potentially the methods and classes you have stashed away in the grab-bag in your application). Those methods are probably megamorphic (many many observed shapes). They probably see all sorts of stuff because they are general-purpose utilities that everybody needs, all the time. One such example in the Ruby core library is the venerable that we saw earlier. Because there are a million different call sites to across your application and each probably passes a totally different block, we’re in an unhappy situation. How can we possibly optimize for so many different blocks? What happened to our code locality? How do we fix this? It’s okay. Code locality still rules. Look at all the various callers of . They all pass a different, but constant 2 , block at the call-site: So the code locality that we need in to optimize the code is one level up at the caller. If we can use that call context , we can specialize the code. YJIT accomplishes this by splitting , which is a very natural transformation for basic block versioning and tracing. Such compilers are good at following code paths as they would be executed and putting together context across method calls. However, YJIT’s heuristic for splitting blocks is based on manual annotations: it will only kick in for certain Ruby library functions specially annotated with (and the name is a bit of a misnomer). The team that builds ZJIT, a method JIT, decided not to add splitting facilities. We could split methods (and blocks), but we have an easier time reasoning about larger code units than YJIT does because we optimize an entire method at once. So instead, ZJIT chooses to get call context by inlining . Method inlining refers to copying the body of the callee into the caller. In the above example, it means copying the body of into each of , , and . I don’t mean the Ruby code and I don’t mean the bytecode: I mean the HIR. ZJIT does this by building the HIR of the callee ( ) into the existing HIR of the caller ( , …). The illustrious Kevin Menard wrote ZJIT’s inliner and he’ll write a post about all the details soon. It’s pretty interesting stuff. For now, we can take a look at the (lightly edited) result of being inlined into . The details don’t matter, but there are two things I want to call out: This is a massive improvement over the previous very generic operations. You may notice that there is still one call in the loop: the call to the block ( ). That’s next on our list to tackle. We are optimistic that we will soon also be able to inline block calls. Then the whole thing will really be just a loop! The code that enables us to reason about which block got passed into the inlined callee ( ) only landed a couple of days ago (July 10, 2026), written by Luke Gruber . This was one of Luke’s first changes to ZJIT. Well, for starters, the microbenchmarks that we use as “performance unit tests” for specific aspects of Ruby went wild. Some benchmarks that were using block-based looping got much faster; they were previously bounded by ZJIT’s block invocation performance. Take a look at the benchmark, which tests that we can fold away the call to Ruby’s built-in method. ZJIT ends up optimizing the method to invoke the block directly, and the block gets optimized to nothing but a guard on the self’s class to make sure it hasn’t changed. Because we had previously optimized the body away, the result of the direct block invocation is a massive speedup: Other benchmarks also kind of stop making sense because of the amount of inlining. Our bmethod benchmark, which benchmarked how fast we can call methods defined with , also stopped measuring anything of use. We’re going to have to rework the benchmark to be more fair… As expected, larger Rails benchmarks don’t see a ton of change; they exercise a diffuse set of features so optimizing any one feature bumps the big benchmarks only a little bit. I am excited to see what happens when we can fully turn and friends into call-less loops! Code locality rules. The inliner helps inject more of it and reason across method calls. ZJIT, a little over one year old, is growing up! :’) We’re still tuning the inliner knobs. Some of the code in this post required tweaking to convince the compiler to inline into because of ’s size. It will take some time for the ZJIT developers to figure out reasonable defaults. Try out our HIR explorer at tryzjit.fly.dev . Try out ZJIT in your application by adding the flag to a Ruby over 4.0. Thanks for reading and see you next time. Mostly. I am glossing over , procs, ifuncs, etc. But the common path is by and away iseq blocks. ↩ It’s not always the case that these methods are called with a constant block iseq. Sometimes they are called with the form, for example. Or with the form. In that case, we can use an inline cache to (with a guard) make it constant once more. But we have not implemented that yet, because it is rarer. ↩ The iteration variable is ! You can see it get used as an array index when it gets unboxed as and passed to . You can also see it get incremented with and the constant . In it has a different name, , which gets checked against the array length. ↩ Though systems may offer very dynamic behavior, people don’t frequently make use of wild features all over the place Most people pass fewer than 4 types of objects through a given method As a corollary, even if there are many classes in a system, code locality is super important, and we can take advantage of that Also, most people do not define, re-define, and otherwise continuously modify method definitions What used to be a dynamic is now what we call because we know from the call context what block is passing to . What used to be a method call is now a loop: the condition check is in , the body is in , and the stuff after the loop is / . If you’re interested, try to find the iteration variable and where it gets incremented. See the footnote 3 for the answer. Mostly. I am glossing over , procs, ifuncs, etc. But the common path is by and away iseq blocks. ↩ It’s not always the case that these methods are called with a constant block iseq. Sometimes they are called with the form, for example. Or with the form. In that case, we can use an inline cache to (with a guard) make it constant once more. But we have not implemented that yet, because it is rarer. ↩ The iteration variable is ! You can see it get used as an array index when it gets unboxed as and passed to . You can also see it get incremented with and the constant . In it has a different name, , which gets checked against the array length. ↩
C#
Web Development
Design
Programming
Writing
Career
AI
Performance
C
Ruby
Security
Web
CSS
Entertainment
Culture
Business
Media
DevOps
Backend
Journalism
Marketing
