What the JIT Actually Does to Your Hot Loop

A Java loop does not have one implementation. It has five, and it moves between them while it runs. If you have ever watched a benchmark get three times faster halfway through, or seen a method that is hot in production look unremarkable in a profiler, you were watching that migration and not seeing it.

This article is about reading those transitions rather than inferring them.

The five versions of your loop

Every method starts in the interpreter, which walks bytecode one instruction at a time and counts two things: how often the method is invoked, and how often a loop inside it branches backwards. Those two counters are the JVM’s entire notion of “hot”.

When a counter crosses its threshold, HotSpot compiles the method with C1, the client compiler. C1 is fast to run and produces mediocre code, and it comes in three flavours: no profiling, limited profiling, and full profiling. That last variant is the important one, because it instruments the compiled code to keep collecting the type and branch data that the next compiler will consume.

When the counters cross a much higher threshold, C2 compiles the method again, this time using everything C1’s instrumentation learned. C2 is slow to run and produces genuinely good code: it inlines aggressively, unrolls loops, hoists invariants, eliminates bounds checks it can prove redundant, and vectorises what it can.

HotSpot numbers these levels 0 to 4. A hot method’s usual route is 0 → 3 → 4 — interpreter, C1 profiled, C2 — and that is what “tiered compilation” means; level 1 takes trivial accessors that never need profiling, and level 2 is the stopgap when C2’s queue is long. The default thresholds are deliberately high, and that is the source of most benchmark confusion: a loop that runs ten thousand iterations may finish before the good version of it exists.

On-stack replacement, or why a long loop gets fast mid-flight

There is an obvious problem with counter-based compilation. Consider a method that is called exactly once and contains a loop running for a minute. Its invocation counter will never move. Waiting for the next call to pick up the compiled version means waiting forever.

HotSpot solves this with on-stack replacement: it compiles the loop body separately, keyed to the bytecode index of the backward branch, then transfers the running frame into the compiled version between iterations. The local variable state is mapped across, and the loop continues at speed without ever returning to its caller.

OSR compilations are marked with a % in the compilation log, and they matter for a practical reason: an OSR compilation is specialised to the loop it was made for, not to the method as a whole. A method that only ever runs through OSR can carry worse code than the same logic split into a method that gets called many times.

Reading the compilation log

The single most useful flag here is one that needs no agent and no tooling:

java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions \
     -XX:+PrintInlining MyApp

The output is one line per compilation event:

    113   34       3       com.example.Parser::next (42 bytes)
    114   35 %     3       com.example.Parser::scan @ 12 (86 bytes)
    287   36       4       com.example.Parser::next (42 bytes)
    288   34       3       com.example.Parser::next (42 bytes)   made not entrant: not used
    412   41       4       com.example.Parser::scan (86 bytes)
    906   36       4       com.example.Parser::next (42 bytes)   made not entrant: uncommon trap

The columns are timestamp in milliseconds, compilation id, flags, tier, and the method with its bytecode size. Read them as a story: next was compiled at tier 3 (C1 with full profiling) at 113ms; scan got an OSR compilation at bytecode index 12 one millisecond later; at 287ms next was recompiled at tier 4 by C2, and the tier 3 version was made not entrant so that new calls go to the new code.

Current JDKs append a reason to that marker, and the reason is the whole message. made not entrant: not used on a lower-tier method is bookkeeping — the tier 4 version replaced it. made not entrant: uncommon trap on a tier 4 method is a deoptimisation: C2 made an assumption that the program then violated. A monomorphic call site that suddenly sees a second implementation class, a branch marked as never-taken that finally gets taken, a null that was assumed impossible — each invalidates the compiled code, and execution falls back to the interpreter to rebuild a profile. A method that oscillates between tier 4 and deoptimisation is doing worse than one that never reached tier 4 at all.

The flag column carries four other characters worth recognising: s for a synchronized method, ! for one with an exception handler, b for a blocking compilation, and n for a native method. None is a problem by itself; each changes what the compiler is willing to do.

Inlining is the decision that matters

If you only look at one thing in -XX:+PrintInlining, look at the failures:

@ 27   com.example.Buffer::readByte (12 bytes)   inline (hot)
@ 31   com.example.Codec::decode (410 bytes)     too big
@ 45   com.example.Handler::apply (18 bytes)     virtual call

Inlining is the enabling optimisation. Escape analysis, constant folding across a call boundary, branch elimination based on a caller’s known argument — none of it happens through a call that was not inlined. HotSpot’s limits are bytecode size limits: roughly 35 bytes for a method that is not obviously hot, roughly 325 for one that is, and a hard ceiling on how deep the chain can go.

This is why “extract that into a helper method” is usually free and occasionally not. A 400-byte method that is called from a hot loop will not be inlined; the same logic split into three smaller methods often will be, and the loop then optimises as one unit.

virtual call is the other common failure, and it is C2 saying it could not resolve the call to a single target. A call site whose type profile has seen too many receiver types stops being a candidate for direct inlining, and the compiler falls back to a guarded dispatch or a real virtual call. Megamorphic call sites in the middle of hot loops are one of the few places where an interface abstraction has a measurable, structural cost.

Two nearby messages mean something different: hot method too big is the same size wall with the frequent-inline limit applied, and accessor is a trivial getter that was inlined by a dedicated fast path.

What to do with this

Three habits follow from the above.

Warm up before measuring. Not as a ritual, but until the compilation log stops producing tier 4 events for the code under test. JMH does this for you and reports the iteration where it stabilised; a hand-rolled System.nanoTime() loop does not.

Treat repeated deoptimisation as a bug. If a hot method keeps getting made not entrant, find the assumption being broken. Usually it is a call site that is monomorphic in testing and polymorphic in production, which is exactly the kind of difference a staging environment hides.

Read the size of your hot methods in bytecode, not in lines. javap -c gives the number the inliner actually uses. A method that reads as short and compiles to 380 bytes is a wall in the middle of your loop, and no amount of instruction-level tuning inside it will matter as much as getting it under the limit.

The compiler is not a black box. It publishes its decisions on request, in a format that takes an afternoon to learn to read, and those decisions explain more production performance mysteries than any amount of reasoning about what the source code “should” cost.