Inlining in Java

What is method inlining?

Inlining is an optimization performed by the Java Just-In-Time compiler.

If you have a method:

public int addPlusOne(int a, int b) {
  return a + b + 1;
}

which you call like this:

public void testAddPlusOne() {
  int v1 = addPlusOne(2, 5);
  int v2 = addPlusOne(7, 13);

  // do something with v1, v2
}

the compiler may decide to replace your function call with the body of the function, so the result would effectively look like this:

public void testAddPlusOne() {
  int v1 = 2 + 5 + 1;
  int v2 = 7 + 13 + 1

  // do something with v1, v2
}

The compiler does this to save the overhead of actually making a function call, which would involve pushing each parameter on to the stack.

This can clearly only be done for non-virtual functions. Consider what would happen if the method was overriden in a sub class and the type of the object containing the method isn't known until runtime...how would the compiler know what code to copy: the base class's method body or the sub class's method body? Since all methods are virtual by default in Java, you can explicitly mark those which cannot be overriden as final (or put them into a final class). This will help the compiler figure out that method will never be overriden, and it is safe to inline. (Note that the compiler can sometimes make this determination for non-final methods as well.)

Also, note the word may in the quote. Final methods aren't guaranteed to be inlineable. There are various ways you can guarantee a method isn't capable of being inlined, but no way to force the compiler to inline. It will almost always know better than you anyway when inlining will help vs. hurt the speed of the resulting code.

See wikipedia for a good overview of benefits and problems.

Inlining in Java

A couple of the other answers have suggested that only final methods can be inlined - this is not true, as HotSpot is smart enough to be able to inline non-final methods so long as they haven't been overridden yet. When a class is loaded which overrides the method, it can undo its optimisation. Obviously making the method final mean that's never required...

Basically let the JVM do its job - it's likely to be a lot better at working out where to inline than you are.

Do you have a situation where you're convinced that the JVM isn't doing a good job? Assuming you're using HotSpot, have you tried using the server version instead of client? That can make a huge difference.

What is inlining?

When executing a given piece of code, whenever you call a standard function the execution time is slightly higher than dumping there the code contained into that function. Dumping every time the whole code contained in a function is on the other end unmainteinable because it obviously leads to a whole mess of duplication of code.

Inlining solves the performance and maintainability issue by letting you declare the function as inline (at least in C++), so that when you call that function - instead of having your app jumping around at runtime - the code in the inline function is injected at compile time every time that given function is called.

Downside of this is that - if you inline big functions which you call a lot of times - the size of your program may significantly increase (best practices suggest to do it only on small functions indeed).

Is there anything that will stop java inlining a method?

If your method is too big, or has method which have been inlined too many times already, your method won't get inlined.

Can all methods be inlined, or are there certain elements of my code that would stop java from inlining a method?

There is a number of parameters which control this.

$ java -XX:+PrintFlagsFinal -version | grep Inline
     bool C1ProfileInlinedCalls                     = true                                {C1 product}
     intx FreqInlineSize                            = 325                                 {pd product}
     bool IncrementalInline                         = true                                {C2 product}
     bool Inline                                    = true                                {product}
    ccstr InlineDataFile                            =                                     {product}
     intx InlineSmallCode                           = 2000                                {pd product}
     bool InlineSynchronizedMethods                 = true                                {C1 product}
     intx MaxInlineLevel                            = 9                                   {product}
     intx MaxInlineSize                             = 35                                  {product}
     intx MaxRecursiveInlineLevel                   = 1                                   {product}
     intx Tier23InlineeNotifyFreqLog                = 20                                  {product}
     bool UseInlineCaches                           = true                                {product}
     bool UseOnlyInlinedBimorphic                   = true                                {C2 product}

Of note: the MaxInlineSize limits the depth of inlining. In general this is not worth increasing as it can increase you code size and slow you program. The FrehInlineSize is the maximum size even frequently called methods will be inlined. I have found that increasing this a little can help for some programs.

The MaxInlineSize is the number of bytes that a small method needs to be to be inlined even if it is not frequently called.

Java inline method that contains private calls

1. in the beginning, there is source code
2. the java compiler turns it into bytecode
3. the jvm reads the bytecode, verifies its integrity, and interprets it. The bytecode of "hot spots" (frequently executed blocks of code) is just-in-time compiled into the instruction set of the CPU the JVM is running on

In Java, most optimization is deferred to the JIT-compiler in step 3 to benefit from statistics gathered during program execution. This includes inlining.

Access modifiers such as private are checked during verification, before the bytecode is interpreted, and before it is JIT-compiled. Since access modifiers have already been checked, the JIT is totally agnostic of their existence, and inlining a private method causes no difficulty.

BTW, if you want to check what the JIT does, you can use hotspot's -XX:+PrintAssembly to have the machine code translated back into assembly code for your inspection.

JVM disable inlining for a particular class or method

You can use the -XX:CompileCommand JVM option to control just-in-time compilation. The option can be used for excluding certain methods (or all methods of a class) to be compiled, and more. From the documentation:

Specifies a command to perform on a method. For example, to exclude the indexOf() method of the String class from being compiled, use the following:

-XX:CompileCommand=exclude,java/lang/String.indexOf

If you only want to prevent method inlining, you can use the dontinline command with the same syntax, e.g.

-XX:CompileCommand=dontinline,java/lang/String.indexOf

The same JVM option is used internally by the popular Java microbenchmark harness, JMH.

Do modern Java compilers/JVM inline functions/methods which are called exactly from one place?

A Java JITC will attempt to inline any functions that appear (based on runtime statistics) to be called often enough to merit it. It doesn't matter whether the function is called in only one place or dozens of places -- each calling site is analyzed separately.

Note that the decision is based on several factors. How big the method is is one -- if there are a lot of potential inlining candidates only the most profitable will be inlined, to avoid "code bloat". But the frequency of the call (multiplied by the perceived expense of the call) is the biggest "score" factor.

One thing that will discourage inlining is obvious polymorphic calls. If a call might be polymorphic it must be "guarded" by code that will execute the original call if the arriving class is not the expected one. If statistics prove that a call is frequently polymorphic (and including all the polymorphic variants is not worthwhile) then it's likely not sufficiently profitable to inline. A static or final method is the most attractive, since it requires no guard.

Another thing that can discourage inlining (and a lot of other stuff) is, oddly enough, a failure to return from the method. If you have a method that's entered and then loops 10 million times internally without returning, the JITC never gets a chance to "swap out" the interpreted method and "swap in" the compiled one. But JITCs overcome this to a degree by using techniques for compiling only part of a method, leaving the rest interpreted.

---

Related Topics