Does Java Have Any Mechanism for a Vm to Trace Method Calls on Itself, Without Using Javaagent, etc

Does java have any mechanism for a VM to trace method calls on itself, without using javaagent, etc?

There is no such API provided by the JVM— even for agents started with -javaagent. The JVM TI is a native interface provided for native agents started with the -agent option or for debuggers. Java agents might use the Instrumentation API which provides the lowlevel feature of class instrumentation but no direct profiling capability.

There are two types of profiling implementations, via sampling and via instrumentation.

Sampling works by recording stack traces (samples) periodically. This does not trace every method call but still detect hot spots as they occur multiple times in the recorded stack traces. The advantage is that it does not require agents nor special APIs and you have the control over the profiler’s overhead. You can implement it via the ThreadMXBean which allows you to get stack traces of all running threads. In fact, even a Thread.getAllStackTraces() would do but the ThreadMXBean provides more detailed information about the threads.

So the main task is to implement an efficient storage structure for the methods found in the stack traces, i.e. collapsing occurrences of the same method into single call tree items.

Here is an example of a very simple sampler working on its own JVM:

import java.lang.Thread.State;
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadInfo;
import java.lang.management.ThreadMXBean;
import java.util.*;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

public class Sampler {
  private static final ThreadMXBean TMX=ManagementFactory.getThreadMXBean();
  private static String CLASS, METHOD;
  private static CallTree ROOT;
  private static ScheduledExecutorService EXECUTOR;

  public static synchronized void startSampling(String className, String method) {
    if(EXECUTOR!=null) throw new IllegalStateException("sampling in progress");
    System.out.println("sampling started");
    CLASS=className;
    METHOD=method;
    EXECUTOR = Executors.newScheduledThreadPool(1);
    // "fixed delay" reduces overhead, "fixed rate" raises precision
    EXECUTOR.scheduleWithFixedDelay(new Runnable() {
      public void run() {
        newSample();
      }
    }, 150, 75, TimeUnit.MILLISECONDS);
  }
  public static synchronized CallTree stopSampling() throws InterruptedException {
    if(EXECUTOR==null) throw new IllegalStateException("no sampling in progress");
    EXECUTOR.shutdown();
    EXECUTOR.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
    EXECUTOR=null;
    final CallTree root = ROOT;
    ROOT=null;
    return root;
  }
  public static void printCallTree(CallTree t) {
    if(t==null) System.out.println("method not seen");
    else printCallTree(t, 0, 100);
  }
  private static void printCallTree(CallTree t, int ind, long percent) {
    long num=0;
    for(CallTree ch:t.values()) num+=ch.count;
    if(num==0) return;
    for(Map.Entry<List<String>,CallTree> ch:t.entrySet()) {
      CallTree cht=ch.getValue();
      StringBuilder sb = new StringBuilder();
      for(int p=0; p<ind; p++) sb.append(' ');
      final long chPercent = cht.count*percent/num;
      sb.append(chPercent).append("% (").append(cht.cpu*percent/num)
        .append("% cpu) ").append(ch.getKey()).append(" ");
      System.out.println(sb.toString());
      printCallTree(cht, ind+2, chPercent);
    }
  }
  static class CallTree extends HashMap<List<String>, CallTree> {
    long count=1, cpu;
    CallTree(boolean cpu) { if(cpu) this.cpu++; }
    CallTree getOrAdd(String cl, String m, boolean cpu) {
      List<String> key=Arrays.asList(cl, m);
      CallTree t=get(key);
      if(t!=null) { t.count++; if(cpu) t.cpu++; }
      else put(key, t=new CallTree(cpu));
      return t;
    }
  }
  static void newSample() {
    for(ThreadInfo ti:TMX.dumpAllThreads(false, false)) {
      final boolean cpu = ti.getThreadState()==State.RUNNABLE;
      StackTraceElement[] stack=ti.getStackTrace();
      for(int ix = stack.length-1; ix>=0; ix--) {
        StackTraceElement ste = stack[ix];
        if(!ste.getClassName().equals(CLASS)||!ste.getMethodName().equals(METHOD))
          continue;
        CallTree t=ROOT;
        if(t==null) ROOT=t=new CallTree(cpu);
        for(ix--; ix>=0; ix--) {
          ste = stack[ix];
          t=t.getOrAdd(ste.getClassName(), ste.getMethodName(), cpu);
        }
      }
    }
  }
}

---

Profilers hunting for every method invocation without going through the debugging API use instrumentation to add notification code to every method they are interested in. The advantage is that they never miss a method invocation but on the other hand they are adding a significant overhead to the execution which might influence the result when searching for hot spots. And it’s way more complicated to implement. I can’t give you a code example for such a byte code transformation.

The Instrumentation API is provided to Java agents only but in case you want to go into the Instrumentation direction, here is a program which demonstrates how to connect to its own JVM and load itself as a Java agent:

import java.io.*;
import java.lang.instrument.Instrumentation;
import java.lang.management.ManagementFactory;
import java.nio.ByteBuffer;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.UUID;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

// this API comes from the tools.jar of your JDK
import com.sun.tools.attach.*;

public class SelfAttacher {

  public static Instrumentation BACK_LINK;

  public static void main(String[] args) throws Exception {
 // create a special property to verify our JVM connection
    String magic=UUID.randomUUID().toString()+'/'+System.nanoTime();
    System.setProperty("magic", magic);
 // the easiest way uses the non-standardized runtime name string
    String name=ManagementFactory.getRuntimeMXBean().getName();
    int ix=name.indexOf('@');
    if(ix>=0) name=name.substring(0, ix);
    VirtualMachine vm;
    getVM: {
      try {
      vm = VirtualMachine.attach(name);
      if(magic.equals(vm.getSystemProperties().getProperty("magic")))
        break getVM;
      } catch(Exception ex){}
 //   if the easy way failed, try iterating over all local JVMs
      for(VirtualMachineDescriptor vd:VirtualMachine.list()) try {
        vm=VirtualMachine.attach(vd);
        if(magic.equals(vm.getSystemProperties().getProperty("magic")))
          break getVM;
        vm.detach();
      } catch(Exception ex){}
 //   could not find our own JVM or could not attach to it
      return;
    }
    System.out.println("attached to: "+vm.id()+'/'+vm.provider().type());
    vm.loadAgent(createJar().getAbsolutePath());
    synchronized(SelfAttacher.class) {
      while(BACK_LINK==null) SelfAttacher.class.wait();
    }
    System.out.println("Now I have hands on instrumentation: "+BACK_LINK);
    System.out.println(BACK_LINK.isModifiableClass(SelfAttacher.class));
    vm.detach();
  }
 // create a JAR file for the agent; since our class is already in class path
 // our jar consisting of a MANIFEST declaring our class as agent only
  private static File createJar() throws IOException {
    File f=File.createTempFile("agent", ".jar");
    f.deleteOnExit();
    Charset cs=StandardCharsets.ISO_8859_1;
    try(FileOutputStream fos=new FileOutputStream(f);
        ZipOutputStream os=new ZipOutputStream(fos)) {
      os.putNextEntry(new ZipEntry("META-INF/MANIFEST.MF"));
      ByteBuffer bb = cs.encode("Agent-Class: "+SelfAttacher.class.getName());
      os.write(bb.array(), bb.arrayOffset()+bb.position(), bb.remaining());
      os.write(10);
      os.closeEntry();
    }
    return f;
  }
 // invoked when the agent is loaded into the JVM, pass inst back to the caller
  public static void agentmain(String agentArgs, Instrumentation inst) {
    synchronized(SelfAttacher.class) {
      BACK_LINK=inst;
      SelfAttacher.class.notifyAll();
    }
  }
}

Load Java Agent In Tests Without JVM Arg

For JDK8 the EA Agent Loader is a better choice and doesn't require the pid.
However, it is discontinued for JDK9.

Kotlin example where my agent happens to be called JavaAgent!

AgentLoader.loadAgentClass(JavaAgent::class.java.name, "")

Attach to current VM using com.sun.tools

The ID of the VM to attach to is a process ID (pid). So, you just need to find pid of the current JVM process.

Here is a way to do this:

String jvmName = ManagementFactory.getRuntimeMXBean().getName();
String jvmPid = jvmName.substring(0, jvmName.indexOf('@'));

VirtualMachine self = VirtualMachine.attach(jvmPid);

Note: since JDK 9 attaching to current process requires setting the system property:

-Djdk.attach.allowAttachSelf=true

Is there any way to judge whether a class has been loaded into jvm?

OK, so I have not done this. But it seems that it would be possible via getAllLoadedClasses() in the Instrumentation API.

But then you would still need to start the JVM with your agent.

Specify -javaagent Option in Manifest

Java 9 adds the Launcher-Agent-Class attribute which can be used on executable JAR files to start an agent before the main class is loaded.

How to remove method body at runtime with ASM 5.2

You are not supposed to invoke the methods of a visitor directly.

The correct way to use a ClassVisitor, is to create a ClassReader with the class file bytes of the class you’re interested in and pass the class visitor to the its accept method. Then, all the visit methods will be called by the class reader according to the artifacts found in the class file.

In this regard, you should not consider the documentation outdated, just because it refers to an older version number. E.g. this document describes that process correctly and it speaks for the library that no fundamental change was necessary between the versions 2 and 5.

Still, visiting a class does not change it. It helps analyzing it and perform actions when encountering a certain artifact. Note that returning null is not an actual action.

If you want to create a modified class, you need a ClassWriter to produce the class. A ClassWriter implements ClassVisitor, also class visitors can be chained, so you can easily create a custom visitor delegating to a writer, that will produce a class file identical to the original one, unless you override a method to intercept the recreation of a feature.

But note that returning null from visitMethod does more than removing the code, it will remove the method entirely. Instead, you have to return a special visitor for the specific method which will reproduce the method but ignore the old code and create a sole return instruction (you are allowed to omit the last return statement in source code, but not the return instruction in the byte code).

private static byte[] disableMethod(Method method) {
    Class<?> theClass = method.getDeclaringClass();
    ClassReader cr;
    try { // use resource lookup to get the class bytes
        cr = new ClassReader(
            theClass.getResourceAsStream(theClass.getSimpleName()+".class"));
    } catch(IOException ex) {
        throw new IllegalStateException(ex);
    }
    // passing the ClassReader to the writer allows internal optimizations
    ClassWriter cw = new ClassWriter(cr, 0);
    cr.accept(new MethodReplacer(
            cw, method.getName(), Type.getMethodDescriptor(method)), 0);

    byte[] newCode = cw.toByteArray();
    return newCode;
}

static class MethodReplacer extends ClassVisitor {
    private final String hotMethodName, hotMethodDesc;

    MethodReplacer(ClassWriter cw, String name, String methodDescriptor) {
        super(Opcodes.ASM5, cw);
        hotMethodName = name;
        hotMethodDesc = methodDescriptor;
    }

    // invoked for every method
    @Override
    public MethodVisitor visitMethod(
        int access, String name, String desc, String signature, String[] exceptions) {

        if(!name.equals(hotMethodName) || !desc.equals(hotMethodDesc))
            // reproduce the methods we're not interested in, unchanged
            return super.visitMethod(access, name, desc, signature, exceptions);

        // alter the behavior for the specific method
        return new ReplaceWithEmptyBody(
            super.visitMethod(access, name, desc, signature, exceptions),
            (Type.getArgumentsAndReturnSizes(desc)>>2)-1);
    }
}
static class ReplaceWithEmptyBody extends MethodVisitor {
    private final MethodVisitor targetWriter;
    private final int newMaxLocals;

    ReplaceWithEmptyBody(MethodVisitor writer, int newMaxL) {
        // now, we're not passing the writer to the superclass for our radical changes
        super(Opcodes.ASM5);
        targetWriter = writer;
        newMaxLocals = newMaxL;
    }

    // we're only override the minimum to create a code attribute with a sole RETURN

    @Override
    public void visitMaxs(int maxStack, int maxLocals) {
        targetWriter.visitMaxs(0, newMaxLocals);
    }

    @Override
    public void visitCode() {
        targetWriter.visitCode();
        targetWriter.visitInsn(Opcodes.RETURN);// our new code
    }

    @Override
    public void visitEnd() {
        targetWriter.visitEnd();
    }

    // the remaining methods just reproduce meta information,
    // annotations & parameter names

    @Override
    public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
        return targetWriter.visitAnnotation(desc, visible);
    }

    @Override
    public void visitParameter(String name, int access) {
        targetWriter.visitParameter(name, access);
    }
}

The custom MethodVisitor does not get chained to the method visitor returned by the class writer. Configured this way, it will not replicate the code automatically. Instead, performing no action will be the default and only our explicit invocations on the targetWriter will produce code.

At the end of the process, you have a byte[] array containing the changed code in the class file format. So the question is, what to do with it.

The easiest, most portable thing you can do, is to create a new ClassLoader, which creates a new Class from these bytes, which has the same name (as we didn’t change the name), but is distinct from the already loaded class, because it has a different defining class loader. We can access such dynamically generated class only through Reflection:

public class BytecodeMods {

    public static void main(String[] args) throws Exception {
        byte[] code = disableMethod(BytecodeMods.class.getMethod("test"));
        new ClassLoader() {
            Class<?> get() { return defineClass(null, code, 0, code.length); }
        }   .get()
            .getMethod("test").invoke(null);
    }

    public static void test() {
        System.out.println("This is a test");
    }

    …

In order to make this example do something more notable than doing nothing, you could alter the message instead,

using the following MethodVisitor

static class ReplaceStringConstant extends MethodVisitor {
    private final String matchString, replaceWith;

    ReplaceStringConstant(MethodVisitor writer, String match, String replacement) {
        // now passing the writer to the superclass, as most code stays unchanged
        super(Opcodes.ASM5, writer);
        matchString = match;
        replaceWith = replacement;
    }

    @Override
    public void visitLdcInsn(Object cst) {
        super.visitLdcInsn(matchString.equals(cst)? replaceWith: cst);
    }
}

by changing

        return new ReplaceWithEmptyBody(
            super.visitMethod(access, name, desc, signature, exceptions),
            (Type.getArgumentsAndReturnSizes(desc)>>2)-1);

to

        return new ReplaceStringConstant(
            super.visitMethod(access, name, desc, signature, exceptions),
            "This is a test", "This is a replacement");

---

If you want to change the code of an already loaded class or intercept it right before being loaded into the JVM, you have to use the Instrumentation API.

The byte code transformation itself doesn’t change, you’ll have to pass the source bytes into the ClassReader and get the modified bytes back from the ClassWriter. Methods like ClassFileTransformer.transform(…) will already receive the bytes representing the current form of the class (there might have been previous transformations) and return the new bytes.

The problem is, this API isn’t generally available to Java applications. It’s available for so-called Java Agents, which must have been either, started together with the JVM via startup options or get loaded dynamically in an implementation-specific way, e.g. via the Attach API.

The package documentation describes the general structure of Java Agents and the related command line options.

At the end of this answer is a program demonstrating how to use the Attach API to attach to your own JVM to load a dummy Java Agent that will give the program access to the Instrumentation API. Considering the complexity, I think, it became apparent, that the actual code transformation and turning the code into a runtime class or using it to replace a class on the fly, are two different tasks that have to collaborate, but whose code you usually want to keep separated.

---

Related Topics