Compile Code Fully in Memory with Javax.Tools.Javacompiler

Compile code fully in memory with javax.tools.JavaCompiler

I've run the above code in Mac OS Java 7. None of them works. So i wrote one
https://github.com/trung/InMemoryJavaCompiler

StringBuilder source = new StringBuilder()
    .append("package org.mdkt;\n")
    .append("public class HelloClass {\n")
    .append("   public String hello() { return \"hello\"; }")
    .append("}");

Class<?> helloClass = InMemoryJavaCompiler.compile("org.mdkt.HelloClass", source.toString());

Is it possible to programmatically compile java source code in memory only?

To start, look at the JavaCompiler API. Basically:

  1. Create the Java class in a string.
  2. Put the string into class that extends SimpleJavaFileObject.
  3. Compile using a JavaCompiler instance.

Finally, call the methods the new class.

Here is an example that works with JDK6+:

import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.lang.reflect.InvocationTargetException;
import java.net.URI;
import java.util.Arrays;

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.SimpleJavaFileObject;
import javax.tools.ToolProvider;
import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaFileObject.Kind;

public class CompileSourceInMemory {
  public static void main(String args[]) throws IOException {
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();

    StringWriter writer = new StringWriter();
    PrintWriter out = new PrintWriter(writer);
    out.println("public class HelloWorld {");
    out.println("  public static void main(String args[]) {");
    out.println("    System.out.println(\"This is in another java file\");");    
    out.println("  }");
    out.println("}");
    out.close();
    JavaFileObject file = new JavaSourceFromString("HelloWorld", writer.toString());

    Iterable<? extends JavaFileObject> compilationUnits = Arrays.asList(file);
    CompilationTask task = compiler.getTask(null, null, diagnostics, null, null, compilationUnits);

    boolean success = task.call();
    for (Diagnostic diagnostic : diagnostics.getDiagnostics()) {
      System.out.println(diagnostic.getCode());
      System.out.println(diagnostic.getKind());
      System.out.println(diagnostic.getPosition());
      System.out.println(diagnostic.getStartPosition());
      System.out.println(diagnostic.getEndPosition());
      System.out.println(diagnostic.getSource());
      System.out.println(diagnostic.getMessage(null));

    }
    System.out.println("Success: " + success);

    if (success) {
      try {
        Class.forName("HelloWorld").getDeclaredMethod("main", new Class[] { String[].class })
            .invoke(null, new Object[] { null });
      } catch (ClassNotFoundException e) {
        System.err.println("Class not found: " + e);
      } catch (NoSuchMethodException e) {
        System.err.println("No such method: " + e);
      } catch (IllegalAccessException e) {
        System.err.println("Illegal access: " + e);
      } catch (InvocationTargetException e) {
        System.err.println("Invocation target: " + e);
      }
    }
  }
}

class JavaSourceFromString extends SimpleJavaFileObject {
  final String code;

  JavaSourceFromString(String name, String code) {
    super(URI.create("string:///" + name.replace('.','/') + Kind.SOURCE.extension),Kind.SOURCE);
    this.code = code;
  }

  @Override
  public CharSequence getCharContent(boolean ignoreEncodingErrors) {
    return code;
  }
}

Use of javax.tools to compile java source at Runtime?

Chances are you're running Java from a JRE directory instead of a JDK directory - you need to run a version which "knows" where the Java compiler is.

So for example, on my Windows boxing, running a tiny test app like this:

import javax.tools.*;

class Test {
    public static void main(String[] args) {
        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
        System.out.println(compiler);
    }
}

The results are:

c:\Users\Jon\Test>"\Program Files\Java\jdk1.7.0_09"\bin\java Test
com.sun.tools.javac.api.JavacTool@1e0f2f6

c:\Users\Jon\Test>"\Program Files\Java\jre7\bin\java" Test
null

As you can see, it's fine when I specifically run the JDK version, but I get null when running the JRE version. Check how you're starting Java.

Can't extend a compiled at runtime class

Compiling HelloWorld succeeds because it only needs its own source. When you try to compile ExtendedHelloWorld it fails because it requires its own source and HelloWorld's source. This can be achieved by storing each class in a HashMap<String, String> where the key is the class name and the value is the class' source code.

I'd advise a couple of changes to your code.

I would deflate your compile method and break it into two different compile methods. The first would be used when you want to compile a class that does not extend a class compiled from memory. The second would be used when you do want to compile a class that extends a class compiled from memory.

/*
 * Method to compile a class which doesn't extend a class that's been compiled from memory.
 */
public static Class<?> compile(String className, String sourceCode, URLClassLoader classLoader) throws Exception {
    return compileHelper(className, classLoader, Arrays.asList(new JavaSourceFromString(className, sourceCode)));
}

/*
 * Method to compile a class which extends a class that's been compiled from
 * memory.
 * 
 * This method takes in the class name, a Set of Map.Entry<String, String>,
 * which contains class names and their sources, and a class loader. This
 * method iterates over the entries in the Set, creates JavaFileObjects from
 * the class names and their sources and adds each JavaFileObject to an
 * ArrayList which will be used in the 'compileHelper' method.
 */
public static Class<?> compile(String className, Set<Map.Entry<String, String>> nameAndSource, URLClassLoader classLoader) throws Exception {
    List<JavaFileObject> compilationUnits = new ArrayList<>();

    for(Entry<String, String> entry : nameAndSource) {
        compilationUnits.add(new JavaSourceFromString(entry.getKey(), entry.getValue()));
    }

    return compileHelper(className, classLoader, compilationUnits);
}

The above methods then call a helper method which which actually compiles the class(es). This method closely resembles your compile method, but the output of diagnostics have been moved to a separate method, printDiagnostics(diagnostics).

/*
 * Helper method that actually does the compiling.
 */
private static Class<?> compileHelper(String className, URLClassLoader classLoader, Iterable<? extends JavaFileObject> compilationUnits) throws Exception {
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
    CompilationTask task = null;
    boolean success = false;

    // debug compilation units section
    System.out.println("Compiling " + className);

    System.out.println(" - compilationUnits ");
    for(JavaFileObject o : compilationUnits) {
        System.out.println("   + " + o.toString());
    }
    // end debug

    task = compiler.getTask(null, null, diagnostics, null, null, compilationUnits);
    success = task.call();

    if (success) {
        System.out.println("Successful compilation of " + className);
        return Class.forName(className.replace(".", "/"), true, classLoader);
    } else {
        System.out.println("Failed while compiling " + className);
        printDiagnostics(diagnostics);
        throw new Exception("It didn't work!");
    }
}

In order to use the methods above you'll need to use a HashMap<String, String> to store the class names and source codes of each class you wish to compile. Then when you're ready to compile make a call to compile passing in the entrySet() from the HashMap, for instance: compile(className, nameAndSource.entrySet(), classLoader)

For example:

public static void main(String args[]) throws Exception {
    Map<String, String> nameAndSource = new HashMap<>();
    URLClassLoader classLoader = URLClassLoader.newInstance(new URL[] { new File("").toURI().toURL() });
    String className;
    StringBuilder sourceCode;

    // HelloWorld class
    className = "HelloWorld";
    sourceCode = new StringBuilder();        
    sourceCode.append("public class HelloWorld {");
    sourceCode.append("  public static void main(String args[]) {");
    sourceCode.append("    System.out.append(\"This is in another java file\");");
    sourceCode.append("  }");
    sourceCode.append("}");

    // pass the class name and source code to 'compile'        
    Class<?> helloWorld = compile(className, sourceCode.toString(), classLoader);

    // add HelloWorld class name and source code to HashMap
    nameAndSource.put(className, sourceCode.toString());

    // ExtendedHelloWorldClass
    className = "ExtendedHelloWorld";
    sourceCode = new StringBuilder();
    sourceCode.append("public class ExtendedHelloWorld extends HelloWorld {");
    sourceCode.append("  public int num = 2;");
    sourceCode.append("}");

    // add ExtendedHelloWorld class name and source code to HashMap
    nameAndSource.put(className, sourceCode.toString());

    // here's where we pass in the nameAndSource entrySet()
    Class<?> extendedHelloWorld = compile(className, nameAndSource.entrySet(), classLoader);

    return;
}

Here's is the complete source code of what I tried to describe above:

public class CompileSourceInMemory {

    /*
     * Method to compile a class which extends a class that's been compiled from
     * memory.
     * 
     * This method takes in the class name, a Set of Map.Entry<String, String>,
     * which contains class names and their sources, and a class loader. This
     * method iterates over the entries in the Set, creates JavaFileObjects from
     * the class names and their sources and adds each JavaFileObject to an
     * ArrayList which will be used the private compile method.
     */
    public static Class<?> compile(String className, Set<Map.Entry<String, String>> nameAndSource, URLClassLoader classLoader) throws Exception {
        List<JavaFileObject> compilationUnits = new ArrayList<>();

        for (Entry<String, String> entry : nameAndSource) {
            compilationUnits.add(newJavaSourceFromString(entry.getKey(), entry.getValue()));
        }

        return compileHelper(className, classLoader, compilationUnits);
    }

    /*
     * Method to compile a class which doesn't extend a class that's been
     * compiled from memory.
     */
    public static Class<?> compile(String className, String sourceCode, URLClassLoader classLoader) throws Exception {
        return compileHelper(className, classLoader, Arrays.asList(new JavaSourceFromString(className, sourceCode)));
    }

    /*
     * Helper method that actually does the compiling.
     */
    private static Class<?> compileHelper(String className, URLClassLoader classLoader, Iterable<? extends JavaFileObject> compilationUnits) throws Exception {
        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
        DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
        CompilationTask task = null;
        boolean success = false;

        // debug compilation units section
        System.out.println("Compiling " + className);

        System.out.println(" - compilationUnits ");
        for (JavaFileObject o : compilationUnits) {
            System.out.println("   + " + o.toString());
        }
        // end debug

        task = compiler.getTask(null, null, diagnostics, null, null, compilationUnits);
        success = task.call();

        if (success) {
            System.out.println("Successful compilation of " + className);
            return Class.forName(className.replace(".", "/"), true, classLoader);
        } else {
            System.out.println("Failed while compiling " + className);
            printDiagnostics(diagnostics);
            throw new Exception("It didn't work!");
        }
    }

    public static void main(String args[]) throws Exception {
        Map<String, String> nameAndSource = new HashMap<>();
        URLClassLoader classLoader = URLClassLoader.newInstance(new URL[] { new File("").toURI().toURL() });
        String className;
        StringBuilder sourceCode;

        // HelloWorld Class
        className = "HelloWorld";
        sourceCode = new StringBuilder();
        sourceCode.append("public class HelloWorld {");
        sourceCode.append("  public static void main(String args[]) {");
        sourceCode.append("    System.out.append(\"This is in another java file\");");
        sourceCode.append("  }");
        sourceCode.append("}");

        // pass the class name and source code to 'compile'
        Class<?> helloWorld = compile(className, sourceCode.toString(), classLoader);

        // add HelloWorld class name and source code to HashMap
        nameAndSource.put(className, sourceCode.toString());

        // ExtendedHelloWorld Class
        className = "ExtendedHelloWorld";
        sourceCode = new StringBuilder();
        sourceCode.append("public class ExtendedHelloWorld extends HelloWorld {");
        sourceCode.append("  public int num = 2;");
        sourceCode.append("}");

        // add ExtendedHelloWorld class name and source code to HashMap
        nameAndSource.put(className, sourceCode.toString());

        // pass the nameAndSource entrySet() to 'compile'
        Class<?> extendedHelloWorld = compile(className, nameAndSource.entrySet(), classLoader);

        // ExtendedExtendedHelloWorld Class
        className = "ExtendedExtendedHelloWorld";
        sourceCode = new StringBuilder();
        sourceCode.append("public class ExtendedExtendedHelloWorld extends ExtendedHelloWorld {");
        sourceCode.append("  public void printNum() { System.out.println(num); }");
        sourceCode.append("}");

        // add ExtendedExtendedHelloWorld class name and source code to HashMap
        nameAndSource.put(className, sourceCode.toString());

        // pass the nameAndSource entrySet() to 'compile'
        Class<?> extendedExtendedHelloWorld = compile(className, nameAndSource.entrySet(), classLoader);
        Object eehw = extendedExtendedHelloWorld.newInstance();

        return;
    }

    private static void printDiagnostics(DiagnosticCollector<JavaFileObject> diagnostics) {
        StringBuilder sb = new StringBuilder("-- Diagnostics --\n");
        for (Diagnostic<?> d : diagnostics.getDiagnostics()) {
            sb.append(String
                    .format("d.getCode() - %s%nd.getKind() - %s%nd.getPosition() - %d%nd.getStartPosition() - %d%nd.getEndPosition() - %d%nd.getSource() - %s%nd.getMessage(null) - %s%n",
                            d.getCode(), d.getKind().toString(),
                            d.getPosition(), d.getStartPosition(),
                            d.getEndPosition(), d.getSource().toString(),
                            d.getMessage(null)));
        }
        System.out.println(sb.append("--").toString());
    }
}

class JavaSourceFromString extends SimpleJavaFileObject {
    final String code;

    JavaSourceFromString(String name, String code) {
        super(URI.create("string:///" + name.replace('.', '/')
                + Kind.SOURCE.extension), Kind.SOURCE);
        this.code = code;
    }

    @Override
    public CharSequence getCharContent(boolean ignoreEncodingErrors) {
        return code;
    }
}

Here is the output of running the code above:

Compiling HelloWorld
 - compilationUnits 
   + JavaSourceFromString[string:///HelloWorld.java]
Successful compilation of HelloWorld
Compiling ExtendedHelloWorld
 - compilationUnits 
   + JavaSourceFromString[string:///ExtendedHelloWorld.java]
   + JavaSourceFromString[string:///HelloWorld.java]
Successful compilation of ExtendedHelloWorld
Compiling ExtendedExtendedHelloWorld
 - compilationUnits 
   + JavaSourceFromString[string:///ExtendedHelloWorld.java]
   + JavaSourceFromString[string:///ExtendedExtendedHelloWorld.java]
   + JavaSourceFromString[string:///HelloWorld.java]
Successful compilation of ExtendedExtendedHelloWorld

JavaCompiler API: access functions/variables outside the compiled program while it's running?

In the end I was able to use Groovy, which is a superset of the Java language, to achieve my goal, so you can write a script in Java or Groovy. Passing in variables and classes, which can of course contain variables and functions, was straight-forward, e.g.

class MyClass {

    int x = 100;

    void myFunc(int r) {

        System.out.println("value from inside script = " + r + " !");
    }
}

void groovy() {

    MyClass q = new MyClass();

    StringWriter writer = new StringWriter();
    PrintWriter out = new PrintWriter(writer);

    out.println("    System.out.println(\"Hello world\" + q.x);");
    out.println("    q.x += 100;");
    out.println("    q.myFunc(123)");

    out.close();

    // call groovy expressions from Java code
    Binding binding = new Binding();
    binding.setVariable("q", q);
    GroovyShell shell = new GroovyShell(binding);

    Object value = shell.evaluate(writer.toString());

    System.out.println("new value = " + q.x);
}

Related Topics

How to Have 2 Jvms Talk to One Another

Executing Another Application from Java

How to Serialize an Object into a String

Spring Aop Not Working for Method Call Inside Another Method

How to Allow Introducing Only Digits in Jtextfield

Get String Character by Index

What Is the Significance of Url-Pattern in Web.Xml and How to Configure Servlet

Final Method Mocking

How to Sort a List by Different Parameters at Different Timed

Add Bean Programmatically to Spring Web App Context

Class Has Been Compiled by a More Recent Version of the Java Environment

Valid Characters in a Java Class Name

Read Properties File Outside Jar File

How to Create a Pojo

How to Access Owl Documents Using Xpath in Java

Com.Jcraft.Jsch.Jschexception: Unknownhostkey

Java Lambdas 20 Times Slower Than Anonymous Classes

Printwriter Append Method Not Appending


Leave a reply



Submit