Shared Memory Between Two Jvms

Shared Memory between two JVMs

Solution 1:

The best solution in my opinion is to use memory mapped files. This allows you to share a region of memory between any number of process, including other non java programs. You can't place java objects into a memory mapped file, unless you serialize them. The following example shows that you can communicate between two different process, but you would need to make it much more sophisticated to allow better communication between the processes. I suggest you look at Java's NIO package, specifically the classes and methods used in the below examples.

Server:

public class Server {

    public static void main( String[] args ) throws Throwable {
        File f = new File( FILE_NAME );

        FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );

        MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );
        CharBuffer charBuf = b.asCharBuffer();

        char[] string = "Hello client\0".toCharArray();
        charBuf.put( string );

        System.out.println( "Waiting for client." );
        while( charBuf.get( 0 ) != '\0' );
        System.out.println( "Finished waiting." );
    }
}

Client:

public class Client {

    public static void main( String[] args ) throws Throwable {
        File f = new File( FILE_NAME );
        FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );

        MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );
        CharBuffer charBuf = b.asCharBuffer();

        // Prints 'Hello server'
        char c;
        while( ( c = charBuf.get() ) != 0 ) {
            System.out.print( c );
        }
        System.out.println();

        charBuf.put( 0, '\0' );
    }

}

---

Solution 2:

Another solution is to use Java Sockets to communicate back and forth between processes. This has the added benefit of allowing communication over a network very easily. It could be argued that this is slower than using memory mapped files, but I do not have any benchmarks to back that statement up. I won't post code to implementing this solution, as it can become very complicated to implement a reliable network protocol and is fairly application specific. There are many good networking sites that can be found with quick searches.

---

Now the above examples are if you want to share memory between two different process. If you just want to read/write to arbitrary memory in the current process, there are some warnings you should know first. This goes against the entire principle of the JVM and you really really should not do this in production code. You violate all safety and can very easily crash the JVM if you are not very careful.

That being said, it is quite fun to experiment with. To read/write to arbitrary memory in the current process you can use the sun.misc.Unsafe class. This is provided on all JVMs that I am aware of and have used. An example on how to use the class can be found here.

How can I share memory between two JVM instances?

Save your graph to disk, then map it into memory with MappedByteBuffer. Both processes should use the same memory, which will be shared with the page cache.

Any concept of shared memory in Java

Since there is no official API to create a shared memory segment, you need to resort to a helper library/DDL and JNI to use shared memory to have two Java processes talk to each other.

In practice, this is rarely an issue since Java supports threads, so you can have two "programs" run in the same Java VM. Those will share the same heap, so communication will be instantaneous. Plus you can't get errors because of problems with the shared memory segment.

Can multiple JVM processes share memory for common classes?

At this time and with the Oracle VM, this isn't possible.

But I agree, it would be a nice feature, especially since Java has all the information it needs to do that automatically.

Of the top of my hat, I think that the JIT is the only reason why this can't work: The JIT takes runtime behavior into account. So if app A uses some code in a different pattern than app B, that would result in different assembler code generated at runtime.

But then, the usual "pattern" is "how often is this code used." So if app A called some method very often and B didn't, they could still share the code because A has already paid the price for optimizing/compiling it.

What you can try is deploy several applications as WAR files into a single VM. But from my experience, that often causes problems with code that doesn't correctly clean up thread locals or shutdown hooks.

How to share an object in multiple jvm?

You can't share an object, because memory addresses are different for two JVM.

You can share a copy of the same object on two jvm, but any change to that object on the first JVM is not reflected to the second jvm.

So if you need to share informations between two JVM you can do that in many different ways:

• Save data to a shared database
• Save data to a shared file
• Save data to an external web service that offer methods to save and read data
• Save data to an external cache system like REDIS

Shared Memory between two JVMs

Solution 1:

The best solution in my opinion is to use memory mapped files. This allows you to share a region of memory between any number of process, including other non java programs. You can't place java objects into a memory mapped file, unless you serialize them. The following example shows that you can communicate between two different process, but you would need to make it much more sophisticated to allow better communication between the processes. I suggest you look at Java's NIO package, specifically the classes and methods used in the below examples.

Server:

public class Server {

    public static void main( String[] args ) throws Throwable {
        File f = new File( FILE_NAME );

        FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );

        MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );
        CharBuffer charBuf = b.asCharBuffer();

        char[] string = "Hello client\0".toCharArray();
        charBuf.put( string );

        System.out.println( "Waiting for client." );
        while( charBuf.get( 0 ) != '\0' );
        System.out.println( "Finished waiting." );
    }
}

Client:

public class Client {

    public static void main( String[] args ) throws Throwable {
        File f = new File( FILE_NAME );
        FileChannel channel = FileChannel.open( f.toPath(), StandardOpenOption.READ, StandardOpenOption.WRITE, StandardOpenOption.CREATE );

        MappedByteBuffer b = channel.map( MapMode.READ_WRITE, 0, 4096 );
        CharBuffer charBuf = b.asCharBuffer();

        // Prints 'Hello server'
        char c;
        while( ( c = charBuf.get() ) != 0 ) {
            System.out.print( c );
        }
        System.out.println();

        charBuf.put( 0, '\0' );
    }

}

---

Solution 2:

Another solution is to use Java Sockets to communicate back and forth between processes. This has the added benefit of allowing communication over a network very easily. It could be argued that this is slower than using memory mapped files, but I do not have any benchmarks to back that statement up. I won't post code to implementing this solution, as it can become very complicated to implement a reliable network protocol and is fairly application specific. There are many good networking sites that can be found with quick searches.

---

Now the above examples are if you want to share memory between two different process. If you just want to read/write to arbitrary memory in the current process, there are some warnings you should know first. This goes against the entire principle of the JVM and you really really should not do this in production code. You violate all safety and can very easily crash the JVM if you are not very careful.

That being said, it is quite fun to experiment with. To read/write to arbitrary memory in the current process you can use the sun.misc.Unsafe class. This is provided on all JVMs that I am aware of and have used. An example on how to use the class can be found here.

Share classes between JVMs

I was not particularly abreast of this topics but I did some research. I think we can say with considerable certainty that it is not practically possible in most JVM's. The question below is similar to yours and it has answers and comments that may be helpful.

Can multiple JVM processes share memory for common classes?

A possible option is you could put the classes you would like to share together in one process and expose their functionality using something like JMX (Java Management Extensions). That way the other processes that are loaded several times would not have to load all the classes. But how much this can reduce the total resource footprint is of course questionable and case dependent.

Share data between two JVMs when application is clustered

As you have Hazelcast already available in your tech stack, use the caching facility of Hazelcast:

Hazelcast is the leading in-memory data grid solution. Its java
caching solution enables organizations to predictably scale
mission-critical enterprise applications by providing in-memory access
to frequently used data. Hazelcast stores frequently accessed data
in-memory and across an elastically scalable data grid. This enables
any network of machines to dynamically cluster and pool both memory
and processors to accelerate application performance.

You can put the set of data to a cache created with Hazelcast, and access the data on other nodes in the cluster.

How to have 2 JVMs talk to one another

Multiple options for IPC:

Socket-Based (Bare-Bones) Networking

• not necessarily hard, but:
  
  • might be verbose for not much,
  • might offer more surface for bugs, as you write more code.
• you could rely on existing frameworks, like Netty

RMI

• Technically, that's also network communication, but that's transparent for you.

Fully-fledged Message Passing Architectures

• usually built on either RMI or network communications as well, but with support for complicated conversations and workflows
• might be too heavy-weight for something simple
• frameworks like ActiveMQ or JBoss Messaging

Java Management Extensions (JMX)

• more meant for JVM management and monitoring, but could help to implement what you want if you mostly want to have one process query another for data, or send it some request for an action, if they aren't too complex
• also works over RMI (amongst other possible protocols)
• not so simple to wrap your head around at first, but actually rather simple to use

File-sharing / File-locking

• that's what you're doing right now
• it's doable, but comes with a lot of problems to handle

Signals

• You can simply send signals to your other project
• However, it's fairly limited and requires you to implement a translation layer (it is doable, though, but a rather crazy idea to toy with than anything serious.

---

Without more details, a bare-bone network-based IPC approach seems the best, as it's the:

• most extensible (in terms of adding new features and workflows to your
• most lightweight (in terms of memory footprint for your app)
• most simple (in terms of design)
• most educative (in terms of learning how to implement IPC). ^{(as you mentioned "socket is hard" in a comment, and it really is not and should be something you work on)}

That being said, based on your example (simply requesting the other process to do an action), JMX could also be good enough for you.

---

Related Topics