Running Jmap Getting Unable to Open Socket File

Running jmap getting Unable to open socket file

jmap vs. jmap -F, as well as jstack vs. jstack -F use completely different mechanisms to communcate with the target JVM.

jmap / jstack

When run without -F these tools use Dynamic Attach Mechanism. This works as follows.

1. Before connecting to Java process 1234, jmap creates a file .attach_pid1234 at the working directory of the target process or at /tmp.

2. Then jmap sends SIGQUIT to the target process. When JVM catches the signal and finds .attach_pid1234, it starts AttachListener thread.

3. AttachListener thread creates UNIX domain socket /tmp/.java_pid1234 to listen to commands from external tools.

4. For security reasons when a connection (from jmap) is accepted, JVM verifies that credentials of the socket peer are equal to euid and egid of JVM process. That's why jmap will not work if run by different user (even by root).

5. jmap connects to the socket, and sends dumpheap command.

6. This command is read and executed by AttachListener thread of the JVM. All output is sent back to the socket. Since the heap dump is made in-process directly by JVM, the operation is really fast. However, JVM can do this only at safepoints. If a safepoint cannot be reached (e.g. the process is hung, not responding, or a long GC is in progress), jmap will timeout and fail.

Let's summarize the benefits and the drawbacks of Dynamic Attach.

Pros.

• Heap dump and other operations are run collaboratively by JVM at the maximum speed.
• You can use any version of jmap or jstack to connect to any other version of JVM.

Cons.

• The tool should be run by the same user (euid/egid) as the target JVM.
• Can be used only on live and healthy JVM.
• Will not work if the target JVM is started with -XX:+DisableAttachMechanism.

---

jmap -F / jstack -F

When run with -F the tools switch to special mode that features HotSpot Serviceability Agent. In this mode the target process is frozen; the tools read its memory via OS debugging facilities, namely, ptrace on Linux.

1. jmap -F invokes PTRACE_ATTACH on the target JVM. The target process is unconditionally suspended in response to SIGSTOP signal.

2. The tool reads JVM memory using PTRACE_PEEKDATA. ptrace can read only one word at a time, so too many calls required to read the large heap of the target process. This is very and very slow.

3. The tool reconstructs JVM internal structures based on the knowledge of the particular JVM version. Since different versions of JVM have different memory layout, -F mode works only if jmap comes from the same JDK as the target Java process.

4. The tool creates heap dump itself and then resumes the target process.

Pros.

• No cooperation from target JVM is required. Can be used even on a hung process.
• ptrace works whenever OS-level privileges are enough. E.g. root can dump processes of all other users.

Cons.

• Very slow for large heaps.
• The tool and the target process should be from the same version of JDK.
• The safepoint is not guaranteed when the tool attaches in forced mode. Though jmap tries to handle all special cases, sometimes it may happen that target JVM is not in a consistent state.

Note

There is a faster way to take heap dumps in forced mode. First, create a coredump with gcore, then run jmap over the generated core file. See the related question.

Java - AttachNotSupportedException: Unable to open socket file: HotSpot VM not loaded

Common reasons for this problem:

• Attach socket /tmp/.java_pid1234 has been removed (e.g. by a scheduled job that periodically cleans up /tmp).
• Target JVM is started with -XX:+DisableAttachMechanism option.
• Garbage Collection or other long VM operation (e.g. Heap Dump) is in progress.
• JVM cannot reach safepoint within attach timeout. This happens rarely, and the problem is typically intermittent.

jstack: Target process not responding

I got it working by doing two things:

1. Changed call to: sudo -u tomcat6 jstack -J-d64 -m pid
2. Replaced OpenJDK with Sun's original sun-6-jdk and sun-6-jre packages

---

Explanation for part 1: I switched to 64-bit mode, used sudo and run the command as Tomcat user.

Note: Part 2 might not be necessary. For some users it seems like part 1 is enough. In fact, try to add just the sudo command first. It might already do the trick.

com.sun.tools.attach.AttachNotSupportedException: Unable to open socket file: target process not responding or HotSpot VM not loaded

Work around for now.

Adding '-XX:+StartAttachListener' to jvm argument fixed the issue.

A similar issue is discussed here at https://code.google.com/p/jmockit/issues/detail?id=136 and http://mail.openjdk.java.net/pipermail/macosx-port-dev/2013-October/006098.html (which talks about a possible regression in jdk7 build)

---

Related Topics