Java: How to Test Methods That Call System.Exit()

Java: How to test methods that call System.exit()?

Indeed, Derkeiler.com suggests:

  • Why System.exit() ?

Instead of terminating with System.exit(whateverValue), why not throw an unchecked exception? In normal use it will drift all the way out to the JVM's last-ditch catcher and shut your script down (unless you decide to catch it somewhere along the way, which might be useful someday).

In the JUnit scenario it will be caught by the JUnit framework, which will report that
such-and-such test failed and move smoothly along to the next.

  • Prevent System.exit() to actually exit the JVM:

Try modifying the TestCase to run with a security manager that prevents calling System.exit, then catch the SecurityException.

public class NoExitTestCase extends TestCase 
{

protected static class ExitException extends SecurityException
{
public final int status;
public ExitException(int status)
{
super("There is no escape!");
this.status = status;
}
}

private static class NoExitSecurityManager extends SecurityManager
{
@Override
public void checkPermission(Permission perm)
{
// allow anything.
}
@Override
public void checkPermission(Permission perm, Object context)
{
// allow anything.
}
@Override
public void checkExit(int status)
{
super.checkExit(status);
throw new ExitException(status);
}
}

@Override
protected void setUp() throws Exception
{
super.setUp();
System.setSecurityManager(new NoExitSecurityManager());
}

@Override
protected void tearDown() throws Exception
{
System.setSecurityManager(null); // or save and restore original
super.tearDown();
}

public void testNoExit() throws Exception
{
System.out.println("Printing works");
}

public void testExit() throws Exception
{
try
{
System.exit(42);
} catch (ExitException e)
{
assertEquals("Exit status", 42, e.status);
}
}
}

Update December 2012:

Will proposes in the comments using System Rules, a collection of JUnit(4.9+) rules for testing code which uses java.lang.System.

This was initially mentioned by Stefan Birkner in his answer in December 2011.

System.exit(…)

Use the ExpectedSystemExit rule to verify that System.exit(…) is called.

You could verify the exit status, too.

For instance:

public void MyTest {
@Rule
public final ExpectedSystemExit exit = ExpectedSystemExit.none();

@Test
public void noSystemExit() {
//passes
}

@Test
public void systemExitWithArbitraryStatusCode() {
exit.expectSystemExit();
System.exit(0);
}

@Test
public void systemExitWithSelectedStatusCode0() {
exit.expectSystemExitWithStatus(0);
System.exit(0);
}
}

Dealing with System.exit(0) in JUnit tests

How I deal with that is to install a security manager that throws an exception when System.exit is called. Then there is code that catches the exception and doesn't fail the test.

public class NoExitSecurityManager
extends java.rmi.RMISecurityManager
{
private final SecurityManager parent;

public NoExitSecurityManager(final SecurityManager manager)
{
parent = manager;
}

public void checkExit(int status)
{
throw new AttemptToExitException(status);
}

public void checkPermission(Permission perm)
{
}
}

And then in the code, something like:

catch(final Throwable ex)
{
final Throwable cause;

if(ex.getCause() == null)
{
cause = ex;
}
else
{
cause = ex.getCause();
}

if(cause instanceof AttemptToExitException)
{
status = ((AttemptToExitException)cause).getStatus();
}
else
{
throw cause;
}
}

assertEquals("System.exit must be called with the value of " + expectedStatus, expectedStatus, status);

Using JUnit on code that terminates

You can use System Rules: "A collection of JUnit rules for testing code which uses java.lang.System."

Among their rules, you have ExpectedSystemExit, below is an example on how to use it. I believe it is a very clean solution.

import org.junit.Rule;
import org.junit.Test;
import org.junit.contrib.java.lang.system.Assertion;
import org.junit.contrib.java.lang.system.ExpectedSystemExit;

public class SystemExitTest {
@Rule
public final ExpectedSystemExit exit = ExpectedSystemExit.none();

@Test
public void noSystemExit() {
//passes
}

@Test
public void executeSomeCodeAFTERsystemExit() {
System.out.println("This is executed before everything.");
exit.expectSystemExit();
exit.checkAssertionAfterwards(new Assertion() {
@Override
public void checkAssertion() throws Exception {
System.out.println("This is executed AFTER System.exit()"+
" and, if exists, the @org.junit.After annotated method!");
}
});
System.out.println("This is executed right before System.exit().");
System.exit(0);
System.out.println("This is NEVER executed.");
}

@Test
public void systemExitWithArbitraryStatusCode() {
exit.expectSystemExit();
System.exit(0);
}

@Test
public void systemExitWithSelectedStatusCode0() {
exit.expectSystemExitWithStatus(0);
System.exit(0);
}

@Test
public void failSystemExit() {
exit.expectSystemExit();
//System.exit(0);
}

}

If you use maven, you can add this to your pom.xml:

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>
<dependency>
<groupId>com.github.stefanbirkner</groupId>
<artifactId>system-rules</artifactId>
<version>1.3.0</version>
</dependency>

Testing if a java program calls System.exist - without exiting Junit

I think it's more inelegant to use System.exit to validate input parameters.
Why not just throw an exception and let the caller of this method handle it ?

How to mock System.exit with PowerMockito?

I think you should replace both the lines in your sample code

PowerMockito.spy(System.class);
PowerMockito.doNothing.....

to

PowerMockito.mockStatic(System.class);

This change works in my local as System.exit does nothing because of the mock on static method.

Also, I hope you are using PrepareForTest annotation

@PrepareForTest(CLASS_UNDER_TEST)

The spy method is to call real methods and have some wrapper around the non-static methods. Since you need a mock for static methods, mockStatic method should be used instead.

Update 1

The PowerMockito mockStatic method by default creates mock for all the static methods within the class. I don't have any clean solution. But, I can suggest a solution which looks ugly but does what is needed i.e only mock specific static method and remaining methods are invoking real methods. PoweMockito's mockStatic method is internally calling DefaultMockCreator to mock the static methods.

@RunWith(PowerMockRunner.class)
public class StaticTest {

@Test
public void testMethod() throws Exception {

// Get static methods for which mock is needed
Method exitMethod = System.class.getMethod("exit", int.class);
Method[] methodsToMock = new Method[] {exitMethod};

// Create mock for only those static methods
DefaultMockCreator.mock(System.class, true, false, null, null, methodsToMock);

System.exit(-1); // This will be mocked
System.out.println(System.currentTimeMillis()); // This will call up real methods
}
}

As per the PowerMockito documentation, the right way to call static void method is -

PowerMockito.mockStatic(SomeClass.class);
PowerMockito.doNothing().when(SomeClass.class);
SomeClass.someVoidMethod();

Reference - https://github.com/powermock/powermock/wiki/Mockito#how-to-stub-void-static-method-to-throw-exception

This should create the mock behaviour for the specific static void method. Unfortunately, this doesn't work for System Class because System class is final. Had it been not final, this would have worked. I tried it and I got this exception -

org.mockito.exceptions.base.MockitoException: 
Cannot mock/spy class java.lang.System
Mockito cannot mock/spy because :
- final class

Code -

@Test
public void testMethod() throws Exception {
PowerMockito.mockStatic(System.class);
PowerMockito.doNothing().when(System.class);
System.exit(-1); // mockito error coming here

System.exit(-1);
System.currentTimeMillis();
}

Scala: How to test methods that call System.exit()?

If you really wish to go with a method using System.exit(), the simplest way to test it was actually called is to replace your SecurityManager with one that'll throw an ExitException (subclassing SecurityException) when System.exit() is called:

class SystemExitSpec

import java.security.Permission

import org.specs2.mutable.Specification
import org.specs2.specification.BeforeAfterAll

sealed case class ExitException(status: Int) extends SecurityException("System.exit() is not allowed") {
}

sealed class NoExitSecurityManager extends SecurityManager {
override def checkPermission(perm: Permission): Unit = {}

override def checkPermission(perm: Permission, context: Object): Unit = {}

override def checkExit(status: Int): Unit = {
super.checkExit(status)
throw ExitException(status)
}
}

abstract class SystemExitSpec extends Specification with BeforeAfterAll {

sequential

override def beforeAll(): Unit = System.setSecurityManager(new NoExitSecurityManager())

override def afterAll(): Unit = System.setSecurityManager(null)
}

test ConverterSpec

import org.specs2.execute.Failure

import scala.io.Source

class ConverterSpec extends SystemExitSpec {

"ConverterSpec" should {

"empty args" >> {
try {
Converter.main(Array[String]())
Failure("shouldn't read this code")
} catch {
case e: ExitException =>
e.status must_== 1
}
1 must_== 1
}
}

How to test the program has quit? [JUnit]

There are several ways to test System.exit() in a JUnit test:

  1. Mock it with PowerMock
  2. Call a method which calls System.exit() and overwrite this method in the test.
  3. Use the System Rules library (kudos go to Stefan Birkner for pointing that out)

The reason is that System.exit() never returns, so control never returns to your test. So you have to make sure that the actual method isn't called.

Make JUnit test fail on System.exit somewhere in another thread

Why does it call System.exit()? And why does it call System.exit() at a level you're covering with your unit test?

If you want to test the exiting code, your tests can spawn a process and check that it has exited as planned. If you want to test other functionality, better structure your application in such a way that the code under test doesn not have such nasty side effects.

In general though, I'd say that having individual threads call System.exit() is a strong indicator of bad architecture. If any thread can instantly kill the entire appication without giving other threads a chance to clean up you'll likely run into quite some problems quite soon.

And if your threads call some global shutdown handler (that you can override in your tests) instead of System.exit() you won't have a problem testing that functionality.

Update: When dealing with legacy code, my first step would be to isolate the System.exit() into a place where it can be replaced in tests. E.g. create a class like this:

// "Singleton" with replaceable instance
public class ShutdownHandler {
private static ShutdownHandler instance = new ShutdownHandler();

public static ShutdownHandler getInstance() {
return instance;
}

public synchronized void shutdown() {
// default implementation
System.exit();
}

public static void setInstance(ShutdownHandler newInstance) {
// (probably also check that this is only called in a test environment)
instance = newInstance;
}
}

Then replace all System.exit() calls with ShutdownHandler.getInstance().shutdown(). That won't change the application's functionality at all. But now you can replace the ShutdownHandler instance in your tests with an implementation that sets some flag + throws an exception instead of exiting -- and your tests can then check that flag to determine whether the app would have exited.

I can also recommend "Working Effectively With Legacy Code" by Robert C. Martin for more ideas on how to turn an old mess into something manageable.



Related Topics



Leave a reply



Submit