What's the Best Mock Framework for Java

What's the best mock framework for Java?

I've had good success using Mockito.

When I tried learning about JMock and EasyMock, I found the learning curve to be a bit steep (though maybe that's just me).

I like Mockito because of its simple and clean syntax that I was able to grasp pretty quickly. The minimal syntax is designed to support the common cases very well, although the few times I needed to do something more complicated I found what I wanted was supported and easy to grasp.

Here's an (abridged) example from the Mockito homepage:

import static org.mockito.Mockito.*;

List mockedList = mock(List.class);
mockedList.clear();
verify(mockedList).clear();

It doesn't get much simpler than that.

The only major downside I can think of is that it won't mock static methods.

Test and Mock framework for Java and .NET

The N/J-series of frameworks (NUnit/JUnit, NMock/JMock, etc.) are typically parallel ports of each other or are based on the same starting principles. Those will definitely let you transfer at least some of your knowledge between them.

Best Mock object framework for EJB junit tests

It's not a mocking framework, but you might like to have a look at OpenEJB, which is a lightweight EJB container suitable for, amongst other things, use in unit tests. It will save you having to mock the container's interfaces, and you can still mock interfaces to other components.

What is the difference between PowerMock, EasyMock and Mockito frameworks?

I give you an explanation that other people probably wont like, but I (and a lot of people I mentioned it to) find/found very helpful: PowerMock is the mocking framework ... that you better not use.

The main advantage from PowerMock is that you can use it to test certain constructs (for example static method invocations) that EasyMock can't mock. Thus: when you want to test 3rd party code that you can't change; and that contains static calls; then you might turn to PowerMock.

But when you write your own code, and you focus on writing testable code; then you will find that the "need to use PowerMock" is absolutely equal to "you did a bad job designing your code". Using static for example directly leads to direct coupling between your classes; and therefore it is hard to get rid of it once it is in.

Don't get me wrong - PowerMock has its place in testing; but its powerful features come at certain cost.

On EasyMock / Mockito: mainly "two different ways" of writing down test cases; the later one leading to tests that many people find easier to read; whereas EasyMock and "strict" mocks allow for writing down test cases that will break quickly upon most changes in your production code (which by itself can be very useful, or very annoying).

Why should I use mock objects (Java)? Do all mocking frameworks serve the same purpose?

The purpose of mocking is to allow you to test your application components in isolation. For example, if you have a service that uses a DAO, you want to be able to test your service without actually having to go to the database via the DAO. You would

1) Test the DAO in isolation, and

2) Mock the DAO in your service tests, so your service can be tested in isolation.

Since your DAO has a clearly defined API, your mock DAO simulates that API. So if you have a findAll type method that your service calls, you can easily mock the DAO to expect a call to findAll and return the appropriate results.

The purpose of all mocking frameworks is the same -- to allow you to set up expectations and assert that those expectations were met in your tests. The details, however, are different between frameworks. I have used EasyMock in actual work environments, and played around with Mockito for answering a few questions on SO. Both are pretty nice. The cost of entry is fairly low -- the idea is always to set up expectations, get the mock to return the results, and then to verify that the expectations were met. You might have to take a day or 2 to understand your mocking framework, but after that you should be able to be productive and figure out what you don't know as you go.

Which is the best isolation framework for Java? JMock, Easymock, Mockito, or other?

I have used Easymock earlier, but now I'm using Mockito. I found Mockito simpler as compared to Easymock. For the detailed comparison of Easymock and Mockito you can refer here

Mocking library/framework that works best in Android?

(Update: Mockito has added Android support as of version 1.9.5 and EasyMock has added Android support as of version 3.2 by factoring out those bits that generate code at runtime and making them pluggable, e.g. by using dexmaker instead of cglib.)

Except for android-mock mentioned by DixonD (which is a fairly young, unproven library), there currently is no solution. You can immediately forget anything based on CGLib (Mockito, plain EasyMock), since CGLib relies on byte code generation and won't work on Dalvik (it also relies on the Java Beans package, which is also not part of Android).

For what it's worth, you could use the very few mock classes coming with Android (like MockContext), but they don't verify behavior, they're just stubs. Their default behavior is to throw a runtime error in every method, so you have to subclass them and override the methods you want to mock.

However, you can still use mocking libraries in non-instrumentation tests, i.e. in your standard unit tests executed on the JVM. You can use PowerMock to mock framework methods, it has support for mocking static methods and constructors, making the mocking as powerful as e.g. in Ruby (just more painful to use).

We use JUnit 4 + PowerMock + Mockito and mock out classes like Context and TextUtils in a base class from which we inherit every normal JUnit test. For instrumentation tests, we create custom mock classes and decide using a factory which implementation (mock or not) to instantiate at runtime.

Comparison between Mockito vs JMockit - why is Mockito voted better than JMockit?

Update Sep 2019: The only mocking framework supported (by default) by Spring Boot is Mockito. If you use Spring, the answer is quite obvious.

I'd say the competition is between JMockit and PowerMock, then Mockito.

I'd leave "plain" jMock and EasyMock because they use only proxy & CGLIB and do not use Java 5 instrumentation like the newer frameworks.

jMock also didn't have a stable release for over 4 years. jMock 2.6.0 required 2 years to go from RC1 to RC2, and then another 2 years before it actually got released.

Regarding Proxy & CGLIB vs instrumentation:

(EasyMock and jMock) are based on java.lang.reflect.Proxy,
which requires an interface to be
implemented. Additionally, they
support the creation of mock objects
for classes through CGLIB subclass
generation. Because of that, said
classes cannot be final and only
overridable instance methods can be
mocked. Most importantly, however,
when using these tools the
dependencies of code under test (that
is, the objects of other classes on
which a given class under test
depends) must be controlled by the
tests, so that mock instances can be
passed to the clients of those
dependencies. Therefore, dependencies
can't simply be instantiated with the
new operator in a client class for
which we want to write unit tests.

Ultimately, the technical limitations
of conventional mocking tools impose
the following design restrictions on
production code:

Each class which may need to be mocked in a test must either implement
a separate interface or not be final.

The dependencies of each class to be tested must either be obtained
through configurable instance creation
methods (factories or a Service
Locator), or be exposed for dependency
injection. Otherwise, unit tests won't
be able to pass mock implementations
of dependencies to the unit under
test.

Since only instance methods can be mocked, classes to be unit tested
cannot call any static methods on
their dependencies, nor instantiate
them using any of the constructors.

The above is copied from http://jmockit.org/about.html . Further, it compares between itself (JMockit), PowerMock, and Mockito in several ways:

There are now other mocking tools for
Java which also overcome the
limitations of the conventional ones,
between them PowerMock, jEasyTest, and
MockInject. The one that comes closest
to the feature set of JMockit is
PowerMock, so I will briefly evaluate
it here (besides, the other two are
more limited and don't seem to be
actively developed anymore).

JMockit vs PowerMock

First of all, PowerMock does not provide a complete API for mocking,
but instead works as an extension to
another tool, which currently can be
EasyMock or Mockito. This is obviously
an advantage for existing users of
those tools.

JMockit, on the other hand, provides entirely new APIs, although
its main API (Expectations) is similar
to both EasyMock and jMock. While this
creates a longer learning curve, it
also allows JMockit to provide a
simpler, more consistent, and easier
to use API.

Compared to the JMockit Expectations API, the PowerMock API is
more "low-level", forcing users to
figure out and specify which classes
need to be prepared for testing (with
the @PrepareForTest({ClassA.class,
...}) annotation) and requiring
specific API calls to deal with
various kinds of language constructs
that may be present in the production
code: static methods
(mockStatic(ClassA.class)),
constructors
(suppress(constructor(ClassXyz.class))),
constructor invocations
(expectNew(AClass.class)), partial
mocks (createPartialMock(ClassX.class,
"methodToMock")), etc.

With JMockit Expectations, all kinds of methods and constructors are
mocked in a purely declarative way,
with partial mocking specified through
regular expressions in the @Mocked
annotation or by simply "un-mocking"
the members with no recorded
expectations; that is, the developer
simply declares some shared "mock
fields" for the test class, or some
"local mock fields" and/or "mock
parameters" for individual test
methods (and in this last case the
@Mocked annotation often won't be
needed).

Some capabilities available in JMockit, such as support for mocking
equals and hashCode, overridden
methods, and others, are currently not
supported in PowerMock. Also, there is
no equivalent to JMockit's ability to
capture instances and mock
implementations of specified base
types as the test executes, without
the test code itself having any
knowledge of the actual implementation
classes.

PowerMock uses custom class loaders (usually one per test class)
in order to generate modified versions
of the mocked classes. Such heavy use
of custom class loaders can lead to
conflicts with third-party libraries,
hence the need to sometimes use the
@PowerMockIgnore("package.to.be.ignored")
annotation on test classes.

The mechanism used by JMockit (runtime instrumentation through a
"Java agent") is simpler and safer,
although it does require passing a
"-javaagent" parameter to the JVM when
developing on JDK 1.5; on JDK 1.6+
(which can always be used for
development, even if deploying on an
older version) there is no such
requirement, since JMockit can
transparently load the Java agent on
demand by using the Attach API.

Another recent mocking tool is
Mockito. Although it does not attempt
to overcome the limitations of older
tools (jMock, EasyMock), it does
introduce a new style of behavior
testing with mocks. JMockit also
supports this alternative style,
through the Verifications API.

JMockit vs Mockito

Mockito relies on explicit calls to its API in order to separate code
between the record (when(...)) and
verify (verify(...)) phases. This
means that any invocation to a mock
object in test code will also require
a call to the mocking API.
Additionally, this will often lead to
repetitive when(...) and
verify(mock)... calls.

With JMockit, no similar calls exist. Sure, we have the new
NonStrictExpectations() and new
Verifications() constructor calls, but
they occur only once per test
(typically), and are completely
separate from the invocations to
mocked methods and constructors.

The Mockito API contains several inconsistencies in the syntax used for
invocations to mocked methods. In the
record phase, we have calls like
when(mock.mockedMethod(args))... while
in the verify phase this same call
will be written as
verify(mock).mockedMethod(args).
Notice that in the first case the
invocation to mockedMethod is made
directly on the mock object, while in
the second case it is made on the
object returned by verify(mock).

JMockit has no such inconsistencies because invocations to
mocked methods are always made
directly on the mocked instances
themselves. (With one exception only:
to match invocations on the same
mocked instance, an onInstance(mock)
call is used, resulting in code like
onInstance(mock).mockedMethod(args);
most tests won't need to use this,
though.)

Just like other mocking tools which rely on method
chaining/wrapping, Mockito also runs
into inconsistent syntax when stubbing
void methods. For example, you write
when(mockedList.get(1)).thenThrow(new
RuntimeException()); for a non-void
method, and doThrow(new
RuntimeException()).when(mockedList).clear();
for a void one. With JMockit, it's
always the same syntax:
mockedList.clear(); result = new
RuntimeException();.

Yet another inconsistency occurs in the use of Mockito spies: "mocks"
that allow the real methods to be
executed on the spied instance. For
example, if spy refers to an empty
List, then instead of writing
when(spy.get(0)).thenReturn("foo") you
will need to write
doReturn("foo").when(spy).get(0). With
JMockit, the dynamic mocking feature
provides similar functionality to
spies, but without this issue since
real methods only get executed during
the replay phase.

In EasyMock and jMock, the first mocking APIs for Java, the focus was
entirely on the recording of expected
invocations of mocked methods, for
mock objects that (by default) do not
allow unexpected invocations. Those
APIs also provide the recording of
allowed invocations for mock objects
that do allow unexpected invocations,
but this was treated as a second-class
feature. Additionally, with these
tools there is no way to explicitly
verify invocations to mocks after the
code under test is exercised. All such
verifications are performed implicitly
and automatically.

In Mockito (and also in Unitils Mock), the opposite viewpoint is
taken. All invocations to mock objects
that may happen during the test,
whether recorded or not, are allowed,
never expected. Verification is
performed explicitly after the code
under test is exercised, never
automatically.

Both approaches are too extreme, and consequently less than optimal.
JMockit Expectations & Verifications
is the only API that allows the
developer to seamlessly choose the
best combination of strict (expected
by default) and non-strict (allowed by
default) mock invocations for each
test.

To be more clear, the Mockito API has the following shortcoming. If you
need to verify that an invocation to a
non-void mocked method happened during
the test, but the test requires a
return value from that method that is
different from the default for the
return type, then the Mockito test
will have duplicate code: a
when(mock.someMethod()).thenReturn(xyz)
call in the record phase, and a
verify(mock).someMethod() in the
verify phase. With JMockit, a strict
expectation can always be recorded,
which won't have to be explicitly
verified. Alternatively, an invocation
count constraint (times = 1) can be
specified for any recorded non-strict
expectation (with Mockito such
constraints can only be specified in a
verify(mock, constraint) call).

Mockito has poor syntax for verifications in order, and for full
verifications (that is, checking that
all invocations to mock objects are
explicitly verified). In the first
case, an extra object needs to be
created, and calls to verify made on
it: InOrder inOrder = inOrder(mock1,
mock2, ...). In the second case, calls
like verifyNoMoreInteractions(mock) or
verifyZeroInteractions(mock1, mock2)
need to be made.

With JMockit, you simply write new VerificationsInOrder() or new
FullVerifications() instead of new
Verifications() (or new
FullVerificationsInOrder() to combine
both requirements). No need to specify
which mock objects are involved. No
extra mocking API calls. And as a
bonus, by calling
unverifiedInvocations() inside an
ordered verification block, you can
perform order-related verifications
that are simply impossible in Mockito.

Finally, the JMockit Testing Toolkit
has a wider scope and more ambitious
goals than other mocking toolkits, in
order to provide a complete and
sophisticated developer testing
solution. A good API for mocking, even
without artificial limitations, is not
enough for productive creation of
tests. An IDE-agnostic, easy to use,
and well integrated Code Coverage tool
is also essential, and that's what
JMockit Coverage aims to provide.
Another piece of the developer testing
toolset which will become more useful
as the test suite grows in size is the
ability to incrementally rerun tests
after a localized change to production
code; this is also included in the
Coverage tool.

(granted, the source may be biased, but well...)

I'd say go with JMockit. It's the easiest to use, flexible, and works for pretty much all cases even difficult ones and scenarios when you can't control the class to be tested (or you can't break it due to compatibility reasons etc.).

My experiences with JMockit have been very positive.

How do Java mocking frameworks work?

I will speak of the framework that I use ( jmock ), but others do something very similar.

How does it know how to create the
mock object?

It does not, you tell the framework that you require a mock object and you give it a type ( ideally an interface ) of the object that you would like to mock. Behind the scenes the mock framework uses a combination of reflection and, sometimes, byte-code rewrites to create a mock object.

Is it done in runtime or generates a
file?

jMock creates it at runtime.

How do you know its behavior?

Mock objects are pretty dumb. You specify the behavior that you expect of them.

And most importantly - what is the
work flow of using such a framework
(what is the step-by-step for creating
a test).

One very important thing that framework must provide is an ability to check that expected behavior has been observed during the test execution.

jmock does it by introducing specific test runner that checks that all expectations on all declared mock objects after the test has finished. If something does not match, the exception is thrown.

Usually the pattern is:

Acquire mock factory ( called Mockery in jmock )out of thin air ( with specific no-args constructor ).
Create required mock objects.
Setup expectation on mock objects
Call the method you are testing, passing mocks instead of real objects
If your method returns some values, check expected value with regular assertXXX methods.

What's the Best Mock Framework for Java