Rust Mock Shootout!

Rust has several different mocking libraries. None is clearly superior to all
of the others. This project compares their feature sets, using an
apples-to-apples approach. Each of several dozen features is implemented with
multiple mocking libraries, if possible. The test results then show which
features work with which library.

The Contenders

Mockers

This is the oldest library in our shootout. Inspired by
GoogleMock, it has an elegant syntax, powerful sequence matching, and a rich
variety of helper methods. It works on stable Rust, though generic methods
require nightly.

Mock_Derive

This was the first Rust mocking library that supported
automatically derive ing the Mock object from the target trait. That saves a
lot of typing! Mock_Derive is still very easy to use, though it lacks any
ability to validate method arguments. It also can’t work with generic traits,
traits with generic methods, traits defined in external crates, or multiple
traits (like T: A + B). However, it does have a few rare features, like the
ability to mock foreign free functions, or traits with static methods.
Unfortunately, it’s maintenance has fallen behind and it no longer compiles on
recent toolchains.

Galvanic-mock

This is part of a suite of testing libraries. Along with
galvanic-test and galvanic-assert, it provides a comprehensive set of testing
functionality for Rust programs. Galvanic-mock itself takes a behavior-driven
approach to testing. It tries to separate the specification from what a mock
does, from how that mock is expected to be used. It’s a good all-around
library: good feature set, good documentation, good ergonomics.

Pseudo

All of the previous libraries had one thing in common: they all
require the nightly compiler. That allows them to do some pretty cool stuff,
like tweak the language’s syntax, but it’s also inherently unstable. No code
that relies on nightly can be guaranteed to work with future compilers. Pseudo
is different. It eschews nightly-dependent features like derive so that it
can work on stable Rust. Unfortunately, that also makes it pretty verbose.

Double

Like Pseudo, Double runs on stable Rust. However, it uses a few
macros to reduce the verbosity. The feature set is pretty similar. In fact,
the entire API is eerily similar. I think one of these crates must’ve copied
from the other (perhaps they both did).

Simulacrum

This is a bit of a different beast. Whereas other mock
libraries try to provide a clean API, Simulacrum actually provides 3 different
APIs. That can be confusing at first, but the result is great power. Unusual
requirements, impossible to meet with the highest-level and most conveient API,
can be satisfied (at greater effort) with the lower-level APIs. Simulacrum also
runs on stable Rust, and manages to do it with less verbosity than Double.
Optional, nightly-dependent, support for derive is a work-in-progress.

Mock-it

Mock-it is Rust’s newest mocking library, and probably its
simplest. Its chief advantage is that its simplicity allows it to run on stable
Rust, though that’s fast becoming a less unique feature. However, the lack of a
high-level API also gives Mock-it some of the power of Simulacrum; it can mock a
struct, for example. Overall, Mock-it has few practical advantages, but it’s
a good starting point for someone looking to build something bigger.

Mocktopus

Mocktopus is a bit of a different beast. Whereas every other
library focuses on mocking trait s, Mocktopus focuses on free functions. In
fact, the only way that it can mock a trait is by manually creating an
implementation of that trait, and then mocking every single function.
Mocktopus also has only the most rudimentary support for expectations. Its
advantages are that it requires very little boiler plate, it can mock struct
s and free functions, and it handles generic functions well.

Mockiato

A fairly new library. The syntax is simple and terse, but it has
few distinct features.

Mockall

Mockall is the newest contender. It aims to have the best feature
set of all of the above libraries, with the most ergonomic interface. It
runs on stable Rust, and uses no unsafe code at all. I wrote it myself,
after writing this shootout and being unsatisfied with all of the other
libraries.

Features

I evaluated about three dozen features evaluated for each library. The first
group are the essential features. These determine the library’s overall
capabilities. While they may not be important to all users, they’re considered
“essential” because a user can’t implement any of these if the library doesn’t
intrinsically support them.

The second group are the convenience features. These are features that a user
can implement in terms of other essential features, for example, “Match
constant” can be implemented by matching with a method). The lack of any of
these features shouldn’t preclude the use of a certain library, but may cause
some annoyance.

The third group of “features” are really more informational in nature:

Feature Matrix

	Double	Galvanic-mock	Mock-it	Mockall	Mockers	Mockiato	Mocktopus	Pseudo	Simulacrum
Essential Features
Associated types	no	yes	yes	yes	yes	no	no	yes	yes
Checkpoints	no	no	no	yes	yes	no	no	yes	yes
Closures	no	no	no	yes	no	no	no	no	no
Reference parameters	no	yes	no	yes	yes	yes	yes	no	yes
Consume parameters	no	no	no	yes	yes	no	yes	no	no
Consume self	no	yes	yes	yes	yes	yes	yes	yes	yes
Doctest	yes	yes	yes	yes	yes	yes	yes	yes	yes
External traits	yes	no	yes	yes	yes	yes	no	yes	yes
Foreign	no	no	no	yes	yes	no	no	no	no
Generic methods	no	no	no	yes	yes	no	yes	no	yes
Generic methods with lifetime parameters	no	no	no	yes	no	yes	yes	no	no
Generic return	no	no	no	yes	yes	yes	yes	no	yes
Generic structs	no	no	yes	yes	no	no	yes	yes	yes
Generic traits	no	yes	yes	yes	no	yes	yes	yes	yes
Inherited traits	yes	yes	yes	yes	yes	no	no	yes	yes
Match function	yes	yes	no	yes	yes	no	yes	yes	yes
Structs	no	no	yes	yes	no	no	yes	yes	yes
Traits	yes	yes	yes	yes	yes	yes	no	yes	yes
Multiple traits	no	yes	yes	yes	yes	no	no	yes	yes
Return call with args	yes	yes	no	yes	yes	no	yes	yes	yes
Return reference	no	no	no	yes	no	yes	no	no	no
Return mutable reference	no	no	no	yes	no	no	no	no	no
Return owned	no	no	no	yes	yes	yes	no	no	no
Return parameters	no	no	no	yes	yes	no	yes	no	yes
Send	no	no	yes	yes	no	yes	no	yes	no
Sequence	single method	no	no	multi object	multi object	single method	no	single method	multi method
Static methods	no	no	yes	yes	yes	no	yes	yes	no
Times range	yes	no	no	yes	yes	yes	no	no	no
Where clauses	no	no	no	yes	no	no	no	no	no
Convenience Features
Derive	no	yes	no	yes	yes	yes	yes	no	no
Fallback	no	no	no	no	no	no	yes	no	no
Impl Trait	no	no	no	yes	no	no	no	no	no
Match combinations	yes	no	no	yes	yes	no	no	no	no
Match constant	yes	no	yes	yes	yes	yes	no	yes	yes
Match operator	yes	yes	no	yes	yes	no	no	no	yes
Match pattern	no	no	no	no	yes	no	no	no	no
Match range	no	no	no	no	yes	no	no	no	no
Match wildcard	yes	yes	yes	yes	yes	yes	no	yes	yes
Mock modules	no	no	no	yes	no	no	yes	no	no
Return a constant	yes	yes	yes	yes	yes	yes	no	yes	no
Return default	yes	no	yes	yes	yes	no	no	yes	no
Return panic	no	yes	no	no	yes	yes	no	no	no
Times once	yes	yes	no	yes	yes	yes	no	no	yes
Times any	yes	yes	no	yes	yes	yes	no	no	yes
Times n	yes	yes	no	yes	yes	yes	no	yes	yes
Times never	yes	yes	no	yes	yes	yes	no	yes	yes
Other
Maximum arguments	12	≥ 16	12	≥ 16	4	≥ 16	≥ 16	12	9
Rustc	stable	nightly	stable	stable	stable	stable	nightly	stable	stable
First release	Dec-12-2017	Aug-13-2017	Mar-11-2018	Jul-3-2019	Apr-6-2016	Feb-11-2019	Sep-5-2017	Mar-23-2017	Dec-17-2017
Tested version	0.2.2	0.1.3	0.3.0	0.5.2	0.13.4	0.9.5	0.7.3	0.2.0	0.3.1
Current version

Feature Definitions

Associated types

Can the library mock a trait with associated types, like
Iterator?

Checkpoints

When validating sequences of method calls, can the library
create checkpoints (aka Eras)? A checkpoint divides expectations
chronologically. All expectations created before the checkpoint must be
satisfied before it, and all expectations created afterwards must be satisfied
after it.

Closures

Can the library mock a method that takes a closure argument, and
execute that closure when checking call arguments and calculating return
values?

Reference parameters

Can a mocked method take its parameters by reference?

Consume parameters

Can a mock method consume its parameters, passing them
by value to an arbitrary function? This is important, for example, to keep the
parameters from drop ping after the mocked method call.

Consume self

Can the library mock a consuming method? A consuming method
is one that takes the self parameter by value, rather than by reference.
into_* methods are a common example.

Doctest

Can the library be used in doc tests? The key difference here is
that doc tests are compiled with cfg=false.

External traits

Can the library mock a trait defined in another module or
crate?

Fallback

Can a mock object proxy certain method calls to a real object?

Foreign

Can the library mock static external functions?

Generic methods

Can the library mock traits with generic methods that
have parameterized arguments, and set expectations for those methods? For
example, a method like fn foo<T>(&self, t: T) -> u32.

Generic method with lifetime parameters

Can the library mock
functions or methods that have generic lifetime parameters, and set
expectations for those methods? For example, a method like
fn foo<'a>(&self, t: T<'a>).

Generic return

Can the library mock traits with generic methods that have
parameterized return values, and set expectations for those methods? For
example, a method like fn foo<T>(&self, x: u32) -> T.

Generic structs

Can the library mock generic structs, like
std::convert::Mutex<T>?

Generic traits

Can the library mock generic traits, like
std::sync::Into<T>?

Impl Trait

Can the library derive mocks for methods that use
-> impl Trait syntax?

Inherited traits

Can the library mock inherited traits like pub trait B: A?

Match function

Can an expectation validate arguments with an arbitrary
function?

Structs

Can the library mock a concrete struct instead of just
trait s? This requires altering the module’s namespace during unit tests.
The test_double crate can do that.

Modules

Can the library mock every function in a module?

Multiple traits

Can the library create a mock that satisfies multiple traits,
so it can be passed to a function like fn foo<T: A + B>(t: T)?

Traits

Can the library create a mock object that implements a trait?

Return call with args

Can a mocked method return a value computed from the
arguments by an arbitrary function?

Return reference

Can a mocked method return a reference with the lifetime
of the mock object?

Return mutable reference

Can a mocked method return a mutable reference
with the lifetime of the mock object?

Return owned

Can a mocked method return ownership of a value that does not
implement Clone?

Return parameters

Can a mocked method modify method arguments provided as
mutable references?

Sequence

Can the library assert that methods are called in a particular
order? This feature is implemented to different degrees by different libraries.
One library only supports validating call order on a method-by-method basis.
Another can validate the call order of different methods of the same mock
object. Only one library can validate the call order of methods of different
objects.

Send

Are mock objects Send? If not, then the library cannot mock a
trait that is.

Static methods

Can the library mock a trait that has a static method? A
static method, also called an “associated function” is one that does not receive
any form of self as a parameter. In Rust, they must be called using
Trait::function() syntax, rather than object.function(). That makes it
impossible for a mock library to set an expectation on such a method. However,
mocking such a trait is still useful for setting expectations on other methods.

Times range

Can the library expect a method to be called a variable number
of times, bounded by a range?

Where clauses

Can the library mock generic traits and methods with where
clauses, and will the expectation still satisfy those where clauses?

Derive

Can the library automatically generate the Mock struct by
derive ing on the trait? This feature can save a lot of typing, but it
invariably requires nightly Rust. It also can’t be used when mocking external
traits.

Match combinattions

Expectations can match boolean combinations of other
validators.

Match constant

Can an expectation match arguments with constants?

Match operator

Can an expectation match arguments with common operators,
like eq , gt , etc?

Match pattern

Can an expectation match enum arguments using patterns?

Match range

Can an expectation match arguments with ranges?

Match wildcard

Can an expectation match any argument?

Return constant

Can an expectation return a constant?

Return default

Can mocked methods automatically return the
Default::default() value?

Return panic

Can expectations panic instead of returning?

Times once

Can an expectation assert that it’s called exactly once?

Times any

Can a mocked method be called any number of times?

Times n

Can an expectation assert that it’s called an arbitrary number of
times?

Times never

Can a mocked method expect to never be called?

Maximum arguments

The maximum number of arguments for a mocked method.

Rustc

Minimum required compiler version. None of these six crates
guarantee a specific version, just “stable”, or “nightly”. Typically, the
crates that require “nightly” will only work with a narrow range of nightly
versions.

First release

Date of the first release on crates.io .

Conclusion

The best mocking library is … none of them. No one library is clearly
superior to all of the others. Every project will need to choose a different
mocking library with the features needed for that particular project. Some
projects may event need to use multiple libraries in combination. But a few
things stand out:

• Now that proc macros have been available on stable Rust for awhile, there
  isn’t any good reason to require nightly anymore. However, a few libraries
  have optional features that are only available when built with nightly
  compiler.
• However, not all libraries make use of proc macros. Those that don’t, like Mock-it, are more verbose to use than those that do.
• On the other hand, proc macros can be difficult to debug when they fail to compile. Galvanic-mock is probably the worst in that respect. A non-proc-macro based library may be easier to debug.
• Validating call sequences is an underappreciated aspect of mocking. Only
  Mockers and Mockall have full support for validating the sequences of multiple
  objects’ methods. However, Simulacrum’s sequence syntax is more elegant.
• Mocking is complicated, and none of these libraries’ authors anticipated every
  possible situation. However, Mock-it and Simulacrum proved surprisingly
  versatile, since they allow the user relatively low-level access into the
  library itself.
• Mockall is the clear winner on features, as it should be, because I wrote it
  with prior knowledge of all the other libraries.
• Mockall and Mockers are probably the easiest libraries to use.

Discussion

Tear me a new one at the Rust Forums