/*
ChibiOS/RT - Copyright (C) 2006,2007,2008,2009,2010,
2011 Giovanni Di Sirio.
This file is part of ChibiOS/RT.
ChibiOS/RT is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
ChibiOS/RT is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see
# Advanced googletest Topics
<!-- GOOGLETEST_CM0016 DO NOT DELETE -->
## Introduction
Now that you have read the [googletest Primer](primer.md) and learned how to
write tests using googletest, it's time to learn some new tricks. This document
will show you more assertions as well as how to construct complex failure
messages, propagate fatal failures, reuse and speed up your test fixtures, and
use various flags with your tests.
## More Assertions
This section covers some less frequently used, but still significant,
assertions.
### Explicit Success and Failure
These three assertions do not actually test a value or expression. Instead, they
generate a success or failure directly. Like the macros that actually perform a
test, you may stream a custom failure message into them.
```c++
SUCCEED();
```
Generates a success. This does **NOT** make the overall test succeed. A test is
considered successful only if none of its assertions fail during its execution.
NOTE: `SUCCEED()` is purely documentary and currently doesn't generate any
user-visible output. However, we may add `SUCCEED()` messages to googletest's
output in the future.
```c++
FAIL();
ADD_FAILURE();
ADD_FAILURE_AT("file_path", line_number);
```
`FAIL()` generates a fatal failure, while `ADD_FAILURE()` and `ADD_FAILURE_AT()`
generate a nonfatal failure. These are useful when control flow, rather than a
Boolean expression, determines the test's success or failure. For example, you
might want to write something like:
```c++
switch(expression) {
case 1:
... some checks ...
case 2:
... some other checks ...
default:
FAIL() << "We shouldn't get here.";
}
```
NOTE: you can only use `FAIL()` in functions that return `void`. See the
[Assertion Placement section](#assertion-placement) for more information.
### Exception Assertions
These are for verifying that a piece of code throws (or does not throw) an
exception of the given type:
Fatal assertion | Nonfatal assertion | Verifies
------------------------------------------ | ------------------------------------------ | --------
`ASSERT_THROW(statement, exception_type);` | `EXPECT_THROW(statement, exception_type);` | `statement` throws an exception of the given type
`ASSERT_ANY_THROW(statement);` | `EXPECT_ANY_THROW(statement);` | `statement` throws an exception of any type
`ASSERT_NO_THROW(statement);` | `EXPECT_NO_THROW(statement);` | `statement` doesn't throw any exception
Examples:
```c++
ASSERT_THROW(Foo(5), bar_exception);
EXPECT_NO_THROW({
int n = 5;
Bar(&n);
});
```
**Availability**: requires exceptions to be enabled in the build environment
### Predicate Assertions for Better Error Messages
Even though googletest has a rich set of assertions, they can never be complete,
as it's impossible (nor a good idea) to anticipate all scenarios a user might
run into. Therefore, sometimes a user has to use `EXPECT_TRUE()` to check a
complex expression, for lack of a better macro. This has the problem of not
showing you the values of the parts of the expression, making it hard to
understand what went wrong. As a workaround, some users choose to construct the
failure message by themselves, streaming it into `EXPECT_TRUE()`. However, this
is awkward especially when the expression has side-effects or is expensive to
evaluate.
googletest gives you three different options to solve this problem:
#### Using an Existing Boolean Function
If you already have a function or functor that returns `bool` (or a type that
can be implicitly converted to `bool`), you can use it in a *predicate
assertion* to get the function arguments printed for free:
<!-- mdformat off(github rendering does not support multiline tables) -->
| Fatal assertion | Nonfatal assertion | Verifies |
| --------------------------------- | --------------------------------- | --------------------------- |
| `ASSERT_PRED1(pred1, val1)` | `EXPECT_PRED1(pred1, val1)` | `pred1(val1)` is true |
| `ASSERT_PRED2(pred2, val1, val2)` | `EXPECT_PRED2(pred2, val1, val2)` | `pred2(val1, val2)` is true |
| `...` | `...` | `...` |
<!-- mdformat on-->
In the above, `predn` is an `n`-ary predicate function or functor, where `val1`,
`val2`, ..., and `valn` are its arguments. The assertion succeeds if the
predicate returns `true` when applied to the given arguments, and fails
otherwise. When the assertion fails, it prints the value of each argument. In
either case, the arguments are evaluated exactly once.
Here's an example. Given
```c++
// Returns true if m and n have no common divisors except 1.
bool MutuallyPrime(int m, int n) { ... }
const int a = 3;
const int b = 4;
const int c = 10;
```
the assertion
```c++
EXPECT_PRED2(MutuallyPrime, a, b);
```
will succeed, while the assertion
```c++
EXPECT_PRED2(MutuallyPrime, b, c);
```
will fail with the message
```none
MutuallyPrime(b, c) is false, where
b is 4
c is 10
```
> NOTE:
>
> 1. If you see a compiler error "no matching function to call" when using
> `ASSERT_PRED*` or `EXPECT_PRED*`, please see
> [this](faq.md#the-compiler-complains-no-matching-function-to-call-when-i-use-assert-pred-how-do-i-fix-it)
> for how to resolve it.
#### Using a Function That Returns an AssertionResult
While `EXPECT_PRED*()` and friends are handy for a quick job, the syntax is not
satisfactory: you have to use different macros for different arities, and it
feels more like Lisp than C++. The `::testing::AssertionResult` class solves
this problem.
An `AssertionResult` object represents the result of an assertion (whether it's
a success or a failure, and an associated message). You can create an
`AssertionResult` using one of these factory functions:
```c++
namespace testing {
// Returns an AssertionResult object to indicate that an assertion has
// succeeded.
AssertionResult AssertionSuccess();
// Returns an AssertionResult object to indicate that an assertion has
// failed.
AssertionResult AssertionFailure();
}
```
You can then use the `<<` operator to stream messages to the `AssertionResult`
object.
To provide more readable messages in Boolean assertions (e.g. `EXPECT_TRUE()`),
write a predicate function that returns `AssertionResult` instead of `bool`. For
example, if you define `IsEven()` as:
```c++
::testing::AssertionResult IsEven(int n) {
if ((n % 2) == 0)
return ::testing::AssertionSuccess();
else
return ::testing::AssertionFailure() << n << " is odd";
}
```
instead of:
```c++
bool IsEven(int n) {
return (n % 2) == 0;
}
```
the failed assertion `EXPECT_TRUE(IsEven(Fib(4)))` will print:
```none
Value of: IsEven(Fib(4))
Actual: false (3 is odd)
Expected: true
```
instead of a more opaque
```none
Value of: IsEven(Fib(4))
Actual: false
Expected: true
```
If you want informative messages in `EXPECT_FALSE` and `ASSERT_FALSE` as well
(one third of Boolean assertions in the Google code base are negative ones), and
are fine with making the predicate slower in the success case, you can supply a
success message:
```c++
::testing::AssertionResult IsEven(int n) {
if ((n % 2) == 0)
return ::testing::AssertionSuccess() << n << " is even";
else
return ::testing::AssertionFailure() << n << " is odd";
}
```
Then the statement `EXPECT_FALSE(IsEven(Fib(6)))` will print
```none
Value of: IsEven(Fib(6))
Actual: true (8 is even)
Expected: false
```
#### Using a Predicate-Formatter
If you find the default message generated by `(ASSERT|EXPECT)_PRED*` and
`(ASSERT|EXPECT)_(TRUE|FALSE)` unsatisfactory, or some arguments to your
predicate do not support streaming to `ostream`, you can instead use the
following *predicate-formatter assertions* to *fully* customize how the message
is formatted:
Fatal assertion | Nonfatal assertion | Verifies
------------------------------------------------ | ------------------------------------------------ | --------
`ASSERT_PRED_FORMAT1(pred_format1, val1);` | `EXPECT_PRED_FORMAT1(pred_format1, val1);` | `pred_format1(val1)` is successful
`ASSERT_PRED_FORMAT2(pred_format2, val1, val2);` | `EXPECT_PRED_FORMAT2(pred_format2, val1, val2);` | `pred_format2(val1, val2)` is successful
`...` | `...` | ...
The difference between this and the previous group of macros is that instead of
a predicate, `(ASSERT|EXPECT)_PRED_FORMAT*` take a *predicate-formatter*
(`pred_formatn`), which is a function or functor with the signature:
```c++
::testing::AssertionResult PredicateFormattern(const char* expr1,
const char* expr2,
...
const char* exprn,
T1 val1,
T2 val2,
...
Tn valn);
```
where `val1`, `val2`, ..., and `valn` are the values of the predicate arguments,
and `expr1`, `expr2`, ..., and `exprn` are the corresponding expressions as they
appear in the source code. The types `T1`, `T2`, ..., and `Tn` can be either
value types or reference types. For example, if an argument has type `Foo`, you
can declare it as either `Foo` or `const Foo&`, whichever is appropriate.
As an example, let's improve the failure message in `MutuallyPrime()`, which was
used with `EXPECT_PRED2()`:
```c++
// Returns the smallest prime common divisor of m and n,
// or 1 when m and n are mutually prime.
int SmallestPrimeCommonDivisor(int m, int n) { ... }
// A predicate-formatter for asserting that two integers are mutually prime.
::testing::AssertionResult AssertMutuallyPrime(const char* m_expr,
const char* n_expr,
int m,
int n) {
if (MutuallyPrime(m, n)) return ::testing::AssertionSuccess();
return ::testing::AssertionFailure() << m_expr << " and " << n_expr
<< " (" << m << " and " << n << ") are not mutually prime, "
<< "as they have a common divisor " << SmallestPrimeCommonDivisor(m, n);
}
```
With this predicate-formatter, we can use
```c++
EXPECT_PRED_FORMAT2(AssertMutuallyPrime, b, c);
```
to generate the message
```none
b and c (4 and 10) are not mutually prime, as they have a common divisor 2.
```
As you may have realized, many of the built-in assertions we introduced earlier
are special cases of `(EXPECT|ASSERT)_PRED_FORMAT*`. In fact, most of them are
indeed defined using `(EXPECT|ASSERT)_PRED_FORMAT*`.
### Floating-Point Comparison
Comparing floating-point numbers is tricky. Due to round-off errors, it is very
unlikely that two floating-points will match exactly. Therefore, `ASSERT_EQ` 's
naive comparison usually doesn't work. And since floating-points can have a wide
value range, no single fixed error bound works. It's better to compare by a
fixed relative error bound, except for values close to 0 due to the loss of
precision there.
In general, for floating-point comparison to make sense, the user needs to
carefully choose the error bound. If they don't want or care to, comparing in
terms of Units in the Last Place (ULPs) is a good default, and googletest
provides assertions to do this. Full details about ULPs are quite long; if you
want to learn more, see
[here](https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/).
#### Floating-Point Macros
<!-- mdformat off(github rendering does not support multiline tables) -->
| Fatal assertion | Nonfatal assertion | Verifies |
| ------------------------------- | ------------------------------- | ---------------------------------------- |
| `ASSERT_FLOAT_EQ(val1, val2);` | `EXPECT_FLOAT_EQ(val1, val2);` | the two `float` values are almost equal |
| `ASSERT_DOUBLE_EQ(val1, val2);` | `EXPECT_DOUBLE_EQ(val1, val2);` | the two `double` values are almost equal |
<!-- mdformat on-->
By "almost equal" we mean the values are within 4 ULP's from each other.
The following assertions allow you to choose the acceptable error bound:
<!-- mdformat off(github rendering does not support multiline tables) -->
| Fatal assertion | Nonfatal assertion | Verifies |
| ------------------------------------- | ------------------------------------- | -------------------------------------------------------------------------------- |
| `ASSERT_NEAR(val1, val2, abs_error);` | `EXPECT_NEAR(val1, val2, abs_error);` | the difference between `val1` and `val2` doesn't exceed the given absolute error |
<!-- mdformat on-->
#### Floating-Point Predicate-Format Functions
Some floating-point operations are useful, but not that often used. In order to
avoid an explosion of new macros, we provide them as predicate-format functions
that can be used in predicate assertion macros (e.g. `EXPECT_PRED_FORMAT2`,
etc).
```c++
EXPECT_PRED_FORMAT2(::testing::FloatLE, val1, val2);
EXPECT_PRED_FORMAT2(::testing::DoubleLE, val1, val2);
```
Verifies that `val1` is less than, or almost equal to, `val2`. You can replace
`EXPECT_PRED_FORMAT2` in the above table with `ASSERT_PRED_FORMAT2`.
### Asserting Using gMock Matchers
[gMock](../../googlemock) comes with a library of matchers for validating
arguments passed to mock objects. A gMock *matcher* is basically a predicate
that knows how to describe itself. It can be used in these assertion macros:
<!-- mdformat off(github rendering does not support multiline tables) -->
| Fatal assertion | Nonfatal assertion | Verifies |
| ------------------------------ | ------------------------------ | --------------------- |
| `ASSERT_THAT(value, matcher);` | `EXPECT_THAT(value, matcher);` | value matches matcher |
<!-- mdformat on-->
For example, `StartsWith(prefix)` is a matcher that matches a string starting
with `prefix`, and you can write:
```c++
using ::testing::StartsWith;
...
// Verifies that Foo() returns a string starting with "Hello".
EXPECT_THAT(Foo(), StartsWith("Hello"));
```
Read this
[recipe](../../googlemock/docs/cook_book.md#using-matchers-in-googletest-assertions)
in the gMock Cookbook for more details.
gMock has a rich set of matchers. You can do many things googletest cannot do
alone with them. For a list of matchers gMock provides, read
[this](../../googlemock/docs/cook_book.md##using-matchers). It's easy to write
your [own matchers](../../googlemock/docs/cook_book.md#NewMatchers) too.
gMock is bundled with googletest, so you don't need to add any build dependency
in order to take advantage of this. Just include `"gmock/gmock.h"`
and you're ready to go.
### More String Assertions
(Please read the [previous](#asserting-using-gmock-matchers) section first if
you haven't.)
You can use the gMock
[string matchers](../../googlemock/docs/cheat_sheet.md#string-matchers) with
`EXPECT_THAT()` or `ASSERT_THAT()` to do more string comparison tricks
(sub-string, prefix, suffix, regular expression, and etc). For example,
```c++
using ::testing::HasSubstr;
using ::testing::MatchesRegex;
...
ASSERT_THAT(foo_string, HasSubstr("needle"));
EXPECT_THAT(bar_string, MatchesRegex("\\w*\\d+"));
```
If the string contains a well-formed HTML or XML document, you can check whether
its DOM tree matches an
[XPath expression](http://www.w3.org/TR/xpath/#contents):
```c++
// Currently still in //template/prototemplate/testing:xpath_matcher
#include "template/prototemplate/testing/xpath_matcher.h"
using prototemplate::testing::MatchesXPath;
EXPECT_THAT(html_string, MatchesXPath("//a[text()='click here']"));
```
### Windows HRESULT assertions
These assertions test for `HRESULT` success or failure.
Fatal assertion | Nonfatal assertion | Verifies
-------------------------------------- | -------------------------------------- | --------
`ASSERT_HRESULT_SUCCEEDED(expression)` | `EXPECT_HRESULT_SUCCEEDED(expression)` | `expression` is a success `HRESULT`
`ASSERT_HRESULT_FAILED(expression)` | `EXPECT_HRESULT_FAILED(expression)` | `expression` is a failure `HRESULT`
The generated output contains the human-readable error message associated with
the `HRESULT` code returned by `expression`.
You might use them like this:
```c++
CComPtr<IShellDispatch2> shell;
ASSERT_HRESULT_SUCCEEDED(shell.CoCreateInstance(L"Shell.Application"));
CComVariant empty;
ASSERT_HRESULT_SUCCEEDED(shell->ShellExecute(CComBSTR(url), empty, empty, empty, empty));
```
### Type Assertions
You can call the function
```c++
::testing::StaticAssertTypeEq<T1, T2>();
```
to assert that types `T1` and `T2` are the same. The function does nothing if
the assertion is satisfied. If the types are different, the function call will
fail to compile, the compiler error message will say that
`T1 and T2 are not the same type` and most likely (depending on the compiler)
show you the actual values of `T1` and `T2`. This is mainly useful inside
template code.
**Caveat**: When used inside a member function of a class template or a function
template, `StaticAssertTypeEq<T1, T2>()` is effective only if the function is
instantiated. For example, given:
```c++
template <typename T> class Foo {
public:
void Bar() { ::testing::StaticAssertTypeEq<int, T>(); }
};
```
the code:
```c++
void Test1() { Foo<bool> foo; }
```
will not generate a compiler error, as `Foo<bool>::Bar()` is never actually
instantiated. Instead, you need:
```c++
void Test2() { Foo<bool> foo; foo.Bar(); }
```
to cause a compiler error.
### Assertion Placement
You can use assertions in any C++ function. In particular, it doesn't have to be
a method of the test fixture class. The one constraint is that assertions that
generate a fatal failure (`FAIL*` and `ASSERT_*`) can only be used in
void-returning functions. This is a consequence of Google's not using
exceptions. By placing it in a non-void function you'll get a confusing compile
error like `"error: void value not ignored as it ought to be"` or `"cannot
initialize return object of type 'bool' with an rvalue of type 'void'"` or
`"error: no viable conversion from 'void' to 'string'"`.
If you need to use fatal assertions in a function that returns non-void, one
option is to make the function return the value in an out parameter instead. For
example, you can rewrite `T2 Foo(T1 x)` to `void Foo(T1 x, T2* result)`. You
need to make sure that `*result` contains some sensible value even when the
function returns prematurely. As the function now returns `void`, you can use
any assertion inside of it.
If changing the function's type is not an option, you should just use assertions
that generate non-fatal failures, such as `ADD_FAILURE*` and `EXPECT_*`.
NOTE: Constructors and destructors are not considered void-returning functions,
according to the C++ language specification, and so you may not use fatal
assertions in them; you'll get a compilation error if you try. Instead, either
call `abort` and crash the entire test executable, or put the fatal assertion in
a `SetUp`/`TearDown` function; see
[constructor/destructor vs. `SetUp`/`TearDown`](faq.md#CtorVsSetUp)
WARNING: A fatal assertion in a helper function (private void-returning method)
called from a constructor or destructor does not does not terminate the current
test, as your intuition might suggest: it merely returns from the constructor or
destructor early, possibly leaving your object in a partially-constructed or
partially-destructed state! You almost certainly want to `abort` or use
`SetUp`/`TearDown` instead.
## Teaching googletest How to Print Your Values
When a test assertion such as `EXPECT_EQ` fails, googletest prints the argument
values to help you debug. It does this using a user-extensible value printer.
This printer knows how to print built-in C++ types, native arrays, STL
containers, and any type that supports the `<<` operator. For other types, it
prints the raw bytes in the value and hopes that you the user can figure it out.
As mentioned earlier, the printer is *extensible*. That means you can teach it
to do a better job at printing your particular type than to dump the bytes. To
do that, define `<<` for your type:
```c++
#include <ostream>
namespace foo {
class Bar { // We want googletest to be able to print instances of this.
...
// Create a free inline friend function.
friend std::ostream& operator<<(std::ostream& os, const Bar& bar) {
return os << bar.DebugString(); // whatever needed to print bar to os
}
};
// If you can't declare the function in the class it's important that the
// << operator is defined in the SAME namespace that defines Bar. C++'s look-up
// rules rely on that.
std::ostream& operator<<(std::ostream& os, const Bar& bar) {
return os << bar.DebugString(); // whatever needed to print bar to os
}
} // namespace foo
```
Sometimes, this might not be an option: your team may consider it bad style to
have a `<<` operator for `Bar`, or `Bar` may already have a `<<` operator that
doesn't do what you want (and you cannot change it). If so, you can instead