Assert That Code Does Not Compile

Assert that code does NOT compile

A good answer is given at the end of a great article "Diagnosable validity" by Andrzej Krzemieński:

A practical way to check if a given construct fails to compile is to do it from outside C++: prepare a small test program with erroneous construct, compile it, and test if compiler reports compilation failure. This is how “negative” unit tests work with Boost.Build. For an example, see this negative test form Boost.Optional library: optional_test_fail_convert_from_null.cpp. In configuration file it is annotated as compile-fail, meaning that test passes only if compilation fails.

SFINAE to assert() that code DOES NOT compile

The following macro lets you rewrite a SFINAE-unfriendly expression such as [](auto&&x) { return x+1; } in a SFINAE-friendly way.

#define RETURNS(...)\
  noexcept(noexcept(__VA_ARGS__))\
  ->decltype(__VA_ARGS__)\
  { return __VA_ARGS__;}

So that lets you rewrite the above lambda expression like this:

[](auto&&x) RETURNS( x+1 )

or, another example:

struct { template<class X> auto operator()(X&&x) RETURNS(x+1) };

and it is SFINAE friendly. RETURNS isn't actually required, but it makes much of the code so much cleaner. There is a c++20 proposal to replace RETURNS with => by SO's own @barry.

Next we need to be able to test if a function object can be called.

namespace details {
  template<class, class, class...>
  struct can_invoke:std::false_type{};
  template<class F, class...Args>
  struct can_invoke<
    F,
    std::void_t<std::result_of_t<F&&(Args&&...)>>,
    Args...
  >:
    std::true_type
  {};
}
template<class F, class...Args>
using can_invoke=details::can_invoke<F,void,Args...>;

we are almost there. (This is the core of the technique; I sometimes use can_apply that takes template<class...>class Z instead of class F here.) c++17 has a similar trait; it can be used instead.

test_invoke takes callable and returns a callable tester. A callable tester takes arguments, and returns true or false types based on "could the original callable be called with these arguments".

template<class F>
constexpr auto test_invoke(F&&){
  return [](auto&&...args) RETURNS( can_invoke< F, decltype(args)... >{} );
}

and here we are. test_invoke can be skipped if you are willing to work with pure types, but working with values can eliminate some bugs.

auto myclass_ctor=[](auto&&...args)RETURNS(myclass_t(decltype(args)(args)...));

myclass_ctor is a callable object that represents constructing myclass_t.

static_assert(!test_invoke(myclass_ctor)("Hello") );

template<class C>
auto ctor=[](auto&&...args)RETURNS(C(decltype(args)(args)...));
static_assert(!test_invoke(ctor<myclass_t>)("Hello") );

this requires constexpr lambda, a c++17 feature but an early one. It can be done without it but it gets ugly. Plus move ctor requirement of elision is annoying to work around in c++14.

To translate to c++14, replace every lambda with a manual function object with appropriate constexpr special member functions. RETURNS applies to operator() just as well, as demonstrated above.

To get around elision move ctor requrement, RETURNS(void( blah )).

Apologies for any tyops; I am on phone.

How to assert that an expression does not compile

After some time, I found following piece of software:

http://docs.codehaus.org/display/JANINO/Home

It looks like it should be possible to do some sort of hack in order compile only the block in question. Right now, my example would not work, anyway, because it yet does not support generics.

Testing an assertion that something must not compile

Not a framework, but Jorge Ortiz (@JorgeO) mentioned some utilities he added to the tests for Foursquare's Rogue library at NEScala in 2012 which support tests for non-compilation: you can find examples here. I've been meaning to add something like this to shapeless for quite a while.

More recently, Roland Kuhn (@rolandkuhn) has added a similar mechanism, this time using Scala 2.10's runtime compilation, to the tests for Akka typed channels.

These are both dynamic tests of course: they fail at (test) runtime if something that shouldn't compile does. Untyped macros might provide a static option: ie. a macro could accept an untyped tree, type check it and throw a type error if it succeeds). This might be something to experiment with on the macro-paradise branch of shapeless. But not a solution for 2.10.0 or earlier, obviously.

Update

Since answering the question, another approach, due to Stefan Zeiger (@StefanZeiger), has surfaced. This one is interesting because, like the untyped macro one alluded to above, it is a compile time rather than (test) runtime check, however it is also compatible with Scala 2.10.x. As such I think it is preferable to Roland's approach.

I've now added implementations to shapeless for 2.9.x using Jorge's approach, for 2.10.x using Stefan's approach and for macro paradise using the untyped macro approach. Examples of the corresponding tests can be found here for 2.9.x, here for 2.10.x and here for macro paradise.

The untyped macro tests are the cleanest, but Stefan's 2.10.x compatible approach is a close second.

Is it possible to ASSERT_DOES_NOT_COMPILE with GTest?

There is a way, but sadly it's probably not the way you want.

My first thought was to try to get SFINAE to discount an overload by expanding an invalid lambda in an unevaluated context. Unhappily (in your case), this is specifically disallowed...

#define CODE { \
 utter garbage \
}
struct test
{
    template<class T>
    static std::false_type try_compile(...) { return{}; }
    template<class T>
    static auto try_compile(int)
    -> decltype([]() CODE, void(), std::true_type());
    { return {}; }
};
struct tag {};
using does_compile = decltype(test::try_compile<tag>(0));

output:

./maybe_compile.cpp:88:17: error: lambda expression in an unevaluated operand
    -> decltype([]() CODE, void(), std::true_type());

So it was back to the drawing board and a good old system call to call out to the compiler...

#include <iostream>
#include <string>
#include <cstdlib>
#include <fstream>
#include <sstream>

struct temp_file {
    temp_file()
    : filename(std::tmpnam(nullptr))
    {}

    ~temp_file() {
        std::remove(filename.c_str());
    }

    std::string filename;
};

bool compiles(const std::string code, std::ostream& reasons)
{
    using namespace std::string_literals;

    temp_file capture_file;
    temp_file cpp_file;

    std::ofstream of(cpp_file.filename);
    std::copy(std::begin(code), std::end(code), std::ostream_iterator<char>(of));
    of.flush();
    of.close();
    const auto cmd_line = "c++ -x c++ -o /dev/null "s + cpp_file.filename + " 2> " + capture_file.filename;
    auto val = system(cmd_line.c_str());

    std::ifstream ifs(capture_file.filename);
    reasons << ifs.rdbuf();
    ifs.close();

    return val == 0;
}

auto main() -> int
{
    std::stringstream reasons1;
    const auto code1 =
R"code(
    #include <iostream>
    int main() {
        return 0;
    }
)code";
    std::cout << "compiles: " << compiles(code1, reasons1) << std::endl;

    std::stringstream reasons2;
    const auto code2 =
R"code(
    #include <iostream>
    int main() {
        FOO!!!!XC@£$%^&*()VBNMYGHH
        return 0;
    }
)code";
    std::cout << "compiles: " << compiles(code2, reasons2) << std::endl;
    std::cout << "\nAnd here's why...\n";
    std::cout << reasons2.str() << std::endl;

    return 0;
}

which in my case gives the following example output:

compiles: 1
compiles: 0

And here's why...
/var/tmp/tmp.3.2dADZ7:4:9: error: use of undeclared identifier 'FOO'
        FOO!!!!XC@£$%^&*()VBNMYGHH
        ^
/var/tmp/tmp.3.2dADZ7:4:19: error: non-ASCII characters are not allowed outside of literals and identifiers
        FOO!!!!XC@£$%^&*()VBNMYGHH
                  ^
2 errors generated.

of course you can add all the necessary macros around the call to compiles() in order to GTESTify it. You will of course have to set command line options on the c-compiler invocation to set the correct paths and defines.

How do I write a test assuring a compilation error?

There isn't currently a way of indicating that a regular test should not compile. And by the look of related issues (#521 and #1994 ) something like #[compile_fail] is unlikely to become available any time soon.

However, there are two other ways to write these tests.

Doctests

Since Rust 1.22, you can make doc tests which are supposed to fail to compile, by labeling the code snippet with compile_fail:

/// Foos a bar.
/// 
/// # Example
///
/// ```compile_fail
/// foo(3); // 3 is not a bar
/// ```
fn foo(bar: Bar) {
}

Compile-test toolkit

The compile-test tools used internally by the Rust project were extracted into the dedicated crate compiletest_rs.

With the boilerplate suggested in the documentation, one can write compile-fail tests in the tests/compile-fail folder:

fn main() {
   let x: bool = 0;
}

static_assert causing program to not compile even though the assert is in the header of a function template

it seems as if my substitution failure is not an error is not correctly working.

The rule is "Substitution Failure Is Not An Error". However, this only applies to the process of finding a type.

What's happening here is that finding a matching type actually succeeds, but it lands on a type that happens to have a failing static_assert(), which causes a compilation error.

There never was any substitution failure to consider as "not an error".

For your ibounded type to be used in SFINAE, it needs to not exist at all when the condition doesn't pass.

Since you are trying to wrap your head around this, I think it's worth going over what it would take for ibounded to work. However, there's a way easier way to do this in practice, see below for details.

Manual SFINAE: (for your curiosity)

// First declare ibounded without defining it
template<int lower, 
         int upper, 
         int val, 
         typename rtype, 
         typename pass=std::true_type>  // <---- magic sauce here
struct ibounded;

// Add a partial specialization for which the pass parameter will be set to true or false depending on the condition.
template<int lower, int upper, int val, typename rtype>
struct ibounded<lower, 
                upper, 
                val, 
                rtype, 
                std::integral_constant<bool, (lower < val && upper > val)>> {
  using TYPE = rtype; 
};

The key point here is having a default value for the pass parameter in the original declaration. Because of it, any attempt to use the ibounded<a, b, c, T> type is actually trying to use ibounded<a, b, c, T, std::true_type>.

Now, onto the partial specialization. It defines the following types:

ibounded<a, b, c, void, std::true_type>, but only when a < c < b
ibounded<a, b, c, void, std::false_type>, but only when a >= c >= b

There doesn't exist a definition of ibounded<a, b, c, T, std::true_type> for which c is not between a and b. All that's left to do is to make sure that you never manually set the value of the pass parameter, and ibounded is good to go as a SFINAE test.

Easier way:
std::enable_if_t<bool, T> is a type that exists as T if the bool is true, and does not exist otherwise. So you can just use:

template<int lower, int upper, int val, typename rtype>
using ibounded = std::enable_if_t<(lower < val && upper > val), rtype>;

It becoming rtype makes using it a lot nicer as well, since you don't have to lookup the type:

template<int filenum_t, int pos> inline auto _init_filename() -> ibounded<0, 9, filenum_t, void> {

Assert That Code Does Not Compile