c++ – Generic test case templates

I often find that when I am writing, refactoring, or reviewing code that I want to do some simple testing. There are many existing test frameworks such as gtest and cppunit but my desire was to create something much simpler and with fewer features.

Specifically, I have two use cases in mind:

  1. I have some test input values with known desired outputs
  2. I have an existing function that I want to optimize but keep correct

For both of these, I’ve create a very simple pair of templated objects after studying my own use of such techniques in existing code. In looking at how I use such code, I find that I have three general uses:

  1. I want pretty color printing to the screen with good values in green and bad ones in red
  2. I want to redirect the output to a file without the pretty colors
  3. I want to run all the tests and just silently get a bool that indicates whether all tests passed

Here are the templates I’d like to get reviewed.

testcase.h

#ifndef EDWARD_TEST_CASE
#define EDWARD_TEST_CASE

#include <string>
#include <string_view>
#include <vector>
#include <iomanip>
#include <iostream>

namespace edward {

static const std::string CSI{"x1b("};
static const std::string RED{CSI + "31m"};
static const std::string GREEN{CSI + "32m"};
static const std::string RESET{CSI + "0m"};

static const std::string badgood(2)(2){ 
    { "(BAD) ", "(OK) " },
    { RED + "(BAD) " + RESET, GREEN + "(OK)  "+ RESET },
}; 

template <class InputType, class OutputType>
class TestCollection {
    struct TestCase {
        InputType input;
        OutputType expected;
    };
public:
    TestCollection(OutputType (*testfunc)(InputType), std::vector<TestCase> tests) 
        : testfunc{testfunc}
        , tests{tests}
    {}
    bool testAll(bool verbose = true, bool color = true) const {
        return verbose ? verboseTest(color) : quietTest();
    }
private:
    bool quietTest() const {
        bool good{true};
        for (const auto& t : tests) {
            good &= (testfunc(t.input) == t.expected);
        }
        return good;
    }
    bool verboseTest(bool color = true) const {
        bool good{true};
        std::size_t i{0};
        for (const auto& t : tests) {
            auto result = testfunc(t.input);
            bool isOK = result == t.expected;
            good &= isOK;
            std::cout << badgood(color)(isOK)
                << "Test #" << i << ": "
                << std::boolalpha << "got "" << result 
                << "", expected "" << t.expected 
                << "" from "" << t.input << ""n";
        }
        return good;
    }
    OutputType (*testfunc)(InputType);
    std::vector<TestCase> tests;
};

template <class InputType, class OutputType>
class DynamicTest {
public:
    DynamicTest(OutputType (*testfunc)(InputType), OutputType (*trustedfunc)(InputType))
        : testfunc{testfunc}
        , trustedfunc{trustedfunc}
    {}
    bool test(InputType in, bool verbose = true, bool color = true) const {
        return verbose ? verboseTest(in, color) : quietTest(in);
    }
private:
    bool quietTest(InputType in) const {
        return testfunc(in) == trustedfunc(in);
    }
    bool verboseTest(InputType in, bool color = true) const {
        OutputType result{testfunc(in)};
        OutputType expected{trustedfunc(in)};
        bool isOK{result == expected};
        std::cout << badgood(color)(isOK)
            << std::boolalpha << "got "" << result 
            << "", expected "" << expected 
            << "" from "" << in << ""n";
        return isOK;
    }

    OutputType (*testfunc)(InputType);
    OutputType (*trustedfunc)(InputType);
};

}; // end of namespace EDWARD

#endif // EDWARD_TEST_CASE

Here is some sample test code that exercises both templates and illustrates the intended use. Note that these use this idiom to decide whether or not to print color:

bool color{static_cast<bool>(isatty(STDOUT_FILENO))};

This works on Linux (or any POSIX compliant) machines but is not, to my knowledge, portable to Windows. Rather than make the template non-portable, I simply omit it from the templates and rely on the caller. Also note that the printing is only done to std::cout in the template. Here again, I found that it’s the only way I used existing code, so I didn’t incorporate features I’ve never used.

main.cpp

#include "testcase.h"
#include <iostream>
#include <string>
#include <algorithm>
#include <cmath>
#include <functional>
#include <unistd.h>

bool isInteger(const std::string& n) {
    if (n.size() > 1) {
        return std::all_of(n.begin() + 1, n.end(), isdigit)
            && (isdigit(n(0)) || (n(0) == '-') || (n(0) == '+'));
    }
    return isdigit(n(0));
}

void one() {
    bool verbose{true};
    bool color{static_cast<bool>(isatty(STDOUT_FILENO))};

    static const edward::TestCollection<const std::string&, bool> tc{ isInteger, {
        {"+", false},
        {"-", false},
        {"0", true},
        {"3", true},
        {"9", true},
        {"a", false},
        {"99a9", false},
        {"9909", true},
        {"", false},
        {"this test lies", true},  // deliberately bad test
        {"-3.14", false},
        {"+32768", true},
        {"-32768", true},
    }};
    auto result{tc.testAll(verbose, color)};
    std::cout << "All tests " << (result ? "passed" : "did NOT pass") << "n";
}

int square_it(int x) {
    return std::pow(x, 2);
}

void two() {
    bool verbose{true};
    bool color{static_cast<bool>(isatty(STDOUT_FILENO))};

    edward::DynamicTest<int, int> dt{()(int x)->int{return x*x;}, square_it };
    bool result{true};
    for (int i{-5}; i < 5; ++i) {
        result &= dt.test(i, verbose, color);
    }
    std::cout << "All tests " << (result ? "passed" : "did NOT pass") << "n";
}

int main() {
    one();
    two();
}

I’m interested in a general review.