Lambdas, or functions as values, have worked their way into mainstream programming and transformed development for many programmers.
It’s rare to see a program that doesn’t
.filter a list with a predicate or
.map to transform from one type to another. This same
as-a-value approach can be applied to automated tests, and I have hopes it will also transform how we program.
Automated tests (e.g. unit tests) have rightfully established themselves as a core development practice. Tests improve the way we design and enable continuous verification of expected behaviors.
The common approach to unit test uses attributes to specially mark language constructs, like classes, as tests. Some external tool then examines the code for these special attributes and runs the tests.
This attribute-based approach works well with the right tooling. However, it caused me to develop some sneaky and false assumptions
- Tests are a special and separate kind of code
- Running tests is magic reserved for special external tooling
- Tests are closed constructs that cannot be effectively reused, composed, or programatically modified
- Test process customization requires either serious meta-programming or the good graces of the framework to provide customization hooks
Tests as Values
All the above assumptions are not essential to tests. Tests can be values in much the same way we now use functions or objects as values. This allows test to be treated like any other value in our code. We can bring all of our good practices from other coding tasks into testing too.
Let’s look at some examples of how test-values are useful.
Tests are just values in Expecto. This means we can put them in a list like we would with any other kind of value
We can, of course, combine lists of tests together if we like.
We can run the list of tests
We can filter tests if we so desire.
Test setup and teardown can be done just by mapping the tests with a new enclosing function. Tests names can also be dynamically prefixed or reformatted using a simple list map.
I’ve named just a few possibilities, and all of this is accomplished using the normal list operations and functions we program with otherwise. There’s no need for advanced meta programming to customize the testing process or operate on a set of tests. Of course, Expecto also provides some semi-magic options for integrating with other test frameworks like XUnit and their test runners.
Looking Forward: Composition
Tests-as-values enable more than testing customizations. It also means that our tests can be created by factories, passed around, and composed into new test lists.
I currently use a kind of test factory to reuse a single test list across different implementations of an interface.
I have hopes that this kind of reuse and composition can raise our level of abstraction in testing. For example, I commonly write very similar and fairly complex tests for fetching some object based on an intersection of tags. Such tests get even more complicated if there are both include and exclude rules.
This kind of test seems ripe for reuse. Behavior varies very little between uses, yet the behaviors are complex with sneaky edge cases. I’d love to compose a test suite like
I’m still uncertain how many tests scenarios are worth the indirection of separating out a standard suite then mapping into that suite’s abstractions. It’s a whole new mindset to approach tests with. So far, it seems like problems mappable to general set theory problems are prime candidates (i.e. search, filter, sort).
I find it wonderful that all the semi-mysterious actions of a testing framework become clear and accessible with tests-as-values. Easy access to the testing pipeline empowers developers to make powerful customizations for their own usage, like my test api library for F#. Looking forward, I hope that test values will improve isolation of cross-cutting expectations into reusable test suites.
All-in-all, I think tests-as-values are a clear win. I certainly notice any project missing them.