UI Mutation and Generative Testing Update

I previously wrote on statistical and mutation testing for UIs. After additional thought, I don’t think UI mutation testing is feasible, at least in the traditional sense. Structure-based generative testing still could be. Lack of Deterministic Errors for Mutation Mutation testing is a tool to measure completeness. It tells us how well our other tests detect possible errors in our code. Mutation testing frameworks measure test suite completeness by crawling the code and injecting breaking changes.

March 27, 2022 · 3 min

Test Case Selection Guide

Testing is a broad and essential topic for Software Developers. There is a dizzying diversity of testing approaches for different scenarios. However, the typical developer’s inner loop focuses on a few kinds of tests and test selection. That is, choosing which tests need to be written.

January 28, 2022 · 4 min

Gherkin-named Tests as Low-bar Acceptance Tests

I previously compared the tradeoffs of Gherkin-based and TestApi-based behavior tests. Recently, I realized there is a simple in-between solution: TestApi with Gherkin-style names.

December 12, 2021 · 2 min

What Tests Measure

My test types diagram sparked concern that mature process focuses on repeated measured improvement, not specific techniques. This is right, but I don’t see the conflict. Tests are a kind of measure, and the diagram identifies common tests (measures) certain actors leverage to meet larger goals. This raises the question, what do the different kinds of tests measure? Consequently, what do they tell us about our system?

December 5, 2021 · 6 min

Governance is a mine of Quality techniques

I recently put together that multiple verification (the quality technique) is effectively the same and as governance techniques. It’s about managing the level of trust and risk we accept from individual contributors.

November 26, 2021 · 3 min

BDD Pushes the right extensibility

I recently read Mark Seemann’s post on how Testability is the Open-Closed Principle. This helped me realize the approach to testing directs the kinds of flexibility in our system, and behavior-based testing pushes the right kind of flexibility.

November 12, 2021 · 1 min

Test Api in F# Improved

I previously wrote on the Test API behavioral testing in F#. More time and thinking has brought about a few key improvements, making the pattern both more concise and standardized.

October 8, 2021 · 5 min

Test Types and Lifecycle Phases

I was thinking about responsibility for different kinds of quality in an organization and I noticed that different kinds of testing line up well with particular roles and software lifecycle phases. I’ve summed it all up in a quick visual.

August 30, 2021 · 5 min

Hypothesis: Statistical and Mutation Testing are Achievable for Visual UI Testing

I have a wide range of techniques for testing code behavior. I can reliably increase quality over time no matter how large the feature set grows and even measure the completeness of my test suite (mutation testing). The same can’t be said for UIs. I have long struggled to automate UI tests at all, and manual tests don’t sustain as the system grows. However, I think I may now have a technique to close the gap.

July 25, 2021 · 4 min

Pros and cons of REPL-centered development

I’ve been working on Clojure lately, and the Clojure community tends to center their development around the REPL. While often useful, this approach also comes with some unforeseen downsides.

June 19, 2021 · 2 min