What Is Property Based Testing

I originally looked into property based testing a while ago. I knew that it was something I wanted to come back to and dig deeper. I got my chance and recently gave a talk on it at Prairie Dev Con Deliver. This conference is focused on Agile methodologies and testing.

Property Based Testing

Property based testing differs from unit testing in that it doesn’t use specific inputs and results. It instead uses a property to determine correct behaviour. I looked up the definition of Property in the dictionary and came up with “a trait belonging to a process with results common to all members of a set of inputs.” This is still a little abstract, but I found adding an example helped me understand further.

First Property

Any positive number multiplied by negative one will have a result less than zero. In this example:

  • The process is multiplication with negative one: (*) -1
  • The input set is any positive number: choose {1 .. }
  • The result is a number less than zero: < 0

These three pieces make up the property we are asserting.

Generative Advantage

One of the big advantages to property based testing over normal testing is the generative aspect of it. Property based testing doesn’t rely on specific example based testing. The system generates values based on the specifications required. Since the system is generating values at each run, the tests are always actively looking for defects. This also lets the system come up with many variations of inputs that we humans may not think of. This means we may detect a defect that can occur when given a large set of data.

A nice feature to some testing libraries, such as .Net’s FsCheck, is shrinking. When a falsifiable input is found, Shrinking allows the system to start minimizing (shrinking) the input values required to generate the defect. The idea is to make the inputs be as manageable as possible for the developer to be able to reproduce the error with minimal effort. The ideal use case would be to take that falsifiable example and put it in a unit test.

Like unit tests, we come up with many different properties of the process we are testing. The difficulty with property based testing is coming up with properties can be time consuming and difficult. On the other hand, the time consuming and thought does allow us to gain greater knowledge of the process we are testing. Thankfully though, there are some common patterns that have emerged as good ways to get started. We will look at the There and Back property in the next post.

Summary

Property based testing is a unique way to ensure your code is as defect free as possible. We’ve defined three requirements for a property. The process is what we are testing. The input set or subset required to test the process. In the example above we used any positive number. The results with some defined commonality of evaluating the process on our input set.