Much ado about testing

# Much ado about testing

Nicolas Rinaudo • [@NicolasRinaudo] • [Besedo]

---

# Software testing

---

## Testing

---

## Testing

---

## Testing

---

## Testing

---

## Testing

---

## System Under Test

> Write a program that, given a number, prints it.
>
> But for multiples of three print `Fizz` instead of the number and for the multiples of  five print `Buzz`.

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i))      "Fizz"
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## System Under Test

> Write a program that, .highlight[given a number], prints it.
>
> But for multiples of three print `Fizz` instead of the number and for the multiples of five print `Buzz`.

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(`i: Int`): String = {
  if(mult3(i))      "Fizz"
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## System Under Test

> Write a program that, given a number, .highlight[prints it].
>
> But for multiples of three print `Fizz` instead of the number and for the multiples of five print `Buzz`.

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i))      "Fizz"
  else if(mult5(i)) "Buzz"
  else             `i.toString`
}
```

---

## System Under Test

> Write a program that, given a number, prints it.
>
> But for .highlight[multiples of three] print `Fizz` instead of the number and for the multiples of five print `Buzz`.

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(`mult3(i)`)      "Fizz"
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## System Under Test

> Write a program that, given a number, prints it.
>
> But for multiples of three print .highlight[`Fizz`] instead of the number and for the multiples of five print `Buzz`.

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i))      `"Fizz"`
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## System Under Test

> Write a program that, given a number, prints it.
>
> But for multiples of three print `Fizz` instead of the number and for the .highlight[multiples of five] print `Buzz`.

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i))      "Fizz"
  else if(`mult5(i)`) "Buzz"
  else             i.toString
}
```

---

## System Under Test

> Write a program that, given a number, prints it.
>
> But for multiples of three print `Fizz` instead of the number and for the multiples of five print .highlight[`Buzz`].

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i))      "Fizz"
  else if(mult5(i)) `"Buzz"`
  else             i.toString
}
```

---

# Example-based testing

---

## Overview

---

## Overview

---

## Overview

---

## Overview

---

## Overview

---

## Overview

---

## Overview

```scala
class ListTests extends AnyFunSuite {
  test("List(3, 2, 1) should sort to List(1, 2, 3)") {
    assert(List(3, 2, 1).sorted == List(1, 2, 3))
  }
}
```

---

## Overview

```scala
class ListTests extends AnyFunSuite {
  test("List(3, 2, 1) should sort to List(1, 2, 3)") {
    assert(`List(3, 2, 1)`.sorted == List(1, 2, 3))
  }
}
```

---

## Overview

```scala
class ListTests extends AnyFunSuite {
  test("List(3, 2, 1) should sort to List(1, 2, 3)") {
    assert(List(3, 2, 1)`.sorted` == List(1, 2, 3))
  }
}
```

---

## Overview

```scala
class ListTests extends AnyFunSuite {
  test("List(3, 2, 1) should sort to List(1, 2, 3)") {
    assert(List(3, 2, 1).sorted == `List(1, 2, 3)`)
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test(`"n multiple of 3 outputs Fizz"`) {
    assert(fizzBuzz(3) == "Fizz")
  }

test(`"n multiple of 5 outputs Buzz"`) {
    assert(fizzBuzz(5) == "Buzz")
  }

test(`"n multiple of neither 3 or 5 outputs n"`) {
    assert(fizzBuzz(7) == "7")
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(`3`) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(`5`) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(`7`) == "7")
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(`fizzBuzz`(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(`fizzBuzz`(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(`fizzBuzz`(7) == "7")
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == `"Fizz"`)
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == `"Buzz"`)
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == `"7"`)
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test(`"n multiple of 3 outputs Fizz"`) {
    assert(`fizzBuzz(3) == "Fizz"`)
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test(`"n multiple of 5 outputs Buzz"`) {
    assert(`fizzBuzz(5) == "Buzz"`)
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }
}
```

---

## FizzBuzz test suite

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test(`"n multiple of neither 3 or 5 outputs n"`) {
    assert(`fizzBuzz(7) == "7"`)
  }
}
```

---

## FizzBuzz test suite

```scala
new FizzBuzzSuite().execute()
```

---

## FizzBuzz test suite

```scala
new FizzBuzzSuite().execute()
// FizzBuzzSuite:
// - n multiple of 3 outputs Fizz
// - n multiple of 5 outputs Buzz
// - n multiple of neither 3 or 5 outputs n
```

---

## Client Demo

```scala
fizzBuzz(15)
```

---

## Client Demo

```scala
fizzBuzz(15)
// res1: String = Fizz
```

---

## Clarifying specifications

n % 3 | n % 5 || Output
-------|-------||----------
 T     | T     || FizzBuzz
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Clarifying specifications

.highlight[n % 3] | n % 5 || Output
-------|-------||----------
 T     | T     || FizzBuzz
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Clarifying specifications

n % 3 | .highlight[n % 5] || Output
-------|-------||----------
 T     | T     || FizzBuzz
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Clarifying specifications

n % 3 | n % 5 || .highlight[Output]
-------|-------||----------
 T     | T     || FizzBuzz
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Clarifying specifications

n % 3 | n % 5 || Output
-------|-------||----------
 .highlight[T]     | T     || FizzBuzz
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Clarifying specifications

n % 3 | n % 5 || Output
-------|-------||----------
 T     | .highlight[T]     || FizzBuzz
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Clarifying specifications

n % 3 | n % 5 || Output
-------|-------||----------
 T     | T     || .highlight[FizzBuzz]
 T     | F     || Fizz
 F     | T     || Buzz
 F     | F     || n

---

## Fixing FizzBuzz

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i))      "Fizz"
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## Fixing FizzBuzz

.diff-rm[
```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
* if(mult3(i))`      "Fizz"`
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```
]

---

## Fixing FizzBuzz

.diff-add[
```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
* if(mult3(i)) `{`
* `  if(mult5(i)) "FizzBuzz"`
* `  else         "Fizz"`
* `}`
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```
]

---

## Fixing FizzBuzz

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(`mult3(i)`) {
    if(`mult5(i)`) `"FizzBuzz"`
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## Fixing FizzBuzz

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(`mult3(i)`) {
    if(mult5(i)) "FizzBuzz"
    else         `"Fizz"`
  }
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## Fixing FizzBuzz

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else             i.toString
}
```

---

## Fixing tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }
}
```

---

## Fixing tests

.diff-add[
```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }
*
* `test("n multiple of 3 and 5 outputs FizzBuzz") {`
* `  assert(fizzBuzz(15) == "FizzBuzz")`
* `}`
}
```
]

---

## Fixing tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(`15`) == "FizzBuzz")
  }
}
```

---

## Fixing tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(`fizzBuzz`(15) == "FizzBuzz")
  }
}
```

---

## Fixing tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == `"FizzBuzz"`)
  }
}
```

---

## Fixing tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Fixing tests

```scala
new FizzBuzzSuite().execute()
```

---

## Fixing tests

```scala
new FizzBuzzSuite().execute()
// FizzBuzzSuite:
// - n multiple of 3 outputs Fizz
// - n multiple of 5 outputs Buzz
// - n multiple of neither 3 or 5 outputs n
// - n multiple of 3 and 5 outputs FizzBuzz
```

---

## Client Demo

```scala
fizzBuzz(-3)
```

---

## Client Demo

```scala
fizzBuzz(-3)
// res3: String = FizzBuzz
```

---

## Key takeaways

Example-based testing is:

* very good at the _oracle problem_.

* very bad at the _test case generation problem_.

* a little bit discouraging...

---

# Generative Testing

---

## Overview

---

## Overview

---

## Overview

---

## Overview

---

## Overview

---

## Overview

---

## Property-Based Testing

---

## Property-Based Testing

```scala
forAll { (is: List[Int]) =>
  is.sorted.diff(is).isEmpty
}
```

---

## Property-Based Testing

```scala
`forAll { (is: List[Int])` =>
  is.sorted.diff(is).isEmpty
}
```

---

## Property-Based Testing

```scala
forAll { (is: List[Int]) =>
  `is.sorted`.diff(is).isEmpty
}
```

---

## Property-Based Testing

```scala
forAll { (is: List[Int]) =>
  is.sorted.`diff(is).isEmpty`
}
```

---

## Property-Based Testing

```scala
val propFizzBuzz = forAll { (i: Int) =>
  if(mult3(i)) {
    if(mult5(i)) fizzBuzz(i) == "FizzBuzz"
    else         fizzBuzz(i) == "Fizz"
  }
  else if(mult5(i)) fizzBuzz(i) == "Buzz"
  else              fizzBuzz(i) == i.toString
}
```

---

## Property-Based Testing

```scala
val propFizzBuzz = `forAll { (i: Int)` =>
  if(mult3(i)) {
    if(mult5(i)) fizzBuzz(i) == "FizzBuzz"
    else         fizzBuzz(i) == "Fizz"
  }
  else if(mult5(i)) fizzBuzz(i) == "Buzz"
  else              fizzBuzz(i) == i.toString
}
```

---

## Property-Based Testing

```scala
val propFizzBuzz = forAll { (i: Int) =>
  if(`mult3(i)`) {
    if(`mult5(i)`) fizzBuzz(i) == `"FizzBuzz"`
    else         fizzBuzz(i) == "Fizz"
  }
  else if(mult5(i)) fizzBuzz(i) == "Buzz"
  else              fizzBuzz(i) == i.toString
}
```

---

## Property-Based Testing

```scala
val propFizzBuzz = forAll { (i: Int) =>
  if(`mult3(i)`) {
    if(mult5(i)) fizzBuzz(i) == "FizzBuzz"
    else         fizzBuzz(i) == `"Fizz"`
  }
  else if(mult5(i)) fizzBuzz(i) == "Buzz"
  else              fizzBuzz(i) == i.toString
}
```

---

## Property-Based Testing

```scala
val propFizzBuzz = forAll { (i: Int) =>
  if(mult3(i)) {
    if(mult5(i)) fizzBuzz(i) == "FizzBuzz"
    else         fizzBuzz(i) == "Fizz"
  }
  else if(`mult5(i)`) fizzBuzz(i) == `"Buzz"`
  else              fizzBuzz(i) == i.toString
}
```

---

## Property-Based Testing

---

## Property-Based Testing

```scala
propFizzBuzz.check()
```

---

## Property-Based Testing

```scala
propFizzBuzz.check()
// + OK, passed 100 tests.
```

---

## Property-Based Testing

```scala
val propFizzBuzz = forAll { (i: Int) =>
* if(mult3(i)) {
*   if(mult5(i)) fizzBuzz(i) == "FizzBuzz"
*   else         fizzBuzz(i) == "Fizz"
* }
* else if(mult5(i)) fizzBuzz(i) == "Buzz"
* else              fizzBuzz(i) == i.toString
}
```

---

## Test oracle

---

## Test oracle

---

## Test oracle

---

## Test oracle

---

## Test oracle

---

## Test oracle

---

## Test oracle

```scala
forAll { (is: List[Int]) =>
  fastSort(is) == is.sorted
}
```

---

## Test oracle

```scala
`forAll { (is: List[Int])` =>
  fastSort(is) == is.sorted
}
```

---

## Test oracle

```scala
forAll { (is: List[Int]) =>
  `fastSort(is)` == is.sorted
}
```

---

## Test oracle

```scala
forAll { (is: List[Int]) =>
  fastSort(is) == `is.sorted`
}
```

---

## Test oracle

```scala
forAll { (is: List[Int]) =>
  fastSort(is) `==` is.sorted
}
```

---

## Test oracle: use case

---

## Test oracle: use case

---

## Test oracle: use case

---

## Test oracle: use case

---

## Test oracle: use case

---

## Test oracle: use case

---

## Test oracle: use case

---

## Test oracle: fizzBuzz

---

## Validity

---

## Validity

---

## Validity

---

## Validity

```scala
forAll { (i: Int) =>
  math.abs(i) >= 0
}
```

---

## Validity

```scala
`forAll { (i: Int)` =>
  math.abs(i) >= 0
}
```

---

## Validity

```scala
forAll { (i: Int) =>
  `math.abs(i)` >= 0
}
```

---

## Validity

```scala
forAll { (i: Int) =>
  math.abs(i) `>= 0`
}
```

---

## Validity: use case

---

## Validity: use case

---

## Validity: use case

---

## Validity: use case

---

## Validity: fizzBuzz

```scala
val propValidFizzBuzz = forAll { (i: Int) =>
  fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
}
```

---

## Validity: fizzBuzz

```scala
val propValidFizzBuzz = `forAll { (i: Int)` =>
  fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
}
```

---

## Validity: fizzBuzz

```scala
val propValidFizzBuzz = forAll { (i: Int) =>
  `fizzBuzz(i)` in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
}
```

---

## Validity: fizzBuzz

```scala
val propValidFizzBuzz = forAll { (i: Int) =>
  fizzBuzz(i) `in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")`
}
```

---

## Validity: fizzBuzz

```scala
propValidFizzBuzz.check()
```

---

## Validity: fizzBuzz

```scala
propValidFizzBuzz.check()
// + OK, passed 100 tests.
```

---

## Involutivity

---

## Involutivity

---

## Involutivity

---

## Involutivity

---

## Involutivity

---

## Involutivity

---

## Involutivity

---

## Involutivity

```scala
forAll { (i: Int) =>
  i.toString.toInt == i
}
```

---

## Involutivity

```scala
`forAll { (i: Int)` =>
  i.toString.toInt == i
}
```

---

## Involutivity

```scala
forAll { (i: Int) =>
  i.`toString`.toInt == i
}
```

---

## Involutivity

```scala
forAll { (i: Int) =>
  i.toString.`toInt` == i
}
```

---

## Involutivity

```scala
forAll { (i: Int) =>
  i.toString.toInt `== i`
}
```

---

## Involutivity: use case

```scala
Track(
   artist  = "Iron Maiden",
   year    = 1982,
   album   = "The Number of the Beast",
   name    = "Children of the Damned"
)
```

---

## Involutivity: use case

```json
{
  "artist" : "Iron Maiden",
  "year"   : 1982,
  "album"  : "The Number of the Beast",
  "name"   : "Children of the Damned"
}
```

---

## Involutivity: use case

```scala
Track(
   artist  = "Iron Maiden",
   year    = 1970,
   album   = "The Number of the Beast",
   name    = "Children of the Damned"
)
```

---

## Involutivity: use case

```scala
Track(
   artist  = "Iron Maiden",
   year    = `1970`,
   album   = "The Number of the Beast",
   name    = "Children of the Damned"
)
```

---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = forAll { (i: Int) =>
  !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
}
```

---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = `forAll { (i: Int)` =>
  !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
}
```

---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = forAll { (i: Int) =>
  `!(mult3(i) || mult5(i))` ==> (fizzBuzz(i).toInt == i)
}
```

---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = forAll { (i: Int) =>
  !(mult3(i) || mult5(i)) `==>` (fizzBuzz(i).toInt == i)
}
```

---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = forAll { (i: Int) =>
  !(mult3(i) || mult5(i)) ==> (`fizzBuzz(i)`.toInt == i)
}
```
---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = forAll { (i: Int) =>
  !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).`toInt` == i)
}
```

---

## Involutivity: fizzBuzz

```scala
val propInvolutiveFizzBuzz = forAll { (i: Int) =>
  !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt `== i`)
}
```

---

## Involutivity: fizzBuzz

```scala
propInvolutiveFizzBuzz.check()
```

---

## Involutivity: fizzBuzz

```scala
propInvolutiveFizzBuzz.check()
// + OK, passed 100 tests.
```

---

## Idempotence

---

## Idempotence

---

## Idempotence

---

## Idempotence

---

## Idempotence

---

## Idempotence

```scala
forAll { (is: List[Int]) =>
  is.sorted == is.sorted.sorted
}
```

---

## Idempotence

```scala
`forAll { (is: List[Int])` =>
  is.sorted == is.sorted.sorted
}
```

---

## Idempotence

```scala
forAll { (is: List[Int]) =>
  `is.sorted` == is.sorted.sorted
}
```

---

## Idempotence

```scala
forAll { (is: List[Int]) =>
  is.sorted == `is.sorted.sorted`
}
```

---

## Idempotence

```scala
forAll { (is: List[Int]) =>
  is.sorted `==` is.sorted.sorted
}
```

---

## Idempotence: use case

`artist`     | `year`     | `album`                 | `name`
--------------|-------------|--------------------------------------------------
Iron Maiden   | 1982        | The Number of the Beast | Children of the Damned
Iron Maiden   | 1982        | The Number of the Beast | Invaders
Iron Maiden   | 1982        | The Number of the Beast | Gangland

```scala
db.delete(
  artist  = "Iron Maiden",
  name    = "Gangland"
)
```

---

## Idempotence: use case

```scala
db.delete(
  artist  = "Iron Maiden",
  name    = "Gangland"
)
```

---

## Idempotence: use case

`artist`     | `year`     | `album`                 | `name`
--------------|-------------|--------------------------------------------------
Iron Maiden   | 1982        | The Number of the Beast | Children of the Damned
       |          |                    |  
       |          |                    |

```scala
db.delete(
  artist  = "Iron Maiden",
  name    = "Gangland"
)
```

---

## Idempotence: fizzBuzz

---

## Invariance

---

## Invariance

---

## Invariance

---

## Invariance

```scala
forAll { (is: List[Int]) =>
  is.sorted.diff(is).isEmpty
}
```

---

## Invariance

```scala
`forAll { (is: List[Int])` =>
  is.sorted.diff(is).isEmpty
}
```

---

## Invariance

```scala
forAll { (is: List[Int]) =>
  `is.sorted`.diff(is).isEmpty
}
```

---

## Invariance

```scala
forAll { (is: List[Int]) =>
  is.sorted.`diff(is).isEmpty`
}
```

---

## Invariance: use case

---

## Invariance: use case

`artist`     | `year`     | `album`                 | `name`
--------------|-------------|--------------------------------------------------
İ̵̺͝ṛ̸̣͕̐͜o̴̢̎̿̓̌n̸̥͙̬̣̾́͛̾̌͘ͅ ̶̻͂M̷̗̣̠̠̎̀ȧ̸̛͔͇̐͘į̵̗̞̩̯̫̇̔̾̋̕͝d̵̟͓͆̅̇̋̐̚ͅe̷̯̪̙̼̩̩͐n̴̛̗̟͈͇̖̎̈̕   | 1̶͍͓̣̔̍́̒̒̅͘͜9̷̬̖̹̗̻͍̋͗̈́͠8̶͓̫̹̹̍2̸̨̲̝̏͌        | T̸̜̟͕̩̘̃̇͂͠h̶̦̓͋̇͝e̴̢̝̱̦͎͗͂̅̓͜ͅ ̶̝̬͠͝Ǹ̵̛̘̺̯̃̈́̍ù̷̱̖m̷̗͙̱̥͇̹͑̎̂͊̈́́̌b̸̢̦̪̽̅̂̑e̴̤̫̝̎̈́̇r̵͇̥̃̃̈͋̎̋͗͜ ̴͇̖̼͙̥̲̹̄̀ò̷͉̠̒f̶̲͇̳͛̔̃̽ ̸͍̉̚͝t̶̼̗̬̞̣̂͗͠͠͝h̴̰͒̾̑͐́͝ě̷̠̤͎̳͍̿͆́̕͝ ̸̭̬̊̐̾̍B̷̨̑é̴̡̟͔͖̏̃̔̓͝͠ą̷͉̫͇̫̈́̍̄́̃s̵͍̀̅͂͂͑̌ẗ̷̡̪̙͉͔́͑͗̇̋̑ | C̵͚̣̬̠̞̝̰̐̍̃h̷̥͛͂͋̚̚ï̴̤l̴͉̳̘̝͖̤͋͝d̴̢̲̫̯͙̺̮́̈́̕r̸̢̗̥͇̻͓͗̋̆́ȇ̸̲̉̆n̵̞̙͎̠̲͕̭̅̃̉̂ ̵͔̦͙̯̎͂̑̚͝õ̷̳̱͈̻̤̠̫̽̓̑̊͒f̸̦͋ͅ ̴̹̖̌ṫ̷̻̼͖͗̉̀̑h̴̹͎̰̬̟͊̍ͅe̸̤̱͛́͑ ̸̞̾͊͠D̵̢̜̬̳̍̑͛̄͆̀̅a̴͙̼̤̤͓͇̟̒̈́͋m̷̡̞̬̞̬͆̔̿n̴̨̢̖͉͓̥̂̄͘ë̴̬͚͎̭̭ͅd̸̢͕̼̗̠̂
Ì̷̧̺͓͕̘͓̯̗̬͈̤̗̝̰͜ȓ̵̨̜̟̱̘̟̬̙̯̱͍̩̭̈̆̅̍̌̿́̈́́̚͜͝ͅo̴̡͗͗͛̑̿̾͒͂̎̆n̶͖̳͖͉̗͕̱͔̱̖̫̽̎͜ͅ ̸̢̨̛͇͚͔̘̦̘̱̮̖̿͑̒̉̽̎́̑̚̚̕M̷̜͖̊͗̅̆̏̈́͛̍̂̉́̃̚̕͝ả̷̡̛̫͔̳̞̻͔͉̜̍̅̈́̆͛͒̋̑͠i̵̢̡͎̯̙͇͍̇͊̃̓̚͜ͅd̸̟̤̼̲̈́̎̑͜ͅȩ̸͓̝̘̙͉̼̺̖̳͓̩̽̕ͅn̴͈̈̃̿͝   | 1̴̛͈̣̲̹͗̾͂͆̒̓͛̋͝9̵̨̢̨̻̲̬͕̲͔͖̘̀̿͝ͅ8̷̩̹̣̦̙͖͚̞̮́̉̑̓̉͑̈̾́͊̀̽͛͘̚2̵̡̘̠̼͙̅̇͛̽͒̀̄̾́̋̎͐͛͝        | T̷̢̟̙̥̬̰͎͚͈͕̬͆̋͊͛̂̓̈́͊̌̀͆̅̐͘͝ȟ̴̛̹̝͕͒͗è̵̛̩͇͎̤̪͍͎̳̺͔̖̝͖͊̏̀̽͝ ̵͕͇͌͌́͛̑̏͊̈̔̿̚̕͠͝N̴̤̥̖̠͍̻̤̣͈̩̞̾̍̉͐͋̇̈̈́̓̈́̚ͅu̵̲̹̙̦̞̎̏͗͋̌͐̈́͘͜͝͝m̶̡̙̖̳̖͇̻̰̼̝̖̎̈̈̄́̀̽̈́͘͜͝b̸̡̙͚̙̮̀̀̅́̔́̿̑͆̂͒̕è̷̫͕̗̰̌̈́̏͗͆͂r̶̡̢̨̛̛̙̖̲͔̥̲͕̈́̀͆̀̅̀̔̃̿̉̽͝͝ ̵̨͍͎͈̹̺̟̠̪̼̟̟͔̜̆̈́̈́o̶̡̧̯̠̜̤͊̒̀͆̒̀̋̅̽̕͜͝ͅf̸͓̥̻͚̘̹͈͉̱̆̅̊̿͑͛́͆̽̋͘͜͜ ̶̣̭͙̯͇̦͂̋͆͆̕ţ̴̡̨̭̭̺̣̲̤͎̩̮̮̥̿́̔́̂͂h̵͉̤͛͊͛͐͒͒ė̶̜̰͓̤̼̖̜̣͎̫̑̍̌́̂̈̌͆̍ ̴̘̞̖̳͕̳͑̀̆̑͜B̷̨͈̹̣̘̰͕͉̰̹͗̔ȩ̷̡̠̙̖̱̜̲̤͚̼͓̠͛̾̓͆̅̒͂̅̀̿͘͘ạ̶̢̨̮̩͇͚͓̀̄̐̈́̇̇͊̄̾̕͝͝s̵̡̛͚̭̞͌̋̐̑͑̓̒̋͊͊͌̈̎̚ṱ̴̫̤̩͂͒̅̋̓̓̿̅̈́̎ | I̴̢̦̠͈̊̆͌͝͝ͅn̶̛͈̺̜͇̲̭͓͉̎̽̓̊̈̈́̾̔́̽̇͘v̷͈̘̞̬̼̲̍͊͊̋̓̕ą̷̰͙͙̖̖̤̬͙̃d̷̠̀́̓͆̽̽̽̓͘ȩ̵̤̱͓̫̻̫̯̪̎͌̇̆̄̓ͅr̴̗̹̪̬̘̪̝̰͆̒͆̋̃̃̏̈͑̅͝ş̴͎̭̻͔̘͈͉̟̻͈̯̩͓̔̐
---

## Invariance: use case

```scala
forAll { (cmds: List[Command]) =>
  cmds.exists { cmd =>
    db.execute(cmd)
    notCorrupt(db)
  }
}
```

---

## Invariance: use case

```scala
`forAll { (cmds: List[Command])` =>
  cmds.exists { cmd =>
    db.execute(cmd)
    notCorrupt(db)
  }
}
```

---

## Invariance: use case

```scala
forAll { (cmds: List[Command]) =>
  cmds.exists { cmd =>
    `db.execute(cmd)`
    notCorrupt(db)
  }
}
```

---

## Invariance: use case

```scala
forAll { (cmds: List[Command]) =>
  cmds.exists { cmd =>
    db.execute(cmd)
    `notCorrupt(db)`
  }
}
```

---

## Invariance: use case

---

## Invariance: fizzBuzz

```scala
val propSafeFizzBuzz = forAll { (i: Int) =>
  Try(fizzBuzz(i)).isSuccess
}
```

---

## Invariance: fizzBuzz

```scala
val propSafeFizzBuzz = `forAll { (i: Int)` =>
  Try(fizzBuzz(i)).isSuccess
}
```

---

## Invariance: fizzBuzz

```scala
val propSafeFizzBuzz = forAll { (i: Int) =>
  Try(`fizzBuzz(i)`).isSuccess
}
```

---

## Invariance: fizzBuzz

```scala
val propSafeFizzBuzz = forAll { (i: Int) =>
  `Try`(fizzBuzz(i))`.isSuccess`
}
```

---

## Invariance: fizzBuzz

```scala
propSafeFizzBuzz.check()
```

---

## Invariance: fizzBuzz

```scala
propSafeFizzBuzz.check()
// + OK, passed 100 tests.
```

---

## Metamorphic relation

---

## Metamorphic relation

---

## Metamorphic relation

---

## Metamorphic relation

---

## Metamorphic relation

---

## Metamorphic relation

---

## Metamorphic relation

```scala
forAll { (i: Int) =>
  mult3(i) == mult3(-i)
}
```

---

## Metamorphic relation

```scala
`forAll { (i: Int)` =>
  mult3(i) == mult3(-i)
}
```

---

## Metamorphic relation

```scala
forAll { (i: Int) =>
  `mult3(i)` == mult3(-i)
}
```

---

## Metamorphic relation

```scala
forAll { (i: Int) =>
  mult3(i) == mult3(`-i`)
}
```

---

## Metamorphic relation

```scala
forAll { (i: Int) =>
  mult3(i) == `mult3`(-i)
}
```

---

## Metamorphic relation

```scala
forAll { (i: Int) =>
  mult3(i) `==` mult3(-i)
}
```

---

## Metamorphic relation: use case

---

## Metamorphic relation: use case

---

## Metamorphic relation: use case

---

## Metamorphic relation: use case

---

## Metamorphic relation: use case

---

## Metamorphic relation: use case

---

## Metamorphic relation: use case

_[Metamorphic testing of RESTful web apis](http://www.lsi.us.es/~segura/files/papers/segura17-tse.pdf)_:

* 3 new unique Spotify bugs.

* 8 new unique YouTube bugs.

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = `forAll { (i: Int)` =>
  fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  `fizzBuzz(i)`.contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") ==> fizzBuzz(`-i`).contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") ==> `fizzBuzz(-i)`.contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i)`.contains("Fizz")` ==> fizzBuzz(-i).contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") `==>` fizzBuzz(-i).contains("Fizz")
}
```

---

## Metamorphic relation: fizzBuzz

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i)`.contains("Fizz")`
}
```

---

## Metamorphic relation: fizzBuzz

```scala
propStableFizz.check()
```

---

## Metamorphic relation: fizzBuzz

```scala
propStableFizz.check()
// ! Falsified after 1 passed tests.
// > ARG_0: -1
// > ARG_0_ORIGINAL: -20
```

---

## Fixing `fizzBuzz`

```scala
def mult3(i: Int): Boolean = !(i % 3 > 0)
def mult5(i: Int): Boolean = !(i % 5 > 0)

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```

---

## Fixing `fizzBuzz`

```scala
def mult3(i: Int): Boolean = `!(i % 3 > 0)`
def mult5(i: Int): Boolean = `!(i % 5 > 0)`

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```

---

## Fixing `fizzBuzz`

```scala
val propMult3 = forAll { (i: Int) =>
  mult3(i) == mult3(-i)
}
```

```scala
propMult3.check()
```

---

## Fixing `fizzBuzz`

```scala
val propMult3 = forAll { (i: Int) =>
  mult3(i) == mult3(-i)
}
```

```scala
propMult3.check()
// ! Falsified after 1 passed tests.
// > ARG_0: -1
// > ARG_0_ORIGINAL: -46
```

---

## Fixing `fizzBuzz`

.diff-rm[
```scala
*def mult3(i: Int): Boolean = `!(i % 3 > 0)`
*def mult5(i: Int): Boolean = `!(i % 5 > 0)`

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]

---

## Fixing `fizzBuzz`

.diff-add[
```scala
*def mult3(i: Int): Boolean = `i % 3 == 0`
*def mult5(i: Int): Boolean = `i % 5 == 0`

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]

---

## Fixing `fizzBuzz`

```scala
def mult3(i: Int): Boolean = i % 3 == 0
def mult5(i: Int): Boolean = i % 5 == 0

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```

---

## Fixing `fizzBuzz`

```scala
val propMult3 = forAll { (i: Int) =>
  mult3(i) == mult3(-i)
}
```

```scala
propMult3.check()
```

---

## Fixing `fizzBuzz`

```scala
val propMult3 = forAll { (i: Int) =>
  mult3(i) == mult3(-i)
}
```

```scala
propMult3.check()
// + OK, passed 100 tests.
```

---

## Fixing `fizzBuzz`

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
}
```

```scala
propStableFizz.check()
```

---

## Fixing `fizzBuzz`

```scala
val propStableFizz = forAll { (i: Int) =>
  fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
}
```

```scala
propStableFizz.check()
// + OK, passed 100 tests.
```

---

## Challenging properties

```scala
class FizzBuzzProps extends Properties("FizzBuzz") {

property("validity") = forAll { (i: Int) =>
    fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
  }

property("involutivity") = forAll { (i: Int) =>
    !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
  }

property("metamorphic") = forAll { (i: Int) =>
    fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
  }

property("invariant") = forAll { (i: Int) =>
    Try(fizzBuzz(i)).isSuccess
  }
}
```

---

## Challenging properties

```scala
class FizzBuzzProps extends Properties("FizzBuzz") {

* property("validity") = forAll { (i: Int) =>
*   fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
* }

property("involutivity") = forAll { (i: Int) =>
    !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
  }

property("metamorphic") = forAll { (i: Int) =>
    fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
  }

property("invariant") = forAll { (i: Int) =>
    Try(fizzBuzz(i)).isSuccess
  }
}
```

---

## Challenging properties

```scala
class FizzBuzzProps extends Properties("FizzBuzz") {

property("validity") = forAll { (i: Int) =>
    fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
  }

* property("involutivity") = forAll { (i: Int) =>
*   !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
* }

property("metamorphic") = forAll { (i: Int) =>
    fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
  }

property("invariant") = forAll { (i: Int) =>
    Try(fizzBuzz(i)).isSuccess
  }
}
```

---

## Challenging properties

```scala
class FizzBuzzProps extends Properties("FizzBuzz") {

property("validity") = forAll { (i: Int) =>
    fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
  }

property("involutivity") = forAll { (i: Int) =>
    !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
  }

* property("metamorphic") = forAll { (i: Int) =>
*   fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
* }

property("invariant") = forAll { (i: Int) =>
    Try(fizzBuzz(i)).isSuccess
  }
}
```

---

## Challenging properties

```scala
class FizzBuzzProps extends Properties("FizzBuzz") {

property("validity") = forAll { (i: Int) =>
    fizzBuzz(i) in Set(i.toString, "Fizz", "Buzz", "FizzBuzz")
  }

property("involutivity") = forAll { (i: Int) =>
    !(mult3(i) || mult5(i)) ==> (fizzBuzz(i).toInt == i)
  }

property("metamorphic") = forAll { (i: Int) =>
    fizzBuzz(i).contains("Fizz") ==> fizzBuzz(-i).contains("Fizz")
  }

* property("invariant") = forAll { (i: Int) =>
*   Try(fizzBuzz(i)).isSuccess
* }
}
```

---

## Challenging properties

```scala
new FizzBuzzProps().check()
```

---

## Challenging properties

```scala
new FizzBuzzProps().check()
// + FizzBuzz.validity: OK, passed 100 tests.
// + FizzBuzz.involutivity: OK, passed 100 tests.
// + FizzBuzz.metamorphic: OK, passed 100 tests.
// + FizzBuzz.invariant: OK, passed 100 tests.
```

---

## Challenging properties

```scala
def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```

---

## Challenging properties

.diff-rm[
```scala
def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
*   else         `"Fizz"`
  }
* else if(mult5(i)) `"Buzz"`
  else              i.toString
}
```
]

---

## Challenging properties

.diff-add[
```scala
def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
*   else         `"Buzz"`
  }
* else if(mult5(i)) `"Fizz"`
  else              i.toString
}
```
]

---

## Challenging properties

```scala
def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Buzz"
  }
  else if(mult5(i)) "Fizz"
  else              i.toString
}
```

---

## Challenging properties

```scala
new FizzBuzzProps().check()
```

---

## Challenging properties

---

## Challenging properties

```scala
val propFizz = forAll { (i: Int) =>
  fizzBuzz(i * 3).contains("Fizz")
}
```

---

## Challenging properties

```scala
val propFizz = `forAll { (i: Int)` =>
  fizzBuzz(i * 3).contains("Fizz")
}
```

---

## Challenging properties

```scala
val propFizz = forAll { (i: Int) =>
  fizzBuzz(`i * 3`).contains("Fizz")
}
```

---

## Challenging properties

```scala
val propFizz = forAll { (i: Int) =>
  `fizzBuzz`(i * 3).contains("Fizz")
}
```

---

## Challenging properties

```scala
val propFizz = forAll { (i: Int) =>
  fizzBuzz(i * 3).`contains("Fizz")`
}
```

---

## Challenging properties

```scala
propFizz.check()
```

---

## Challenging properties

```scala
propFizz.check()
// ! Falsified after 0 passed tests.
// > ARG_0: -1
// > ARG_0_ORIGINAL: -96
```

---

## Key takeaways

Property-based testing is:

* very good at the _test case generation problem_.

* good at the _oracle problem_.

* hard to get the hang of.

---

# Generative Example-Based Tests

---

## Example-Based Tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Example-Based Tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
* test("n multiple of 3 outputs Fizz") {
*   assert(fizzBuzz(3) == "Fizz")
* }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Example-Based Tests

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(`fizzBuzz(3) == "Fizz"`)
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Naive property

```scala
val propFizzBuzz3 =
  fizzBuzz(3) == "Fizz"
```

---

## Naive property

```scala
val propFizzBuzz3 =
  fizzBuzz(`3`) == "Fizz"
```

---

## Naive property

.diff-add[
```scala
val propFizzBuzz3 =
* `forAll(const(3)) { i =>`
* `  `fizzBuzz(3) == "Fizz"
* `}`
```
]

---

## Naive property

```scala
val propFizzBuzz3 =
  forAll(const(3)) { `i` =>
    fizzBuzz(3) == "Fizz"
  }
```

---

## Naive property

```scala
val propFizzBuzz3 =
  forAll(`const(3)`) { i =>
    fizzBuzz(3) == "Fizz"
  }
```

---

## Naive property

.diff-rm[
```scala
val propFizzBuzz3 =
  forAll(const(3)) { i =>
*   fizzBuzz(`3`) == "Fizz"
  }
```
]

---

## Naive property

.diff-add[
```scala
val propFizzBuzz3 =
  forAll(const(3)) { i =>
*   fizzBuzz(`i`) == "Fizz"
  }
```
]

---

## Naive property

```scala
val propFizzBuzz3 =
  forAll(const(3)) { i =>
    fizzBuzz(i) == `"Fizz"`
  }
```

---

## Naive property

```scala
val propFizzBuzz3 =
  forAll(const(3)) { i =>
    fizzBuzz(i) == "Fizz"
  }
```

---

## Naive property

```scala
val propFizzBuzz3 =
  forAll(`const(3)`) { i =>
    fizzBuzz(i) == "Fizz"
  }
```

---

## Multiples of 3

```scala
val genMult3: Gen[Int] =
  arbitrary[Int].
    map(_ * 3).
    filter(_ % 5 != 0)
```

---

## Multiples of 3

```scala
val genMult3: Gen[Int] =
  `arbitrary[Int]`.
    map(_ * 3).
    filter(_ % 5 != 0)
```

---

## Multiples of 3

```scala
val genMult3: Gen[Int] =
  arbitrary[Int].
    `map(_ * 3)`.
    filter(_ % 5 != 0)
```

---

## Multiples of 3

```scala
val genMult3: Gen[Int] =
  arbitrary[Int].
    map(_ * 3).
    `filter(_ % 5 != 0)`
```

---

## Multiples of 3

.diff-rm[
```scala
val propFizzBuzz3 =
* forAll(`const(3)`) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Multiples of 3

.diff-add[
```scala
val propFizzBuzz3 =
* forAll(`genMult3`) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Multiples of 3

```scala
val propFizzBuzz3 =
  forAll(genMult3) { i =>
    fizzBuzz(i) == "Fizz"
  }
```

---

## Generic property

```scala
val propFizzBuzz3 =
  forAll(`genMult3`) { i =>
    fizzBuzz(i) == "Fizz"
  }
```

---

## Generic property

```scala
val propFizzBuzz3 =
  forAll(genMult3) { i =>
    fizzBuzz(i) == `"Fizz"`
  }
```

---

## Generic property

.diff-rm[
```scala
*`val propFizzBuzz3` =
  forAll(genMult3) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Generic property

.diff-add[
```scala
*`def propFizzBuzz(gen: Gen[Int])` =
  forAll(genMult3) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Generic property

.diff-rm[
```scala
def propFizzBuzz(gen: Gen[Int]) =
* forAll(`genMult3`) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Generic property

.diff-add[
```scala
def propFizzBuzz(gen: Gen[Int]) =
* forAll(`gen`) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Generic property

.diff-add[
```scala
*def propFizzBuzz(gen: Gen[Int]`, expected: String`) =
  forAll(gen) { i =>
    fizzBuzz(i) == "Fizz"
  }
```
]

---

## Generic property

.diff-rm[
```scala
def propFizzBuzz(gen: Gen[Int], expected: String) =
  forAll(gen) { i =>
*   fizzBuzz(i) == `"Fizz"`
  }
```
]

---

## Generic property

.diff-add[
```scala
def propFizzBuzz(gen: Gen[Int], expected: String) =
  forAll(gen) { i =>
*   fizzBuzz(i) == `expected`
  }
```
]

---

## Generic property

```scala
def propFizzBuzz(gen: Gen[Int], expected: String) =
  forAll(gen) { i =>
    fizzBuzz(i) == expected
  }
```

---

## Generic property

```scala
val propFizzBuzz3 =
  forAll(genMult3) { i =>
    fizzBuzz(i) == "Fizz"
  }
```

---

## Generic property

.diff-rm[
```scala
val propFizzBuzz3 =
*`  forAll(genMult3) { i =>`
*`    fizzBuzz(i) == "Fizz"`
*`  }`
```
]

---

## Generic property

.diff-add[
```scala
val propFizzBuzz3 =
*`  propFizzBuzz(`
*`    gen      = genMult3,`
*`    expected = "Fizz"`
*`  )`
```
]

---

## Generic property

```scala
val propFizzBuzz3 =
  propFizzBuzz(
    `gen      = genMult3`,
    expected = "Fizz"
  )
```

---

## Generic property

```scala
val propFizzBuzz3 =
  propFizzBuzz(
    gen      = genMult3,
    `expected = "Fizz"`
  )
```

---

## Generic property

```scala
val propFizzBuzz3 =
  propFizzBuzz(
    gen      = genMult3,
    expected = "Fizz"
  )
```

---

## Multiples of 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

* test("n multiple of 5 outputs Buzz") {
*   assert(fizzBuzz(5) == "Buzz")
* }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Multiples of 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(`5`) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Multiples of 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == `"Buzz"`)
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Multiples of 5

```scala
val genMult5: Gen[Int] =
  arbitrary[Int].
    map(_ * 5).
    filter(_ % 3 != 0)
```

---

## Multiples of 5

```scala
val genMult5: Gen[Int] =
  `arbitrary[Int]`.
    map(_ * 5).
    filter(_ % 3 != 0)
```

---

## Multiples of 5

```scala
val genMult5: Gen[Int] =
  arbitrary[Int].
    `map(_ * 5)`.
    filter(_ % 3 != 0)
```

---

## Multiples of 5

```scala
val genMult5: Gen[Int] =
  arbitrary[Int].
    map(_ * 5).
    `filter(_ % 3 != 0)`
```

---

## Multiples of 5

```scala
val propFizzBuzz5 =
  propFizzBuzz(
    gen      = genMult5,
    expected = "Buzz"
  )
```

---

## Multiples of 5

```scala
val propFizzBuzz5 =
  propFizzBuzz(
    `gen      = genMult5`,
    expected = "Buzz"
  )
```

---

## Multiples of 5

```scala
val propFizzBuzz5 =
  propFizzBuzz(
    gen      = genMult5,
    `expected = "Buzz"`
  )
```

---

## Multiples of neither 3 nor 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

* test("n multiple of neither 3 or 5 outputs n") {
*   assert(fizzBuzz(7) == "7")
* }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Multiples of neither 3 nor 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(`7`) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Multiples of neither 3 nor 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == `"7"`)
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == "FizzBuzz")
  }
}
```

---

## Multiples of neither 3 nor 5

```scala
val genMultNone: Gen[Int] =
  arbitrary[Int].
    filter(_ % 5 != 0).
    filter(_ % 3 != 0)
```

---

## Multiples of neither 3 nor 5

```scala
val genMultNone: Gen[Int] =
  `arbitrary[Int]`.
    filter(_ % 5 != 0).
    filter(_ % 3 != 0)
```

---

## Multiples of neither 3 nor 5

```scala
val genMultNone: Gen[Int] =
  arbitrary[Int].
    `filter(_ % 5 != 0)`.
    filter(_ % 3 != 0)
```

---

## Multiples of neither 3 nor 5

```scala
val genMultNone: Gen[Int] =
  arbitrary[Int].
    filter(_ % 5 != 0).
    `filter(_ % 3 != 0)`
```

---

## Multiples of neither 3 nor 5

```scala
val propFizzBuzzNone =
  propFizzBuzz(
    gen      = genMultNone,
    expected = ???
  )
```

---

## Multiples of neither 3 nor 5

```scala
val propFizzBuzzNone =
  propFizzBuzz(
    `gen      = genMultNone`,
    expected = ???
  )
```

---

## Multiples of neither 3 nor 5

```scala
val propFizzBuzzNone =
  propFizzBuzz(
    gen      = genMultNone,
    `expected = ???`
  )
```

---

## Multiples of neither 3 nor 5

```scala
def propFizzBuzz(gen: Gen[Int], expected: String) =
  forAll(gen) { i =>
    fizzBuzz(i) == expected
  }
```

---

## Multiples of neither 3 nor 5

.diff-rm[
```scala
*def propFizzBuzz(gen: Gen[Int], `expected: String`) =
  forAll(gen) { i =>
*   fizzBuzz(i) == `expected`
  }
```
]

---

## Multiples of neither 3 nor 5

.diff-add[
```scala
*def propFizzBuzz(gen: Gen[Int], `oracle: Int => String`) =
  forAll(gen) { i =>
*   fizzBuzz(i) == `oracle(i)`
  }
```
]

---

## Multiples of neither 3 nor 5

```scala
def propFizzBuzz(gen: Gen[Int], oracle: Int => String) =
  forAll(gen) { i =>
    fizzBuzz(i) == oracle(i)
  }
```

---

## Multiples of neither 3 nor 5

.diff-rm[
```scala
val propFizzBuzzNone =
  propFizzBuzz(
    gen      = genMultNone,
*   `expected = ???`
  )
```
]

---

## Multiples of neither 3 nor 5

.diff-add[
```scala
val propFizzBuzzNone =
  propFizzBuzz(
    gen      = genMultNone,
*   `oracle   = _.toString`
  )
```
]

---

## Multiples of neither 3 nor 5

```scala
val propFizzBuzzNone =
  propFizzBuzz(
    gen      = genMultNone,
    oracle   = _.toString
  )
```

---

## Multiples of 3 and 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

* test("n multiple of 3 and 5 outputs FizzBuzz") {
*   assert(fizzBuzz(15) == "FizzBuzz")
* }
}
```

---

## Multiples of 3 and 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(`15`) == "FizzBuzz")
  }
}
```

---

## Multiples of 3 and 5

```scala
class FizzBuzzSuite extends AnyFunSuite {
  test("n multiple of 3 outputs Fizz") {
    assert(fizzBuzz(3) == "Fizz")
  }

test("n multiple of 5 outputs Buzz") {
    assert(fizzBuzz(5) == "Buzz")
  }

test("n multiple of neither 3 or 5 outputs n") {
    assert(fizzBuzz(7) == "7")
  }

test("n multiple of 3 and 5 outputs FizzBuzz") {
    assert(fizzBuzz(15) == `"FizzBuzz"`)
  }
}
```

---

## Multiples of 3 and 5

```scala
val genMult15: Gen[Int] =
  arbitrary[Int].
    map(_ * 3 * 5)
```

---

## Multiples of 3 and 5

```scala
val genMult15: Gen[Int] =
  `arbitrary[Int]`.
    map(_ * 3 * 5)
```

---

## Multiples of 3 and 5

```scala
val genMult15: Gen[Int] =
  arbitrary[Int].
    `map(_ * 3 * 5)`
```

---

## Multiples of 3 and 5

```scala
val propFizzBuzz15 =
  propFizzBuzz(
    gen    = genMult15,
    oracle = _ => "FizzBuzz"
  )
```

---

## Multiples of 3 and 5

```scala
val propFizzBuzz15 =
  propFizzBuzz(
    `gen    = genMult15`,
    oracle = _ => "FizzBuzz"
  )
```

---

## Multiples of 3 and 5

```scala
val propFizzBuzz15 =
  propFizzBuzz(
    gen    = genMult15,
    `oracle = _ => "FizzBuzz"`
  )
```

---

## Key takeaways

* you don't need to learn a new way to write tests (right away).

* you can adapt existing tests with little work.

* you will acquire the skills you need as you do so.

---

# In closing

---

## If you only remember 1 slide...

Property-based testing:

* finds more flaws than example-based testing.

* doesn't need to be harder than example-based testing.

* importantly, is a lot of fun!

---

# Questions?

Nicolas Rinaudo • [@NicolasRinaudo] • [Besedo]

---

# Metamorphic testing

---

## Profile moderation

---

## Profile moderation

---

## Profile moderation

---

## Profile moderation

---

## Profile moderation

---

## Profile moderation

---

## Metamorphic testing

---

## Metamorphic testing

---

## Metamorphic testing

---

## Metamorphic testing

---

## Metamorphic testing

---

## Compiler validation

```scala
def mult3(i: Int): Boolean = i % 3 == 0
def mult5(i: Int): Boolean = i % 5 == 0

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
```

---

## Compiler validation

```scala
println(fizzBuzz(`5`))
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
// Buzz
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
// `Buzz`
```

---

## Compiler validation

.coverage-on[
```scala
*def mult3(i: Int): Boolean = i % 3 == 0
*def mult5(i: Int): Boolean = i % 5 == 0

def fizzBuzz(i: Int): String = {
* if(mult3(i)) {
    if(mult5(i)) "FizzBuzz"
    else         "Fizz"
  }
* else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]

---

## Compiler validation

.coverage-off[
```scala
def mult3(i: Int): Boolean = i % 3 == 0
def mult5(i: Int): Boolean = i % 5 == 0

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
*   if(mult5(i)) "FizzBuzz"
*   else         "Fizz"
  }
  else if(mult5(i)) "Buzz"
* else              i.toString
}
```
]

---

## Compiler validation

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
*   `if(mult5(i)) "FizzBuzz"`
*   `else         "Fizz"`
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]

---

## Compiler validation

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
*   `"Fizz"`
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]
---

## Compiler validation

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
*   `"Fizz"`
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]

---

## Compiler validation

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
*   `"Zzif"`
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```
]

---

## Compiler validation

```scala
def mult3(i: Int): Boolean = i % 3 == 0
def mult5(i: Int): Boolean = i % 5 == 0

def fizzBuzz(i: Int): String = {
  if(mult3(i)) {
    "Zzif"
  }
  else if(mult5(i)) "Buzz"
  else              i.toString
}
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
```

---

## Compiler validation

```scala
println(fizzBuzz(`5`))
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
// Buzz
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
// `Buzz`
```

---

## Compiler validation

```scala
println(fizzBuzz(5))
// `Zzif`
```

---

## Compiler validation

_[Compiler Validation via Equivalence Modulo Inputs](http://vuminhle.com/pdf/pldi14-emi.pdf)_:

* 79 new unique gcc bugs.

* 68 new unique LLVM bugs.

* 42 new unique scalac bugs.

---

## Key takeaways

Metamorphic Testing:

* helps generating test cases when it's too expensive.

* helps with the oracle problem.

* a good word to use when you want to sound smart.

---

# Questions?

Nicolas Rinaudo • [@NicolasRinaudo] • [Besedo]

[Besedo]:https://twitter.com/besedo_official
[@NicolasRinaudo]:https://twitter.com/NicolasRinaudo