Kata: Generating bingo cards with clojure.spec, clojure/test.check, RDD and TDD

Clojure Developers Barcelona has been running for several years now. Since we're not many yet, we usually do mob programming sessions as part of what we call "sagas". For each saga, we choose an exercise or kata and solve it during the first one or two sessions. After that, we start imagining variations on the exercise using different Clojure/ClojureScript libraries or technologies we feel like exploring and develop those variations in following sessions. Once we feel we can't imagine more interesting variations or we get tired of a given problem, we choose a different problem to start a new saga. You should try doing sagas, they are a lot of fun!

Recently we've been working on the Bingo Kata.

The initial implementation

These were the tests we wrote to check the randomly generated bingo cards:

	(ns bingo.card-test
	(:require
	[clojure.test :refer :all]
	[bingo.card :as card]))
	(defn- check-column [& {:keys [column num-elements min max]}]
	(is (= num-elements (count column)))
	(is (apply distinct? column))
	(is (every? #(> (inc max) % (dec min)) column)))
	(deftest generating-a-bingo-card
	(let [card (card/create)]
	(testing "column B contains 5 different numbers between 1 and 15 inclusive"
	(check-column :column (:b card) :num-elements 5 :min 1 :max 15))
	(testing "column I contains 5 different numbers between 16 and 30 inclusive"
	(check-column :column (:i card) :num-elements 5 :min 16 :max 30))
	(testing "column N contains 4 different numbers between 31 and 45 inclusive"
	(check-column :column (:n card) :num-elements 4 :min 31 :max 45))
	(testing "column G contains 5 different numbers between 46 and 60 inclusive"
	(check-column :column (:g card) :num-elements 5 :min 46 :max 60))
	(testing "column O contains 5 different numbers between 61 and 75 inclusive"
	(check-column :column (:o card) :num-elements 5 :min 61 :max 75))))

view raw old_card_test.clj hosted with ❤ by GitHub

and the code we initially wrote to generate them was something like (we didn't save the original one):

	(ns bingo.card)
	(defn create []
	{:b (take 5 (shuffle (range 1 16)))
	:i (take 5 (shuffle (range 16 31)))
	:n (take 4 (shuffle (range 31 46)))
	:g (take 5 (shuffle (range 46 61)))
	:o (take 5 (shuffle (range 61 76)))})

view raw old-card.clj hosted with ❤ by GitHub

As you can see the tests are not concerned with which specific numeric values are included on each column of the bingo card. They are just checking that they follow the specification of a bingo card. This makes them very suitable for property-based testing.

Introducing clojure.spec

In the following session of the Bingo saga, I suggested creating the bingo cards using clojure.spec.

spec is a Clojure library to describe the structure of data and functions. Specs can be used to validate data, conform (destructure) data, explain invalid data, generate examples that conform to the specs, and automatically use generative testing to test functions.

For a brief introduction to this wonderful library see Arne Brasseur's Introduction to clojure.spec talk.

I'd used clojure.spec at work before. At my current client Green Power Monitor, we've been using it for a while to validate the shape (and in some cases types) of data flowing through some important public functions of some key name spaces. We started using pre and post-conditions for that validation (see Fogus' Clojure’s :pre and :post to know more), and from there, it felt as a natural step to start using clojure.spec to write some of them.

Another common use of clojure.spec specs is to generate random data conforming to the spec to be used for property-based testing.

In the Bingo kata case, I thought that we might use this ability of randomly generating data conforming to the spec in production code. This meant that instead of writing code to randomly generating bingo cards and then testing that the results were as expected, we might describe the bingo cards using clojure.spec and then took advantage of that specification to randomly generate bingo cards using clojure.test.check's generate function.

So with this idea in our heads, we started creating a spec for bingo columns on the REPL bit by bit (for the sake of brevity what you can see here is the final form of the spec):

	(require '[clojure.spec.alpha :as s])
	(s/def ::b (s/and vector?
	#(= 5 (count %))
	(partial every? int?)
	#(< (apply max %) 16)
	#(> (apply min %) 0)
	#(apply distinct? %)))
	=> :bingo.card-spec/b
	(s/explain-str ::b [1 2 3 4 5])
	=> "Success!\n"
	(s/explain-str ::b [1 2 3 4 "a"])
	=> "val: [1 2 3 4 \"a\"] fails spec: :bingo.card-spec/b predicate: (partial every? int?)\n"
	(s/explain-str ::b [0 2 3 4 5])
	=> "val: [0 2 3 4 5] fails spec: :bingo.card-spec/b predicate: (> (apply min %) 0)\n"
	(s/explain-str ::b [1 2 3 4 16])
	=> "val: [1 2 3 4 16] fails spec: :bingo.card-spec/b predicate: (< (apply max %) 16)\n"
	(s/explain-str ::b [1 2 3 4 1])
	=> "val: [1 2 3 4 1] fails spec: :bingo.card-spec/b predicate: (apply distinct? %)\n"
	(s/explain-str ::b [1 2 3 4])
	=> "val: [1 2 3 4] fails spec: :bingo.card-spec/b predicate: (= 5 (count %))\n"

view raw repl-column-spec-version-1.clj hosted with ❤ by GitHub

then we discovered clojure.spec's coll-of function which allowed us to simplify the spec a bit:

	(s/def ::b (s/and (s/coll-of int? :kind vector? :count 5 :distinct true)
	#(< (apply max %) 16)
	#(> (apply min %) 0)))

view raw simplified-initial-column-spec.clj hosted with ❤ by GitHub

Generating bingo cards

Once we thought we had it, we tried to use the column spec to generate columns with clojure.test.check's generate function, but we got the following error:

ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.

Of course we were trying to find a needle in a haystack...

After some trial and error on the REPL and reading the clojure.spec guide, we found the clojure.spec's int-in function and we finally managed to generate the bingo columns:

	(require '[clojure.test.check.generators :as gen])
	=> nil
	(require '[clojure.spec.alpha :as s])
	=> nil
	(s/def ::b (s/and (s/coll-of int? :kind vector? :count 5 :distinct true)
	#(< (apply max %) 16)
	#(> (apply min %) 0)))
	=> :user/b
	(gen/generate (s/gen ::b))
	ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4739)
	(s/def ::b (s/and (s/coll-of pos-int? :kind vector? :count 5 :distinct true)
	#(< (apply max %) 16)
	#(> (apply min %) 0)))
	=> :user/b
	(gen/generate (s/gen ::b))
	ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4739)
	(s/def ::b (s/coll-of (s/int-in 0 16) :kind vector? :count 5 :distinct true))
	=> :user/b
	(gen/generate (s/gen ::b))
	=> [10 11 1 14 8]
	(gen/generate (s/gen ::b))
	=> [12 9 14 1 13]
	(gen/generate (s/gen ::b))
	=> [8 11 3 15 14]
	(gen/generate (s/gen ::b))
	=> [8 11 13 15 10]
	(gen/generate (s/gen ::b))
	=> [9 7 11 13 10]
	(gen/generate (s/gen ::b))
	=> [8 15 11 12 13]

view raw repl-column-spec-version-2.clj hosted with ❤ by GitHub

Then we used the spec code from the REPL to write the bingo cards spec:

	(ns bingo.card-spec
	(:require
	[clojure.spec.alpha :as s]))
	(defn create-column-spec [& {:keys [min max n-elements]}]
	(s/coll-of
	(s/int-in min (inc max)) :kind vector? :count n-elements :distinct true))
	(s/def ::b (create-column-spec :min 1 :max 15 :n-elements 5))
	(s/def ::i (create-column-spec :min 16 :max 30 :n-elements 5))
	(s/def ::n (create-column-spec :min 31 :max 45 :n-elements 4))
	(s/def ::g (create-column-spec :min 46 :max 60 :n-elements 5))
	(s/def ::o (create-column-spec :min 61 :max 75 :n-elements 5))
	(s/def ::bingo-card (s/keys :req-un [::b ::i ::n ::g ::o]))

view raw card_spec.clj hosted with ❤ by GitHub

in which we wrote the create-column-spec factory function that creates column specs to remove duplication between the specs of different columns.

With this in place the bingo cards could be created in a line of code:

	(ns bingo.card
	(:require
	[bingo.card-spec :as card-spec]
	[clojure.test.check.generators :as gen]
	[clojure.spec.alpha :as s]))
	(def generator (s/gen ::card-spec/bingo-card))
	(defn create []
	(gen/generate generator))

view raw bingo-card.clj hosted with ❤ by GitHub

Introducing property-based testing

Property-based tests make statements about the output of your code based on the input, and these statements are verified for many different possible inputs.
Jessica Kerr (Property-based testing: what is it?)

Having the specs it was very easy to change our bingo card test to use property-based testing instead of example-based testing just by using the generator created by clojure.spec:

	(ns bingo.card-creation-test
	(:require
	[clojure.test :refer :all]
	[bingo.card :as card]
	[clojure.test.check.clojure-test :refer [defspec]]
	[clojure.test.check.properties :as prop]))
	(defn- check-column [& {:keys [column num-elements min max]}]
	(is (= num-elements (count column)))
	(is (apply distinct? column))
	(is (every? #(> (inc max) % (dec min)) column)))
	(def valid-bingo-card
	(prop/for-all [card card/generator]
	(check-column :column (:b card) :num-elements 5 :min 1 :max 15)
	(check-column :column (:i card) :num-elements 5 :min 16 :max 30)
	(check-column :column (:n card) :num-elements 4 :min 31 :max 45)
	(check-column :column (:g card) :num-elements 5 :min 46 :max 60)
	(check-column :column (:o card) :num-elements 5 :min 61 :max 75)))
	(defspec generated-bingo-cards-are-valid
	200
	valid-bingo-card)

view raw card-creation-test.clj hosted with ❤ by GitHub

See in the code that we're reusing the check-column function we wrote for the example-based tests.

This change was so easy because of:

1. clojure.spec can produce a generator for clojure/test.check from a given spec
.
2. The initial example tests, as I mentioned before, were already checking the properties of a valid bingo card. This means that they weren't concerned with which specific numeric values were included on each column of the bingo card, but instead, they were just checking that the cards followed the rules for a bingo card to be valid.

Going fast with REPL driven development (RDD)

The next user story of the kata required us to check a bingo card to see if its player has won. We thought this might be easy to implement because we only needed to check that the numbers in the card where contained by the set of called numbers, so instead of doing TDD, we played a bit on the REPL did REPL-driven development (RDD):

	Loading src/bingo/card_spec.clj... done
	Loading src/bingo/card.clj... done
	(in-ns 'bingo.card)
	=> #object[clojure.lang.Namespace 0x938dc60 "bingo.card"]
	(def a-card (create))
	=> #'bingo.card/a-card
	a-card
	=> {:b [12 13 11 8 15], :i [27 23 25 30 28], :n [38 44 43 33], :g [48 56 54 57 49], :o [70 63 68 62 73]}
	(def numbers [12 13 11 8 15 27 23 25 30 28 38 44 43 33 48 56 54 57 49 70 63 68 62 73])
	=> #'bingo.card/numbers
	(vals a-card)
	=> ([12 13 11 8 15] [27 23 25 30 28] [38 44 43 33] [48 56 54 57 49] [70 63 68 62 73])
	(->> a-card
	vals)
	=> ([12 13 11 8 15] [27 23 25 30 28] [38 44 43 33] [48 56 54 57 49] [70 63 68 62 73])
	(->> a-card
	vals
	flatten)
	=> (12 13 11 8 15 27 23 25 30 28 38 44 43 33 48 56 54 57 49 70 63 68 62 73)
	(->> a-card
	vals
	flatten
	(every? (set numbers)))
	=> true
	(->> a-card
	vals
	flatten
	(every? (set (rest numbers))))
	=> false
	(->> a-card
	vals
	flatten
	(every? (set [])))
	=> false
	(->> a-card
	vals
	flatten
	(every? (set numbers)))
	=> true

view raw repl-checking-bingo.clj hosted with ❤ by GitHub

Once we had the implementation working, we copied it from the REPL into its corresponding name space

	(ns bingo.card
	(:require
	[bingo.card-spec :as card-spec]
	[clojure.test.check.generators :as gen]
	[clojure.spec.alpha :as s]))
	(def generator (s/gen ::card-spec/bingo-card))
	(defn create []
	(gen/generate generator))
	(defn bingo? [card numbers]
	(->> card
	vals
	flatten
	(every? (set numbers))))

view raw card-with-bingo-checking.clj hosted with ❤ by GitHub

and wrote the quicker but ephemeral REPL tests as "permanent" unit tests:

	(ns bingo.checking-bingo-test
	(:require
	[clojure.test :refer :all]
	[bingo.card :as card]))
	(deftest checking-card-has-bingo
	(let [a-card {:b [12 13 11 8 15],
	:i [27 23 25 30 28],
	:n [38 44 43 33],
	:g [48 56 54 57 49],
	:o [70 63 68 62 73]}]
	(is (card/bingo? a-card
	[12 13 11 8 15 27 23 25 30 28 38 44 43 33 48 56 54 57 49 70 63 68 62 73]))
	(is (not (card/bingo? a-card
	[12 13 11 8 15 27 23 25 30 28 38 44 43 33 48 56 54 57 49 70])))
	(is (not (card/bingo? a-card [])))))

view raw checking_bingo_test.clj hosted with ❤ by GitHub

In this case RDD allowed us to go faster than TDD, because RDD's feedback cycle is much faster. Once the implementation is working on the REPL, you can choose which REPL tests you want to keep as unit tests.

Some times I use only RDD like in this case, other times I use a mix of TDD and RDD following this cycle:

1. Write a failing test (using examples that a bit more complicated than the typical ones you use when doing only TDD).
2. Explore and triangulate on the REPL until I made the test pass with some ugly but complete solution.
3. Refactor the code.

Other times I just use TDD.

I think what I use depends a lot on how easy I feel the implementation might be.

Last details

The last user story required us to create a bingo caller that randomly calls out Bingo numbers. To develop this story, we used TDD and an atom to keep the not-yet-called numbers. These were our tests:

	(ns bingo.caller-test
	(:require
	[clojure.test :refer :all]
	[bingo.caller :as caller]))
	(defn- check [number]
	(is (>= 75 number 1))
	(is (not (caller/in-bag? number))))
	(deftest called-numbers
	(testing "the called numbers are between 1 and 75 inclusive and are different each time"
	(caller/reset-numbers-bag!)
	(check (caller/call-number!))
	(check (caller/call-number!)))
	(testing "after calling number 75 times, all numbers between 1 and 75 are present"
	(caller/reset-numbers-bag!)
	(let [called-numbers (atom #{})]
	(dotimes [_ 75]
	(swap! called-numbers conj (caller/call-number!)))
	(is (= @called-numbers (set (range 1 76)))))))

view raw caller_test.clj hosted with ❤ by GitHub

and this was the resulting code:

	(ns bingo.caller)
	(def ^:private numbers-bag (atom nil))
	(defn call-number! []
	(let [number (rand-nth @numbers-bag)]
	(swap! numbers-bag #(remove #{number} %))
	number))
	(defn in-bag? [number]
	((set @numbers-bag) number))
	(defn reset-numbers-bag! []
	(reset! numbers-bag (range 1 76)))

view raw caller.clj hosted with ❤ by GitHub

And it was done! See all the commits here if you want to follow the process (many intermediate steps happened on the REPL). You can find all the code on GitHub.

Summary

This experiment was a lot of fun because we got to play with both clojure.spec and clojure/test.check, and we learned a lot. While explaining what we did, I talked a bit about property-based testing and how I use REPL-driven development.

Thanks again to all my colleagues in Clojure Developers Barcelona!