Using coeffects in re-frame

Event handlers, collectively, provide the control logic in a re-frame application. Since the mutation of the application state is taken care of by re-frame, these event handlers can be pure functions that given the current value of the application state as first argument and the event (with its payload) as second argument, provide a new value for the application state.

This is important because the fact that event handlers are pure functions brings great advantages:

• Local reasoning, which decreases the cognitive load required to understand the code.
• Easier testing, because pure functions are much easier to test.
• Events replay-ability, you can imagine a re-frame application as a reduce (left fold) that proceeds step by step. Following this mental model, at any point in time, the value of the application state would be the result of performing a reduce over the entire collection of events dispatched in the application up until that time, being the combining function for this reduce the set of registered event handlers.

However, to build a program that does anything useful, it's inevitable to have some side-effects and/or side-causes. So, there would be cases in which event handlers won't be pure functions.

In this post, we'll focus on side-causes.

"Side-causes are data that a function, when called, needs but aren't in its argument list. They are hidden or implicit inputs."

Let's see an example:

	(ns visual-spelling.play.answer.handlers
	(:require
	[re-frame.core :as re-frame]
	[visual-spelling.db :as db]
	[visual-spelling.words-in-use :as words-in-use]))
	(re-frame/reg-event-db
	:word-ready-to-check
	(fn [db _]
	(update
	(db/save-user-answer
	(js/Date.now) ;; <- side-cause!!
	db)
	:words-in-use
	words-in-use/check-current-word)))

view raw impure-handler-due-to-side-cause.cljs hosted with ❤ by GitHub

The event handler for the :word-ready-to-check event, that we are registering using reg-event-db, is not pure because it's using the value returned by JavaScript's Date.now function instead of getting it through its arguments. To make matters worse, in this particular case, this side-cause also makes the event handler untestable because JavaScript's Date.now function returns a different value each time it's called.

To be able to test this event handler we'll have to somehow stub the function that produces the side-cause. In ClojureScript, there are many ways to do this (using with-redefs, with-bindings, a test doubles library like CircleCI's bond, injecting and stubbing the dependency, etc.).

In this example, we chose to make the dependency that the handler has on a function that returns the current timestamp explicit and inject it into the event handler which becomes a higher order function.

	(ns visual-spelling.play.answer.handlers
	(:require
	[visual-spelling.db :as db]
	[visual-spelling.words-in-use :as words-in-use]))
	(defn word-ready-to-check-handler [time-fn db _]
	(update (db/save-user-answer (time-fn) db)
	:words-in-use
	words-in-use/check-current-word))

view raw handler-time-fn-injected.cljs hosted with ❤ by GitHub

In the previous code example, notice that the event handler now receives as its first parameter a function that returns the current timestamp, time-fn.

And this would be how before registering that event handler with reg-event-db, we perform a partial application to inject JavaScript's Date.now function into it:

	(ns visual-spelling.play.answer.register-handlers
	(:require
	[re-frame.core :as re-frame]
	[visual-spelling.play.answer.handlers :as handlers]))
	(re-frame/reg-event-db
	:word-ready-to-check
	(partial handlers/word-ready-to-check-handler js/Date.now))

view raw registering-handler-time-fn-injected.cljs hosted with ❤ by GitHub

Using the same technique, we can now inject a stub of the time-fn function into the event handler in order to test it:

	(deftest test-word-ready-to-check-handler
	(testing "when answer is correct, saves current word and updates status"
	(let [time-fn (stub-time-fn :any-timestamp)
	right-answered-word (make-word
	:correct-word "o"
	:status :all-gaps-answered
	:parts (parts {:correct-answer "o"
	:options ["a" "o" "e" "u"]
	:user-answer "o"}))
	db (build-db :current-word right-answered-word)
	resulting-db (answer-handlers/word-ready-to-check-handler
	time-fn db :no-event)]
	(check-status
	resulting-db :all-right-answers)
	(check-results-contains
	resulting-db (results/correct "o" :any-timestamp)))))

view raw testing-handler-stubbed-time-fn.cljs hosted with ❤ by GitHub

This is the code of the home-made factory to create stubs for time-fn that we used in the previous test:

	(ns visual-spelling.tests-helpers)
	(defn stub-time-fn [& timestamps]
	(let [a-ts (atom timestamps)]
	(fn []
	(let [ts (first @a-ts)]
	(swap! a-ts rest)
	ts))))

view raw stub-time-fn.cljs hosted with ❤ by GitHub

We've seen how using a test double makes the event handler testable again, but at the price of introducing more complexity.

The bottom line problem is that the event handler is not a pure function. This makes us lose not only the easiness of testing, as we've seen, but also the rest of advantages cited before: local reasoning and events replay-ability. It would be great to have a way to keep event handlers pure, in the presence of side-effects and/or side-causes.

Since re-frame's 0.8.0 (2016.08.19) release, this problem has been solved by introducing the concept of effects and coeffects. Effects represent what your program does to the world (side-effects) while coeffects track what your program requires from the world (side-causes). Now we can write effectful event handlers that keep being pure functions.

Let's see how to use coeffects in the previous event handler example. As we said, coeffects track side-causes, (see for a more formal definition Coeffects The next big programming challenge).

At the beginning, we had this impure event handler:

	(ns visual-spelling.play.answer.handlers
	(:require
	[re-frame.core :as re-frame]
	[visual-spelling.db :as db]
	[visual-spelling.words-in-use :as words-in-use]))
	(re-frame/reg-event-db
	:word-ready-to-check
	(fn [db _]
	(update
	(db/save-user-answer
	(js/Date.now) ;; <- side-cause!!
	db)
	:words-in-use
	words-in-use/check-current-word)))

view raw impure-handler-due-to-side-cause.cljs hosted with ❤ by GitHub

then we wrote a testable but still impure version:

	(ns visual-spelling.play.answer.handlers
	(:require
	[visual-spelling.db :as db]
	[visual-spelling.words-in-use :as words-in-use]))
	(defn word-ready-to-check-handler [time-fn db _]
	(update (db/save-user-answer (time-fn) db)
	:words-in-use
	words-in-use/check-current-word))

view raw handler-time-fn-injected.cljs hosted with ❤ by GitHub

Thanks to coeffects, we can eliminate the side-cause, passing the timestamp input through the argument list of the event handler. The resulting event handler is a pure function:

	(ns visual-spelling.play.answer.handlers
	(:require
	[visual-spelling.db :as db]
	[visual-spelling.words-in-use :as words-in-use]))
	(defn word-ready-to-check-handler [cofx _]
	(let [db (update (db/save-user-answer (:timestamp cofx) (:db cofx))
	:words-in-use
	words-in-use/check-current-word)]
	{:db db}))

view raw handler-using-timestamp-coeffect-no-destructuring.cljs hosted with ❤ by GitHub

This event handler receives, as its first parameter, a map of coeffects, cofx, which contains two coeffects, represented as key/value pairs. One of them is the current timestamp which is associated to the :timestamp key, and the other one is the application state associated to the :db key. The second argument is, as in previous examples, the event with its payload.

The map of coeffects, cofx, is the complete set of inputs required by the event handler to perform its computation. Notice how the application state is just another coeffect.

This is the same event handler but using destructuring (which is how I usually write them):

	(ns visual-spelling.play.answer.handlers
	(:require
	[visual-spelling.db :as db]
	[visual-spelling.words-in-use :as words-in-use]))
	(defn word-ready-to-check-handler [{:keys [db timestamp]} _]
	(let [db (update (db/save-user-answer timestamp db)
	:words-in-use
	words-in-use/check-current-word)]
	{:db db}))

view raw handler-using-timestamp-coeffect.cljs hosted with ❤ by GitHub

How does this work? How does the coeffects map get passed to the event handler?

We need to do two things previously:

First, we register the event handler using re-frame's reg-event-fx instead of reg-event-db.

When you use reg-event-db to associate an event id with the function that handles it, its event handler, that event handler gets as its first argument, the application state, db.

While event handlers registered via reg-event-fx also get two arguments, the first argument is a map of coeffects, cofx, instead of the application state. The application state is still passed in the cofx map as a coeffect associated to the :db key, it's just another coeffect. This is how the previous pure event handler gets registered:

	(ns visual-spelling.play.answer.register-handlers
	(:require
	[re-frame.core :as re-frame]
	[visual-spelling.play.answer.handlers :as handlers]))
	(re-frame/reg-event-fx
	:word-ready-to-check
	[(re-frame/inject-cofx :timestamp)] ;; <- interceptor injects coeffect
	handlers/word-ready-to-check-handler)

view raw reg-event-using-coeffect.cljs hosted with ❤ by GitHub

Notice the second parameter passed to reg-event-fx. This is an optional parameter which is a vector of interceptors. Interceptors are functions that wrap event handlers implementing middleware by assembling functions, as data, in a collection. They "can look after cross-cutting concerns helping to "factor out commonality, hide complexity and introduce further steps into the 'Derived Data, Flowing' story promoted by re-frame".

In this example, we are passing an interceptor created using re-frame's inject-cofx function which returns an interceptor that will load a key/value pair (coeffect id/coeffect value) into the coeffects map just before the event handler is executed.

Second, we factor out the coeffect handler, and then register it using re-frame's reg-cofx. This function associates a coeffect id with the function that injects the corresponding key/value pair into the coeffects map. This function is known as the coeffect handler. For this example, we have:

	(ns visual-spelling.play.answer.register-handlers
	(:require
	[re-frame.core :as re-frame]))
	(re-frame/reg-cofx
	:timestamp
	(fn [cofx _]
	(assoc cofx :timestamp (js/Date.now))))

view raw timestamp-coeffect.cljs hosted with ❤ by GitHub

Since the event handler is now a pure function, it becomes very easy to test it:

	(deftest test-word-ready-to-check-handler
	(testing "when answer is correct, saves current word and updates status"
	(let [right-answered-word (make-word
	:correct-word "o"
	:status :all-gaps-answered
	:parts (parts {:correct-answer "o"
	:options ["a" "o" "e" "u"]
	:user-answer "o"}))
	cofx {:db (build-db :current-word right-answered-word)
	:timestamp :any-timestamp}
	resulting-fx (answer-handlers/word-ready-to-check-handler
	cofx :no-event)]
	(check-status
	resulting-fx :all-right-answers)
	(check-results-contains
	resulting-fx (results/correct "o" :any-timestamp)))))

view raw test-word-ready-to-check-handler.cljs hosted with ❤ by GitHub

Notice how we don't need to use tests doubles anymore in order to test it. Thanks to the use of coeffects, the event handler is a pure function, and we can just pass any timestamp to it in its arguments.

More importantly, we've also regained the advantages of local reasoning and events replay-ability that comes with having pure event handlers.

In future posts, we'll see how we can do something similar using effects to keep events handlers pure in the presence of side-effects.

Interesting Talk: "ClojureScript Made Easy"

I've just watched this very interesting talk by Jonathan Boston

• ClojureScript Made Easy

Interesting Talk: "Leaving side effects aside"

I've just watched this wonderful talk by Juanma Serrano

• Leaving side effects aside

These are the slides.

Refactoring tests using builder functions in Clojure/ClojureScript

Using literals in your tests can have some advantages, such as, readability and traceability.

While this is true when the data are simple, it's less so when the data are nested, complex structures. In that case, using literals can hinder refactoring and thus become an obstacle to adapting to changes.

The problem with using literals for complex, nested data is that the knowledge about how to build such data is spread all over the tests. There are many tests that know about the representation of the data.

In that scenario, nearly any change in the representation of those data will have a big impact on the tests code because it will force us to change many tests.

This is an example of a test using literals, (from a ClojureScript application using re-frame, I'm developing with Francesc), to prepare the application state (usually called db in re-frame):

	(deftest test-right-and-wrong-word-handlers
	(let [co-fx {:db
	{:language :catalan
	:words-in-use {:current-word {:correct-word "ess"
	:answer-ts :any-ts
	:parts [(text/make "e")
	(tests-helpers/answered-gap "ss" [] 1 "ss")]}
	:current-word-index 1
	:visited-words {}
	:words {"first-word" {}
	"current-word" {:description "current-word"
	:parts ["e"
	{:correct-answer "ss"
	:options ["c", "ç", "k", "s", "ss", "z", "q"]}]}
	"next-word" {:description "next-word"
	:parts ["e"
	{:correct-answer "ss"
	:options ["c", "ç", "k", "s", "ss", "z", "q"]}]}}
	:words-keys ["first-word" "current-word" "next-word"]}}}]
	(testing "that right word handler selects the next word"
	(check-current-word-is-new
	(answer-handlers/right-word-handler co-fx :no-event)
	:description "next-word" :current-word-index 2))
	(testing "that wrong word handler repeats the same word"
	(check-current-word-is-new
	(answer-handlers/wrong-word-handler co-fx :no-event)
	:description "current-word" :current-word-index 1))))

view raw using-literals.cljs hosted with ❤ by GitHub

As you can see, this test knows to much about the structure of db.

There were many other tests doing something similar at some nesting level of the db.

To make things worse, at that moment, we were still learning a lot about the domain, so the structure of the db was suffering changes with every new thing we learned.

The situation was starting to be painful, since any refactoring provoke many changes in the tests, so we decided to fix it.

What we wanted was a way to place all the knowledge about the representation of the db in just one place (i.e., remove duplication), so that, in case we needed to change that representation, the impact of the change would be absorbed by changing only one place.

A nice way of achieving this goal in object-oriented code, and at the same time making your tests code more readable, is by using test data builders which use the builder pattern, but how can we do these builders in Clojure?

Option maps or function with keyword arguments are a nice alternative to traditional builders in dynamic languages such as Ruby or Python.

In Clojure we can compose functions with keyword arguments to get very readable builders that also hide the representation of the data.

We did TDD to write these builder functions. These are the tests for one of them, the db builder:

	(ns visual-spelling.test-helpers.db-builder-test
	(:require
	[cljs.test :refer-macros [deftest testing is]]
	[visual-spelling.test-helpers.db-helpers :as db-helpers]
	[visual-spelling.test-helpers.db-builder :refer [make-db]]
	[visual-spelling.db :as db]))
	(deftest test-db-builder
	(is (= (make-db) db/default-db))
	(is (= (make-db :results :some-results)
	(assoc db/default-db :results :some-results)))
	(is (= (make-db :language :some-language)
	(assoc db/default-db :language :some-language)))
	(let [db (make-db :words {:some-word {} :another-word {}})]
	(is (= (db-helpers/extract-from-db db :words)
	{:some-word {} :another-word {}}))
	(is (= (set (db-helpers/extract-from-db db :words-keys))
	(set [:some-word :another-word]))))
	(is (= (db-helpers/extract-from-db
	(make-db :current-word-index :some-index) :current-word-index)
	:some-index))
	(is (= (db-helpers/extract-from-db
	(make-db :current-word :some-word) :current-word)
	:some-word))
	(is (= (make-db :words-in-use :some-words-in-use)
	(assoc db/default-db :words-in-use :some-words-in-use)))
	(is (= (make-db :words-per-session :some-words-per-session)
	(assoc db/default-db :words-per-session :some-words-per-session)))
	(is (= (make-db :visited-words :some-:visited-words)
	(assoc db/default-db :visited-words :some-:visited-words))))

view raw db_builder_test.cljs hosted with ❤ by GitHub

and this is the db builder code:

	(ns visual-spelling.test-helpers.db-builder
	(:require
	[visual-spelling.db :as db]))
	(defn- property-used? [v]
	(not= v ::not-used))
	(defn- assoc-if-used [m k v]
	(if (property-used? v)
	(assoc m k v)
	m))
	(defn make-words-in-use
	[default-word-in-use words-in-use words current-word-index current-word]
	(let [words-keys (if (property-used? words) (keys words) ::not-used)]
	(if (property-used? words-in-use)
	words-in-use
	(-> default-word-in-use
	(assoc-if-used :words-keys words-keys)
	(assoc-if-used :words words)
	(assoc-if-used :current-word-index current-word-index)
	(assoc-if-used :current-word current-word)))))
	(defn make-db
	[& {:keys [language words current-word-index current-word
	visited-words results words-in-use words-per-session]
	:or {language ::not-used
	words ::not-used
	current-word-index ::not-used
	current-word ::not-used
	visited-words ::not-used
	results ::not-used
	words-in-use ::not-used
	words-per-session ::not-used}}]
	(let [words-in-use (make-words-in-use
	(:words-in-use db/default-db)
	words-in-use
	words current-word-index current-word)]
	(-> db/default-db
	(assoc :words-in-use words-in-use)
	(assoc-if-used :language language)
	(assoc-if-used :results results)
	(assoc-if-used :words-per-session words-per-session)
	(assoc-if-used :visited-words visited-words))))

view raw db_builder.cljs hosted with ❤ by GitHub

which is using associative destructuring on the function's optional arguments to have a function with keyword arguments, and creating proper default values using the :or keyword (have a look at the material of a talk about destructuring I did some time ago for Clojure Barcelona Developers community).

After creating builder functions for some other data used in the project, our test started to read better and to be robust against changes in the structure of db.

For instance, this is the code of the test I showed at the beginning of the post, but now using builder functions instead of literals:

	(deftest test-right-and-wrong-word-handlers
	(let [db (make-db
	:language :catalan
	:current-word (make-word
	:correct-word "ess"
	:answer-ts :any-ts
	:parts (parts "e"
	{:correct-answer "ss"
	:user-answer "ss"}))
	:current-word-index 1
	:words {"first-word" (make-raw-word)
	"current-word" (make-raw-word
	:description "current-word"
	:parts (raw-parts
	"e"
	{:correct-answer "ss"
	:options ["c", "ç", "k", "s", "ss", "z", "q"]}))
	"next-word" (make-raw-word
	:description "next-word"
	:parts (raw-parts
	"e"
	{:correct-answer "ss"
	:options ["c", "ç", "k", "s", "ss", "z", "q"]}))})
	co-fx {:db db}]
	(testing "that right word handler selects the next word"
	(check-current-word-is-new
	(right-word-handler co-fx :no-event) :description "next-word" :current-word-index 2))
	(testing "that wrong word handler repeats the same word"
	(check-current-word-is-new
	(wrong-word-handler co-fx :no-event) :description "current-word" :current-word-index 1))))

view raw using-builders.cljs hosted with ❤ by GitHub

which not only hides the representation of db but also eliminates details that are not used in particular tests, while still being as easy to read, if not easier, than the version using literals.

We have seen how, by composing builder functions and using them in our tests, we managed to reduce the surface of the impact that changes in the representation of data might have on our tests. Builder functions absorb the impact of those changes, and enable faster refactoring, and, by doing so, enable adapting to changes faster.

Interesting Talk: "Dependency management in Clojure"

I've just watched this great talk by Holger Schauer

• Dependency management in Clojure

Interesting Talk: "Growing Clojure systems. From functions to microservices"

I've just watched this very interesting talk by Marcin Bilski

• Growing Clojure systems. From functions to microservices

Interesting Interview: "Objects, Functions, Virtual Machines, IDEs and More"

I've just watched this very interesting interview with Dave Thomas and Erik Meijer

• Objects, Functions, Virtual Machines, IDEs and More

Interesting Talk: "Native mobile apps with ClojureScript"

I've just watched this very interesting talk by Viktor Eriksson

• Native mobile apps with ClojureScript