Showing posts with label Learning. Show all posts
Showing posts with label Learning. Show all posts

Tuesday, August 23, 2022

Example of role tests in JavaScript with Jest

In this post we’ll show our last example applying the concept of role tests, this time in JavaScript using Jest. Have a look at our previous posts on this topic.

This example comes from a deliberate practice session we did recently with some developers from Audiense with whom we’re doing Codesai’s Practice Program in JavaScript twice a month.

Similar to what we did in our previous example of role tests in Java, we wrote the following tests to develop two different implementations of the TransactionsRepository port while solving the Bank Kata: the InMemoryTransactionsRepository and the NodePersistTransactionRepository.

These are their tests, respectively:

As what happened in our previous post, both tests contain the same test cases since both tests document and protect the contract of the same role, TransactionsRepository, which InMemoryTransactionsRepository and NodePersistTransactionRepository implement.

Again we’ll use the concept of role tests to remove that duplication, and make the contract of the role we are implementing more explicit.

Although Jest does not have something equivalent or similar to the RSpec’s shared examples functionality we used in our previous example in Ruby, we can get a very similar result by composing functions.

First, we wrote the behavesLikeATransactionRepository function. This function contains all the test cases that document the role and protect its contract, and receives as a parameter a testContext object containing methods for all the operations that will vary in the different implementations of this integration test.

Notice that in the case of Jest we are using composition, whereas we used inheritance in the case of Junit.

Then, we called the behavesLikeATransactionRepository function from the previous tests and implemented a particular version of the methods of the testContext object for each test.

This is the new code of InMemoryTransactionsRepositoryTest:

And this is the new code of NodePersistTransactionRepository after the refactoring:

This new version of the tests not only reduces duplication, but also makes explicit and protects the behaviour of the TransactionsRepository role. It also makes less error prone the process of adding a new implementation of TransactionsRepository because just by using the behavesLikeATransactionRepository function, you’d get a checklist of the behaviour you need to implement in order to ensure substitutability, i.e., to ensure the Liskov Substitution Principle is not violated.

These role tests using composition are also more readable than the Junit ones, in my opinion at least :)

Acknowledgements.

I’d like to thank Audiense’s deliberate practice group for working with us on this kata, and my colleague Rubén Díaz for co-facilitating the practice sessions with me.

Thanks to my Codesai colleagues for reading the initial drafts and giving me feedback.

References.

Simple example of property-based testing

Introduction.

We were recently writing tests to characterise a legacy code at a client that was being used to encrypt and decrypt UUIDs using a cipher algorithm. We have simplified the client’s code to remove some distracting details and try to highlight the key ideas we’d like to transmit.

In this post we’ll show how we used parameterized tests to test the encryption and decryption functions, and how we applied property-based testing to explore possible edge cases using a very useful pattern to discover properties.

Finally, we’ll discuss the resulting tests.

Testing the encryption and decryption functions.

We started by writing examples to fixate the behaviour of the encryption function. Notice how we used parameterized tests[1] to avoid the duplication that having a different test for each example would have caused:

Then we wrote more parameterized tests for the decryption function. Since the encryption and decryption functions are inverses of each other, we could use the same examples that we had used for the encryption function. Notice how the roles of being input and expected output change for the parameters of the new test.

Exploring edge cases.

You might wonder why we wanted to explore edge cases. Weren’t the parameterized tests enough to characterise this legacy code?

Even though, the parameterized tests that we wrote for both functions were producing a high test coverage, coverage only “covers” code that is already there. We were not sure if there could be any edge cases, that is, inputs for which the encryption and decryption functions might not behave correctly. We’ve found edge cases in the past even in code with 100% unit test coverage.

Finding edge cases is hard work which sometimes might require exploratory testing by specialists. It would be great to automatically explore some behaviour to find possible edge cases so we don’t have to find them ourselves or some QA specialists. In some cases, we can leverage property-based testing to do that exploration for us[2].

One of the most difficult parts of using property-based testing is finding out what properties we should use. Fortunately, there are several approaches or patterns for discovering adequate properties to apply property-based testing to a given problem. Scott Wlaschin wrote a great article in which he explains several of those patterns[3].

It turned out that the problem we were facing matched directly to one of the patterns described by Wlaschin, the one he calls “There and back again”(also known as “Round-tripping” or “Symmetry” pattern).

According to Wlaschin “There and back again” properties “are based on combining an operation with its inverse, ending up with the same value you started with”. As we said before, in our case the decryption and encryption functions were inverses of each other so the “There and back again” pattern was likely to lead us to a useful property.

There and back again diagram.

Once we knew which property to use it was very straightforward to add a property-based test for it. We used the jqwik library. We like it because it has very good documentation and it is integrated with Junit.

Using jqwik functions we wrote a generator of UUIDs (have a look at the documentation on how to generate customised parameters), we then wrote the decrypt_is_the_inverse_of_encrypt property:

By default jqwik checks the property with 1000 new randomly generated UUIDs every time this test runs. This allows us to gradually explore the set of possible examples in order to find edge cases that we have not considered.

Discussion.

If we examine the resulting tests we may think that the property-based tests have made the example-based tests redundant. Should we delete the example-based tests and keep only the property-based ones?

Before answering this question, let’s think about each type of test from different points of view.

Understandability.

Despite being parameterized, it’s relatively easy to see which inputs and expected outputs are used by the example-based tests because they are literal values provided by the generateCipheringUuidExamples method. Besides, this kind of testing was more familiar to the team members.

In contrast, the UUID used by the property-based tests to check the property is randomly generated and the team was not familiar with property-based testing.

Granularity.

Since we are using a property that uses the “There and back again” pattern, if there were an error, we wouldn’t know whether the problem was in the encryption or the decryption function, not even after the shrinking process[4]. We’d only know the initial UUID that made the property fail.

This might not be so when using other property patterns. For instance, when using a property based on the “The test oracle” pattern, we’d know the input and the actual and expected outputs in case of an error.

In contrast, using example-based testing it would be very easy to identify the location of the problem.

Confidence, thoroughness and exploration.

The example-based tests specify behaviour using concrete examples in which we set up concrete scenarios, and then check whether the effects produced by the behaviour match what we expect. In the case of the cipher, we pass an input to the functions and assert that their output is what we expect. The testing is reduced just to the arbitrary examples we were able to come up with, but there’s a “gap between what we claim to be testing and what we’re actually testing”[5]: why those arbitrary examples? Does the cipher behave correctly for any possible example?

Property-based testing “approach the question of correctness from a different angle: under what preconditions and constraints (for example, the range of input parameters) should the functionality under test lead to particular postconditions (results of a computation), and which invariants should never be violated in the course?”[6]. With property-based testing we are not limited to the arbitrary examples we were able to come up with as in example-based testing. Instead, property-based testing gives us thoroughness and the ability to explore because it’ll try to find examples that falsify a property every time the test runs. I think this ability to explore makes them more dynamic.

Implementation independence.

The example-based tests depend on the implementation of the cypher algorithm, whereas the property-based tests can be used for any implementation of the cypher algorithm because the decrypt_is_the_inverse_of_encrypt property is an invariant of any cipher algorithm implementation. This makes the property-based tests ideal to write a role test[7] that any valid cipher implementation should pass.

Explicitness of invariants.

In the case of the cipher there’s a relationship between the encryption and decryption functions: they are inverses of each other.

This relationship might go completely untested using example-based testing if we use unrelated examples to test each of the functions. This means there could be changes to any of the functions that may violate the property while passing the independent separated tests of each function.

In the parameterized example-based tests we wrote, we implicitly tested this property by using the same set of examples for both functions just changing the roles of input and expected output for each test, but this is limited to the set of examples.

With property-based testing we are explicitly testing the relation between the two functions and exploring the space of inputs to try to find one that falsifies the property of being inverses of each other.

Protection against regressions.

Notice that, in this case, If we deleted the example-based tests and just kept the property-based test using the decrypt_is_the_inverse_of_encrypt property, we could introduce a simple regression by implementing both functions, encrypt and decrypt, as the identity function. That obviously wrong implementation would still fulfil the decrypt_is_the_inverse_of_encrypt property, which means that the property-based test using decrypt_is_the_inverse_of_encrypt property is not enough on its own to characterise the desired behaviour and protect it against regressions. We also need to at least add example-based testing for one of the cipher functions, either encrypt or decrypt. Notice that this might happen for any property based on “There and back again” pattern. This might not hold true for different contexts and property patterns.

What we did.

Given the previous discussion, we decided to keep both example-based and property-based tests in order to gain exploratory power while keeping familiarity, granularity and protection against regressions.

Summary.

We’ve shown a simple example of how we applied JUnit 5 parameterized tests to test the encryption and decryption functions of a cipher algorithm for UUIDs.

Then we showed a simple example of how we can use property-based testing to explore our solution and find edge cases. We also talked about how discovering properties can be the most difficult part of property-based testing, and how there are patterns that can be used to help us to discover them.

Finally, we discussed the resulting example-based and property-based tests from different points of view.

We hope this post will motivate you to start exploring property-based testing as well. If you want to learn more, follow the references we provide and start playing. Also have a look at the other posts exploring property-based testing in our blog we have written in the past.

Acknowledgements.

I’d like to thank my Codesai colleagues for reading the initial drafts and giving me feedback.

Notes.

[1] The experience of writing parameterized tests using JUnit 5 is so much better than it used to be with JUnit 4!

[2] Have a look at this other post in which I describe how property-based tests were able to find edge cases that I had not contemplated in a code with 100% test coverage that had been written using TDD.

[3] Scott Wlaschin’s article, Choosing properties for property-based testing, is a great post in which he manages to explain the patterns that have helped him the most, to discover the properties that are applicable to a given problem. Besides the “There and back again” pattern, I’ve applied the [“The test oracle”][https://fsharpforfunandprofit.com/posts/property-based-testing-2/#the-test-oracle] on several occasions. Some time ago, I wrote a post explaining how I used it to apply property-based testing to an implementation of a binary search tree. Another interesting article about the same topic is Property-based Testing Patterns, Sanjiv Sahayam.

[4] “Shrinking is the mechanism by which a property-based testing framework can be told how to simplify failure cases enough to let it figure out exactly what the minimal reproducible case is.” from chapter 8 of Fred Hebert’s PropEr Testing online book

[5] From David MacIver’s In praise of property-based testing post. According to David MacIver “the problem with example-based tests is that they end up making far stronger claims than they are actually able to demonstrate”.

[6] From Johannes Link’s Know for Sure with Property-Based Testing post.

[7] Have a look at our recent post about role tests.

References.

Monday, February 14, 2022

Notes on LSP from Agile Principles, Practices and Patterns book

I continue sharing my notes on SOLID to prepare the ground for the upcoming The Big Branch Theory Podcast episode about Liskov Substitution Principle.

Ok, so these are the raw notes I took while reading the chapter devoted to Liskov Substitution Principle (LSP) in Robert C. Martin’s Agile Principles, Practices and Patterns in C# book (I added some personal annotations between brackets):

  • “The primary mechanisms behind the OCP are abstraction and polymorphism” <- [but in some languages inheritance is needed to have polymorphism]

  • ”.. questions addressed by the LSP”

    • “What are the desgin rules that govern this particular use of inheritance hierarchies?”

    • “What are the characteristics of the best inheritance hierarchies?”

    • “What are the traps that will cause us to create hierarchies that do not conform to OCP?”

  • LSP -> “Subtypes must be substitutable for their base types”

  • “Violating LSP often results in the use of runtime type checking in a manner that grossly violates OCP”

  • “a violation of LSP is a latent violation of OCP”

  • “… more subtle way of violating LSP” -> “.. use of IS-A relationship is sometimes thought to be one of the fundamental techniques of OOA, a term frequently used but seldom defined […]. However this kind of thinking can lead to some subtle yet significant problems. Generally, these problems are not foreseen until we see them in code”

  • Invariants -> “those properties that must always be true regardless of state”

  • “… when the creation of a derived class causes us to make changes to the base class, it often implies that the design is faulty”

  • “Validity is not intrinsic”

    • “LSP leads us to a very important conclusion: A model, viewed in isolation, cannot be meaningfully validated. The validity of a model can be expressed only in terms of its clients.

    • “When considering whether a particular design is appropriate, one cannot simply view the solution in isolation. One must view it in terms of the reasonable assumptions made by the users of that design”

    • “Therefore, as with all other principles, it is often best to defer all but the most obvious LSP violations until the related fragility has been smelled”

  • “IS-A is about behavior” -> “… it is behavior that software is really all about. LSP makes it clear than in OOD, the IS-A relationship pertains to behavior that can be reasonably assumed and that clients depend on” <- [related to behavioural approach to modelling shown in David West’s Object Thinking, or Rebecca Wirfs-Brock’s Designing Object-Oriented Software]

  • “How do you know what your clients will really expect? There is a technique for making those reasonable assumptions explicit and thereby enforcing LSP…” -> Design By Contract (DBC)

  • “Using DBC, the author of a class explicitly states the contract of that class. The contract informs the author of any client code if the behaviors that can be relied on. The contract is specified by declaring preconditions and postconditions for each method. The preconditions must be true for the method to execute. On completion, the method guarantees that the postconditions are true.”

  • “…the rule for preconditions and postconditions of derivatives, as stated by Meyer, is: ‘A routine redeclaration [in a derivative] may only replace the original precondition by one equal or weaker, and the original postcondition by one equal or stronger’” <- “X is weaker than Y if X does not enforce all the constraints of Y. It does not matter how many new constraints X enforces”

  • “In other words, when using an object through its base class interface, the user knows only the preconditions and postconditions of the base class. Thus, derived objects must not expect such users to obey preconditions that are stronger than those required by the base class. Also, derived classes must conform to all the postconditions of the base. That is, their behaviors and outputs must not violate any of the constraints established for the base class.” “Users of the base class must not be confused by the output of the derived class.” <- [a form of the Least Astonishment Principle]

  • “Contracts can […] be specified by writing unit tests. By thoroughly testing the behavior of a class, the unit tests make the behavior of the class clear. Authors of the client code will want to review the unit tests in order to know what to reasonably assume about the classes they are using”

  • “It’s a big advantage not to have to know or care what kind of [sth] you are using. It means that the programmer can decide which kind of [sth] is needed in each particular instance, and none of the client functions will be affected by that decision”

  • “…the problem with conventions: they have to be continually resold to each developer”

  • “There are occasions when it is more expedient to accept a subtle flaw in polymorphic behavior than to attempt to manipulate the design into complete LSP compliance. Accepting compromise instead of pursuing perfection is an engineering trade-off. A good engineer learns when compromise is more profitable that perfection. However, conformance to LSP should not be surrendered lightly. The guarantee that a subclass will always work where its base classes are used is a powerful way to manage complexity. Once it is forsaken we must consider each subclass individually.

  • “Factoring is a powerfull tool. If qualities can be factored out of two subclassses, there is the distinct possibility that other classes will show up later that need those qualities too”

  • Rebecca Wirfs-Brock, on factoring:”

    • “We can state that if a set of classes all support a common responsibility, they should inherit that responsibility from a common superclass”

    • “If a common superclass does not already exist, create one, and move the common responsibility to it. After all such a class is demonstrably useful […]. Isn’t it conceivable that a later extension of your system might add a new subclass that will support those same responsibilitties in a new way? This new superclass will probably be an abstract class”

  • “Some simple heuristics can give you some clues about LSP violations. These heuristics all have to do with derivative classes that somehow remove functionality from their base class. A derivative that does less that its base is usually not substitutable for that base and therefore violates LSP” <- “The presence of degenerate functions in derivatives is not always indicative of an LSP violation, but it’s worth looking at them when they occur” [see Refused Bequest code smell]

  • “The OCP is at the heart of many of the claims made for OOD. […] The LSP is one of the prime enablers of OCP”

  • “The substitutability of subtypes allows a module, expressed in terms of a base type, to be extensible without modification. That substitutability must be sth that developers can depend on implicitly. Thus, the contract of the base type has to be well and prominently understood, if not explicitly enforced, by the code”

  • “The […] IS-A is too broad to act as a definition of a subtype. The true definition of a subtype is substitutable, where substitutability is defined by either an explicit or implicit contract

This post was also published in Codesai's blog.

Saturday, January 30, 2021

Notes on OCP from Agile Principles, Practices and Patterns book

Some time ago I wrote a post sharing my notes on SRP from Agile Principles, Practices and Patterns book because I was making an effort to get closer to the sources of some object-oriented concepts. I didn’t continue sharing my notes on SOLID because I thought they might not be interesting for our readers. However, seeing the success of the Single responsibility ¿Principle? episode of The Big Branch Theory Podcast for which I used my notes on SRP, I’ve decided to share the rest of my notes on SOLID on Codesai’s blog.

Ok, so these are the raw notes I took while reading the chapter devoted to Open-closed Principle in Robert C. Martin’s Agile Principles, Practices and Patterns in C# book (I added some personal annotations between brackets):

  • OCP -> “Software entities (classes, modules, functions, etc) should be open for extension but closed for modification”

  • “When a single change to a program results in a cascade of changes to dependent modules, the design smells of fragility” <- [No local consequences. See Beck’s Local Consequences principle from Implementation Patterns] “OCP advises us to refactor the system so that further changes of that kind will not cause more modifications. If OCP is applied well, further changes of that kind are achieved by adding new code, not by changing old code that already works”

  • “It’s possible to create abstractions that are fixed and yet represent an unbounded group of possible behaviors”

  • “[A module that uses such abstractions] can be closed for modification, since it depends on an abstraction that is fixed. Yet the behavior of the module can be extended by creating new derivatives of the abstraction”

  • “Abstract classes are more closely associated to their clients than to the classes that implement them” <- [related with Separated Interface from Fowler’s P of EAA]

  • ”[Strategy and Template Method patterns] are the most common ways to satisfy OCP. They represent a clear separation of generic functionality from the detailed implementation of that functionality”

  • Anticipation

    • “[When a] program conforms to OCP. It is changed by adding code rather than by changing existing code”

    • “In general no matter how “closed” a module is, there will always be some kind of change against which it is not closed”

    • “Since closure <- [“closure” here means protection against a given axis of variation or change, see Craig Larman’s Protected Variation: The Importance of Being Closed] can’t be complete, it must be strategic. That is the designer must choose the kinds of changes against which to close the design, must guess at the kinds of changes that are most likely, and then construct abstractions to protect against those changes.”

    • “This is not easy. It amounts to making educated guesses about the likely kinds of changes that the application will suffer over time.” “Also conforming to OCP is expensive. It takes development time and money to create the appropriate abstractions. These abstractions also increase the complexity of the software design”

    • “We want to limit the application of OCP to changes that are likely”

    • “How do we know which changes are likely? We do the appropriate research, we ask the appropriate questions, and we use our experience and common sense.” <- [also requires knowing about the domain. A bit easier to predict in technological boundaries. Listen to the conversation in Single Responsibility ¿Principle?] podcast] “And after all that, we wait until the changes happen!” <- [see Yagni] “We don’t want to load the design with lots of unnecessary abstractions. Rather we want to wait until we need the abstraction and then put them in”

  • “Fool me once”

    • “… we initially write our code expecting it not to change. When a change occurs, we implement the abstractions that protect us from future changes of that kind.” <- [One heuristic: we get to OCP through refactoring to avoid Speculative Generality. Most useful heuristic in unknown territory.]

    • “If we decide to take the first bullet, it is to our advantage to get the bullets flying early and frequently. We want to know what changes are likely before we are very far down the development path. The longer we wait to find out what kind of changes are likely, the more difficult it will be to create the appropriate abstractions.”

    • “Therefore, we need to stimulate changes”

      • “We write tests first” -> “testing is one kind of usage of the system. By writing tests first, we force the system to be testable. Therefore, changes in testability will not surprise us later. We will have built the abstractions that make the system testable. We are likely to find that many of these abstractions will protect us from other kinds of changes later.” <- [incrementally (tests “right after”) might also work]

      • “We use short development cycles”

      • “We develop features before infrastructure and frequently show those feature to stake-holders”

      • “We develop the most important features first”

      • “We release the software early and often”

  • “Closure is based on abstraction”

  • “Using a data-driven approach to achieve closure” <- [OCP is not only an OO principle, see Craig Larman’s Protected Variation: The Importance of Being Closed for more]

    • “If we must close the derivatives […] from knowledge of one another, we can use a table-driven approach”

    • “The only item that is not closed against [the rule that involves] the various derivatives is the table itself. An that table can be placed in its own module, separated from all the other modules, so that changes to it do not affect any of the other modules”

  • “In many ways the OCP is at the heart of OOD.”

  • “Yet conformance to [OCP] is not achieved by using an OOP language. Nor is it a good idea to apply rampant abstraction to every part of the application. Rather it requires a dedication on the part of the developers to apply abstraction only to those parts of the program that exhibit frequent change. <- [applying Beck’s Rate of Change principle from Implementation Patterns]”

  • “Resisting premature abstraction is as important as abstraction itself <- [related to Sandi Metz’s “duplication is far cheaper than the wrong abstraction”]”

For me getting closer to the sources of SOLID principles was a great experience that helped me to remove illusions of knowledge I had developed due to the telephone game effect caused by initially learning about SOLID through blog posts and talks. I hope these notes on OCP might be useful to you as well, and motivate you to read a bit closer to some of the sources.

This post was also published in Codesai's blog.There's also a previous version of this post in this blog.

Tuesday, January 26, 2021

Saturday, January 23, 2021

Solving the Beverages Prices Refactoring kata (2): limiting the options on the menu

Introduction.

This is the second and last post in a series of posts showing a possible solution to the Beverages Prices Refactoring kata that I recently developed with some people from Women Tech Makers Barcelona with whom I’m working through Codesai’s Practice Program twice a month.

In the previous post we introduced a design based on composition that fixed the Combinatorial Explosion code smell and produced a flexible solution applying the decorator design pattern. There was a potential problem in that solution because the client code, the code that needed to find out the price of the beverages, knew[1] too much about how to create and compose beverages and supplements.

Have a look, for instance, at the following line new WithCream(new WithMilk(new Coffee())). It knows about three classes and how they are being composed. In the case of this kata, that might not be a big problem, since the client code is only comprised of a few tests, but, in a larger code base, this problem might spread across numerous classes generating a code smell known as Creation Sprawl[2] In this post, we’ll try to reduce client knowledge of concrete component and decorator classes and their composition by encapsulating all the creational knowledge behind a nice, readable interface that we’ll keep all the complexity of combining the supplements (decorators) and beverages (components) hidden from the client code.

Another more subtle problem with this design based on composition has to do with something that we have lost: the fact that not all combinations of beverages and supplements were allowed on the menu. That knowledge was implicitly encoded in the initial inheritance hierarchy, and disappeared with it. In the current design we can dynamically create any combination of beverages and supplements, including those that were not included in the original menu, like, for instance a tea with cinnamon, milk and cream (doing new WithCinnamon(new WithCream(new WithMilk(new Tea())))) which you might find delicious :).We’ll also explore possible ways to recover that limitation of options.

We’ll start by examining some creational patterns that are usually applied along with the decorator design pattern.

Would the Factory pattern help?

In order to encapsulate the creational code and hide its details from client code, we might use the factory pattern described by Joshua Kerievsky in his Refactoring to Patterns. A factory is a class that implements one or more Creation Methods. A Creation Method is a static or non-static method the creates and returns an object instance[3].

We might apply the Encapsulate Classes with Factory refactoring[4] to introduce a factory class with an interface which provided a creation method for each entry on the menu, that is, it would have a method for making coffee, another one for making tea, another one for making coffee with milk, and so on, and so forth.

Before starting to refactor, let’s think a bit about the consequences of introducing this pattern to assess if it will leave us in a better design spot or not. At first sight, introducing the factory pattern seems to simplify client code and reduce the overall coupling because it encapsulates all the creational logic hiding the complexity related to composing decorators and components behind its interface which solves the first problem we discussed in the introduction. The second one, limiting the combinations of beverages and supplements to only the ones available on the menu, is solved just by limiting the methods in the interface of the factory.

However, it would also create a maintenance problem somehow similar to the initial combinatorial explosion code smell we were trying to avoid when we decided to introduce the decorator design pattern. As we said, the interface of the factory would have a method for each combination of beverages and supplements available on the menu. This means that to add a new supplement we’d have to multiply the number of Creation methods in the interface of the factory by two. So, we might say that, introducing the factory pattern, we’d get an interface suffering from a combinatorial explosion of methods[5].

Knowing that, we might conclude that a solution using the factory pattern would be interesting only when having a small number of options or if we didn’t expect the number of supplements to grow. As we said in the previous post, we think it likely that we’ll be required to add new supplements so we prefer a design that is easy to evolve along the axis of change of adding new supplements[6]. This means the factory pattern is not the way to go for us this time because it’s not flexible enough for our current needs. We’ll have to explore more flexible alternatives[7].

Let’s try using the Builder design pattern.

The builder design pattern is often used for constructing complex and/or composite objects[8]. Using it we might create a nice readable interface to compose the beverages and supplements bit by bit. Like the factory pattern, a builder would encapsulate the complexity of combining decorators from the client code. Unlike the factory pattern, a builder allows to construct the composite following a varying process. It’s this last characteristic that will avoid the combinatorial explosion of methods that made us discard the factory pattern.

In this case you can introduce the builder by applying the Encapsulate Composite with Builder. Let’s have a look at how we implemented it:

Notice how we keep the state of the partially composed object and apply the decorations incrementally until it’s returned by the make method. Notice also how the beverage is the initial state in the process of creating the composite object.

These are the tests after introducing the builder design pattern:

Notice the fluent interface that we decided for the builder. Although a fluent interface is not a requirement to write a builder, we think it reads nice.

As we said before, using a builder does not suffer from the combinatorial explosion of methods that the factory pattern did. The builder design pattern is more flexible than the factory pattern which makes it more suitable for composing components and decorators.

Still, our success is only partial because the builder can create any combination of beverages and supplements. A drawback of using a builder instead of a factory is usually that clients require to have more domain knowledge. In this case, the current solution forces the client code to hold a bit of domain knowledge: it knows which combinations of beverages and supplements are available on the menu.

We’ll fix this last problem in the next section.

A hybrid solution combining factory and builder patterns.

Let’s try to limit the possible combinations of beverages and supplements to the options on the menu by combining the creation methods of the factory pattern and the builder design pattern.

To do so, we added to BeverageMachine the creation methods, coffee, tea and hotChocolate, that create different builders for each type of beverage: CoffeeBuilder, TeaBuilderand HotChocolateBuilder, respectively. Each of the builders has only the public methods to select the supplements which are possible on the menu for a given type of beverage.

Notice that we chose to write the builders as inner classes of the BeverageMachine class. They could have been independent classes, but we prefer inner classes because the builders are only used by BeverageMachine and this way they don’t appear anywhere else.

This is the first design that solves the problem of limiting the possible combinations of beverages and supplements to only the options on the menu. It still encapsulates the creational logic and still reads well. In fact the tests haven’t changed at all because BeverageMachine’s public interface is exactly the same.

However, the new builders present duplication: the code related to supplements that can be used with different beverages and the code in the make method.

What is different for the clients that call the coffee method and the clients that call the tea or hotChocolate methods are the public methods they can use on each builder, that is, their interfaces. When we had only one builder, we had an interface with methods that were not interesting for some of its clients.

By having three builders we segregated the interfaces so that no client was forced to depend on methods it does not use[9]. However we didn’t need to introduce classes to segregate the interfaces, we could have just used, well, interfaces. As we’ll see in the next section using interfaces would have avoided the duplication in the implementation of the builders.

Segregating interfaces better by using interfaces.

As we said, instead of directly using three different builder classes, it’s better to use three interfaces, one for each kind of builder. That would also comply with the Interface Segregation Principle, but, using the interfaces helps us avoid having duplicated code in the implementation of the builders, because we can write only one builder class, Beverage Machine, that implements the three interfaces.

Notice how, in the creation methods, we feed the base beverage into BeverageMachine through its constructor, and how each of those creation methods return the appropriate interface. Notice also that BeverageMachine’s public interface remains the same, so this refactor won’t change the tests at all. You can check the resulting builder interfaces in Gist: TeaBuilder, HotChocolateBuilder and CoffeeBuilder.

Conclusions.

In this last post of the series dedicated to the Beverages Prices Refactoring kata, we’ve explored different ways to avoid creation sprawl, reduce coupling with client code and reduce implicit creational domain knowledge in client code. In doing so, we have learned about and applied several creational patterns (factory pattern, and builder design pattern), and some related refactorings. We have also used some design principles (such as coupling, open-closed principle or interface segregation principle), and code smells (such as combinatorial explosion or creation sprawl) to judge different solutions and guide our refactorings.

Aknowledgements.

I’d like to thank the WTM study group, and especially Inma Navas for solving this kata with me. Thanks also to my Codesai colleagues and to Inma Navas for reading the initial drafts and giving me feedback and to Amelia Hallsworth for the picture.

Notes.

[1] Knowledge here means coupling or connascence.

[2] Creation Sprawl is a code smell that happens when the knowledge for creating an object is spread out across numerous classes, so that creational responsibilities are placed in classes that should now be playing any role in object creation. This code smell was described by Joshua Kerievsky in his Refactoring to Patterns book.

[3] Don’t confuse the Factory Pattern with design patterns with similar names like Factory method pattern or Abstract factory pattern. These two design patterns are creational patterns described in the Design Patterns: Elements of Reusable Object-Oriented Software book.

A Factory Method is “a non-static method that returns a base class or an interface type and that is implemented in a hierarchy to enable polymorphic creation” whereas an Abstract Factory is “an interface for creating fqamiñlies of related or dependent objects without specifying their concrete classes”.

In the Factory Pattern a Factory is “any class that implements one or more Creation Methods” which are “static or non-static methods that create and return an object instance”. This definition is more general. Every Abstract Factory is a Factory (but not the other way around), and every Factory Method is a Creation Method (but not necessarily the reverse). Creation Method also includes what Martin Fowler called “factory method” in Refactoring (which is not the Factory Method design pattern) and Joshua Bloch called “static factory” (probably a less confusing name than Fowler’s one) in Effective Java.

[4] Presented in the fourth chapter of Refactoring to Patterns that is dedicated to Creational Patterns. [5] If you remember the previous post, before introducing the decorator design pattern, we suffered from a combinatorial explosion of classes (adding a new supplement meant multiplying the number of classes by two). Now, the factory interface (its public methods) would suffer a combinatorial explosion of methods.

[6] In other words: that it’s [protected against that type of variation (https://www.martinfowler.com/ieeeSoftware/protectedVariation.pdf).

[7] This is common when working with creational patterns. All of them encapsulate knowledge about which concrete classes are used and hide how instances of these classes are created and put together, but some are more flexible than others. It’s usual to start using a Factory pattern and evolve toward the other creational patterns as we realize more flexibility is needed.

[8] We have devoted several posts to builders: Remove data structures noise from your tests with builders, Refactoring tests using builder functions in Clojure/ClojureScript, In a small piece of code, The curious case of the negative builder.

[9] Following the Interface Segregation Principle that states that “no client should be forced to depend on methods it does not use”.

References.

Books

Articles

Monday, November 30, 2020

Solving the Beverages Prices Refactoring kata (1): composition over inheritance

Introduction.

We are going to show a possible solution to the Beverages Prices Refactoring kata that we developed recently with some people from Women Tech Makers Barcelona with whom I’m doing Codesai’s Practice Program twice a month.

The Beverages Prices Refactoring kata shows an example of inheritance gone astray. The initial code computes the price of the different beverages that are sold in a coffee house. There are some supplements that can be added to those beverages. Each supplement increases the price a bit. Not all combinations of drinks and supplements are possible.

As part of the kata, we are asked to add an optional cinnamon supplement that costs 0.05€ to all our existing catalog of beverages. We are also advised to refactor the initial code a bit before introducing the new feature. Let’s see why.

The initial code.

To get an idea of the kind of problem we are facing, we’ll have a look at the code. There are 8 files: a Beverage interface and 7 classes, one for each type of beverage and one for each allowed combination of beverages and supplements.

Initial code files

A closer look reveals that the initial design uses inheritance and polymorphism to enable the differences in computing the price of each allowed combination of beverage and supplements. This is the inheritance hierarchy:

Class diagram showing the inheritance hierarchy in the initial code

If that diagram is not enough to scare you, have a quick look at the unit tests of the code:

First, we make the change easy[1].

Given the current design, if we decided to add the new feature straight away, we would end up with 14 classes (2 times the initial number of classes). If you think about it, this would happen for each new supplement we decided to add. It would be the same for each new supplement we were required to add: we would be forced to double the number of classes, that means that to add n supplements more would mean multiplying the initial number of classes by 2n.

This exponential growth in the number of classes is a typical symptom of a code smell called Combinatorial Explosion[2]. In this particular case the problem is caused by using inheritance to represent the pricing of beverages plus supplements.

In order to introduce the new cinnamon supplement, we thought sensible to do a bit of preparatory refactoring first in order to remove the Combinatorial Explosion code smell. The recommended refactoring for this code smell is Replace Inheritance with Delegation which leads to a code that uses composition instead of inheritance to avoid the combinatorial explosion. If all the variants[3] (supplements) keep the same interface, we’d be creating an example of the decorator design pattern[4].

The decorator pattern provides an alternative to subclassing for extending behavior. It involves a set of decorator classes that wrap concrete components and keep the same interface that the concrete components. A decorator changes the behavior of a wrapped component by adding new functionality before and/or after delegating to the concrete component.

Applying the decorator pattern design to compute the pricing of beverages plus supplements, the beverages would correspond to the concrete components, tea, coffee and hot chocolate; whereas the supplements, milk and cream would correspond to the decorators.

New design class diagram using the decorator design pattern

We can obtain the behavior for a given combination of supplements and beverage by composing supplements (decorators) with a beverage (base component).

For instance, if we had the following WithMilk decorator for the milk supplement pricing,

we would compose it with a Tea instance to create the behavior that computes the price of a tea with milk:

A nice thing about decorators is that, since they have the same interface as the component they wrap, they are transparent for the client code[6] which never has to know that it’s dealing with a decorator. This allows us to pass them around in place of the wrapped object which makes it possible to compose behaviors using as many decorators as we like. The following example shows how to compose the behavior for computing the price of a coffee with milk and cream[7].

After applying the Replace Inheritance with Delegation refactoring we get to a design that uses composition instead of inheritance to create all the combinations of supplements and beverages. This fixes the Combinatorial Explosion code smell.

You can have a look at the rest of the test after this refactoring in this gist.

Then, we make the easy change.

Once we had the new design based in composition instead of inheritance in place, adding the requested feature is as easy as creating a new decorator that represents the cinnamon supplement pricing:

code after adding the new feature

Remember that using the initial design adding this feature would have involved multiplying by two the number of classes.

What have we gained and lost with this refactoring?

After the refactoring we have a new design that uses the decorator design pattern. This is a flexible alternative design to subclassing for extending behavior that allows us to add behavior dynamically to the objects wrapped by the decorators.

Thanks to this runtime flexibility we managed to fix the Combinatorial Explosion code smell and that made it easier to add the new feature. Now, instead of multiplying the number of cases by two, adding a new supplement only involves adding one new decorator class that represents the new supplement pricing. This new design makes the client code open-closed to the axis of change of adding new supplements.

On the flip side, we have introduced some complexity[8] related to creating the different compositions of decorators and components. At the moment this complexity is being managed by the client code (notice the chains of news in the tests snippets above).

There’s also something else that we have lost in the process. In the initial design only some combinations of beverages and supplements were allowed. This fact was encoded in the initial inheritance hierarchy. Now with our decorators we can dynamically add any possible combination of beverages and supplements.

All in all, we think that the refactoring leaves us in a better spot because we’ll be likely required to add new supplements, and there are usual improvements we can make to the design to isolate the client code from the kind of complexity we have introduced.

Conclusion.

We have shown an example of preparatory refactoring to make easier the addition of a new feature, and learned about the Combinatorial Explosion code smell and how to fix it using the decorator design pattern to get a new design in which we have protected the client code[9] against variations involving new supplements.

In a future post we will show how to encapsulate the creation of the different compositions of decorators and components using builders and/or factories to hide that complexity from client code, and show how we can limit again the allowed combinations that are part of the menu.

Aknowledgements.

I’d like to thank the WTM study group, and especially Inma Navas for solving this kata with me.

Thanks to my Codesai colleagues and Inma Navas for reading the initial drafts and giving me feedback and to Lisa Fotios for her picture.

Notes.

[1] This and the next header come from Kent Beck’s quote:

“For each desired change, make the change easy (warning: this may be hard), then make the easy change”

[2] You can find this code smell described in Bill Wake’s wonderful Refactoring Workbook.

[3] In this context variant means a variation in behavior. For instance, each derived class in a hierarchy is a variant of the base class.

[4] Have a look at the chapter devoted to the decorator design pattern in the great Head First Design Patterns. It’s the most didactic and fun explanation of the pattern I’ve ever found. This kata is heavily inspired in the example used in that chapter to explain the pattern.

[5] We could have also solved this problem composing functions instead of objects, but we wanted to practice with objects in this solution of the kata. That might be an interesting exercise for another practice session.

[6] In this context client code means code that uses objects implementing the interface that both the components and decorators implement, which in the kata would correspond to the Beverage interface.

[7] Also known as a “cortado leche y leche” in Gran Canaria :)

[8] Complexity is most often the price we pay for flexibility. That’s why we should always assess if the gains are worth the price.

[9] Protected variations is another way to refer to the open-closed principle. I particularly prefer that way to refer to this design principle because I think it is expressed in a way that relates less to object orientation. Have a look at Craig Larman’s great article about it: Protected Variation: The Importance of Being Closed

References.

Monday, July 29, 2019

Playing Fizzbuzz with property-based testing

Introduction.

Lately, I’ve been playing a bit with property-based testing.

I practised doing the FizzBuzz kata in Clojure and used the following constraints for fun[1]:

  1. Add one property at a time before writing the code to make the property hold.
  2. Make the failing test pass before writing a new property.

The kata step by step.

To create the properties, I partitioned the first 100 integers according to how they are transformed by the code. This was very easy using two of the operations on sets that Clojure provides (difference and intersection).

The first property I wrote checks that the multiples of 3 but not 5 are Fizz:

and this is the code that makes that test pass:

Next, I wrote a property to check that the multiples of 5 but not 3 are Buzz (I show only the new property for brevity):

and this is the code that makes the new test pass:

Then, I added a property to check that the multiples of 3 and 5 are FizzBuzz:

which was already passing with the existing production code.

Finally, I added a property to check that the rest of numbers are just casted to a string:

which Id made pass with this version of the code:

The final result.

These are the resulting tests where you can see all the properties together:

You can find all the code in this repository.

Conclusions.

It was a lot of fun doing this kata. It is a toy example that didn’t make me dive a lot into clojure.check’s generators documentation because I could take advantage of Clojure’s set functions to write the properties.

I think the resulting properties are quite readable even if you don’t know Clojure. On the other hand, the resulting implementation is probably not similar to the ones you’re used to see, and it shows Clojure’s conciseness and expressiveness.

Footnotes:

[1] I'm not saying that you should property-based testing with this constraints. They probably make no sense in real cases. The constraints were meant to make it fun.

Saturday, May 25, 2019

The curious case of the negative builder

Recently, one of the teams I’m coaching at my current client, asked me to help them with a problem, they were experiencing while using TDD to add and validate new mandatory query string parameters[1]. This is a shortened version (validating fewer parameters than the original code) of the tests they were having problems with:

and this is the implementation of the QueryStringBuilder used in this test:

which is a builder with a fluid interface that follows to the letter a typical implementation of the pattern. There are even libraries that help you to automatically create this kind of builders[2].

However, in this particular case, implementing the QueryStringBuilder following this typical recipe causes a lot of problems. Looking at the test code, you can see why.

To add a new mandatory parameter, for example sourceId, following the TDD cycle, you would first write a new test asserting that a query string lacking the parameter should not be valid.

So far so good, the problem comes when you change the production code to make this test pass, in that momento you’ll see how the first test that was asserting that a query string with all the parameters was valid starts to fail (if you check the query string of that tests and the one in the new test, you’ll see how they are the same). Not only that, all the previous tests that were asserting that a query string was invalid because a given parameter was lacking won’t be “true” anymore because after this change they could fail for more than one reason.

So to carry on, you’d need to fix the first test and also change all the previous ones so that they fail again only for the reason described in the test name:

That’s a lot of rework on the tests only for adding a new parameter, and the team had to add many more. The typical implementation of a builder was not helping them.

The problem we’ve just explained can be avoided by chosing a default value that creates a valid query string and what I call “a negative builder”, a builder with methods that remove parts instead of adding them. So we refactored together the initial version of the tests and the builder, until we got to this new version of the tests:

which used a “negative” QueryStringBuilder:

After this refactoring, to add the sourceId we wrote this test instead:

which only carries with it updating the valid method in QueryStringBuilder and adding a method that removes the sourceId parameter from a valid query string.

Now when we changed the code to make this last test pass, no other test failed or started to have descriptions that were not true anymore.

Leaving behind the typical recipe and adapting the idea of the builder pattern to the context of the problem at hand, led us to a curious implementation, a “negative builder”, that made the tests easier to maintain and improved our TDD flow.

Acknowledgements.

Many thanks to my Codesai colleagues Antonio de la Torre and Fran Reyes, and to all the colleagues of the Prime Services Team at LIFULL Connect for all the mobs we enjoy together.

Footnotes:

[1] Currently, this validation is not done in the controller anymore. The code showed above belongs to a very early stage of an API we're developing.
[2] Have a look, for instance, at lombok's' @Builder annotation for Java.

Tuesday, May 14, 2019

Killing mutants to improve your tests

At my current client we’re working on having a frontend architecture for writing SPAs in JavaScript similar to re-frame’s one: an event-driven bus with effects and coeffects for state management[1] (commands) and subscriptions using reselect’s selectors (queries).

One of the pieces we have developed to achieved that goal is reffects-store. Using this store, React components can be subscribed to given reselect’s selectors, so that they only render when the values in the application state tracked by the selectors change.

After we finished writing the code for the store, we decided to use mutation testing to evaluate the quality of our tests. Mutation testing is a technique in which, you introduce bugs, (mutations), into your production code, and then run your tests for each mutation. If your tests fail, it’s ok, the mutation was “killed”, that means that they were able to defend you against the regression caused by the mutation. If they don’t, it means your tests are not defending you against that regression. The higher the percentage of mutations killed, the more effective your tests are.

There are tools that do this automatically, stryker[2] is one of them. When you run stryker, it will create many mutant versions of your production code, and run your tests for each mutant (that’s how mutations are called in stryker’s’ documentation) version of the code. If your tests fail then the mutant is killed. If your tests passed, the mutant survived. Let’s have a look at the the result of runnning stryker against reffects-store’s code:

Notice how stryker shows the details of every mutation that survived our tests, and look at the summary the it produces at the end of the process.

All the surviving mutants were produced by mutations to the store.js file. Having a closer look to the mutations in stryker’s output we found that the functions with mutant code were unsubscribeAllListeners and unsubscribeListener. After a quick check of their tests, it was esay to find out why unsubscribeAllListeners was having surviving mutants. Since it was a function we used only in tests for cleaning the state after each test case was run, we had forgotten to test it.

However, finding out why unsubscribeListener mutants were surviving took us a bit more time and thinking. Let’s have a look at the tests that were exercising the code used to subscribe and unsubscribe listeners of state changes:

If we examine the mutations and the tests, we can see that the tests for unsubscribeListener are not good enough. They are throwing an exception from the subscribed function we unsubscribe, so that if the unsubscribeListener function doesn’t work and that function is called the test fails. Unfortunately, the test passes also if that function is never called for any reason. In fact, most of the surviving mutants that stryker found above have are variations on that idea.

A better way to test unsubscribeListener is using spies to verify that subscribed functions are called and unsubscribed functions are not (this version of the tests includes also a test for unsubscribeAllListeners):

After this change, when we run stryker we got the following output:

No mutants survived!! This means this new version of the tests is more reliable and will protect us better from regressions than the initial version.

Mutation testing is a great tool to know if you can trust your tests. This is event more true when working with legacy code.

Acknowledgements.

Many thanks to Mario Sánchez and Alex Casajuana Martín for all the great time coding together, and thanks to Porapak Apichodilok for the photo used in this post and to Pexels.

Footnotes:

[1] See also reffects which is the synchronous event bus with effects and coeffects we wrote to manage the application state.
[2] The name of this tool comes from a fictional Marvel comics supervillain Willian Stryker who was obsessed with the eradication of all mutants.

Sunday, April 7, 2019

The Beverages Prices Refactoring kata: a kata to practice refactoring away from an awful application of inheritance.

I created the Beverages Prices Refactoring kata for the Deliberate Practice Program I’m running at Lifull Connect offices in Barcelona (previously Trovit). Its goal is to practice refactoring away from a bad usage of inheritance.

The code computes the price of the different beverages that are sold in a coffe house. There are some supplements that can be added to those beverages. Each supplement increases the price a bit. Not all combinations of drinks and supplements are possible.

Just having a quick look at the tests of the initial code would give you an idea of the kind of problems it might have:

If that’s not enough have a look at its inheritance hierarchy:

To make things worse, we are asked to add an optional cinnamon supplement that costs 0.05€ to all our existing catalog of beverages. We think we should refactor this code a bit before introducing the new feature.

We hope you have fun practicing refactoring with this kata.