Showing posts with label Connascence. Show all posts
Showing posts with label Connascence. Show all posts

Sunday, October 18, 2020

Sleeping is not the best option

Introduction.

Some time ago we were developing a code that stored some data with a given TTL. We wanted to check not only that the data was stored correctly but also that it expired after the given TTL. This is an example of testing asynchronous code.

When testing asynchronous code we need to carefully coordinate the test with the system it is testing to avoid running the assertion before the tested action has completed[1]. For example, the following test will always fail because the assertion in line 30 is checked before the data has expired:

In this case the test always fails but in other cases it might be worse, failing intermittently when the system is working, or passing when the system is broken. We need to make the test wait to give the action we are testing time to complete successfully and fail if this doesn’t happen within a given timeout period.

Sleeping is not the best option.

This is an improved version of the previous test in which we are making the test code wait before the checking that the data has expired to give the code under test time to run:

The problem with the simple sleeping approach is that in some runs the timeout might be enough for the data to expire but in other runs it might not, so the test will fail intermittently; it becomes a flickering test. Flickering tests are confusing because when they fail, we don’t know whether it’s due to a real bug, or it is just a false positive. If the failure is relatively common, the team might start ignoring those tests which can mask real defects and completely destroy the value of having automated tests.

Since the intermittent failures happen because the timeout is too close to the time the behavior we are testing takes to run, many teams decide to reduce the frequency of those failures by increasing the time each test sleeps before checking that the action under test was successful. This is not practical because it soon leads to test suites that take too long to run.

Alternative approaches.

If we are able to detect success sooner, succeeding tests will provide rapid feedback, and we only have to wait for failing tests to timeout. This is a much better approach than waiting the same amount of time for each test regardless it fails or succeeds.

There are two main strategies to detect success sooner: capturing notifications[2] and polling for changes.

In the case we are using as an example, polling was the only option because redis didn’t send any monitoring events we could listen to.

Polling for changes.

To detect success as soon as possible, we’re going to probe several times separated by a time interval which will be shorter than the previous timeout. If the result of a probe is what we expect the test pass, if the result we expect is not there yet, we sleep a bit and retry. If after several retries, the expected value is not there, the test will fail.

Have a look at the checkThatDataHasExpired method in the following code:

By polling for changes we avoid always waiting the maximum amount of time. Only in the worst case scenario, when consuming all the retries without detecting success, we’ll wait as much as in the just sleeping approach that used a fixed timeout.

Extracting a helper.

Scattering ad hoc low level code that polls and probes like the one in checkThatDataHasExpired throughout your tests not only make them difficult to understand, but also is a very bad case of duplication. So we extracted it to a helper so we could reuse it in different situations.

What varies from one application of this approach to another are the probe, the check, the number of probes before failing and the time between probes, everything else we extracted to the following helper[3]:

This is how the previous tests would look after using the helper:

Notice that we’re passing the probe, the check, the number of probes and the sleep time between probes to the AsyncTestHelpers::assertWithPolling function.

Conclusions.

We showed an example in Php of an approach to test asynchronous code described by Steve Freeman and Nat Pryce in their Growing Object-Oriented Software, Guided by Tests book. This approach avoids flickering test and produces much faster test suites than using a fixed timeout. We also showed how we abstracted this approach by extracting a helper function that we are reusing in our code.

We hope you’ve found this approach interesting. If you want to learn more about this and several other techniques to effectively test asynchronous code, have a look at the wonderful Growing Object-Oriented Software, Guided by Tests book[4].

Acknowledgements.

Thanks to my Codesai colleagues for reading the initial drafts and giving me feedback and to Chrisy Totty for the lovely cat picture.

Notes.

[1] This is a nice example of Connascence of Timing (CoT). CoT happens when when the timing of the execution of multiple components is important. In this case the action being tested must run before the assertion that checks its observable effects. That's the coordination we talk about. Check our post about Connascence to learn more about this interesting topic.
[2] In the capturing notifications strategy the test code "observes the system by listening for events that the system sends out. An event-based assertion waits for an event by blocking on a monitor until gets notified or times out", (from Growing Object-Oriented Software, Guided by Tests book).
Some time ago we developed some helpers using the capturing notifications strategy to test asynchronous ClojureScript code that was using core.async channels. Have a look at, for instance, the expect-async-message assertion helper in which we use core.async/alts! and core.async/timeout to implement this behaviour. The core.async/alts! function selects the first channel that responds. If that channel is the one the test code was observing we assert that the received message is what we expected. If the channel that responds first is the one generated by core.async/timeout we fail the test. We mentioned these async-test-tools in previous post: Testing Om components with cljs-react-test.
[3] Have a look at the testing asynchronous systems examples in the GOOS Code examples repository for a more object-oriented implementation of helper for the polling for changes strategy, and also examples of the capturing notifications strategy.
[4] Chapter 27, Testing Asynchronous code, contains a great explanation of the two main strategies to test asynchronous code effectively: capturing notifications and polling for changes.

References.

Sunday, September 3, 2017

Data clumps, primitive obsession and hidden tuples

During the writing of a recent post about connascence for Codesai's blog some of us were discussing whether we could consider a data clump a form of Connascence of Meaning (CoM) or not. In the end, we agreed that data clumps are indeed a form of CoM and that introducing a class for the missing abstraction reduces their connascence to Connascence of Type (CoT).

I had wondered in the past why we use a similar refactoring to eliminate both primitive obsession and data clump smells. Thinking about them from the point of view of connascence has helped me a lot to understand why.

I had also an alternative and curious reasoning to get to the same conclusion, in which a data clump gets basically reduced to an implicit form of primitive obsession. The reasoning is as it follows:

The concept of primitive obsession might be extended to consider the collections that a given language offers as primitives. In such cases, encapsulating the collection reifies a new concept that might attract code that didn't have where to "live" and thus was scattered all over. So far so good.

From the point of view of connascence, primitive obsession is a form of CoM that we transform into CoT by introducing a new type and then we might find Connascence of Algorithm (CoA) that we'd remove by moving the offending code inside the new type.

The composing elements of a data clump only make sense when they go together. This means that they're conceptually (but implicitly) grouped. In this sense a data clump could be seen as a "hidden or implicit tuple".

Having this "hidden collection" in mind is now easier to see how closely related the data clump and primitive obsession smells are. In this sense, we remove a data clump by encapsulating a collection, its "implicit or hidden tuple", inside a new class. Again, from the point of view of connascence, this encapsulation reduces CoM to CoT and might make evident some CoA that will make us move some behavior into the new class that becomes a value object.

This "implicit tuple" reasoning helped me to make more explicit the mental process that was leading me to end up doing very similar refactorings to remove both code smells.

However I think that CoM unifies both cases much more easily than relating the two smells.

The fact that the collection (the grouping of the elements of a data clump) is implicit also makes it more difficult to recognize a data clump as CoM in the first place. That's why I think that a data clump is a more implicit example of CoM than primitive obsession, and, thus, we might consider its CoM to be stronger than the primitive obsession's one.

A curious reasoning, right?

Tuesday, August 22, 2017

In a small piece of code

This post appeared originally on Codesai’s Blog.

In a previous post we talked about positional parameters and how they can suffer from Connascence of Position, (CoP). Then we saw how, in some cases, we might introduce named parameters to remove the CoP and transform it into Connascence of Name, (CoN), but always being careful to not hiding cases of Connascence of Meaning, (CoM). In this post we’ll focus on languages that don’t provide named parameters and see different techniquess to remove the CoP.

Let’s see an example of a method suffering of CoP:

In languages without named parameters (the example is written in Java), we can apply a classic[1] refactoring technique, Introduce Parameter Object, that can transform CoP into CoN. In this example, we introduced the ClusteringParameters object:

which eliminates the CoP transforming it into CoN:

In this particular case, all the parameters passed to the function were semantically related, since they all were parameters of the clustering algorithm, but in many other cases all the parameters aren’t related. So, as we saw in our previous post for named parameters, we have to be careful of not accidentally sweeping hidden CoM in the form of data clumps under the rug when we use the Introduce Parameter Object refactoring.

In any case, what it’s clear is that introducing a parameter object produces much less expressive code than introducing named parameters. So how to gain semantics while removing CoP in languages without named parameters?

One answer is using fluent interfaces[2] which is a technique that is much more common than you think. Let’s have a look at the following small piece of code:

This is just a simple test. However, just in this small piece of code, we can find two examples of removing CoP using fluent interfaces and another example that, while not removing CoP, completely removes its impact on expressiveness. Let’s look at them with more detail.

The first example is an application of the builder pattern using a fluent interface[3].

Applying the builder pattern provides a very specific[4] internal DSL that we can use to create a complex object avoiding CoP and also getting an expressiveness comparable or even superior to the one we’d get using named parameters.

In this case we composed two builders, one for the SafetyRange class:

and another for the Alarm class:

Composing builders you can manage to create very complex objects in a maintanable and very expressive way.

Let’s see now the second interesting example in our small piece of code:

This assertion using hamcrest is so simple that the JUnit alternative is much clearer:

but for more than one parameter the JUnit interface starts having problems:

Which one is the expected value and which one is the actual one? We never manage to remember…

Using hamcrest removes that expressiveness problem:

Thanks to the semantics introduced by hamcrest[6], it’s very clear that the first parameter is the actual value and the second parameter is the expected one. The internal DSL defined by hamcrest produces declarative code with high expressiveness. To be clear hamcrest is not removing the CoP, but since there are only two parameters, the degree of CoP is very low[7]. The real problem of the code using the JUnit assertion was its low expressiveness and using hamcrest fixes that.

For us it’s curious to see how, in trying to achieve expressiveness, some assertion libraries that use fluent interfaces have (probably not being aware of it) eliminate CoP as well. See this other example using Jasmine:

Finally, let’s have a look at the last example in our initial small piece of code which is also using a fluent interface:

This is Mockito’s way of defining a stub for a method call. It’s another example of fluent interface which produces highly expressive code and avoids CoP.

Summary.

We started seeing how, in languages that don’t allow named parameters, we can remove CoP by applying the Introduce Parameter Object refactoring and how the resulting code was much less expressive than the one using the Introducing Named Parameters refactoring. Then we saw how we can leverage fluent interfaces to remove CoP while writing highly expressive code, mentioned internal DSLs and showed you how this technique is more common that one can think at first by examining a small piece of code.

References.

Books.

Posts.

Footnotes:

[2] Of course, fluent interfaces are also great in languages that provide named parameters.
[3] Curiosly there're alternative ways to implement the builder pattern that use options maps or named parameters. Some time ago we wrote about an example of using the second way: Refactoring tests using builder functions in Clojure/ClojureScript.
[4] The only purpose of that DSL is creating one specific type of object.
[5] For us the best explanation of the builder pattern and how to use it to create maintanable tests is in chapter 22, Constructing Complex Test Data, of the wonderful Growing Object-Oriented Software Guided by Tests book.
[6] hamcrest is a framework for writing matcher objects allowing 'match' rules to be defined declaratively. We love it!

Monday, July 31, 2017

Two examples of Connascence of Position

This post appeared originally on Codesai’s Blog.

As we saw in our previous post about connascence, Connascence of Position (CoP) happens when multiple components must be adjacent or appear in a particular order. CoP is the strongest form of static connascence, as shown in the following figure.
Connascence forms sorted by descending strength (from Kevin Rutherford's XP Surgery).
A typical example of CoP appears when we use positional parameters in a method signature because any change in the order of the parameters will force to change all the clients using the method.

The degree of the CoP increases with the number of parameters, being zero when we have only one parameter. This is closely related with the Long Parameters List smell.
In some languages, such as Ruby, Clojure, C#, Python, etc, this can be refactored by introducing named parameters (see Introduce Named Parameter refactoring)[1].

Now changing the order of parameters in the signature of the method won’t force the calls to the method to change, but changing the name of the parameters will. This means that the resulting method no longer presents CoP. Instead, now it presents Connascence of Name, (CoN), which is the weakest form of static connascence, so this refactoring has reduced the overall connascence.

The benefits don’t end there. If we have a look at the calls before and after the refactoring, we can see how the call after introducing named parameters communicates the intent of each parameter much better. Does this mean that we should use named parameters everywhere?

Well, it depends. There’re some trade-offs to consider. Positional parameters produce shorter calls. Using named parameters gives us better code clarity and maintainability than positional parameters, but we lose terseness[2]. On the other hand, when the number of parameters is small, a well chosen method name can make the intent of the positional arguments easy to guess and thus make the use of named parameters redundant.

We should also consider the impact that the degree and locality of each instance of CoP[3] can have on the maintainability and communication of intent of each option. On one hand, the impact on maintainability of using positional parameters is higher for public methods than for private methods (even higher for published public methods)[4]. On the other hand, a similar reasoning might be made about the intent of positional parameters: the positional parameters of a private method in a cohesive class might be much easier to understand than the parameters of a public method of a class a client is using, because in the former case we have much more context to help us understand.

The communication of positional parameters can be improved a lot with the parameter name hinting feature provided by IDEs like IntelliJ. In any case, even though they look like named parameters, they still are positional parameters and have CoP. In this sense, parameter name hinting might end up having a bad effect in your code by reducing the pain of having long parameter lists.

Finally, moving to named parameters can increase the difficulty of applying the most frequent refactoring: renaming. Most IDEs are great renaming positional parameters, but not all are so good renaming named parameters.

A second example.

There are also cases in which blindly using named parameters can make things worse. See the following example:

The activate_alarm method presents CoP, so let’s introduce named parameters as in the previous example:

We have eliminated the CoP and now there’s only CoN, right?

In this particular case, the answer would be no. We’re just masking the real problem which was a Connascence of Meaning (CoM) (a.k.a. Connascence of Convention). CoM happens when multiple components must agree on the meaning of specific values[5]. CoM is telling us that there might be a missing concept or abstraction in our domain. The fact that the lower_threshold and higher_threshold only make sense when they go together, (we’re facing a case of data clump), is an implicit meaning or convention on which different methods sharing those parameters must agree, therefore there’s CoM.

We can eliminate the CoM by introducing a new class, Range, to wrap the data clump and reify the missing concept in our domain reducing the CoM to Connascence of Type (CoT)[6]. This refactoring plus the introduction of named parameters leaves with the following code:

This refactoring is way better than only introducing named parameters because it does not only provides a bigger coupling reduction by going down in the scale from from CoP to CoT instead of only from CoP to CoM, but also it introduces more semantics by adding a missing concept (the Range object).

Later we’ll probably detect similarities[7] in the way some functions that receives the new concept are using it and reduce it by moving that behavior into the new concept converting it in a value object. It’s in this sense that we say that value objects attract behavior.

Summary.

We have presented two examples of CoP, a “pure” one and another one that was really hiding a case of CoM. We have related CoP and CoM with known code smells, (Long Parameters List, Data Clump and Primitive Obsession), and introduced refactorings to reduce their coupling and improve their communication of intent. We have also discussed a bit, about when and what we need to consider before applying these refactorings.

References.

Talks.

Books.

Posts.

Footnotes.

:
[1] For languages that don't allow named parameters, see the Introduce Parameter Object refactoring.
[3] See our previous post About Connascence.
[4] For instance, Sandi Metz recommends in her POODR book to "use hashes for initialization arguments" in constructors (this was the way of having named parameters before Ruby 2.0 introduced keyword arguments).
[5] Data Clump and Primitive Obsession smells are examples of CoM.
[6] Connascence of Type, (CoT), happens when multiple components must agree on the type of an entity.
[7] Those similarities in the use of the new concept are examples of Conascence of Algorithm which happens when multiple components must agree on a particular algorithm.

Thursday, January 26, 2017

About Connascence

This post appeared originally on Codesai’s Blog.

Lately at Codesai we’ve been studying and applying the concept of connascence in our code and even have done an introductory talk about it. We’d like this post to be the first of a series of posts about connascence.

 

1. Origin.

The concept of connascence is not new at all. Meilir Page-Jones introduced it in 1992 in his paper Comparing Techniques by Means of Encapsulation and Connascence. Later, he elaborated more on the idea of connascence in his What every programmer should know about object-oriented design book from 1995, and its more modern version (same book but using UML) Fundamentals of Object-Oriented Design in UML from 1999.
Ten years later, Jim Weirich, brought connascence back from oblivion in a series of talks: Grand Unified Theory of Software Design, The Building Blocks of Modularity and Connascence Examined. As we’ll see later in this post, he did not only bring connascence back to live, but also improved its exposition.
More recently, Kevin Rutherford, wrote a very interesting series of posts, in which he talked about using connascence as a guide to choose the most effective refactorings and about how connascence can be a more objective and useful tool than code smells to identify design problems[1].

 

2. What is connascence?

The concept of connascence appeared in a time, early nineties, when OO was starting its path to become the dominant programming paradigm, as a general way to evaluate design decisions in an OO design. In the previous dominant paradigm, structured programming, fan-out, coupling and cohesion were fundamental design criteria used to evaluate design decisions. To make clear what Page-Jones understood by these terms, let’s see the definitions he used:
Fan-out is a measure of the number of references to other procedures by lines of code within a given procedure.
Coupling is a measure of the number and strength of connections between procedures.
Cohesion is a measure of the “single-mindedness” of the lines of code within a given procedure in meeting the purpose of that procedure.
According to Page-Jones, these design criteria govern the interactions between the levels of encapsulation that are present in structured programming: level-1 encapsulation (the subroutine) and level-0 (lines of code), as can be seen in the following table from Fundamentals of Object-Oriented Design in UML.

Encapsulation levels and design criteria in structured programming.

However, OO introduces at least level-2 encapsulation, (the class), which encapsulates level-1 constructs (methods) together with attributes. This introduces many new interdependencies among encapsulation levels, which will require new design criteria to be defined, (see the following table from Fundamentals of Object-Oriented Design in UML).

Encapsulation levels and design criteria in OO.

Two of these new design criteria are class cohesion and class coupling, which are analogue to the structured programing’s procedure cohesion and procedure coupling, but, as you can see, there are other ones in the table for which there isn’t even a name.
Connascence is meant to be a deeper criterion behind all of them and, as such, it is a general way to evaluate design decisions in an OO design. This is the formal definition of connascence by Page-Jones:
Connascence between two software elements A and B means either
  1. that you can postulate some change to A that would require B to be changed (or at least carefully checked) in order to preserve overall correctness, or
  2. that you can postulate some change that would require both A and B to be changed together in order to preserve overall correctness.
In other words, there is connascence between two software elements when they must change together in order for the software to keep working correctly.
We can see how this new design criteria can be used for any of the interdependencies among encapsulation levels present in OO. Moreover, it can also be used for higher levels of encapsulation (packages, modules, components, bounded contexts, etc). In fact, according to Page-Jones, connascence is applicable to any design paradigm with partitioning, encapsulation and visibility rules[2].

 

3. Forms of connascence.

Page-Jones distinguishes several forms (or types) of connascence.
Connascence can be static, when it can be assessed from the lexical structure of the code, or dynamic, when it depends on the execution patterns of the code at run-time.
There are several types of static connascence:
  • Connascence of Name (CoN): when multiple components must agree on the name of an entity.
  • Connascence of Type (CoT): when multiple components must agree on the type of an entity.
  • Connascence of Meaning (CoM): when multiple components must agree on the meaning of specific values.
  • Connascence of Position (CoP): when multiple components must agree on the order of values.
  • Connascence of Algorithm (CoA): when multiple components must agree on a particular algorithm.
There are also several types of dynamic connascence:
  • Connascence of Execution (order) (CoE): when the order of execution of multiple components is important.
  • Connascence of Timing (CoTm): when the timing of the execution of multiple components is important.
  • Connascence of Value (CoV): when there are constraints on the possible values some shared elements can take. It’s usually related to invariants.
  • Connascence of Identity (CoI): when multiple components must reference the entity.
Another important form of connascence is contranascence which exists when elements are required to differ from each other (e.g., have different name in the same namespace or be in different namespaces, etc). Contranascence may also be either static or a dynamic.

 

4. Properties of connascence.

Page-Jones talks about two important properties of connascence that help measure its impact on maintanability:
  • Degree of explicitness: the more explicit a connascence form is, the weaker it is.
  • Locality: connascence across encapsulation boundaries is much worse than connascence between elements inside the same encapsulation boundary.
A nice way to reformulate this is using what it’s called the three axes of connascence[3]:

4.1. Degree.

The degree of an instance of connascence is related to the size of its impact. For instance, a software element that is connascent with hundreds of elements is likely to become a larger problem than one that is connascent to only a few.

4.2 Locality.

The locality of an instance of connascence talks about how close the two software elements are to each other. Elements that are close together (in the same encapsulation boundary) should typically present more, and higher forms of connascence than elements that are far apart (in different encapsulation boundaries). In other words, as the distance between software elements increases, the forms of connascence should be weaker.

4.3 Stregth.

Page-Jones states that connascence has a spectrum of explicitness. The more implicit a form of connascence is, the more time consuming and costly it is to detect. Also a stronger form of connascence is usually harder to refactor. Following this reasoning, we have that stronger forms of connascence are harder to detect and/or refactor. This is why static forms of connascence are weaker (easier to detect) than the dynamic ones, or, for example, why CoN is much weaker (easier to refactor) than CoP.
The following figure by Kevin Rutherford shows the different forms of connascence we saw before, but sorted by descending strength.

 Connascence forms sorted by descending strength (from Kevin Rutherford's XP Surgery).

 

5. Connascence, design principles and refactoring.

Connascence is simpler than other design principles, such as, the SOLID principles, Law of Demeter, etc. In fact, it can be used to see those principles in a different light, as they can be seen using more fundamental principles like the ones in the first chapter of Kent Beck’s Implementation Patterns book.
We use code smells, which are a collection of code quality antipatterns, to guide our refactorings and improve our design, but, according to Kevin Rutherford, they are not the ideal tool for this task[4]. Sometimes connascence might be a better metric to reason about coupling than the somewhat fuzzy concept of code smells.
Connascence gives us a more precise vocabulary to talk and reason about coupling and cohesion[5], and thus helps us to better judge our designs in terms of coupling and cohesion, and decide how to improve them. In words of Gregory Brown, “this allows us to be much more specific about the problems we’re dealing with, which makes it it easier to reason about the types of refactorings that can be used to weaken the connascence between components”.
It provides a classification of forms of coupling in a system, and even better, a scale of the relative strength of the coupling each form of connascence generates. It’s precisely that scale of relative strengths what makes connascence a much better guide for refactoring. As Kevin Rutherford says:
"because it classifies the relative strength of that coupling, connascence can be used as a tool to help prioritize what should be refactored first"
Connascence explains why doing a given refactoring is a good idea.

 

6. How should we apply connascence?

Page-Jones offers three guidelines for using connascence to improve systems maintanability:
  1. Minimize overall connascence by breaking the system into encapsulated elements.
  2. Minimize any remaining connascence that crosses encapsulation boundaries.
  3. Maximize the connascence within encapsulation boundaries.
According to Kevin Rutherford, the first two points conforms what he calls the Page-Jones refactoring algorithm[6].
These guidelines generalize the structured design ideals of low coupling and high cohesion and is applicable to OO, or, as it was said before, to any other paradigm with partitioning, encapsulation and visibility rules.
They might still be a little subjective, so some of us, prefer a more concrete way to apply connascence using, Jim Weirich’s two principles or rules:
  • Rule of Degree[7]: Convert strong forms of connascence into weaker forms of connascence.
  • Rule of Locality: As the distance between software elements increases, use weaker forms of connascence.

 

7. What’s next?

In future posts, we’ll see examples of concrete forms of conasscence relating them with design principles, code smells, and refactorings that might improve the design.

Footnotes:
[1] See Kevin Rutherford's great post The problem with code smells.
[2] This explains the titles Jim Weirich chose for his talks: Grand Unified Theory of Software Design and The Building Blocks of Modularity.
[4] Again see Kevin Rutherford's great post The problem with code smells.
[5] The concepts of coupling and cohesion can be hard to grasp, just see this debate about them Understanding Coupling and Cohesion hangout.
[6] See Kevin Rutherford's post The Page-Jones refactoring algorithm.
[7] Even though he used the word degree, he was actually talking about strength.

 

References.

Books.

Papers.

Talks.

Posts.

Others.

Sunday, May 31, 2015

Interesting Talk: "Red, Green, ... now what ?!"

I've just watched this wonderful practical demonstration of using connascence to guide the refactoring step of TDD by James Jeffries and Kevin Rutherford: These are the slides.

These other slides explain more about each type of connascence.