You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Starting with Kotest 5.7, the non-deterministic testing functions have moved to the `kotest-assertions-core` module, and
9
+
are available under the new package `com.sksamuel.kotest.assertions.nondeterministic`. The previous iterations of these
10
+
functions are still available, but deprecated.
11
+
:::
7
12
8
-
When testing non-deterministic code, a common use case is "I expect this code to pass after a short period of time".
13
+
Testing non-deterministic code can be hard. You might need to juggle threads, timeouts, race conditions, and the
14
+
unpredictability of when events are happening.
9
15
10
-
For example, if you were testing a IO operation, you might need to wait until the IO operation has flushed.
16
+
For example, if you were testing that an asynchronous file write was completed successfully, you need to wait until the
17
+
write operation has completed and flushed to disk.
11
18
12
-
Sometimes you can do a Thread.sleep but this is isn't ideal as you need to set a sleep threshold high enough so that it
13
-
won't expire prematurely on a slow machine. Plus it means that your test will sit around waiting on the timeout even if
14
-
the code completes quickly on a fast machine.
19
+
Some common approaches to these problems are:
15
20
16
-
Or you can roll a loop and sleep and retry and sleep and retry, but this is just boilerplate slowing you down.
21
+
* Using callbacks which are invoked once the operation has completed. The callback can be then used to assert that the
22
+
state of the system is as we expect. But not all operations provide callback functionality.
17
23
18
-
Another common approach is to use countdown latches and this works fine if you are able to inject the latches in the
19
-
appropriate places but it isn't always possible to have the code under test trigger a latch.
24
+
* Block the thread using `Thread.sleep` or suspend a function using `delay`, waiting for the operation to complete.
25
+
The sleep threshold needs to be set high enough to be sure the operations will have completed on a fast or slow
26
+
machine. Plus it means that your test will sit around waiting on the timeout even if
27
+
the code completes quickly on a fast machine.
20
28
21
-
As an alternative, kotest provides the `eventually` function and the `Eventually` configuration which periodically test
22
-
the code ignoring your specified exceptions and ensuring the result satisfies an optional predicate, until the timeout
23
-
is eventually reached or too many iterations have passed. This is flexible and is perfect for testing nondeterministic
24
-
code.
29
+
* Use a loop with a sleep and retry and a sleep and retry, but then you need to write boilerplate to track number of
30
+
iterations, handle certain exceptions and fail on others, ensure the total time taken has not exceeded the max and so
31
+
on.
25
32
33
+
* Use countdown latches and block threads until the latches are released by the non-determistic operation. This can
34
+
work well if you are able to inject the latches in the appropriate places, but just like callbacks, it isn't always
35
+
possible to have the code to be tested integrate with a latch.
26
36
27
-
### Examples
37
+
As an alternative to the above solutions, kotest provides the `eventually` function which solves the common use case of
38
+
_"**I expect this code to pass after a short period of time**"_.
28
39
40
+
Eventually works by periodically invoking a given lambda, ignoring specified exceptions, until the lambda passes, or a
41
+
timeout is reached, or too many
42
+
iterations have passed. This is flexible and is perfect for testing nondeterministic code. Eventually can be customized
43
+
with regards to the types of exceptions to handle, how the lambda is considered a success or failure, with a listener,
44
+
and so on.
29
45
30
-
#### Simple examples
46
+
##API
31
47
32
-
Let's assume that we send a message to an asynchronous service.
33
-
After the message is processed, a new row is inserted into user table.
48
+
There are two ways to use eventually. The first is simply providing a duration, using the Kotlin `Duration` type,
49
+
followed by the code that should eventually pass without an exception being raised.
34
50
35
-
We can check this behaviour with our `eventually` function.
51
+
For example:
36
52
37
53
```kotlin
38
-
classMyTests : ShouldSpec() {
39
-
init {
40
-
should("check if user repository has one row after message is sent") {
41
-
sendMessage()
42
-
eventually(5.seconds) {
43
-
userRepository.size() shouldBe 1
44
-
}
45
-
}
46
-
}
54
+
eventually(5.seconds) {
55
+
userRepository.getById(1).name shouldBe "bob"
47
56
}
48
57
```
49
58
50
-
#### Exceptions
59
+
The second is by providing a config block. This method should be used when you need to
60
+
set more options than just the duration. It also allows the config to be shared between multiple invocations of
61
+
eventually.
62
+
63
+
For example:
64
+
65
+
```kotlin
66
+
val config = eventuallyConfig {
67
+
duration =1.seconds
68
+
interval =100.milliseconds
69
+
}
51
70
52
-
By default, `eventually` will ignore any `AssertionError` that is thrown inside the function (note, that means it won't catch `Error`).
53
-
If you want to be more specific, you can tell `eventually` to ignore specific exceptions and any others will immediately fail the test.
71
+
eventually(config) {
72
+
userRepository.getById(1).name shouldBe "bob"
73
+
}
74
+
```
54
75
55
-
Let's assume that our example from before throws a `UserNotFoundException` while the user is not found in the database.
56
-
It will eventually return the user when the message is processed by the system.
76
+
## Configuration Options
57
77
58
-
In this scenario, we can explicitly skip the exception that we expect to happen until the test passed, but any other exceptions would
59
-
not be ignored. Note, this example is similar to the former, but if there was some other error, say a ConnectionException for example, this would cause
60
-
the eventually block to immediately exit with a failure message.
78
+
### Durations and Intervals
61
79
80
+
The duration is the total amount of time to keep trying to pass the test. The `interval` allows us to
81
+
specify how often the test should be attempted. So if we set duration to 5 seconds, and interval to 250 millis,
82
+
then the test would be attempted at most `5000 / 250 = 20` times.
Alternatively, rather than specifying the interval as a fixed number, we can pass in a function. This allows us to
92
+
perform some kind of backoff, or anything else we need.
75
93
94
+
For example, to use a fibonacci increasing interval, starting with 100ms:
76
95
77
-
#### Predicates
96
+
```kotlin
97
+
val config = eventuallyConfig {
98
+
duration =5.seconds
99
+
intervalFn =100.milliseconds.fibonacci()
100
+
}
101
+
```
102
+
103
+
### Initial Delay
78
104
79
-
In addition to verifying a test case eventually runs without throwing, we can also verify the result and treat a non-throwing result as failing.
105
+
Usually `eventually` starts executing the test block immediately, but we can add an initial delay before the first
106
+
iteration using `initialDelay`, such as:
80
107
81
108
```kotlin
82
-
classMyTests : StringSpec({
83
-
"check that predicate eventually succeeds in time" {
84
-
var i = 0
85
-
eventually<Int>(25.seconds, predicate = { it == 5 }) {
86
-
delay(1.seconds)
87
-
i++
88
-
}
89
-
}
90
-
})
109
+
val config = eventuallyConfig {
110
+
initialDelay =1.seconds
111
+
}
91
112
```
92
113
93
-
#### Sharing configuration
114
+
###Retries
94
115
95
-
Sharing the configuration for eventually is a breeze with the `Eventually` data class. Suppose you have classified the operations in your
96
-
system to "slow" and "fast" operations. Instead of remembering which timing values were for slow and fast we can set up some objects to share between tests
97
-
and customize them per suite. This is also a perfect time to show off the listener capabilities of `eventually` which give you insight
98
-
into the current value of the result of your producer and the state of iterations!
116
+
In addition to bounding the number of invocations by time, we can do so by iteration count. In the following example
117
+
we retry the operation 10 times, or until 8 seconds has expired.
99
118
100
119
```kotlin
101
-
val slow =EventuallyConfig<ServerResponse, ServerException>(5.minutes, interval =25.milliseconds.fibonacci(), exceptionClass =ServerException::class)
102
-
val fast = slow.copy(duration =5.seconds)
120
+
val config = eventuallyConfig {
121
+
initialDelay =8.seconds
122
+
retries =10
123
+
}
124
+
125
+
eventually(config) {
126
+
userRepository.getById(1).name shouldBe "bob"
127
+
}
128
+
```
129
+
130
+
### Specifying the exceptions to trap
131
+
132
+
By default, `eventually` will ignore any `AssertionError` that is thrown inside the function (note, that means it won't
133
+
catch `Error`). If you want to be more specific, you can tell `eventually` to ignore specific exceptions and any others
134
+
will immediately fail the test. We call these exceptions, the _expected exceptions_.
135
+
136
+
For example, when testing that a user should exist in the database, a `UserNotFoundException` might be thrown
137
+
if the user does not exist. We know that eventually that user will exist. But if an `IOException` is thrown, we don't
138
+
want to keep retrying as this indicates a larger issue than simply timing.
103
139
104
-
classFooTests : StringSpec({
105
-
val logger = logger("FooTests")
106
-
val fSlow = slow.copy(listener = { i, t -> logger.info("Current $i after {${t.times} attempts")})
140
+
We can do this by specifying that `UserNotFoundException` is an exception to suppress.
0 commit comments