To write [marble tests], or not to write, that is the question.
Every Software Developer should know the right answer. Tests are our safeguard in case of human errors. The Holy Grail of a working software. The key here is to write accurate tests. The ones that will check (almost) every possible scenario. This is relatively easy for a synchronous operations, but gets more complicated when we have to test asynchronous-muli-threaded-time-dependent-stream-of-data (I just made up this name, don’t bother).
Even though TypeScript (effectively JavaScript) is described as single-threaded language, which it is not by the way – JS is just a code, not the runtime. Well it’s kinda single-threaded, but that’s because of the engine1… Either way, we have a concept called asynchronicity. Asynchronous operations allow developers to dispatch some tasks to what may seem to be a separate thread. This brings up the topic of completion time, and how to make sure the data we just parsed gave proper results.
A long long time ago humans invented Promises. Although we don’t always fulfill them in the real world, we expect our software to do it better than us. So the Promises are like one-time tasks that have to do some calculations and eventually give us the result (or throw an error). This is not very interesting. What really is interesting are the Observables – a super-duper-time-traveling-thing (again, just my imagination) which feeds us with the data all day long!
Great, so we talked a bit about some high-level tech side of the programmers life. Let’s go into the details of Observables and how to test them, in the end, that’s the main topic of this post.
The most popular framework to work with Observables is ReactiveX. It has implementations for many popular languages like Java, Scala, .NET, and of course JavaScript. ReactiveX is a library to compose asynchronous and event-based programs using observable sequences.
The whole concept is based on the Observable type, it’s satellite types (Subjects etc.) and operators inspired by Array type (map, reduce, filter etc.). If you’re not familiar with it head to http://reactivex.io and take a look. ReactiveX not only allows us to easily use Observables in our code. It also gives us an ability to transform data received in time. This means that we can grab a stream of data, manipulate it freely (we can change values, skip values, add values) and return in the same time frames (or different ones!) the results. What a mess!
Given that Observables are time-based constructs, there was a challenge on how to demonstrate their flow when using RX to manipulate the stream. It ended up on using marble diagrams. The marble diagram consists of two timelines and a function. The function may be simple x*10 but may do also some advanced logic in order to get the most of the data we received in the stream. Below you can see an example of the diagram. The incoming values (1,2,3,4) are being modified by the map() function and then placed on the second timeline which is holding the results. This is a very simple usage, but the easiest one to draw as a diagram.
While on paper this seems like an easy thing, in the real world we usually don’t just “x * 10” data. Instead we process more advanced objects, we select data that we want, we even fetch other things, and retry if there’s an error. Yes, you can retry the whole timeline of the Observable, it’s like having a Multiverse where the same things happen in a bit different time! Great stuff…
Let’s move to some examples. Let’s say we want to make an HTTP library, but based on Observables rather than Promises. We have the fetch() function but it returns a Promise. So we proceed to write some code that will make the Observable from it.
After a few hours of pain we came up with the following code:
function fromFetch(fetch) {
return new Observable<any>(observer => {
fetch.then(response => {
const reader = response.body.getReader();
const readStream = () => {
reader.read().then(result => {
if (result.value) {
observer.next(result.value);
}
if (result.done) {
observer.complete();
} else {
readStream();
}
})
}
readStream();
});
})
}Good thing is that it’s working, at least we think it does. So how to write a test to make sure it will always work? We can’t just write a test the “normal” way, because we need to have a single exit point of the test case. We can of course store some state in variables, add to that state whenever Observable emits data, and on Observable completion just call it a day, but that approach produces a bunch of non-reusable code. So, how to approach it the ReactiveX-way? The key here are the marbles. Those are probably the best possible visualization of the data stream. So, if we can draw them, we can also write them. Here, have yourself a marble: (ab|)
Nice one, right? So, what exactly those chars mean? The brackets encapsulate single marble (single time frame). Everything inside are values that are expected to arrive: a, b and special char “|” which indicates that the Observable should call complete() in this time frame. The a and b may actually refer to some complex objects… there’s a lot to explain here. The syntax is very simple, although powerful.
Let’s go back to our fromFetch() function. As we know (hopefully!), we shouldn’t put real HTTP requests inside a test. Those will slow them down, and probably make them fail from time to time. Instead let’s assume we have some reliable library that mocks the built-in fetch() function.
So, our test case should look similar to this one:
it('streams response', marbles(m => {
// Given
const request = fetchMock('...');
// When
const source = fromFetch(request).pipe(mapTo("Received mock bytes"));
const result = m.cold(‘a---b---c---d---|’, { a: "Received mock bytes", … })
// Then
m.expect(source).toBeObservable(result);
}));We’re starting with creating mock of our fetch() request. The mock is designed to stream four arrays of random bytes. We’re mapping it to the string because we don’t need to test the actual values returned. Then we’re putting it inside our to-be-tested function, and on the next line magic happens.
What we’re doing here is creating a cold Observable2. The first argument is a string representation of a marble diagram. Here we’re expecting 4 values (brackets are optional if we expect only one value in a frame) to be emitted in 4-frame intervals, and a closing signal at the end. The second parameter is an object describing what “a,b,c…” actually should be, so in our case it’s an object with strings that we mapped our request to. The result Observable is the stream that we’re expecting to receive after running the request Observable. On the last line we’re running an expectation that will take care of testing whether those two Observables actually match. It will compare time frames, event types, and received values.
So… that’s it! We have written our first marble test. We can now sleep well knowing that our fromFetch() function is properly transforming a fetch() into an Observable.
Of course there’s a lot more to cover, but there are tons of Internet resources talking about it, so go ahead and read some more if you’re interested.
Happy coding!
1) Want to do multithreading in JS? Check this out: going-multithread-with-node-js
2) cold-vs-hot-observables
