Math.random()

Word documents, The Relay Way™

2017-01-01T02:00:00+00:00

TL;DR Sometimes you have to set aside your developer work and do some office stuff. And sometimes that includes generating Word reports with dynamic contents. What if you could bridge Word, GraphQL and JavaScript to help you with these chores? docx-templates helps you do exactly that.

Every once in a while you need to produce a Microsoft Word document with dynamic contents. Maybe it’s a project report, or the boring economic section of a tender. Chances are you’re working within a team, and you absolutely need your document to be in the de facto standard for non-nerdy output: Word.

You don’t want to write hundreds of tables by hand, so you start thinking about docx automation. And since you’re a proud Node.js developer, you stumble upon these types of modules from the top of npm’s search results:

Fully programmatic generators, like docx, which provide functions to create paragraphs, text runs, and so on.
Template-based generators, like docx-templates, which let you start with an ordinary Word document, adding commands as needed to insert dynamic contents.

The benefits of the second approach are numerous:

You don’t need the library to support every tiny little feature of Word. The input template will be used transparently, so the output will be compatible with whatever version of Office you’re using.
You don’t need to write code to generate static parts of your document.
You can even write documents à la Relay. What am I talking about? Well, just keep on reading!

Colocating queries

One of the key ideas in Relay is colocation. It’s literally front and center in its home page, and what it basically means is that data requirements (the query) are declared very close to the components that need the data.

If we translate this idea to report generation, wouldn’t it be nice if we just wrote the data query in the document itself? A very simple template would look like this:

Query colocation

We’re using the Star Wars API with the GraphQL wrapper for maximum convenience, but docx-templates doesn’t really care about the language you choose for queries, as long as they eventually resolve to some data.

With the data requirements in place, we can write a short JavaScript snippet to execute the embedded query and create the report.

// swapi.js
const createReport = require('docx-templates');
require('isomorphic-fetch');  // assuming a modern Node, you need no Promise polyfill

createReport({
  template: process.argv[2],
  data: query =>
    fetch('http://graphql-swapi.parseapp.com', {
      method: 'POST',
      headers: {
        Accept: 'application/json',
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ query }),
    })
    .then(res => res.json())
    .then(res => res.data),
});

Run this script with node swapi swapi-simple-template.docx and voilà, both trilogies come up in your report (alas, no Force Awakens data yet in the GraphQL wrapper 😱):

A simple report generated with docx-templates

A more complex example

GraphQL is really great for describing complex data requirements. Imagine you want to summarize all Star Wars films with some basic data and a table of characters with their species, home planet, and ships (i.e. vehicles with hyperdrive) they appear in. Really nerdy stuff, huh? No problem with GraphQL:

+++QUERY { allFilms { films {
  title, releaseDate, director, producers, openingCrawl
  characterConnection { characters {
    name
    species { name }
    homeworld { name }
    starshipConnection { starships { name crew passengers } }
  } }
} } }+++

Your Word template might look like this (right after the query):

A complete template with embedded JS

This templates demonstrates other features of docx-templates:

You can embed JavaScript in your INS (shorthand notation: =), FOR and other commands.
You can define aliases (ALIAS) for complete commands, which you can then resolve with the *<alias-name> shorthand. We found it quite useful when templating tables.
Loop variables (e.g. $film, $character) can be used from embedded JavaScript snippets, just like any other property of the query result.

Want to see the generated document? Just run node swapi swapi-complex-template.docx and there it is, in all its geeky splendor:

A complete report generated with docx-templates

A final word

docx-templates is a young project and still evolving with our needs. If you use it and have suggestions or —even better— want to contribute, please visit its GitHub repo!

Picture this: snapshot testing

2016-09-27T12:00:00+00:00

TL;DR If you’re not testing your React (Native) components at all, or are relying on insufficient, high-level smoke tests, you’re not alone. Writing those unit tests must be one of the less rewarding experiences for a developer. But wait: snapshots to the rescue! Jest’s newest release features snapshot testing – and might well change your mind.

Jest’s latest release has caused quite a stir, and understandably so: its snapshot testing feature alone is something of a mind-bender. If you’re like me, maybe you didn’t write unit tests for your React (Native) components for a number of valid reasons:

“They’re hard to write”
“They’re brittle”
“They don’t bring so much value if you have at least some integration tests”

Starting with Jest 14, though, you may need to revisit those arguments. Snapshots are trivial to use, they can easily be updated, and they provide full coverage of your components, dynamic behavior included. How do they work? Even that can be explained in just two bullets:

The first time your test renders a component, a snapshot is taken and saved to a file.
Upon re-run, the new render is diffed against the snapshot. If there’s a mismatch, you choose which one to keep (and maybe correct your application).

On top of that, Jest provides a comprehensive testing framework, good performance, top-notch diffs and error context and lots of CLI goodies. Even coverage results out of the box! No wonder it got rave reviews.

Haven’t tried snapshots yet? Let’s give them a spin. The following examples are loosely based on Mady, an application for software internationalization. Until now, Mady had absolutely no tests covering its React components.

A hello-world component

We’ll try snapshots on a very simple component first. Mady’s Header is a stateless functional component showing the application name and a couple of buttons. Lots of styles to be tested, but not much more.

The full unit test for this component is so short you might think it’s a joke:

/* eslint-env jest */
import React from 'react';
import renderer from 'react-test-renderer';
import Header from '../header';

it('renders correctly', () => {
  const tree = renderer.create(<Header />).toJSON();
  expect(tree).toMatchSnapshot();
});

Run Jest, and a new snapshot will be created:

exports[`renders correctly 1`] = `
<div
  style={
    Object {
      "WebkitFlex": "0 0 2.5em",
      "alignItems": "center",
      "backgroundColor": "#dcd6ff",
      "display": "flex",
      "flex": "0 0 2.5em",
      "flexDirection": "row",
      "padding": "5px 8px"
    }
  }>
  ...
</div>
`;

That’s all there is to it, and it will cover every tiny little CSS property you may be using. Change a color? Jest will warn you. Nudge the header one pixel down? Warning again. Tell Jest to update the snapshot for you, and you’re done.

Note: with the current version of React, you may need to add this at the beginning: jest.mock('react-dom').

Mocking children components

Now let’s test a slightly more complicated component. Mady’s TranslatorRow renders the original message to be translated, as well as several translations. But the translations themselves, which are editable, are handled by a child Translation component.

Mady’s Header and TranslatorRow components in action

We are writing unit tests for TranslatorRow, so we don’t want Translations to be rendered as well (they will have unit tests themselves). But we do want to know which props are passed down to them.

Jest’s mocking functionality comes in handy for this. Let’s use it on Translation:

jest.mock('../translation', () => jest.fn((props) =>
  <div dataMockType="Translation" {...props} />
));

Notice how we manage to include in the snapshot the props that are being passed down: {...props}. Notice also the dataMockType attribute, which I found useful for easily identifying mocked components in the snapshot.

Now, one of the actual unit tests may look like this:

import TranslatorRow from '../translatorRow';

const MSG_WITH_TRANSLATIONS = {
  id: 'msgId',
  text: 'A message',
  translations: [
    { id: 'id1', lang: 'es', translation: 'Un mensaje' },
    { id: 'id2', lang: 'ca', translation: 'Un missatge' },
  ],
};

it('renders correctly a message with translations', () => {
  const tree = renderer.create(
    <TranslatorRow
      msg={MSG_WITH_TRANSLATIONS}
      langs={['es', 'ca']}
    />
  ).toJSON();
  expect(tree).toMatchSnapshot();
});

If we look at the stored snapshot, we’ll see that an empty div was correctly instantiated for each translation, together with the correct props:

...
<div
  dataMockType="Translation"
  lang="es"
  msg={...}
  translation={
    Object {
      "id": "id1",
      "lang": "es",
      "translation": "Un mensaje"
    }
  } />
...
<div
  dataMockType="Translation"
  lang="ca"
  msg={...}
  translation={
    Object {
      "id": "id2",
      "lang": "ca",
      "translation": "Un missatge"
    }
  } />

Snapshotting Relay containers

In the real application, TranslatorRow is a Relay container, so it expects to be passed a relay prop. Now, the Relay Higher-Order Component (HOC) lives outside our application and we may not even know the full shape of that prop.

I found it more straightforward to just leave the HOC out of our unit test. For this purpose, our default export in translatorRow.js is the wrapped TranslatorRow, but we also expose the unwrapped version as a named export.

const fragments = {
  msg: () => Relay.QL`
    fragment on Message {
      id
      text
      ${Translation.getFragment('msg')}
      translations(first: 100000) { edges { node {
        id
        lang
        ${Translation.getFragment('translation')}
      }}}
    }
  `,
};

class TranslatorRow extends React.PureComponent {
  // ...
}

export default Relay.createContainer(TranslatorRow, { fragments });
export { TranslatorRow as _TranslatorRow };  // just for unit tests!

The snapshot test and the saved snapshot file will be exactly as before, except that we now need to use the named export and conform to Relay’s connection specs:

import { _TranslatorRow as TranslatorRow } from '../translatorRow';

const MSG_WITH_TRANSLATIONS = {
  id: 'msgId',
  text: 'A message',
  translations: { edges: [
    { node: {
      { id: 'id1', lang: 'es', translation: 'Un mensaje' },
    } },
    { node: {
      { id: 'id2', lang: 'ca', translation: 'Un missatge' },
    } },
  ] },
};

// ...

Testing dynamic behavior with snapshots

React components may respond to user interaction in different ways. One component might update its state and re-render, whereas another one might just call a function passed in as a prop.

One thing they have in common, though, is that they need to attach event handlers to DOM elements to be notified of user interactions. And those handlers happen to be accessible in Jest and integrate well with snapshot testing; isn’t that awesome?

Dynamic tests on a self-rendering component

In this example, TranslatorRow is hoverable and re-renders on mouseEnter and mouseLeave:

class TranslatorRow extends React.PureComponent {
  constructor() {
    super();
    this.state = { hovered: false };
  }

  render() {
    return (
      <div
        className={this.state.hovered ? 'hovered' : ''}
        onMouseEnter={() => this.setState({ hovered: true })}
        onMouseLeave={() => this.setState({ hovered: false })}
      >
        {/* ... */}
      </div>
    );
  }
}

To test the component’s dynamic behavior, Jest gives us access to the full rendered tree, including event handlers we can manually trigger:

it('changes its class upon mouseEnter/mouseLeave', () => {
  const component = renderer.create(
    <TranslatorRow
      msg={MSG_WITH_TRANSLATIONS}
      langs={['es', 'ca']}
    />
  );
  let tree = component.toJSON();
  expect(tree).toMatchSnapshot(); // snapshot 1

  tree.props.onMouseEnter();
  tree = component.toJSON();      // re-renders
  expect(tree).toMatchSnapshot(); // snapshot 2

  tree.props.onMouseLeave();
  tree = component.toJSON();      // re-renders
  expect(tree).toMatchSnapshot(); // snapshot 3
});

Jest will store not one, but three snapshots for this test, capturing all details of how the component should render in the idle and hovered states.

Dynamic tests with callback props

In this second example, TranslatorRow responds to click events by calling its onClick prop with its message ID:

class TranslatorRow extends React.PureComponent {
  render() {
    const { onClick, msg } = this.props;
    return (
      <div onClick={() => onClick(msg.id)}>
        {/* ... */}
      </div>
    );
  }
}

To test this behavior, we use Jest mock (or spy) functions:

it('reacts correctly to click events', () => {
  const spyOnClick = jest.fn();
  const tree = renderer.create(
    <TranslatorRow
      msg={MSG_WITH_TRANSLATIONS}
      langs={['es', 'ca']}
      onClick={spyOnClick}
    />
  ).toJSON();
  tree.props.onClick();
  expect(spyOnClick).toBeCalledWith(MSG_WITH_TRANSLATIONS.id);
});

OK, we’re not using snapshots at all in this example, but I included it to give you a broader picture.

Conclusions

This has barely scratched the surface of what Jest can do. After a period in which it was broadly perceived as hard to use and limited to Facebook’s own use cases, Chris Pojer et al did a great rewrite. It’s not just hype: give it a test drive!

I hope this post gave you some ideas to finally address those virgin areas of your project that have never seen a test. I’ll whisper this to your ear: snapshot testing can even… be fun!

Untangling spaghetti logs

2016-07-18T12:00:00+00:00

TL;DR Logging might not be super-exciting, but when you hit your head against a wall during debugging, you sometimes wish you’d used better tools. Storyboard 2.0 has just been released and provides a flexible architecture, a powerful feature set and a novel approach to ~~storytelling~~ logging.

OK, so it might not look like the most sexy subject to blog about. When you are building the most exciting new application, why would you even want to think about logging and not just… go out play Pokémon?

The fact is: sooner or later, just as with testing, you will need to give it some thought. In a typical client-server web application, you will start having concurrent sessions. And even at the client side, where you have only one user, simultaneous actions will start making your logs less and less readable. Debugging, or just grasping what’s going on, can quickly turn into a nightmare if you don’t take the necessary precautions.

Even if you’re not building a web app, you may be overwhelmed by your logs. In my current project, I’m developing a Node-only application with multiple bidirectional interfaces (an interactive SSH session, a proprietary TCP link, an interactive CLI conversation with the user), all of them generating events like crazy.

In an ideal world, I’d like to have an easy-to-use library that would let me filter logs by relevance or source module, attach objects, group them hierarchically in stories (“John clicks on Buy” + “Goes to check-out” ⊂ “John visits the store”), persist them to a file/database, send them to a separate process for trend analysis… The icing on the cake? I’d love to see what’s happening at the client and the server in a single place (how about a Chrome DevTools extension for that?).

Well… have you checked out Storyboard yet?

The Storyboard DevTools in action

A flexible logging architecture

Storyboard is based on a hub component that receives messages and broadcasts them to all attached listeners, or plugins. Despite their name, listeners can also send messages to the hub, making them quite powerful.

Let’s say we have a typical client-server application. You could configure the isomorphic Storyboard library like this for development (legend at the bottom):

Storyboard logging configuration for a client-server application

In this architecture, the server-side hub receives stories and relays them to console, disk and database. In addition, server logs are sent via WebSockets to the client-side hub, which relays server and client stories to the console as well as to the Storyboard DevTools.

Signalling messages are exchanged between the DevTools extension and the WebSocket Server in both directions. Such messages are required for user authentication or, more interestingly, to configure server-side log filters from the extension itself. If you want to get more logs from a particular server module, you can just tweak the filter (say, from *:INFO to *:INFO, suspect:DEBUG) and logs from the suspect module will begin to pour in, no reset required.

Other example architectures

In some cases, you might not be able to include Storyboard in your existing project because your application might not be written in JavaScript, or because you simply don’t want to change it. In this scenario, even though some benefits of the library will be lost, you can still use the Storyboard CLI, which wraps your application’s default input/output streams (stdin, stdout, stderr):

Logging configuration for a Node-only application with the Storyboard CLI tool

This even works with a shell! Just try sb --server bash and you’re instantly sharing your shell session with others via HTTP.

As a final example, here is a Storyboard configuration with support for mobile clients. In this case, the WebSocket client is configured with uploadClientStories: true. Uploaded stories will not pollute the server’s logs as they are written to console or file, but they will get saved to the database for postprocessing, and re-broadcast to other clients with the Storyboard DevTools extension.

Logging configuration including upload of mobile client logs

Conclusion

Logging is something we should keep in mind when developing an application, be it large or small, same as testing, security and other cross-cutting concerns. The flexible architecture of Storyboard may help you untangle your logs and squeeze the most out of them!

Painless immutability

2016-06-16T12:00:00+00:00

TL;DR With the popularity of Immutable.js and Seamless-immutable going through the roof, who needs yet another immutability library? Maybe you, if you don’t want to lock read access to your data through a non-native API, need reasonable write performance and an easy-to-use API. That’s exactly the balance Timm tries to strike.

We’ve read the posts, watched the videos, understood the concepts. Immutability can be really useful in many scenarios; for example:

We perform some lengthy computations on data and want to memoize the results, reusing them until inputs change.
We want to implement an Undo/Redo functionality in an elegant way.
We want to untangle our application’s state and use something like Redux with time travel, serialization, rehydration, you name it.
We’re stuck with our React application performance and have heard of shouldComponentUpdate’s wonders.

In all of these cases, using immutable data is probably the way to go. It can make code more predictable, allows trivial object comparisons, and feels right at home with functional programming. What else could we ask for?

As it turns out, immutable operations don’t come without their own drawbacks. They’ll never be as fast as in-place mutations, no matter how hard you try. You’ll need to bend your mind around some algorithms, especially recursive ones. And it may become a pain to work with in some cases, depending on the tools/libraries you work with.

But what if we found a way to alleviate the cognitive burden on the developer, keep performance reasonable, and allow us to reap the benefits of immutability? Enter Timm, a tiny library that tries to solve some of these trade-offs.

Trade-off #1: seamlessness vs. protection

Immutable.js is one of the big names in the immutability arena, a rock-solid library that offers you everything you’ll ever need and then some more. Performance is also excellent, considering the many things it does, and its objects (Maps, Lists and many others) are opaque, so they’re protected against your accidental mutations.

However, after using it in several projects I’ve always felt the friction of mixing Immutable.js’ Maps and Lists with native JS objects and arrays. It’d be impossible to count how many times I got strange runtime errors because I forgot to use Map’s getter (image.get('url')), instead of plain old dot notation (image.url), or a warning because I took the familiar array length property instead of List’s size.

So: seamless or protected? Timm goes for seamless. It would be possible to have native and protected objects via Object.freeze(), but there seems to be some performance bottlenecks (or alternatively different behaviours in Development and Production mode, which seems just as bad). In conclusion: you can accidentally mutate your objects (watch out!), but:

You will be able to access your data in a straightforward way; and
You won’t be locked in to a particular library, for something as ubiquitous as data management.

Trade-off #2: simplicity/size vs. write performance

Immutable.js aims at being both comprehensive and performant. Optimising read/write speeds in immutable operations while keeping an attractive API is not trivial, and Immutable.js uses advanced techniques (persistent data structures and structural sharing) to achieve just that.

While not providing all the bells and whistles, Timm is 10 times smaller (~ 1 kB minified and compressed) and uses much more basic tools, although it also focuses on performance. Reads are as fast as native (you don’t even need the library for that), and writes are also reasonably fast (benchmarks):

On small arrays/objects: same performance as Immutable.js for shallow writes, becoming increasingly faster than Immutable.js for deeper writes.
On large arrays or objects (see Array: write in the figure below, where arrays have 1000 elements): increasingly slower than Immutable.js as arrays become longer or objects have more properties. This is where Immutable.js really shines!
Faster (even extremely faster) than Seamless-immutable, which also uses native objects.

Read/write benchmarks for Immutable.js, Timm and Seamless-immutable

…and what about the JS spread operator?

Finally, why not exploit ES6’s shiny new array spread operator and its upcoming sibling for objects? After all, native syntax can’t be beaten! Why not do the following?

const obj = { foo: 'foo', bar: 'bar' };

// Why not do this...
const obj2 = { ...obj, bar: 'boo' };

// ...instead of this?
const obj2 = timm.merge(obj, { bar: 'boo' });

The object spread operator certainly seems nicer. However, beware:

const obj = { foo: 'foo', bar: 'bar' };

// Future spread operator without changes
const obj2 = { ...obj, bar: 'bar' };
console.log(obj2 === obj);
// -> false

// Timm without changes
const obj2 = timm.merge(obj, { bar: 'bar' });
console.log(obj2 === obj);
// -> true!

The spread operator always creates a new object, no matter whether we are actually modifying the original one. By contrast, Timm and Immutable.js only instantiate a new one (lazily) when they detect that the operation is in fact mutating the object; this way, you’ll create fewer objects, alleviating memory requirements and garbage collection events. And what’s more: you’ll get fewer false positives when checking whether an object has changed.

Conclusion

After all this, which is the most suitable option?

Don’t forget the obvious: do you really need immutability? Probably yes, but use your best judgement. If performance is crucial and you can sacrifice code elegance and the other nice perks immutability brings along, well… just mutate in peace! I mean, in place.
If you need a complete, battle-tested library and don’t mind the lock-in associated to a non-native API even for reads: use Immutable.js.
If you prefer plain arrays/objects, use Timm.
If your typical use cases involve more reading than writing, use Timm as well.
If you do a lot of writes on very long arrays or fat objects, use Immutable.js.