CouchDB to store and promote configuration changes

This is the seventh article from a new series about Node.js. In the previous article we started talking about how you can deliver configuration data to your Node.js processes. In this one we’ll see how we can use CouchDB to deliver configuration changes without needing to restart your processes.

Enjoy!‍

CouchDB is a key-value store where the values are arbitrary JSON documents. One CouchDB server can have an arbitrary number of databases, each database an arbitrary number of documents, indexes by key. Each document has a revision number: every time one document is updated, the document revision number gets incremented. This can come in handy for clients detecting stale data and conflicts. CouchDB also has some other interesting features:

• each database has a monotonic sequence number, which increases each time a document gets inserted, updated or deleted.
• CouchDB stores every version of every document, alongside it’s revision number

Also, a CouchDB database provides a continuous changes feed. Given a sequence number, this feed provides every change done in the database since. If the sequence number is not provided, CouchDB will giev us all the changes the database has gone through since creation.

Since CouchDB stores every version of every document, a database occupies an ever-increasing amount of space. To mitigate this you can frequently compact a database, losing all history.

Given that any document is an arbitrary JSON document and that it provides a changes feed, CouchDB can be used for storing and propagating changes throughout processes. Let’s see how.

Installing and setting up CouchDB

First, to try out the following examples, you will need to have a CouchDB server running on your computer. For that you can head out to the following URL that has links to the most used OSs:

http://docs.couchdb.org/en/1.6.1/install/index.html

Once you have it installed, start CouchDB. By default CouchDB is installed in “Admin party” mode, meaning that you don’t need a username and password to be able to perform any operation.

CouchDB talks HTTP, which means that you can use an HTTP command-line client like curl

If you don’t have curl installed you can try to install it via your favorite package manager (if you have one) or head out to the official curl download page: http://curl.haxx.se/download.html

Using curl we can now query our server:

‍Here we can see that CouchDB replies with a welcome JSON document.

Creating a configuration database

Now that we have a server, we need to create a database where we will store our configuration documents. We’ll aptly call this database config and create it using curl:‍

‍And that’s all we will need, really. Now we have a database where we can store our configuration.

Let’s say that we want to store some configuration data related to an e-mail sending service. We can then create the respective configuration document:‍

‍To create this document we could use curl, but an easier way is to use the administration interface provided by CouchDB, available at http://127.0.0.1:5984/_utils/:

You can now click on the config database and click on the New Document button. You can then paste the configuration document above and click on the Save Document button.

When starting a document creation through Futon, CouchDB creates a random document ID for you, but in our case we want to override that value with email.send, making this the effective document ID.

You can now query the configuration value using curl:‍

‍Getting the changes

Now that we have a way of updating the configuration, we need a way to distribute the configuration throughout our Node processes. For that we will use the follow package, that we can set on a new package.json manifest for our new app:

package.json:

‍‍We should now install it:‍

‍Now we will create a singleton config module which:

• exports the configuration and
• is responsible for updating itself when the value in CouchDB changes

config.js:

Let’s analyse this module in parts.

First, we require some modules that we will need later:‍

‍‍Then we create and export an event emitter:‍

‍Now we need to start following the CouchDB changes feed:

‍This piece of code starts listening to changes on the base URL defined in the environment variable CONFIG_SOURCE_URL. If such a variable doesn't exist, it defaults to using the local host (which is convenient during development).

We’re also telling the follow to start since the sequence number 0, which is the same as asking it to feed us the entire history of the config database.

Then we’re listening to the change event on the feed, effectively calling out onConfigChange function for each change on the config database.

Finally, we start following the feed by calling feed.follow().

After all this, we’re exporting a stop function that stops the feed:

‍This function allows us to stop reading from the config, which can be useful when shutting down the process.

Now, the only remaining piece of code to analyse is our onConfigChange function that handles configuration changes from CouchDB:

Every time there is a change, this function gets invoked with a change document describing the change. We start by extracting the id of the changed document. If the change was a document deletion, we delete that key from our exported config object. If this is not a deletion, it's an update or an insert, in which case we update the respective key on our exported config document.

After making the change, we emit a change event to allow our clients to be notified of changes.

I> By listening to this change event, a configuration client can, for instance, detect that it needs to change the hostname of a database server and connect to that server.

Let’s now watch this in action. First, start our module directly:‍

‍‍You will almost immediately see the first change on our database:

You can now use the Couchdb Futon admin interface to change the configuration value:

Once you save the document (using the Save Document button), you will see that change in your Node process output:

‍Client example

Here is an example of a client of our config module:

Next Article

This article completes the series about configuration patterns in Node.js.

In the next series I’ll start the analysis of some networking patterns. In particular, in the next article we’ll start by building a simple TCP service, which we’ll then make evolve throughout future articles.

This article was extracted from the Configuration Patterns, a book from the Node Patterns series.