Introducing frock: Easy fake services for a microservices environment

Over the holiday break, my team open sourced a core piece of our development tooling: frock, a framework for easily managing a suite of fake services. If you work in a service-oriented architecture, it's a tool worth evaluating.

Solving complexity in your development environment

Urban Airship uses microservices—a very common pattern for a SaaS company: we have a number of services which encompass a small amount of functionality, and serve requests from other services over well-defined protocols. This is great in terms of scalability and stability, but can be problematic for developers and their environments.

My team works on Go—the Urban Airship Engagement dashboard product—which has the distinction of being one of the earliest Urban Airship code bases. It contains a lot of functionality: message composition and reporting, billing and usage reports, and service management (such as the recently-released Urban Airship Connect). Go talks to a lot of services, and this meant trouble for my team's development environment.

Previously, our environment consisted of a monolithic Virtual Machine running on Vagrant, which was provisioned with all of the dashboard's dependent services. It had a number of problems:

  • It was slow. There were a large number of services and their dependencies, including several different databases that all needed to be running.
  • It was fragile. My team is not comprised of experts in the JVM, which runs a substantial portion of our stack. Upstream changes to services would break our development environments, causing many of us to develop a severe fear of updating.
  • It accounted for a lot of developer downtime. When a developer's environment was down, they were often unable to work. The complexity of the environment meant that downtime wasn't uncommon. This created a lot of wasted time and developer frustration.
  • It wasn't easy to get data into the environment. When developing, you want to have a wide variety of data to test your code on. The previous environment often required you to run in test data manually, and by communicating directly with the service that housed the data.

frock was the tool written to help us solve all of the above difficulties.

What is frock?

frock allows us to stand up fake services with data that we define, and to write the fake services in a language and runtime that my team is comfortable with: JavaScript and Node.js.

At its core, frock provides just a few pieces of functionality:

  • A highly configurable router, which makes intercepting calls to services straightforward.
  • A plugin-based architecture that doesn't strongly dictate how you write your fake services.
  • Out-of-the-box generic plugins for common tasks, namely a static file server and a proxy server.
  • Support for HTTP and socket services
  • HTTP middleware support to customize the behavior of a service or a route; an example being the delay middleware which can introduce latency into your request being served.

All of this is configured via a JSON configuration file (named frockfile.json by convention), and shared via a repository which includes all of our custom fake services. frock takes cues from grunt and gulp in the sense that it's a utility with configuration that lives alongside your projects.

For HTTP fakes, a frockfile consists of a simple service and route definition, an example:

{
  "servers": [
    {
      "port": 8080,
      "routes": [
        {
          "path": "/api/devices",
          "methods": ["GET"],
          "handler": "frock-static",
          "options": {
            "file": "./fixtures/devices.json",
            "contentType": "application/json"
          }
        },
        {
          "path": "*",
          "methods": "any",
          "handler": "frock-proxy",
          "options": {
            "url": "http://localhost:8052"
          }
        }
      ]
    }
  ]
}

This is a highly-simplified version of a service definition in our frockfile. When visiting the Go dashboard, you are actually hitting a high-availability proxy: this proxy may direct traffic to different services internally, where the primary service is Go but some API routes are served by secondary services.

Rather than run this secondary service locally, frock allows us to pick out that route and replace it with a fake. In the above case, the fake is just a static file, served by the frock-static generic static server plugin. All other requests are directed to the locally-running Go development server, which is running on port 8052.

Writing frock plugins

frock is actually our second platform for solving this problem; the first came in the form of a tool called "multimock", which was a monolithic Python application that spun up multiple service threads and passed them through some generic transformation functions prior to arriving at your custom handlers. It solved a lot of problems, but we rewrote it as frock. Why? Writing plugins in it was too difficult due to upstream assumptions the tool made before your handlers were called.

When writing frock, the core tenet was that "writing plugins should be easy, and with as few built-in assumptions as possible". frock accomplishes that goal by imposing very little on your plugins, and exposing the simplicity of Node.js's HTTP module directly to the implementor.

The simplest possible frock HTTP plugin, which just responds "hello world!" to any request that hits its route:

// file ./hello-world.js
module.exports = createPlugin

function createPlugin (frock, logger, options) {
  return handler

  function handler (req, res) {
    res.end('hello world!')
  }
}

If you've ever written a request handler in Node.js before, this will look very clear; all that a frock plugin consists of is a factory function that returns a route handler.

In the above frockfile example, lets swap this plugin in place for our static file server:

{
  "servers": [
    {
      "port": 8080,
      "routes": [
        {
          "path": "/api/devices",
          "methods": ["GET"],
          "handler": "./hello-world"
        },
        {
          "path": "*",
          "methods": "any",
          "handler": "frock-proxy",
          "options": {
            "url": "http://localhost:8052"
          }
        }
      ]
    }
  ]
}

Now any request to http://localhost:8080/api/devices will respond "hello world!".

How frock has helped us

Working on a large feature release can get complicated: you're often waiting on dependent services to be ready and are working from a spec of how those services will work — in a fast-paced project this means you're running up to some integration date, when you'll finally put all the components in place and (hopefully) they'll all talk to each other. Connect was one such project, and frock helped the web team to build something we were confident would work come integration day. (And it did!)

With Connect, my team was responsible for the configuration interface in the Go dashboard. This interface is where customers configure ready-made integrations, or create access credentials for custom integrations that they've written. Using frock, we were able to write the single-page frontend application first — we had a working prototype very early, before any of the backend services were ready:

  • Defining the API spec the frontend would use to communicate with the backend web application.
  • Implemented a fake of the backend services in frock, so the frontend could be prototyped; this prototype wasn't a throw-away, it directly evolved into the application you interact with today.
  • In tandem, the backend web service was implemented, written against fakes that represented the microservices that we'd be eventually communicating with. These fakes became part of our development environment, and are still used and expanded on today.

None of this is new

Nothing that's described above is revolutionary, but frock was born out of a desire to do the above simply. Give it a look; we think you'll find it useful in your development environments. The frock README contains a quick-start guide, as well as a list of comprehensive docs and examples to get you started using frock in your projects and implementing your own plugins.