Technology

Little server patterns: Dependency parameters

Dependency parameters
May 20, 2022

Read time 9 min

I’ve written a lot of server code over the last years, from serving national news (caches upon caches) and searching media content (why, Elasticsearch, why) to live-debugging AI-driven waste collection (just use Postgres, they said).

Regardless of domain, the same kinds of problems tend to come up in almost every project, and I gravitate toward the same kind of patterns to resolve them. These aren’t big, new, or wacky patterns. They are simple, surprisingly uncommon but uncommonly useful patterns. In this blog series, I’ll share some of them with you.

The first pattern relates to how server code is configured.

Problem: global configurations feed accidental complexity

Many functions in server applications depend on external configuration or derived state, such as:

  • Data store URLs, connections, pools, and clients (for databases, caches, queues)
  • Loggers and log levels
  • External API URLs and credentials
  • Authentication credentials
  • Global time

The configuration itself is often passed in to the application as environment variables or read from a configuration file.

The most common pattern I see for handling this within the app is to use a single global configuration: functions load environment variables directly or indirectly through a global configuration value. Derived configuration, like database connections or loggers, may be stored separately as global singletons.

For example (in TypeScript):

// config.ts
export const config = {
  apiKey: process.env.API_KEY,
  apiPassword: process.env.API_PASSWORD,
  cacheUrl: process.env.CACHE_URL,
  productUrl: process.env.PRODUCT_URL,
  serverPort: process.env.SERVER_PORT,
}
// server.ts
import {config} from './config'
import {cacheClient} from './cache'
import {logger} from './logger'

const fetchCachedProductList = (): Promise<Array | null> => {
  return cacheClient.get('productskey')
    .catch(e => {
      logger.error('Error fetching products from cache', e)
      return null
    })
}

const fetchProductList = (): Promise<Array> => {
  return fetch(`${config.productUrl}/products`)
}

const getProductList = (request: Request) => {
  return fetchCachedProductList()
    .then(fetchProductList)
    .catch(error => {
      logger.error(`Unknown error fetching articles: ${error}`)
    })
}

This approach can work just fine, especially in smaller projects. However, there is a surprising amount of complexity that falls out of using global configurations. Individual tests often require different configurations or dependencies, but global configurations are shared between tests, and environment variables are difficult to change within tests. Testing with global configurations requires a host of workarounds to allow different configurations and responses for tests.

Arguably global configurations cause all of the following unnecessary complexity:

  • Global HTTP capture. Tests often verify the handling of specific responses from external APIs without calling the APIs themselves to simulate varied situations. When test configuration is global, you likely need also to intercept network requests and mock network responses globally via some library.
  • Global module import capture. Another workaround to mock shared dependencies is hack directly into the module import mechanism via a library or language feature to mock modules and their functions directly.
  • Global time capture. Functions and tests that depend on specific times may need a library or other means to capture and mock global time.

In turn, the use of global capture hacks or libraries can cause the following issues:

  • Bugs related to shared global state. Global capture of HTTP requests, module imports, and datetime functions require resetting the global state between tests so that changes from one test don’t affect other tests. From experience it’s easy to make mistakes and end up with tests failing or passing incorrectly because an API was mocked out in another test, or time changed between tests, or something failed in the process of resetting global state.
  • Slower non-parallel tests. Globally-shared state or global HTTP/module/datetime mocks can be difficult or impossible to run in parallel without tests affecting each other. It’s often simpler to run tests sequentially without the potential speed-up from running tests in parallel.
  • Poor test coverage. When understanding, changing, and resetting global configurations makes it hard to write detailed tests, you don’t write them.

I may be over-selling the point. Libraries and language tools for global capture can be convenient and useful for other reasons, and there are more or less sensible ways to structure projects using these tools as well. But here is another approach.

Pattern: dependency parameters are straightforward and flexible

A pattern that I like to use in place of global configurations is to pass the specific configuration dependencies to the functions that need them as the first parameter. The above example might look like this:

type FetchCachedProductListCfg = {
  cacheClient: CacheClient
  logger: Logger
}

const fetchCachedProductList = (cfg: FetchCachedProductListCfg): Promise<Array | null> => {
  const { cacheClient, logger } = cfg
  return cacheClient.get('products')
    .catch(e => {
      logger.error('Error fetching products from cache', e)
      return null
    })
}

const fetchProductList = (productUrl: string): Promise<Array> => {
  return fetch(`${productUrl}/products`)
}

type GetProductListCfg = {
  cacheClient: CacheClient
  logger: Logger
  productUrl: string
}

const getProductList = (cfg: GetProductListCfg) => (request: Request) => {
  return fetchCachedProductList(cfg)
    .then(() => fetchProductList(cfg.productUrl))
    .catch(error => {
      cfg.logger.error(`Unknown error fetching articles: ${error}`)
    })
}

By convention, the “configuration” of a function comes in as the first argument, often curried for convenience when there are other arguments. If there is only a single or two configuration values they can be passed as arguments directly, but if the list grows larger it’s convenient to wrap them up in a record as above. This can also be a place to think about whether your function actually needs those dependencies, or if the dependencies should be refactored to avoid passing unnecessary detail to client functions.

What’s nice about this?

  • It’s straightforward and explicit. Functions take in parameters and do things with them, without having to consider any global context. Dependencies are explicit. Passed dependencies also tend to be the stateful and side-effectful bits of the system, so you end up essentially with a type signature that gives you a list of the side effects that a function might perform.
  • Fewer libraries and less mocking complexity. Tests don’t need to hack the language and environment to mock out modules or functions, capture HTTP requests, or modify global time. You can just pass in the specific dependencies needed by a test. This makes it much easier to do fine-grained testing. On the other hand this might require more work on your part to set up the mocking by e.g. wrapping up some configurations like API parameters into curried functions or client objects and creating tooling for passing in mocked clients in tests (see below discussion).
  • Fast parallel tests. If you need it, it’s easier to write tests in a parallelizable way by e.g. passing different database schemas or ports to different tests.
  • Better test coverage. It’s easier to write more fine-grained tests, so you probably will.
  • Less risk of calling production APIs. You pass in the specific dependencies each function needs so there’s less chance of a global setting or default being used where it shouldn’t.

Applying the “dependency parameter” pattern

In general, functions that depend on other external dependencies can receive those dependencies as wrapped values: client records, partially-applied functions, or client objects. For example:

  • Mocking HTTP responses: to mock HTTP responses, you can wrap API URLs and credentials in a client object or pass partially-applied functions down to their callers.
  • Mocking global time: functions that require the current time can instead receive a function that will return the current time. This way for testing you can pass in functions that return whatever time you want, while in production the function will just return the current time.

This approach means that every function that uses even a logger requires a tree of function calls with a logger as the first parameter up to the root of your application. This can be a bit of hassle. For smaller projects in particular, it might feel like a lot of unnecessary ceremony. And maybe it is. But this is also easiest to set up as a “premature optimization”, and the benefits come more as the project grows. It can be introduced piecemeal for specific parts of the application, but fully adopting the idea is a major refactoring in most cases if the project was built using another approach.

It can also be tempting to share configurations between many functions, or even pass a single large configuration to every function. This is a huge anti-pattern and basically negates all of the benefits of the approach. Essentially, you end up with a global singleton anyway – but now, you have to pass it around everywhere instead of just fetching it from a global source. Each function should take in only exactly the dependencies it requires.

Isn’t this just dependency injection? 

Yes. This pattern is a basic application of dependency injection or inversion of control for backend development, but those terms tend to include a lot of other baggage that is unnecessary in many cases, especially in more functional languages or styles.

The pattern also comes up elsewhere. Scott Wlaschin in Domain Modeling Made Functional discusses the same idea in a general functional context. Stuart Sierra’s Component Model in Clojure similarly emphasizes passing stateful dependencies to components, though it is focused more on the problem of managing stateful life cycles in a dynamic, REPL-based programming environment. Gary Bernhardt covers similar ground in his Boundaries talk about a “Functional Core, Imperative Shell”.

Pure languages like PureScript or Haskell don’t force you to adopt this pattern, but they do nudge you toward the idea by encapsulating side effects in Effect or IO or the like. Reading environment variables or setting up clients is often effectful, so in these languages it can be easier to create them once and pass them as parameters to functions. Reader monads or, more generally, monad transformer stacks in Haskell or PureScript could also be seen as an extension of this idea. Instead of manually passing values into each function, the values are passed automatically through the Reader monad. This is convenient, but I think it’s also an easy way to reintroduce a lot of the problems of global singletons. It makes state implicit again and, by making it harder to pass very narrow, specific dependencies to single functions, it tends to encourage sharing larger configurations. Though there can still be other advantages to adopting these kind of application architectures.

Next in the little server patterns series I will talk about failing quickly.

Sign up for our newsletter

Get the latest from us in tech, business, design – and why not life.