Technology

A matter of optics – Introduction to safe and easy data manipulation

A human character looking at shapes and arrows.
Illustration by Daniel Savage.
August 2, 2021

Read time 13 min

Have you heard about optics but never really understood what they’re useful for? Do you have the impression that they’re only useful in Haskell or Scala, or require you to have a degree in mathematics to understand what’s going on? So did I. But after learning about optics, I found out they can be a simple and powerful tool – and understanding them is only a matter of getting familiar with a few, simple ideas.

This post explains basic concepts around optics and covers a few different optic types and what you can do with them. It explains optics terminology and helps you understand the differences between different kinds of optics. It also gives a glimpse into the power and simplicity that can be achieved with optics compared to manual data manipulation.

For example, with optics, you could come up with more readable Redux reducers if your state contains complex data. Or you could try a different take on UI programming with Harmaja and Lonna, which has built-in support for optics-ts lenses and isomorphisms. You can find use cases for optics from many places, some of them not obvious at first glance. The sky’s the limit!

To get the most out of this post, you should be familiar with programming and have an open mind towards terms that might sound hard but under the hood are just names for simple and beautiful constructions. Throughout this post, examples are given in terms of the optics-ts library for TypeScript, but knowledge of TypeScript is not required to understand the content.

What are optics?

Optics are a way to reference locations inside data structures. These locations are called focuses, and an optic can have any number of focuses, depending on its type. When you have an optic at hand, you can use it to query for the values the optic focuses on, and modify the values in an immutable manner.

Optics are composable, which means you can combine them to create new optics. The most widely known use of optics is to “drill in” to a deeply nested data structure, which is an example of what composition can achieve. However, composition unlocks many other, more exciting possibilities!

Before diving in, let’s establish some terminology and basic concepts.

Basics

An optic is an entity that supports some operations, behaves in a certain way, and is composable with another optic.A lens that focuses on a property of an object is an optic. An isomorphism that reverses a string is an optic.

An optic acts on one or more locations inside data structures. We call those locations the focuses of the optic. Some optics focus on the whole data structure, some focus on a part of it, some may have multiple focuses inside one input data structure, and some may have no focuses on a given input.

An optic type is a group of optics that share some properties. Lens and prism are optic types, for example.

An optic operates on a piece of data, which we call the input. For example, if a lens focuses on the title property of an object and the input is:

{ title: "Hello, World! }

then the value at the optic’s focus is the string “Hello, World!”. If the input is:

{ title: 42 }

then the value at the optic’s focus is the number 42. So the actual, concrete focus or focuses depend on the input.

Immutability means that when writing through an optic to modify the input, we don’t modify the input in place but instead return a new piece of data which is the same as the input with the modifications applied. Furthermore, if possible, the unchanged parts of the data are reused in the output instead of deeply copying the whole data structure.

Optics are stand-alone entities that abstract away the details of how they operate. To actually do something with an optic, you need to read or write through it. Optics-ts provides functions O.get(optic)(input) to get the value at the optic’s focus, and O.set(optic)(value)(input) to write the value or values at the optic’s focus. Both reading and writing can be combined with O.modify(optic)(fn)(input) where fn is a function that gets as an argument the focus of the optic, and returns a value to replace it with. The set of available operations differs depending on the optic type.

(Side note: The aforementioned functions O.get, O.set and O.modify in optics-ts are curried, meaning that you pass one argument at a time, and get a new function as the result. Only when you’ve passed all the arguments like this, the actual result is returned. This makes it possible to partially apply a function, by leaving some arguments out. For example, O.set(optic)(value) gives you a function that takes the input as a parameter. It can be passed around, given as an argument to other functions, etc.)

We say that we read through an optic when we use e.g. O.get to read the value at its focus, and write through the optic when using O.set. The read direction is when the optic is used for reading, and the write direction is when the optic is used for writing. Both directions are involved with O.modify.

Composing means that two or more optics are put together to create a new optic, which combines the functionality of both the optics. For example, we can compose optics a and b to create optic c. When we read through c, optic a is applied first. Optic b is then applied to the value or values that optic a produced to get the final value or values.

Optic types

There are a number of optic types. Different types behave differently and can be used to do different things with data. Let’s cover some common types next.

Isomorphism

The conceptually simplest optic type is an isomorphism. It applies a function on its input in the read direction, and the inverse function in the write direction. An isomorphism has one focus, which is its whole input.

An example of isomorphism would be one that reverses the characters of its input string. When read or written through, "Hello, world!" would become "!dlrow ,olleH".

Another example would be an isomorphism that adds a constant number to its input. When read through, the constant would be added to the input, and when written through, the constant would be subtracted.

Lens

A lens has one focus inside its input. Assume you have the following object:

type Book = {
  title: string
  isbn: string
  author: {
    name: string
  }
}

A lens could let you, for example, focus on the title property. It could also let you focus on a sub-object that has the title and isbn properties. If you have two lenses, first focusing on an author property and the second focusing on a name property, by composing them you get a lens that focuses on the author.name property. Composing lenses like this let you dig deeper into hierarchical data.

As an example, let’s consider the lens that focuses on the author.name property.

import * as O from 'optics-ts'

const optic = O.optic_<Book>()
  .prop('author')
  .prop('name')

// This is the input data
const input: Book = {
  title: "The Hitchhiker's Guide to the Galaxy"
  isbn: "978-0345391803",
  author: {
    name: "Douglas Adams"
  }
}

O.get(optic)(input)
// "Douglas Adams"
O.set(optic)("Arthur Dent")(input)
// {
//   title: "The Hitchhiker’s Guide to the Galaxy"
//   isbn: "978-0345391803",
//   author: {
//     name: "Arthur Dent"
//   }
// }
O.modify(optic)(str => str.length + 29)(input)
// {
//   title: "The Hitchhiker’s Guide to the Galaxy"
//   isbn: "978-0345391803",
//   author: {
//     name: 42
//   }
// }

Note that in the last example, the author name, which is a string, was replaced by a number. In general, it’s not required that the data is kept the same when writing through optics. When an optic allows this, it’s called a polymorphic optic, as opposed to a monomorphic optic with which the type must stay the same. A write operation is polymorphic if it changes the type of the value. In optics-ts, you must use O.optic_() to build polymorphic optics.

Prism

A prism may or may not have a focus in its input. Reading through a prism that doesn’t have a focus doesn’t return a value, and writing through a prism that doesn’t have a focus returns the input unmodified. We say that a prism matches if it has a focus on the input, and doesn’t match if there’s no focus in the input.

An example of a prism is one that focuses on the first element of an array. When read through, it returns the first element of the input, and when written through, it replaces the first element in the input with the new value. But if the input is an empty array, there is no first element, so the prism doesn’t match. In optics-ts, this prism’s name is head.

Another example prism would be one that focuses on a single branch of a union type, as the number in type T = number | undefined. The prism would match if the input was indeed a number, and fail to match if it was undefined. When written through, it would only actually do anything if the input was a number. This prism is called optional in optics-ts.

Because a prism might not have a focus, it doesn’t support the O.get operation. Instead, the operation is called O.preview, and it returns undefined if the prism doesn’t match.

Traversal

A traversal has zero or more focuses. Reading through a traversal returns zero or more values, and writing through it modifies all the focuses in the input.

An example would be a traversal that focuses on all elements of an array (elems in optics-ts), and could be used to modify all of them in a single operation. Writing a constant value through this traversal with O.set would replace all the array elements with that value, and using O.modify would be equal to mapping over the array elements.

Another example of a traversal is one that focuses on the words of a string (words in optics-ts). There would be as many focuses as there are words in the input. The traversal could then be used to for example capitalize each word with O.modify.

The read operation of traversals is O.collect, and it returns an array containing all the focuses of the traversal in the given input.

The power of composition

The true power of optics rises from the composition. As already shown, composing lenses allow one to dig deeper into a hierarchical data structure. But composition is in no way limited to this!

Rules of composition

Thinking of what happens when composing different types of optics, like when composing a prism with a traversal, yields a hierarchy for optics:

Isomorphism <- Lens <- Prism <- Traversal

When two optics are composed, the result is the one of the two which appears rightmost in the hierarchy. For example, if you compose a lens and a prism, the result is a prism. If you compose a traversal with an isomorphism, the result is a traversal.

Why? The number of focuses is one way to look at it. A lens has one focus, and if you apply a prism after a lens the number of focuses may drop to zero. Likewise, if you compose a traversal with an isomorphism, the number of focuses stays as zero or more.

The difference between an isomorphism and a lens is subtle. Both have one focus, but the isomorphism can only act on its whole input, whereas the lens can focus on a “smaller area” inside the input. When writing through a lens, it has to “keep track” of the parts of the input it left out when passing its focus to the next optic in the composition chain, so that when it receives a value to replace its focus with, it can put it into the “context” of the whole input. So in a way, the lens is “stateful” within a single operation. This statefulness puts the lens after the isomorphism in our hierarchy.

There’s actually one more optic type in the hierarchy we haven’t talked about: equivalence. It’s really the simplest of all optics, as it does nothing. It just returns its input as-is in both read and write directions. It acts as the identity with respect to composition. That is, composing an equivalence with any other optic type always yields that other optic type.

Adding equivalence yields our final optics hierarchy:

Equivalence <- Isomorphism <- Lens <- Prism <- Traversal

In optics-ts, the “root” of all optics operations, O.optic, returns an equivalence. Composition happens by calling methods, starting from the equivalence.

Number of focuses

As shown above, composing optics affects the number of focuses, and thus, the optic type. Composing with an isomorphism or a lens keeps the number of focuses the same, but focusing with a prism or a traversal (0..1 and 0..n focuses) can actually reduce the number of focuses.

This has some interesting applications. For example, imagine you have a traversal that focuses on all words of a string:

const optic = O.optic<string>().words()
const input = "This is a string with some shorter and some longer words"

O.collect(optic)(input)
// ["This", "is", "a", "string", "with", "some", "shorter", "and", "some", "longer", "words"]

We can compose this traversal with a prism that matches only on strings that are at least 5 characters long:

const optic2 = O.optic<string>()
  .words()
  .when(s => s.length >= 5)
O.collect(optic2)(input)
// ["string", "shorter", "longer", "words"]

With the same input, we’re left with 4 focuses. Now we can for example transform the focuses to upper case:

O.modify(optic2)(s => s.toUpperCase())(input)
// "This is a STRING with some SHORTER and some LONGER WORDS"

Another way of composing

Composing is not limited to feeding the output of the previous optic as an input to the next. There’s a lens called partsOf, which lets you focus on an array of the focuses of a given traversal. This unlocks some very powerful ways to transform data.

Let’s assume we have a library of books and want to do something to all the authors of those books:

type Book = {
  title: string
  author: {
    name: string
  }
}
type Library = {
  books: Book[]
}
const optic = O.optic<Library>()
  .prop('books')
  .partsOf(o => o.elems().prop('author').prop('name'))

The focus of this optic is now an array that contains all the author names. This makes it easy to, for example, rewrite the authors in alphabetical order:

const input = {
  books: [
    {
      title: "The Lord of the Rings",
      author: { name: "J.R.R. Tolkien" }
    },
    {
      title: "Vingt mille lieues sous les mers",
      author: { name: "Jules Verne" }
    },
    {
     title: "The Hitchhiker's Guide to the Galaxy",
     author: { name: "Douglas Adams" }
    },
  ],
}
 O.modify(optic)(names => [...names].sort())(input)
// {
//   books: [
//     {
//       title: "The Lord of the Rings",
//       author: { name: "Douglas Adams" }
//     },
//     {
//       title: "Vingt mille lieues sous les mers",
//       author: { name: "J.R.R. Tolkien" }
//     },
//     {
//       title: "The Hitchhiker's Guide to the Galaxy",
//       author: { name: "Jules Verne" }
//     },
//   ],
// }

What next?

This post gave a brief introduction to optics terminology and optics types and gave some examples of their power. After reading it, I hope you can more easily follow other material about optics.

If you’ve become interested in using optics with TypeScript, check out optics-ts on GitHub.

You can also listen to me talking about this topic in this Fork Pull Merge Push podcast episode.

Never miss a post