Technology

Working with JSON in Scala can be as simple as in JavaScript

May 21, 2015

Read time 3 min

Whatever your project is, nowadays you almost certainly need to work with JSON. And why not? It’s extremely simple to learn and to work with, with only six (or seven if you separate integers and doubles) data types. Although JSON is so useful and ubiquitous, how is it that working with it is still a pain in the ass in Scala?

Simple JSON, complex libraries

There are many good JSON libraries for Scala, the likes of spray-json and Json4s. However, they mask the point of JSON: simplicity. For example, adding json4s-jackson to the project’s dependencies downloads almost 2 megabytes of binaries. This amount of code enables features beyond imagination. Here’s an example:

val marker = JsonMethods.parse("""{"title": "tsers", "location": {"lat": 60.166, "lng": 24.942}}""")
println("Latitude is: " + (marker  "location"  "lat").extract[Double])

So, looks simple and clean, but has one major problem: it does not compile. Why? It’s because Json4s requires implicit val formats = org.json4s.DefaultFormats so that it can extract dates from JSON. But why should I need to know that if I want to export a double from my marker object? JSON doesn’t even have dates!

The use of these fancy DSLs may also cause unwanted bugs if you’re not familiar with them. Here’s a real-life example:

val doc1 = JsonMethods.parse("""{"keywords": [{"value": "foo"}, {"value": "bar"}]}""")
println("Keywords: " + (doc1  "keywords"  "value").extract[Set[String]])
val doc2 = JsonMethods.parse("""{"keywords": [{"value": "tsers"}]}""")
println("Keywords: " + (doc2  "keywords"  "value").extract[Set[String]])

The second line prints “Keywords: Set(foo, bar)”, but the surprise comes when executing the fourth line. Instead of “Keywords: Set(tsers)”, you’ll get this:

org.json4s.package$MappingException: Expected collection but got JString(tsers) for root JString(tsers) and mapping Set[String]

…that actually caused a new production deployment in our project. So, be careful when using DSLs.

Another scary part of DSLs and Scala implicits is that a single import can change the entire behaviour of your code. When producing JSON, be sure of which of the following imports is being used:

import org.json4s.JsonDSL._
import org.json4s.JsonDSL.WithDouble._
import org.json4s.JsonDSL.WithBigDecimal._

Native experience with Scala dynamics

How do you avoid these complex DSLs? JSON is just a nested data structure containing objects, arrays and primitives. The main problem is Scala’s strict type system that does not allow access to nested values without a defined schema. Fortunately, Scala 2.10 introduced dynamic invocations that solve the problem.

Using the dynamics is very easy. After enabling it and extending your class with the Dynamic trait, the compiler automatically transforms the code into calls of applyDynamic, applyDynamicNamed, selectDynamic and updateDynamic. For JSON’s case it looks like:

import scala.language.dynamics

object JValue { def parse(json: String): JValue = ... }
class JValue extends Dynamic {
def applyDynamic(field: String)(key: Any): JValue = {
  // return array element or object field value
}
def selectDynamic(field: String): JValue = {
  // return object field value
}
def toDouble: Double = {
  // return value or throw exception if not double
}
...
}

val marker = JValue.parse("""{"title": "tsers", "location": {"lat": 60.166, "lng": 24.942}}""")
println("Latitude is: " + marker.location.lat.toDouble)
println("Latitude is: " + marker.selectDynamic("location").selectDynamic("lat").toDouble) // opened form

If JValue implements Traversable trait, it allows also other cool stuff like clean manipulation of arrays:

val doc = Zeison.parse("""{"keywords": [{"value": "foo"}, {"value": "bar"}]}""")
println("Keywords: " + doc.keywords.map(_.value.toStr).toSet)

Almost like native JavaScript experience – and with zero implicits!

Conclusion

Although Scala has a strict type system that restricts the processing of dynamic JSON documents, it’s still possible to build lightweight and almost native JavaScript JSON experiences without hidden implicits and complex DSLs. For more information, you can see the Zeison project that uses the techniques described above.

However, there is no such thing as a silver bullet. There are still many cases when higher level functions (such as Json4s transformers) are needed and it’s up to the developer’s expertise to make the desicion which one to use.

But whatever the case is, just do not try to use a sledgehammer to crack a nut.

Never miss a post