Scala View Server

Scala View Server

The default implementation of the query server in CouchDB uses Javascript running via Mozilla SpiderMonkey. However, language aficionados always find a way to push their own favorite into any accessible option. People have developed query servers for Ruby, Php, Python and Common Lisp.

scouchdb gives one for Scala. You can now write map and reduce scripts for CouchDB views in Scala. Here is a usual session using ScalaTest ..

// create some records in the store
http(test doc Js("""{"item":"banana","prices":{"Fresh Mart":1.99,"Price Max":0.79,"Banana Montana":4.22}}"""))
http(test doc Js("""{"item":"apple","prices":{"Fresh Mart":1.59,"Price Max":5.99,"Apples Express":0.79}}"""))
http(test doc Js("""{"item":"orange","prices":{"Fresh Mart":1.99,"Price Max":3.19,"Citrus Circus":1.09}}"""))

// create a design document
val d = DesignDocument("power", null, Map[String, View]())
d.language = "scala"

// a sample map function in Scala
val mapfn1 = """(doc: dispatch.json.JsValue) => {
  val it = sjson.json.JsBean.fromJSON(doc, Some(classOf[scouch.db.TestBeans.Item_1])); 
    for (st <- it.prices)
      yield(List(it.item, st._2))
}"""

// another map function
val mapfn2 = """(doc: dispatch.json.JsValue) => {
    import dispatch.json.Js._; 
    val x = Symbol("item") ? dispatch.json.Js.str;
    val x(x_) = doc; 
    val i = Symbol("_id") ? dispatch.json.Js.str;
    val i(i_) = doc;
    List(List(i_, x_)) ;
  }"""

Now the way the protocol works is that when the view functions are stored in the view server, CouchDB starts sending the documents one by one and every function gets invoked on every document. So once we create a design document and attach the view with the above map functions, the view server starts processing the documents based on the line based protocol with the main server. And if we invoke the views using scouchdb API as ..

http(test view(
  Views builder("power/power_lunch") build))

and

http(test view(

  Views builder(“power/mega_lunch”) build))

we get back the results based on the queries defined in the map functions. Have a look at the project home page for a complete description of the sample session that works with Scala view functions.

Reduce functions work similarly in Scala. The current implementation does not support rereduce .. watch out for more updates .. Here is a sample view query using map and reduce functions in Scala ..

// create the design document
val d = DesignDocument("big", null, Map[String, View]())
d.language = "scala"

// map function in Scala
val mapfn1 = """(doc: dispatch.json.JsValue) => {
  val it = sjson.json.JsBean.fromJSON(doc, Some(classOf[scouch.db.TestBeans.Item_1])); 
    for (st <- it.prices)
      yield(List(it.item, st._2))
}"""

// reduce function in Scala   
val redfn1 = """(key: List[(String, String)], values: List[dispatch.json.JsNumber], rereduce: Boolean) => {
  values.foldLeft(BigDecimal(0.00))((s, f) => s + (f match { case dispatch.json.JsNumber(n) => n }))
}"""

// create the view    
val vi_1 = new View(mapfn1, redfn1)

Create the document as shown earlier. Then use the view to do the query ..

val ls1 = 
  http(test view(
    Views.builder("big/big_lunch")
         .build))
ls1.size should equal(1)

reduce, by default returns only one row through a computation on the result set returned by map. The above query does not use grouping and returns 1 row as the result. You can also use view results grouping and return rows grouped by keys ..

val ls1 = 
  http(test view(
    Views.builder("big/big_lunch")
         .options(optionBuilder group(true) build) // with grouping
         .build))
ls1.size should equal(3)

Views and Scala Objects

scouchdb views are now seamlessly interoperable with Scala objects. The view API offers capabilities to construct Scala objects either as a post-process on the JSON results of the view or as a direct output from the view query itself.

Here is an example ..

The following map function returns the car make as the key and the car object as the value ..

// map function
val redCars =
  """(doc: dispatch.json.JsValue) => {
    val car = sjson.json.JsBean.fromJSON(doc, Some(classOf[scouch.db.CarSaleItem]));
    if (car.color.contains("Red")) List(List(car.make, car)) else Nil
 }"""

The following query returns JSON corresponding to the car objects being returned from the view ..

val ls1 = http(carDb view(
  Views builder("car_views/red_cars") build))

On the client side, we can have a simple map function that converts the returned collection into a collection of the specific class objects .. Here we have a collection of `CarSaleItem` objects ..

val objs =
  ls1.map { car =>
    val x = Symbol("value") ? obj
    val x(x_) = car
    JsBean.toBean(x_, classOf[CarSaleItem])._3
  }

But it gets better than this .. we can also have direct Scala objects being fetched from the view query directly through scouchdb API ..

// ls1 is now a list of CarSaleItem objects
val ls1 = http(carDb view(
  Views builder("car_views/red_cars") build, classOf[CarSaleItem]))

Note the class being passed as an additional parameter in the view API. Similar stuff is also being supported for views having reduce functions. This makes scouchdb more seamless for interoperability between JSON storage layer and object based application layer.

Setting up the View Server

The view server is an external program which will communicate with the CouchDB server. In order to set our scouchdb query server, we need to have the appropriate configurations set in initialization files.

• The common place to do custom settings for couchdb is local.ini. This can usually be found under /usr/local/etc/couchdb folder. Under [query_servers] in local.ini, you need to specify the command that couchdb will execute to run the view server. You need to include all jars that are required to run the Scala View Server. In the current version, the jars required are scouchdb-0.6.jar, sjson-0.7.jar, dispatch-json-0.7.4.jar, dispatch-http-json-0.7.4.jar and dispatch-http-0.7.4.jar. The Scala View Server also needs to have enough knowledge to run your map/reduce Scala scripts. The environment variable CDB_VIEW_CLASSPATH is used for this purpose. Add all jars necessary to run your map/reduce scripts here. Have a look at the following sample setting in my system.There has been some changes in the configuration files since CouchDB 0.9 – check out the wiki for them. In my system, I set the view server path as follows in local.ini ..
  

  
  
  [query_servers]
  
  scala=$SCALA_HOME/bin/scala -classpath <path to scouchdb jar/scouchdb-tests jar/sjson jar/dispatch-json jar> -DCDB_VIEW_CLASSPATH=<path to scouchdb jar/scouchdb-tests jar/sjson jar/dispatch-json jar> scouch.db.VS “/tmp/view.txt”
  
  

• scala is the language of query server that needs to be registered with CouchDB. Once you start futon after registering scala as the language, you should be able to see “scala” registered as a view query language for writing map functions.
• The classpath points to the jar where you deploy scouchdb and is used to run the view server of couchdb.
• couch.db.VS is the main program that interacts with the CouchDB server. Currently it takes as argument one file name where it sends all statements that it exchanges with the CouchDB server. If it is not supplied, all interactions are routed to the stderr.
• another change that I needed to make was setting of the os_process_timeout value. The default is set to 5000 (5 seconds). I made the following changes in local.ini ..
  

  
  
  [couchdb]
  
  os_process_timeout=20000
  
  

1. The passed in setting for CDB_VIEW_CLASSPATH should point to all jars that need to be passed to the Scala interpreter in order to execute the map/reduce functions.