Skip to content

New generators from avro files

Closed Jul 16, 2016 100% complete

In this milestone we implement a ScalaCheck generator from a file in the local file system that contains the test cases. We use Avro as the serialization format for the test cases in order to store the test cases together with a its schema, and because this serialization format offers a rich data model.

The main difficult here is that the generator needs …

In this milestone we implement a ScalaCheck generator from a file in the local file system that contains the test cases. We use Avro as the serialization format for the test cases in order to store the test cases together with a its schema, and because this serialization format offers a rich data model.

The main difficult here is that the generator needs to be stateful, as it needs to keep a cursor to the record in the input file that will be used to generate the next test case. ScalaCheck doesn't offer any feature for stateful generators, and the base trait Gen for generators is a sealed Scala trait, so we cannot implement it directly, but we can implement another class together with an implicit conversion to Gen. That is done for the simpler case of a generator FromIterableGen that generates the element of an iterator in order, in https://gist.github.com/juanrh/23b813b2b31bb20f6254. This should be polished by:

  • defining a trait for generators with state
  • reimplementing FromIterableGen with that trait. Add a boolean parameter defaultToLast to set whether the generator should fail in subsequent calls when all the elements have already been generated, or it should return the last value in the input iterable after that
  • implementing FromAvroFileGen with that trait. A first version takes a String with the path to the file, and defaultToLast. This generator generates a value for each of the records in the input file.

To be determined:

  • Ideally both the json and binary avro encoding would be supported in the files.

This milestone is closed.

No open issues remain. View closed issues or see open milestones in this repository.