Getting "java.lang.StackOverflowError" when reading avro file

[imlk]

I got "java.lang.stackOverflowError" when using sqlContext.load(filePath, "com.databricks.spark.avro") to read a specific avro file to a DataFrame.
However, I could not reproduce this issue on the sample episode.avro file that you provided; thus it seems like this is something related to the schema of the avro file. I realize that the schema of the avro file is much more complicated than the schema of episode.avo file.
But in the meanwhile, when I was using either the avro-tool.jar tool that provided by Apache or the Python API, it can read and parse our avro file without any issues.
Do you think there is a compatibility issue between the spark-avro and the avro file schema we use ? Is there any workaround that might fix it for now ? Below is part of the error stack. Thanks !

15/05/11 17:00:20 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/05/11 17:00:20 INFO SparkUI: Started SparkUI at http://10.0.1.11:4040
15/05/11 17:00:20 INFO Executor: Starting executor ID  on host localhost
15/05/11 17:00:20 INFO AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@10.0.1.11:57777/user/HeartbeatReceiver
15/05/11 17:00:20 INFO NettyBlockTransferService: Server created on 57779
15/05/11 17:00:20 INFO BlockManagerMaster: Trying to register BlockManager
15/05/11 17:00:20 INFO BlockManagerMasterActor: Registering block manager localhost:57779 with 2.9 GB RAM, BlockManagerId(, localhost, 57779)
15/05/11 17:00:20 INFO BlockManagerMaster: Registered BlockManager
Exception in thread "main" java.lang.StackOverflowError
    at scala.collection.mutable.ArrayBuffer.(ArrayBuffer.scala:47)
    at scala.collection.mutable.ArrayBuffer.(ArrayBuffer.scala:62)
    at scala.collection.mutable.Buffer$.newBuilder(Buffer.scala:44)
    at scala.collection.generic.GenericTraversableTemplate$class.newBuilder(GenericTraversableTemplate.scala:64)
    at scala.collection.AbstractTraversable.newBuilder(Traversable.scala:105)
    at scala.collection.TraversableLike$class.filter(TraversableLike.scala:262)
    at scala.collection.AbstractTraversable.filter(Traversable.scala:105)
    at scala.collection.TraversableLike$class.filterNot(TraversableLike.scala:274)
    at scala.collection.AbstractTraversable.filterNot(Traversable.scala:105)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:72)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:51)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:50)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:50)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:58)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:74)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:51)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:50)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:50)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:74)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:51)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:50)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:50)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:58)
    at com.databricks.spark.avro.SchemaConverters$.toSqlType(SchemaConverters.scala:74)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:51)
    at com.databricks.spark.avro.SchemaConverters$$anonfun$1.apply(SchemaConverters.scala:50)

[aarondav]

Try to increase the driver's JVM stack size, adding something like -Xss5m to the Java driver options. It is likely that your schema is both deeply nested and that we use several levels of stack per nesting level of schema.

[imlk]

Thank you. While even I tried increasing the JVM stack size all the way to -Xss1000m, it throws the stackOverflowError in the end still. Bigger JVM stack size could run for longer, though.
The avrò file I am using, has about 4-5 nesting level in the schema.

[aarondav]

Ah, 4-5 levels of nesting should not have caused this issue. It's not possible that your schema is somehow recursive, right? Would it be possible to get access to the schema?

[imlk]

@aarondav Thank you. I am not allowed to give the schema, but I will try to see if I could find out the problem of the schema and let you know.

[ajetivar]

I have the same issue as well. Wondering if @aarondav found a way to handle this? My schema is also nested with about 6-7 levels. The moment I load the file it throws a stack overflow exception.

[aarondav]

I don't have any new information on the topic..

[ajetivar]

@aarondav Are nested(recursive) schema's supported? If yes, how do I specify a max_depth so that the stack does not blow up ? ex:- Trade -> Product -> Underlyer -> Product -> Underlyer .....

[mkanchwala]

Hi Guys,

I've been also getting same issue : #110

Please let me know what should I do.

Thanks

[dmitriy-sbry]

Hello folks,

Are there any updates on this? I'm running into the same issues with a deeply-nested Avro schema.

Thank you!

[dmitriy-sbry]

**Update: My issue was caused by a circular dependency within the Avro Schema.

[JoshRosen]

Okay, so it sounds like #110 is not, in fact, a duplicate of this issue but is a different StackOverflow in a different phase of query processing. This ticket is now pretty old and it doesn't sound like there are any reported new reproductions of this, so I'm going to optimistically assume this is fixed in newer versions and will close this now. Please file a new ticket if this is still a problem with current version of Spark or spark-avro.

[atm]

I'm also encountering this issue, and I concur with @dmitriy-sbry that this is indeed due to nested records in the Avro schema. However, this is legal under Avro, it's just that the spark-avro schema processor doesn't support this.

You can reproduce this easily just by using an example schema from the Avro docs (https://avro.apache.org/docs/1.8.2/spec.html#schema_record), i.e.:

{
  "type": "record",
  "name": "LongList",
  "aliases": ["LinkedLongs"],                      // old name for this
  "fields" : [
    {"name": "value", "type": "long"},             // each element has a long
    {"name": "next", "type": ["null", "LongList"]} // optional next element
  ]
}

If you put the above schema in a file called NestedRecord.avsc, and then run this Scala code, you'll get a stack overflow:

import org.apache.avro.Schema
import com.databricks.spark.avro.SchemaConverters
import java.io.File

val schema = new Schema.Parser().parse(new File("NestedRecord.avsc"))
SchemaConverters.toSqlType(schema)

@JoshRosen - this seems like a legit bug to me. If you agree, would you mind reopening this issue?