Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

persistent memory cache issue #317

Closed
zhouyuan opened this issue May 12, 2021 · 1 comment · Fixed by #346
Closed

persistent memory cache issue #317

zhouyuan opened this issue May 12, 2021 · 1 comment · Fixed by #346
Labels
bug Something isn't working

Comments

@zhouyuan
Copy link
Collaborator

py4j.protocol.Py4JJavaError: An error occurred while calling o87.persist.
: java.lang.NoSuchMethodError: org.apache.spark.sql.execution.columnar.InMemoryRelation$.apply(Lorg/apache/spark/storage/StorageLevel;Lorg/apache/spark/sql/execution/QueryExecution;Lscala/Option;)Lorg/apache/spark/sql/execution/columnar/InMemoryRelation;
        at org.apache.spark.sql.execution.CacheManager.$anonfun$cacheQuery$2(CacheManager.scala:102)
        at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:772)
        at org.apache.spark.sql.execution.CacheManager.cacheQuery(CacheManager.scala:97)
        at org.apache.spark.sql.Dataset.persist(Dataset.scala:3187)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at py4j.Gateway.invoke(Gateway.java:282)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.GatewayConnection.run(GatewayConnection.java:238)
        at java.lang.Thread.run(Thread.java:748)

@zhouyuan
Copy link
Collaborator Author

@rui-mo here's the issue on cache

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants