-
Notifications
You must be signed in to change notification settings - Fork 196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
avro.shaded not in fatJar causing java.lang.NoClassDefFoundError #393
Comments
davidrabinowitz
added a commit
to davidrabinowitz/spark-bigquery-connector
that referenced
this issue
May 12, 2021
davidrabinowitz
added a commit
that referenced
this issue
May 12, 2021
Fixed by PR #394 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I'm using spark-bigquery-with-dependencies 0.20.0 and facing a
When calling com.google.cloud.spark.bigquery.SchemaConverters#toBigQuerySchema
It seems this class does an import on
avro.shaded.com.google.common.base.Preconditions
But
avro.shaded
is not included into the FatJar.If my searches are fine :
org.apache.avro
comes withhadoop-common
itselft coming withbeam-sdks-java-io-hadoop-common
but as a provided dependency. This is the reason why it's not embedded into the fatJar.To fix the issue I argue to replace com.google.cloud.spark.bigquery.SchemaConverters:18
import avro.shaded.com.google.common.base.Preconditions;
with
import com.google.common.base.Preconditions;
Thanks ;)
The text was updated successfully, but these errors were encountered: