-
Notifications
You must be signed in to change notification settings - Fork 711
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem with spark-nlp #995
Comments
What is |
Also, please complete the template we provide, we need that information in order to reproduce and help. |
@maziyarpanahi
Unfortunately, I couldn't complete the template and I stuck here. |
@maziyarpanahi |
Thanks. What is your Operating System (with distribution and version)? This seems to be an issue with In the meantime please add these two configs to your Spark session:
|
@maziyarpanahi |
Got it. So the cluster is up and running and you just run that code inside of a Jupyter? Is there a way to share that coredump with us? (it seems there might be something installed already that has a conflict with the C++ in libtensorflow_framework.so.1) |
@maziyarpanahi |
It says it was written here: |
@maziyarpanahi
|
OK, no worries. |
@maziyarpanahi |
Thank you, but just in case other PyPI packages can come with either one of them, could you please check all the PyPI packages in that environment to be sure? |
@maziyarpanahi |
I meant if you activate the same Python environment, and run |
@maziyarpanahi |
Great, thank you very much. We are working on this to reproduce and find a workaround. |
@maziyarpanahi |
@albertoandreottiATgmail It seems Ubuntu 18 has something that conflicts with TensorFlow similar to this issue: tensorflow/tensorflow#24976 @m-developer96 I will try to reproduce this on a fresh Ubuntu 18 updated to the latest today/tomorrow |
Hello @m-developer96 , that signal the process is receiving is supposed to happen when the process runs an instruction that the current architecture cannot handle. jose@machine:~/.ivy2$ unzip ./cache/org.tensorflow/libtensorflow_jni/jars/libtensorflow_jni-1.15.0.jar org/tensorflow/native/linux-x86_64/libtensorflow_jni.so jose@machine:~/.ivy2$ ldd org/tensorflow/native/linux-x86_64/libtensorflow_jni.so and post the paths you get on your system? |
I am facing the same issue when trying to train using BertEmbeddings from spark nlp. I am using centOS and spark-nlp version 2.6.2. Is this issue fixed and is there a solution? |
@phoenix1391 Would you mind creating a new issue with a complete template for us to reproduce this issue? (the more info we have the more chance we can reproduce this, especially the OS and its version). Unfortunately, there is no new update on this issue but I am hoping with your new issue we can reproduce it or say what is not compatible at least. |
This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 5 days |
Hi!
I'm using this example to create my own sentiment classifier but when I want to execute the below code, I got an error.
I tested it with
UniversalSentenceEncoder
but got the same error.The error:
I used standalone cluster mode with one master and 3 slaves with 4G memory and 4 core for each one at first. Then I used one master and one slave with 10G memory and 6 core for each one. But still got the same error.
My spark initialization:
How can I fix it?
Thanks for your help :)
The text was updated successfully, but these errors were encountered: