Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Get Space Error Operation timed out #40

Closed
shutter-cp opened this issue Feb 24, 2022 · 5 comments
Closed

Get Space Error Operation timed out #40

shutter-cp opened this issue Feb 24, 2022 · 5 comments

Comments

@shutter-cp
Copy link

shutter-cp commented Feb 24, 2022 •
edited
Loading

能插入部分数据，有大概30个tag，能插入9个，后续就不断的time out

论坛

22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:16:50 ERROR Executor: Exception in task 0.0 in stage 34.0 (TID 539)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 3.0 in stage 34.0 (TID 542)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 11.0 in stage 34.0 (TID 550)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 6.0 in stage 34.0 (TID 545)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 7.0 in stage 34.0 (TID 546)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 10.0 in stage 34.0 (TID 549)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 8.0 in stage 34.0 (TID 547)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 4.0 in stage 34.0 (TID 543)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 2.0 in stage 34.0 (TID 541)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 1.0 in stage 34.0 (TID 540)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 5.0 in stage 34.0 (TID 544)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR Executor: Exception in task 9.0 in stage 34.0 (TID 548)
com.facebook.thrift.transport.TTransportException: java.net.ConnectException: Operation timed out (Connection timed out)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:206)
	at com.vesoft.nebula.client.meta.MetaClient.getClient(MetaClient.java:148)
	at com.vesoft.nebula.client.meta.MetaClient.freshClient(MetaClient.java:168)
	at com.vesoft.nebula.client.meta.MetaClient.getSpace(MetaClient.java:230)
	at com.vesoft.nebula.connector.nebula.MetaProvider.getVidType(MetaProvider.scala:66)
	at com.vesoft.nebula.connector.writer.NebulaWriter.<init>(NebulaWriter.scala:40)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriter.<init>(NebulaVertexWriter.scala:18)
	at com.vesoft.nebula.connector.writer.NebulaVertexWriterFactory.createDataWriter(NebulaSourceWriter.scala:28)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:113)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Operation timed out (Connection timed out)
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:607)
	at com.facebook.thrift.transport.TSocket.open(TSocket.java:201)
	... 18 more
22/02/24 17:16:50 ERROR TaskSetManager: Task 4 in stage 34.0 failed 1 times; aborting job
22/02/24 17:16:50 ERROR WriteToDataSourceV2Exec: Data source writer com.vesoft.nebula.connector.writer.NebulaDataSourceVertexWriter@5cf70fbf is aborting.
22/02/24 17:16:50 ERROR NebulaDataSourceVertexWriter: NebulaDataSourceVertexWriter abort
22/02/24 17:16:50 ERROR WriteToDataSourceV2Exec: Data source writer com.vesoft.nebula.connector.writer.NebulaDataSourceVertexWriter@5cf70fbf aborted.
22/02/24 17:16:50 ERROR Utils: Aborting task
org.apache.spark.TaskKilledException
	at org.apache.spark.TaskContextImpl.killTaskIfInterrupted(TaskContextImpl.scala:149)
	at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:36)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage7.sort_addToSorter_0$(Unknown Source)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage7.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$13$$anon$1.hasNext(WholeStageCodegenExec.scala:636)
	at org.apache.spark.sql.execution.aggregate.SortAggregateExec$$anonfun$doExecute$1$$anonfun$3.apply(SortAggregateExec.scala:80)
	at org.apache.spark.sql.execution.aggregate.SortAggregateExec$$anonfun$doExecute$1$$anonfun$3.apply(SortAggregateExec.scala:77)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndexInternal$1$$anonfun$13.apply(RDD.scala:845)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndexInternal$1$$anonfun$13.apply(RDD.scala:845)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
	at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:100)
	at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:99)
	at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:435)
	at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:441)
	at org.apache.spark.sql.execution.columnar.CachedRDDBuilder$$anonfun$1$$anon$1.hasNext(InMemoryRelation.scala:125)
	at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221)
	at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)
	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1165)
	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1156)
	at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1091)
	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1156)
	at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:882)
	at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:357)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:308)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:346)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:310)
	at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:100)
	at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:99)
	at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:435)
	at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:441)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source)
	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
	at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$13$$anon$1.hasNext(WholeStageCodegenExec.scala:636)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$$anonfun$run$3.apply(WriteToDataSourceV2Exec.scala:117)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$$anonfun$run$3.apply(WriteToDataSourceV2Exec.scala:116)
	at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1394)
	at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:146)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
	at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
22/02/24 17:16:50 ERROR DataWritingSparkTask: Aborting commit for partition 16 (task 555, attempt 0, stage 34.0)
22/02/24 17:16:50 ERROR NebulaVertexWriter: insert vertex task abort.
22/02/24 17:16:50 ERROR DataWritingSparkTask: Aborted commit for partition 16 (task 555, attempt 0, stage 34.0)
22/02/24 17:18:05 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:18:05 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:18:05 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:18:05 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:18:05 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
22/02/24 17:18:05 ERROR MetaClient: Get Space Error: java.net.ConnectException: Operation timed out (Connection timed out)
@wey-gu
Copy link
Contributor

wey-gu commented Feb 25, 2022

请问 spark 运行在容器网络 nebula-net 么？

可能是无法访问metaclient，因为写路径只需要 graphD、graphclient，而默认不在容器网络里，默认compose的配置下，只有 一个 graphD 外部可以连接

@shutter-cp
Copy link
Author

请问火花在容器网络星云网上运行？

可能无法访问元数据，因为写路径只需要图形和图形客户端，而默认不在容器网络中。默认合成配置下，只有图形外部可以连接

spark 运行在我自己笔记本，他可以插入一部分数据进去

@wey-gu
Copy link
Contributor

wey-gu commented Feb 25, 2022

您好，问题是这样的，Spark 的写入过程和其他 SDK 一样，都是通过 graphD，但是它的读取路径（因为spark connector 被认为读取的场景是为了 anlayitical 的场景使用的）是不通过 graphD，直接去读 metad/storageD。

所以您可以写入不意味着可读。

详细可以参考我在这个视频里介绍的 https://www.bilibili.com/video/av509305299

@wey-gu
Copy link
Contributor

wey-gu commented Feb 25, 2022 •
edited
Loading

Sorry I assumed you could speak Chinese, if not, this: https://www.youtube.com/watch?v=hlNzrbqTcP4 is the equivalent information in English to that video.

@jamieliu1023 jamieliu1023 mentioned this issue Feb 26, 2022
Closed
@Sophie-Xie
Copy link
Contributor

Close it first, and open it again if there is new information.

@wey-gu wey-gu mentioned this issue Dec 3, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@wey-gu @shutter-cp @Sophie-Xie