Skip to content
This repository has been archived by the owner on Mar 10, 2023. It is now read-only.

AbortionError(code=StatusCode.FAILED_PRECONDITION, details="Attempting to use uninitialized value conv2d/kernel [[Node: conv2d/kernel/read = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](conv2d/kernel)]] [[Node: ArgMax/_3 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_94_ArgMax", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]") #7

Open
ps-account opened this issue Aug 9, 2018 · 4 comments

Comments

@ps-account
Copy link

Hi, I built the model in TF-serving-gpu, copied it to the server, and it starts fine.

When calling I get an error however:

root@a0bf24294b1b:/tf-serving-example/tensorflow-serving-example# python python/grpc_mnist_client.py --image ./data/0.png --model mnist --host localhost
[[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
44. 140. 254. 183. 12. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 125.
229. 253. 253. 253. 65. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 68. 164. 217.
254. 253. 253. 253. 227. 164. 78. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 64. 246. 253. 253.
254. 253. 182. 253. 253. 253. 182. 78. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 72. 240. 253. 253. 253.
254. 68. 21. 178. 245. 253. 253. 243. 52. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 72. 232. 253. 253. 248. 223.
74. 13. 0. 0. 119. 253. 253. 253. 77. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 224. 253. 253. 253. 128. 0.
0. 0. 0. 0. 36. 214. 253. 253. 232. 56. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 6. 161. 251. 253. 253. 116. 4. 0.
0. 0. 0. 0. 0. 38. 253. 253. 253. 104. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 130. 253. 253. 248. 128. 4. 0. 0.
0. 0. 0. 0. 0. 7. 162. 253. 253. 104. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 36. 217. 253. 253. 223. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 134. 253. 253. 192. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 211. 254. 254. 223. 44. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 135. 255. 254. 255. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 87. 246. 253. 253. 55. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 134. 253. 253. 253. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 114. 253. 253. 246. 27. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 134. 253. 253. 253. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 254. 253. 253. 133. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 86. 191. 253. 253. 182. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 254. 253. 240. 16. 0. 0. 0. 0. 0. 0.
0. 0. 5. 75. 120. 245. 253. 253. 240. 73. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 148. 253. 249. 109. 22. 0. 0. 0. 0. 18.
31. 154. 184. 253. 253. 253. 253. 253. 190. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 105. 253. 253. 253. 218. 134. 134. 134. 134. 205.
254. 253. 253. 253. 253. 253. 253. 126. 28. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 6. 212. 253. 253. 253. 253. 253. 253. 253. 253.
255. 253. 253. 253. 253. 242. 84. 1. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 49. 158. 253. 253. 253. 253. 253. 253. 253.
236. 208. 199. 129. 164. 52. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 7. 104. 236. 253. 253. 164. 226. 104.
62. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]
Traceback (most recent call last):
File "python/grpc_mnist_client.py", line 56, in
run(args.host, args.port, args.image, args.model, args.signature_name)
File "python/grpc_mnist_client.py", line 34, in run
result = stub.Predict(request, 10.0)
File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 309, in call
self._request_serializer, self._response_deserializer)
File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 195, in _blocking_unary_unary
raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.FAILED_PRECONDITION, details="Attempting to use uninitialized value conv2d/kernel
[[Node: conv2d/kernel/read = IdentityT=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
[[Node: ArgMax/_3 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_94_ArgMax", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]")

@ps-account
Copy link
Author

@ps-account
Copy link
Author

( It could be related that I'm building this in TF-serving-gpu 1.9.0 )

@ps-account
Copy link
Author

it probably is tensorflow/serving#594

will try again later either moving your example to 1.9.0 or moving my tf-serving-gpu to 1.8

@ps-account
Copy link
Author

Hmmm, I did a pip install tensorflow-gpu==1.9.0 and run tf-serving-gpu 1.9.0 but I got the same error

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant