Executor sample gRPC local loggger fails

[glindsell]

Running the gRPC local logger sample found here:

https://github.com/SeldonIO/seldon-core/tree/master/executor/samples/local/logger

The following error occurs on the final step make grpc_test :

cd ../../../proto && grpcurl -d '{"data":{"ndarray":[[1.0,2.0]]}}' -plaintext -proto ./prediction.proto  0.0.0.0:5000 seldon.protos.Seldon/Predict
ERROR:
  Code: Unavailable
  Message: transport is closing
Makefile:32: recipe for target 'grpc_test' failed
make: *** [grpc_test] Error 1

The output from the local executor make run_grpc_executor :

../../../executor --sdep seldon-model --namespace default --predictor example --file ./model_grpc.yaml --http_port 8000 --grpc_port 5000 --transport grpc
{"level":"info","ts":1588148404.286985,"logger":"entrypoint","msg":"Flags","transport":"grpc"}
{"level":"info","ts":1588148404.2870219,"logger":"entrypoint","msg":"Trying to get predictor from file"}
ERRO[0000] open /etc/podinfo/annotations: no such file or directoryFailed to load annotations  source="main.go:215"
{"level":"info","ts":1588148404.2898488,"logger":"entrypoint","msg":"Starting","worker":1}
{"level":"info","ts":1588148404.289894,"logger":"entrypoint","msg":"Starting","worker":2}
{"level":"info","ts":1588148404.289902,"logger":"entrypoint","msg":"Starting","worker":3}
{"level":"info","ts":1588148404.289908,"logger":"entrypoint","msg":"Starting","worker":4}
{"level":"info","ts":1588148404.289913,"logger":"entrypoint","msg":"Starting","worker":5}
{"level":"info","ts":1588148404.2921605,"logger":"entrypoint","msg":"Running http probes only server ","port":8000}
{"level":"info","ts":1588148404.2922218,"logger":"entrypoint","msg":"Running grpc server ","port":5000}
{"level":"info","ts":1588148404.2925289,"logger":"entrypoint","msg":"Setting max message size ","size":2147483647}
{"level":"info","ts":1588148404.29388,"logger":"SeldonRestApi","msg":"Listening","Address":"0.0.0.0:8000"}
panic: interface conversion: interface {} is nil, not string

goroutine 121 [running]:
github.com/seldonio/seldon-core/executor/predictor.(*PredictorProcess).logPayload(0xc0005da730, 0xc000186df0, 0xa, 0xc000331180, 0x17fa33b, 0x7, 0x1a39220, 0xc0002ae200, 0xc0005a8420, 0xc0005a8420)
	/home/gl/tmp/multiplex/seldon-core/executor/predictor/predictor_process.go:263 +0x3f9
github.com/seldonio/seldon-core/executor/predictor.(*PredictorProcess).Predict(0xc0005da730, 0xc0000f2a00, 0x1a39220, 0xc0002ae200, 0xc000598640, 0x0, 0x0, 0x203000)
	/home/gl/tmp/multiplex/seldon-core/executor/predictor/predictor_process.go:271 +0x328
github.com/seldonio/seldon-core/executor/api/grpc/seldon.GrpcSeldonServer.Predict(0x1a72c60, 0xc0000e9040, 0xc000202120, 0x1a5e200, 0xc0002da160, 0xc000100000, 0x7ffceab1b13f, 0x7, 0x1a4f8a0, 0xc00032ff20, ...)
	/home/gl/tmp/multiplex/seldon-core/executor/api/grpc/seldon/server.go:37 +0x28e
github.com/seldonio/seldon-core/executor/api/grpc/seldon/proto._Seldon_Predict_Handler.func1(0x1a4f8a0, 0xc00032ff20, 0x1774b20, 0xc000256140, 0x181688d, 0x1d, 0x1a4f8a0, 0xc00032ff20)
	/home/gl/tmp/multiplex/seldon-core/executor/api/grpc/seldon/proto/prediction.pb.go:1538 +0x86
github.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1(0x1a4f8a0, 0xc00032f620, 0x1774b20, 0xc000256140, 0xc000598520, 0xc000598580, 0x484d16, 0x5ea938f1, 0x1f39637a, 0x1a66399e06b)
	/home/gl/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.1.0/tracing/opentracing/server_interceptors.go:31 +0x165
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x1a4f8a0, 0xc00032f620, 0x1774b20, 0xc000256140, 0x20, 0x20, 0x1699760, 0xc000256140)
	/home/gl/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.1.0/chain.go:25 +0x63
github.com/seldonio/seldon-core/executor/api/metric.(*ServerMetrics).UnaryServerInterceptor.func1(0x1a4f8a0, 0xc00032f620, 0x1774b20, 0xc000256140, 0xc000598520, 0xc0005985a0, 0x12eda2a, 0x1699760, 0xc0005985c0, 0xc000598520)
	/home/gl/tmp/multiplex/seldon-core/executor/api/metric/server.go:43 +0xa0
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x1a4f8a0, 0xc00032f620, 0x1774b20, 0xc000256140, 0xc0002b6000, 0x0, 0xc0003ddac0, 0x40e408)
	/home/gl/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.1.0/chain.go:25 +0x63
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1(0x1a4f8a0, 0xc00032f620, 0x1774b20, 0xc000256140, 0xc000598520, 0xc000598580, 0xc000081b30, 0x48df88, 0x16cf060, 0xc00032f620)
	/home/gl/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.1.0/chain.go:34 +0xd5
github.com/seldonio/seldon-core/executor/api/grpc/seldon/proto._Seldon_Predict_Handler(0x15eb960, 0xc000592200, 0x1a4f8a0, 0xc00032f620, 0xc00020e180, 0xc0002b0330, 0x1a4f8a0, 0xc00032f620, 0xc000118c40, 0x1e)
	/home/gl/tmp/multiplex/seldon-core/executor/api/grpc/seldon/proto/prediction.pb.go:1540 +0x14b
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000511200, 0x1a6b2e0, 0xc000511380, 0xc0002b6000, 0xc0002b0450, 0x2692c60, 0x0, 0x0, 0x0)
	/home/gl/go/pkg/mod/google.golang.org/grpc@v1.27.0/server.go:1024 +0x4f4
google.golang.org/grpc.(*Server).handleStream(0xc000511200, 0x1a6b2e0, 0xc000511380, 0xc0002b6000, 0x0)
	/home/gl/go/pkg/mod/google.golang.org/grpc@v1.27.0/server.go:1313 +0xd97
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000441d0, 0xc000511200, 0x1a6b2e0, 0xc000511380, 0xc0002b6000)
	/home/gl/go/pkg/mod/google.golang.org/grpc@v1.27.0/server.go:722 +0xbb
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/home/gl/go/pkg/mod/google.golang.org/grpc@v1.27.0/server.go:720 +0xa1
Makefile:26: recipe for target 'run_grpc_executor' failed
make: *** [run_grpc_executor] Error 2

[glindsell]

The odd thing here is that the metrics sample gRPC commands succeed:
https://github.com/SeldonIO/seldon-core/blob/master/executor/samples/local/metrics/Makefile

And are the exact same as the logger sample gRPC commands which fail:
https://github.com/SeldonIO/seldon-core/blob/master/executor/samples/local/logger/Makefile

[glindsell]

#1764 #1782