combiner receives struct_pb2.ListValue instead of numpy list

[Ridooo]

I am building a seldon deployment inference graph consists of a combiner which receives data from two child models. the models output a numpy list which is received by the combiner as google.protobuf.struct_pb2.ListValue. should it be received at the combiner the same type as it was sent from the model?

Model's output:
[[[211, 267, 320]], [[197, 251, 302]]] >> numpy listcomp2 combiner input:

Received in combiner:
[[values {number_value: 211.0}
values { number_value: 267.0}
values { number_value: 320.0}]
[values { number_value: 197.0}
values { number_value: 251.0}
values { number_value: 302.0}]]

seldon-core verion used in the combiner and model is '0.3.1'

[ukclivecox]

Yes that is strange. The code for the standard aggregate is

seldon-core/python/seldon_core/seldon_methods.py

Lines 214 to 220 in f0a940d

	for msg in request.seldonMessages:
	(features, meta, datadef, data_type) = extract_request_parts(msg)
	features_list.append(features)
	names_list.append(datadef.names)
	client_response = client_aggregate(user_model, features_list, names_list)
	return construct_response(user_model, False, request.seldonMessages[0], client_response)

[Ridooo]

Here how to simulate the case:

from seldon_core.proto import prediction_pb2
from seldon_core.utils import array_to_grpc_datadef , extract_request_parts
from seldon_core.utils import array_to_list_value
import numpy as np

#output= [[[[311, 371, 430], [310, 372, 431]], [[214, 283, 348], [221, 290, 357]]],[[['EUS'], ['EUS']], [['EUS'], ['EUS']]]]
#output= [[[311, 371, 430], [310, 372, 431]], [[214, 283, 348], [221, 290, 357]]]
output1 = [[311],['hello']] 
output2 = [[311,350],['hello']]
output = np.array(output1)

meta = prediction_pb2.Meta()
names = []
default_data_type = "ndarray"
data = array_to_grpc_datadef(default_data_type, output, names)
seldon_msg = prediction_pb2.SeldonMessage(data=data, meta=meta)

features = extract_request_parts(seldon_msg) 
# here is what happens inside grpc_datadef_to_array
print(seldon_msg.WhichOneof("data_oneof"))
print(seldon_msg.data.WhichOneof("data_oneof"))
#np.array(np.array(msg.data.ndarray)[0][0][0])
np.array(seldon_msg.data.ndarray)

when you are using the output1,features will be:
array([['311'],
['hello']], dtype='<U5')
which is good, however, if you use output2, then features which what will be received by the combiner this input:

array([values {
number_value: 311.0
}
values {
number_value: 350.0
}
,
values {
string_value: "hello"
}
], dtype=object)

[Ridooo]

This issue can be fixed by using seldon_message_to_json() instead of np.array() in this function grpc_datadef_to_array for data_type == "ndarray"

[ukclivecox]

Can you test on latest 0.4.0 release?

[axsaucedo]

@Ridooo this is now fixed with #600 via #817.