To pass customized input json without the mandatory jsonData parameter

[divyadilip91]

Hi ,
Currently I see that seldon accepts input with keywords data, bindata,strdata or jsonData.
So it is a compulsory step to include our input data within the keyword otherwise seldon will consider the message as an invalid json.
Eg: if my input json to be passed is {"text":"hi"}, I need to enclose this with jsonData {"jsonData":{"text":"hi"}} and pass to seldon.
Could there be a way where in we can avoid this and pass the json without a restriction?
Please do reply.

[ukclivecox]

The executor service orchestrator is protocol agnostic for REST. The need would be for the python wrapper to not try to parse the request. @RafalSkolasinski is this possible now?

For gRPC there is the ability (in master) to have custom protos see: https://docs.seldon.io/projects/seldon-core/en/latest/examples/customdata_example.html

[RafalSkolasinski]

Not that I am aware. The whole reasoning behind having it as data (that would include ndarray or tensor in example) or strData/jsonData is to allow for different kind of inputs.

The jsonData field is serving exactly the purpose of passing json to the model. Features passed to in example predict function are extracted as

    elif "jsonData" in request:
        data_type = "jsonData"
        features = request["jsonData"]

This is also because wrapper uses top-level keys to pass additional information between different nodes in the graph - like custom tags.

Are there any downsides to having {"jsonData":{"text":"hi"}}?

edit: It should be possible using predict_raw, see Clive's message bellow

[divyadilip91]

Hi @cliveseldon and @RafalSkolasinski ,
Thankyou for your reply . There are no heavy downsides to following this format, but this query arised with the thought that, incase we need to create customer specific microservices using seldon and the customer requires their own input format of dictionary to be passed, would it not be an issue to restrict with a mandatory key within which only, their required format of input dictionary can be passed(same with strdata and bindata}. This may also cause consistency problems with other microservices that had been following a specific format and user readability too. So thought if there could be an extra method (function) where in messages can be passed in custom specific format without getting restricted with mandatory keywords.

It would be helpful if you could clarify the necessity of validating the type of data passed based on a keyword , rather than storing the type of data passed as input and validating the type which would avoid the usage of keywords.

elif "jsonData" in request: data_type = "jsonData" features = request["jsonData"] elif "strData" in request: data_type = "strData" features = request["strData"] elif "binData" in request: data_type = "binData" features = base64.b64decode(request["binData"])

Expecting your reply

[ukclivecox]

I think if you provide a predict_raw method it should work:

seldon-core/python/seldon_core/seldon_methods.py

Lines 85 to 91 in 88714de

	if hasattr(user_model, "predict_raw"):
	try:
	response = user_model.predict_raw(request)
	handle_raw_custom_metrics(response, seldon_metrics, is_proto)
	return response
	except SeldonNotImplementedError:
	pass

Can you try? @divyadilip91

[divyadilip91]

@cliveseldon But predict_raw requires a mandatory parameter rite.

I just tried passing the raw json {"text":"hi"} but it gave a bad microservice data error. Is there something that I have missed?

[divyadilip91]

this is my code basically, but received error

def predict_raw(self, request):
arg=request.get("image_base64")

my input json to be passed is {"image_base64":"base64string"}

[RafalSkolasinski]

It would be helpful if you post what errors you are getting, thanks!

[RafalSkolasinski]

It does work fine for me as long as the returned type from predict_raw is of type dict.

[RafalSkolasinski]

Please post both request output and microservice logs.

[divyadilip91]

Oh is it..Ok then let me check whats missing in my program..Will get back in few mins

[divyadilip91]

@cliveseldon @RafalSkolasinski
Cool I got it working finally...my new seldon service with the predict_raw() function was refering to the old service that used predict() function. By mistake i forgot to change the service name in the new seldon deployment yaml, hence i was getting the error" invalid data type with microservice bad data error. "

Thankyou so much for your help and immediate response.

[ukclivecox]

@divyadilip91 glad its working