Support multiple types of responses in predict()

[RobertLucian]

Closes #913.

---

checklist:

• run make test and make lint
• test manually (i.e. build/push all images, restart operator, and re-deploy APIs)
• update docs and add any new files to summary.md (view in gitbook after merging)

[RobertLucian]

What has to be kept in mind in the back of the mind is that with this method, other protocols for pickle can be used too: 1, 2, 3, and 4 (with the exclusion of 5). 5 cannot be used because the serving container is still based on Python 3.6.x).

Anyhow, the speedup over protocol 0 (in my experiments) is this:

• Protocol 1: ~0.5x speedup.
• Protocol 2: ~0.5x speedup.
• Protocol 3: ~3x speedup.
• Protocol 4: ~3x speedup.
• Protocol 5: ~6x speedup - but is not accessible yet.

So assuming protocol 4, which is 3 times as fast as protocol 0, and no json.dumps for it, the serialization process for a ~27 MB DataFrame takes about 58 ms. That's a very good improvement over the 820 ms using protocol 0 & dumps.json. And even much better if it weren't for pickle (aka using df.to_dict and json.dumps): ~8000 ms.