PyTorch Hub General Seldon Deployment Example

[axsaucedo]

PyTorch Hub General Seldon Deployment Example

This PR contains an example that shows how it's possible to support most of the currently available PyTorch Hub models, and virtually all that follows the standard model structure. The README.md can be found here, and similarly the notebook for the example.

Context on PyTorch

PyTorch Hub is the latest initiative from Facebook's PyTorch team to tackle the challenge of reproducibility, allowing developers to expose their models through a hubconf.py file.

Codebases with a hubconf.py file that are added to the PyTorch Hub repo are added to the list (which can be viewed withtorch.hub.list()) can be imported by simply running the command torch.hub.load("USER/REPO_NAME", "MODEL_NAME", pretrained=True).

Initial thoughts on General Architecture of PyTorch hub and Seldon

The approach that was taken was quite standard actually. Because there was a relatively good level of standardisation of the models they accept in their hub (with exception with others like the BERT tokenizers and util functions) so it was possible to create a very simple Python wrapper that allows the developer/user to define the model to deploy with the parameters in the seldondeployment config file. Below is just a high level overview of how pytorchHub as the module in the PyTorch library serves as an interface with the github repos, and then the trigger (which is referred to as "GitOps) is basically manual in this example, but here is where you could basically have a GitOps tool like ArgoCD making sure that any new changes result in the relevant updates.

Ideas on the general structure for examples

ONe thing to note in this example is that the installation instructions are different. Given that we have quite a lot of examples that reference to the installation (which most of the times is the same) I thought of a way of referring users towards a general installation page/notebook, which would allow us to then reduce the amount of work required when changing or updating installation instructions.

[ukclivecox]

The seldon_client uses gateway_endpointnow. We probably need to check this in the examples.

[axsaucedo]

Good catch @cliveseldon, I was using SeldonClient 0.2.7, I have now updated it to 0.3.0, and have added an explicit definition through the requirements-dev.txt so that people that go through this tutorial make sure they have the right configuration.

[axsaucedo]

By the way @cliveseldon what are your thoughts on the format suggested for the "Setup/install Seldon" section, which basically would references to another page. The idea there was that it could reduce the amount of work required when updates need to be put in place.

[ukclivecox]

A separate install reference rather than in each notebook makes sense. I will create a separate issue for this.

[axsaucedo]

This example is currently having issues when the models take longer to download than 20s, which is outlined in this issue #674

[ukclivecox]

Can we use ModelUri for auto-download?

[axsaucedo]

After going through the pytorch load module it seems that the download of the model is too embedded with the code, and cannot be downloaded by the init container.

[seldondev]

failed to trigger Pull Request pipeline

• failed to create agent
• failed to calculate in repo config
• failed to load trigger config for repository SeldonIO/seldon-core for ref 0b19e11
• failed to find any lighthouse configuration files in repo SeldonIO/seldon-core at sha 0b19e11
• failed to process repo SeldonIO/seldon-core refref 0b19e11
• failed to list files in directory /var/tmp/gitrepo283418320/.lighthouse
• open /var/tmp/gitrepo283418320/.lighthouse
• no such file or directory