provide a graph-level model metadata

[RafalSkolasinski]

Currently #1671 which closes #1638 will only implemented model metadata on single graph components.

It would be nice to also have metadata for the whole graph where input would be the input of first node and output the output of the last node.

Open questions are how it should work with router and combiner elements in the graph.

Anyway, probably the easiest is to implement in executor a logic that would go to /metadata endpoints of each component and expose graph-level metadata.

[RafalSkolasinski]

Including some notes on possible implementation that would make related PR easier to understand.
Note that structure of the graph-level metadata in initial implementation (linked PR) does not correspond exactly to the format below - > feedback on best format appreciated!

Graph-level input / output shapes from model-level metadata

We could leverage a model-level metadata to derive graph-level inputs and outputs shapes.

1. Single - node deployment

In case of single-node graph model-level inputs and outputs, x and y, will simply be also the graph-level inputs and outputs

Deployment Manifest

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-model
spec:
  name: model
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: example/model:version
          name: my-model
     graph:
      name: my-model
      type: MODEL
      children: []      
    name: default
    replicas: 1

Model Level:

$model_meta = {
  "name": "model-name",
  "versions": ["model-version"],
  "platform": "platform-name",
  "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 4]}],
  "outputs": [{"name": "output", "datatype": "BYTES", "shape": [1]}]
}

Graph Level:

$graph_meta = {
  "deployment-name": "seldon-model",
  "spec-name": "model",
  "graphs": [
    {
      "graph-name": "default",
      "components": [
        {
          "name": "my-model",
          "version": "model-version",
        }
      ],
      "graph": {
        "name": "my-model",
        "type": "MODEL",
        "children": [],
      },
      "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 4]}],
      "outputs": [{"name": "output", "datatype": "BYTES", "shape": [1]}],
    }
  ]
}

2. Multi - node deployment: chained nodes

In case of multi-node graph with multiple models chained such that output of one model is input of the next one

The graph-level input x will be first model’s input x and graph-level output y will be the last model’s output y2=y.

We will also have additional constraint that shapes of outputs in earlier model must match the shape of input in later model, it is y1=x2.

Deployment Manifest

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-model
spec:
  name: model
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: example/model-1:version
          name: my-model-1
        - image: example/model-2:version
          name: my-model-2          
     graph:
      name: my-model-1
      type: MODEL     
      children:
        - name: my-model-2         
          type: MODEL
          children: []      
    name: default
    replicas: 1

Model Level:

$model_meta_1 = {
  "name": "model-name-1",
  "versions": ["model-1-version"],
  "platform": "platform-name",
  "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 4]}],
  "outputs": [{"name": "output", "datatype": "BYTES", "shape": [1, 3]}]
}

$model_meta_2 = {
  "name": "model-name-2",
  "versions": ["model-2-version"],
  "platform": "platform-name",
  "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 3]}],
  "outputs": [{"name": "output", "datatype": "BYTES", "shape": [3]}]
}

Graph Level:

$graph_meta = {
  "deployment-name": "seldon-model",
  "spec-name": "model",
  "graphs": [
    {
      "graph-name": "default",
      "components": [
        {
          "name": "my-model-1",
          "version": "model-1-version",
        },
        {
          "name": "my-model-2",
          "version": "model-2-version",
        }        
      ],
      "graph": {
        "name": "my-model-1",
        "type": "MODEL",
        "children": [
          {
            "name": "my-model-2",
            "type": "MODEL",
            "children": []
          }
        ]
      },
      "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 4]}],
      "outputs": [{"name": "output", "datatype": "BYTES", "shape": [1]}],
    }
  ]
}

3. Multi - node deployment: combiner

In case of multi-node graph with combiner linked to few models

Input x is first passed to models and their outputs, y1 and y2 passed to combiner which returns the graph output y:

Models should expect inputs of same shape: graph-level input x

Output of combiner defines the graph-level output y

Deployment Manifest

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: seldon-model
spec:
  name: model
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: example/combiner:version
          name: my-combiner
        - image: example/model-1:version
          name: my-model-1
        - image: example/model-2:version
          name: my-model-2          
     graph:
      name: my-combiner
      type: COMBINER     
      children:
        - name: my-model-1         
          type: MODEL
          children: []        
        - name: my-model-2         
          type: MODEL
          children: []      
    name: default
    replicas: 1

Model Level:

$model_meta_1 = {
  "name": "model-name-1",
  "versions": ["model-1-version"],
  "platform": "platform-name",
  "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 3]}],
  "outputs": [{"name": "output", "datatype": "BYTES", "shape": [1, 4]}]
}

$model_meta_2 = {
  "name": "model-name-2",
  "versions": ["model-2-version"],
  "platform": "platform-name",
  "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 3]}],
  "outputs": [{"name": "output", "datatype": "BYTES", "shape": [1, 3]}]
}

$combiner = {
  "name": "model-combiner",
  "versions": ["combiner-version"],
  "platform": "platform-name",
  "inputs": [
    # this match the output of the first model
    {"name": "input", "datatype": "BYTES", "shape": [1, 4]},
    
    # this match the output of the second model
    {"name": "input", "datatype": "BYTES", "shape": [1, 3]}
  ],
  "outputs": [{"name": "output", "datatype": "BYTES", "shape": [2]}]
}

Graph Level:

$graph_meta = {
  "deployment-name": "seldon-model",
  "spec-name": "model",
  "graphs": [
    {
      "graph-name": "default",
      "components": [
        {
          "name": "my-combiner",
          "version": "combiner-version",
        },
        {
          "name": "my-model-1",
          "version": "model-1-version",
        },
        {
          "name": "my-model-2",
          "version": "model-2-version",
        }        
      ],
      "graph": {
        "name": "my-combiner",
        "type": "COMBINER",
        "children": [
          {
            "name": "my-model-1",
            "type": "MODEL",
            "children": []
          },
          {
            "name": "my-model-2",
            "type": "MODEL",
            "children": []
          }          
        ]
      },
      # this match the input to each model
      "inputs": [{"name": "input", "datatype": "BYTES", "shape": [1, 3]}],
      
      # this match the output of the combiner
      "outputs": [{"name": "output", "datatype": "BYTES", "shape": [2]}],
    }
  ]
}

4. Multi - node deployment: router

In case of multi-node graph with outer

the router passes input x to one of the models, model 1 or model 2 and returns output of that model, yor y2, as the graph-level output y. It may not be unreasonable to expect both (or more) models to:

accept same shaped input x

return same shaped output y, it is shape(y1) = shape(y2)should be satisfied

Detailed json exmaple

This wouldn’t be much different from the above. Will be added later for clarification.

5. Miscellaneous combination of the above

A more complicated graph may consist of the basic elements describe above.
In such situation a more complex graph should be divided into a smaller pieces corresponding to the basic elements listed above.