Declarative Programming based Service Chains

[twood02]

Service chains can be a great way to test a great deal of complex functionality in the openNetVM system. The idea is for a program to be written (probably in /examples) that allows for .json configuration file for running different chains of NFs pooled together. We are envisioning a python program that can parse JSON data to start circular or linear chain functionality with different example NFs. We will keep building on this, and would be a great place for a new researcher to start.

[catherinemeadows]

@kevindweb @dennisafa

Before going any further with the python program that can parse JSON data to start a circular or linear chain, I wanted to come up with an example config file to make sure that the format is correct. I also wanted to discuss for circular chains, if the program should error check (i.e. check service and destination id's and tell the user that the chain isn't actually circular) or if the program should automatically correct for this if it knows that the user is trying to start a circular chain (i.e. correct service and destination id's so that the chain is now circular).
Linear chain example config:

{
    "speed_tester": {
        "serviceid": 1,
        "destinationid": 2,
        "numpackets": 16000
    },
    
    "simple_forward": {
        "serviceid": 2,
        "destinationid": 3
    },

    "basic_monitor": {
        "serviceid": 3
    }
}

[dennisafa]

@kevindweb @dennisafa

Before going any further with the python program that can parse JSON data to start a circular or linear chain, I wanted to come up with an example config file to make sure that the format is correct. I also wanted to discuss for circular chains, if the program should error check (i.e. check service and destination id's and tell the user that the chain isn't actually circular) or if the program should automatically correct for this if it knows that the user is trying to start a circular chain (i.e. correct service and destination id's so that the chain is now circular).
Linear chain example config:

{
    "speed_tester": {
        "serviceid": 1,
        "destinationid": 2,
        "numpackets": 16000
    },
    
    "simple_forward": {
        "serviceid": 2,
        "destinationid": 3
    },

    "basic_monitor": {
        "serviceid": 3
    }
}

I think that it would be cool if the user can give linear or circular as an arg and then have the script auto-correct the ID's dynamically. The format looks good to me.

[twood02]

A major challenge here is that the parameters you are including, such as numpackets are NF specific, so there will not be an easy way for your script to know how to translate that to a specific command line parameter.

A long term solution for this would be to make all of our NFs support either abbreviated command line arguments (e.g., -p) like we do now and verbose arguments (e.g., --numpackets). In that case we could automatically translate the JSON based parameters into exactly what needs to be run on the command line. Incidentally, this would also help with the NF-based JSON config parsing. Currently it only has limited support for NF-specific parameters because we don't have a way to map to the abbreviated arguments.

Let's consider the two extremes for how the JSON files could be defined. Right now you are on the end of the spectrum that makes the JSON config author's job very easy, but your python code will be more complex and will need to know how to convert every argument for every application into command line arguments.

The other extreme would be something like this:
(I've made things up because I don't remember the exact commands we use)

{
    "speed_tester": {
        "directory": "examples/speed_tester/",
        "executable": "go.sh",
        "parameters": "-d 2 -s 1 -p 16000"
    },
    
    "simple_forward": {
        "directory": "examples/simple_forward/",
        "executable": "go.sh",
        "parameters": "-d 3 -s 2"
    },

    "basic_monitor": {
        "directory": "examples/simple_forward/",
        "executable": "go.sh",
        "parameters": "-s 3"
    }
}

This is a lot less friendly to write and understand, but it would be very easy for the python script to parse it and know exactly what to run.

Is there a midpoint which will give us the best of both of these? Maybe have built in support for easily defining our example NFs, but also support "custom" entries which are written out like above?

[kevindweb]

@catherinemeadows and I were talking about this. I think it's ok if the author has a little more complexity in writing the parameters. The usage function in most NFs will be clear enough that the author messed up their NF-specific arguments. If the executable was run incorrectly, we stop the experiment and print out the usage function for the failed NF. Here's what I'm imagining for the JSON config files though.

{
    "speed_tester": [
           {
                    "parameters": "-d 3 -s 1 -p 16000"
            }, {
                    "parameters": "-d 4 -s 2 -p 16000"
            }
    ],
    
    "simple_forward": [
            {
                    "parameters": "-d 2 -s 3"
            }
     ],

    "basic_monitor": [
        {
                "parameters": "-s 4"
        }
    ]
}

I don't think we need the directory, because that's the same as the actual NF name (application name), and the go scripts are always the default executable script. Each NF should have an array of how many NFs are being chained together of that NF type. For speed tester here, we have two NFs being run with different parameters. It would be cleaner than having multiple sections with duplicate "speed_tester". I can imagine having a 5 speed_tester chain made much easier with this model.

[twood02]

resolved by #218