Parameter properties

[jmafoster1]

Could we have different types of causal variables (inputs, outputs, meta variables, etc., maybe custom ones for really advanced users) and corresponding methods is_input, is_output, etc.? That'd be quite useful for some of my data generation stuff.

[jmafoster1]

It's also just occurred to me that, for metavariables, we should have some way to encode how to calculate them from model inputs, e.g. average_age = f(location) where f corresponds to looking it up from covasim's dataset. I'm not sure how best to do this, though, since different metavariables will depend on different inputs. I guess you could just have it be function which takes a set of variables and then pass the whole input configuration through.

[AndrewC19]

This all makes sense to me and I can see two options for implementing it.

1. We could add a new class that is a child of CausalDAG which includes this additional information in its constructor.
2. We could add this information to the scenario object, such that each random variable can be mapped to a type (input, output, meta etc.). Alternatively, we could modify the scenario to become a nested dict of the form {inputs: {x1: c1, ..., xn: cn}, outputs: [y1, ..., yk], metas: [m1, ..., ml]}, where inputs maps constraints to a subset of inputs, outputs is the list of outputs involved in the scenario, and metas is the list of metas involved in the scenario.

For now, I think the second option makes the most sense, since the purpose of the scenario object is to carry the information corresponding to a causal DAG that is useful for testing.

Regarding the implementation of the meta-variable functions, I have only one idea at the moment. We could make an abstract class MetaVariable which has a single abstract method form_meta_var which the user must override e.g.:

from abc import ABC, abstractmethod

class MetaVariable(ABC):
    
    def __init__(self, name):
        self.name = name

    @abstractmethod
    def form_meta_var(self, df):
        pass

class MyVar(MetaVariable):

    def form_meta_var(self, df, x1, x2)
        # do something with execution data (df) using inputs x1 and x2 

For each meta-variable, the user would have to implement a class as above, including a function that can be used to form the meta-variable.

Without implementing a MetaVariable abstract class, I don't think there is a clean way to get a function for every meta-variable. Every other way would rely heavily on the user. For example, we could leave it to the users to implement the functions themselves and map those to the meta-variables in the scenario, e.g.: {inputs: {x1: c1, ..., xn: cn}, outputs: [y1, ..., yk], metas: [m1: f1, ..., ml: fl]}, where f1 and fl are functions.

Therefore, it probably makes the most sense to create a class for each type of variable: inputs, outputs, and meta. The output class would just be a container (at the moment), the input class could handle the application of constraints, and the meta class would work as above. I am still reluctant to reflect these changes in the causal DAG class, though. I would rather keep this all in the scenario.

What are your thoughts?

[jmafoster1]

I agree about the "one class for each type of variable" thing and quite like this idea. Now I've had a more detailed look at the implementation of Scenario, I'm not entirely happy with it. It looks like Scenario is trying to do the same thing I was trying to do in the first year of my PhD, i.e. ascribe constraints to variables. This works for very simple constraints of the form X < 4 or whatever, but really doesn't scale to more complex constraints. What would we do if we wanted to relate multiple variables, e.g. num_infected <= pop_size? Would we manually have to add symmetric constraints to both variables? This quickly becomes unweildy.

What I ended up doing was having a set of variables and a set of constraints over them, and that's what I suggest we do here too. We could have the Scenario constructor take those two things as arguments. We could create a Variable abstract class and extend it with Input, Output, and Meta, and the user could add custom extensions if they wanted. Each variable could have a distribution attribute, meta variables can have their form_meta function to populate a dataframe column, etc. We could also have a method in scenario called something like constraints_over which takes a variable and returns all the constraints which concern that variable (if we're careful about how constraints are implemented). I suggest we express constraints using Z3 if we're going to be using it anyway. It's pretty friendly and we get constraint solving "for free" as it were...

[AndrewC19]

The original purpose of Scenario is for users to place constraints over a subset of the input variables that, collectively, characterise the usage scenario-under-test. Ultimately, we want to use these constraints to filter observational data and, in your case, to generate test data. I am happy for us to change the implementation so long as these two requirements can be fulfilled.

I am not sure what you mean by having a set of variables and a set of constraints over them. I thought that is what the current implementation achieves, given that it is simply a dictionary. Could you provide some more information?

[jmafoster1]

As I understand it currently, a Scenario takes a dictionary which is a mapping from Variables -> Constraints? In the situation where we have constraints which relate multiple variables, we'd have to have separate constraints for each variable. If we instead had Scenarios take a set of variables which are in scope (i.e. the inputs, outputs, and metavariables of the model) and a set of constraints over such variables, we would not explicitly be assigning each constraint to a variable, and thus could freely relate variables in constraints without having to do anything complicated. Does that make sense?

[AndrewC19]

Thanks for explaining, that makes a bit more sense now. To clarify, suppose we have variables a, b, c and three constraints: a > 100, a > b c < b/2. Are you suggesting we have something like:scenario = Scenario(variables={a, b, c}, constraints={a>100, a>b, c<b/2}), but with the constraints implemented in a Z3-compatible format?

In essence, we have the same functionality as before but it is more flexible in that we can now introduce constraints over variables directly.

[jmafoster1]

Yes, that's exactly what I'm suggesting. The variables a, b, and c, would then be appropriate extensions of an abstract Variable class depending on whether they were inputs, outputs, or metas.