Assign NUTS sampler to remaining variables

[brandonwillard]

We need to assign the NUTS sampler to the remaining (continuous) variables here in aemcmc.basic.construct_sampler.

[rlouf]

I am currently not sure how to handle the sampler parameters in this situation. The caller, either machine or human, has to know that the sampling step requires extra parameters. I was first thinking of passing instructions in the form of a string, but this is not a good idea, at least not in AeMCMC: it complicates handling of these step functions by a program downstream.

A better idea is to deal with the distinction at the type level e.g. with SamplingAlgorithm and ParametrizedSamplingAlgorithm. NUTS would be a ParametrizedSamplingAlgorithm and contain the necessary information about its parameters (expected type, rank and shape for instance). These classes can return instructions that humans can understand with their __str__ method.

Something along the lines of (for a single variable):

# To be continued
class NUTS(ParametrizedSamplingAlgorithm):


    def __init__(self, variable):
        ndims = variable.shape[0]

        self.parameters = {}
        self.parameters["step_size"] = (None, config.floatX)}
        self.parameters["inverse_mass_matrix"] = [
            (ndims,config.floatX),
            ((ndims,ndims), config.floatX)
        ]

        # Reparametrize the variables here

    def  construct_step(self):
        """Returns the step function for the given variable"""
        pass

    def __str__(self):
        """Print infotmation about this sampling step:
         1. The variables being assigned the algorithms
         2. The properties of the parameters
         3. The algorithm
         4. The transformations applied.
        """
        pass

The adaptation that NUTS requires is an extra difficulty, but we don't need to figure this out to have a first working version of the sampler builder. I currently leans towards letting the caller handle this using the sampling step types since sampling steps and parameter updates are decoupled. We shouldn't make opinionated decisions about which adaptation algorithm to use in AeMCMC anyway.

(In lack of a better place, I'll also add here that our rewrites need to carry information that is both machine readable: "what distributions are involved?" "What kind of rewrite is this", which mean we could export a database of relation should we want to. And also readable for humans who will need to understand how we built samplers)

[brandonwillard]

I am currently not sure how to handle the sampler parameters in this situation. The caller, either machine or human, has to know that the sampling step requires extra parameters. I was first thinking of passing instructions in the form of a string, but this is not a good idea, at least not in AeMCMC: it complicates handling of these step functions by a program downstream.

Can you provide pseudo-code examples for the relevant scenarios(s)? Without something like that, I'm not confident that I would end up addressing all the relevant concerns together.

[rlouf]

I'll do better than that and open a PR shortly with what I'm more or less sure of and add the explanation there.