Instance generator

[GMMDMDIDEMS]

Hi, is there a function to automatically generate multiple (unique) instances of a specific RDDL domain?

[mike-gimelfarb]

Hi,
Unfortunately, at this time there is no generator support outside the limited subset of 2023 IPC domains. Doing this for each domain is quite time consuming and none of us has the bandwidth currently. I've been thinking of whether it's possible to design a general instance generator that will work for all problems, provided the ranges of fluents (their distributions, etc) are specified somehow to the generator. But I don't have a concrete way to make this generate reasonable and feasible instances across all domains without some sort of custom tweaking. If you have any specific proposal how this could be achieved, please share them!

[haz]

Kind of feels like this could be a nice addition to the next version of RDDL. Initial state generation.

[ssanner]

Good ("interesting", non-trivial, not over-constrained) instance generation is very domain specific; I think it's something that each domain designer has to provide per domain.

…

On Sun, May 12, 2024 at 8:11 AM Christian Muise ***@***.***> wrote: Kind of feels like this could be a nice addition to the next version of RDDL. Initial state generation. — Reply to this email directly, view it on GitHub <#256 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRA3JDJPGBXGZCUGYNHZE3ZB5L73AVCNFSM6AAAAABG74YC5GVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TIMBUHE3TM> . You are receiving this because you are subscribed to this thread.Message ID: <pyrddlgym-project/pyRDDLGym/repo-discussions/256/comments/9404976@ github.com>

[mike-gimelfarb]

I agree with Scott that generating instances on the fly is very domain specific, I am not sure the language will support it. I like the idea of sampling the initial state and I think it will be useful, particularly for the RL community as many domains are defined with a random initial state in open-ai gym for example. This allows better evaluation of the generalization of agents and particularly when learning policies. From language perspective it seems easy enough - just add a block to the domain that specifies how the state variables are initialized (or in the instance?) which supports expressions. however it means internally many things will need to change across the ecosystem to accommodate this: many solvers currently don’t handle it so they will need to be updated. This is a major change, and I’m not sure we want to push another major change given we have already done this going from 1.4 to 2.0 recently. If we do it, it might make sense to just bundle up everything we want to add to the language and call it version 3.0 in that case.

[haz]

Ya, I think we're all on the same page. Domain-specific instance generation. The recommendation would be to have dedicated syntax for making that sampling easy. Already there's a split between the domain/instance files -- if there's an "instance generator" file with some fancy RDDL syntax, then a stand-alone bit of software could take a domain/IG file and produce K instances. Then the solvers work as they always have.

Instead of writing custom software in whatever language for instance generation, you'd have RDDL syntax at your disposal for instance generation. Elegant ;)

[mike-gimelfarb]

Well, here is one proposal nested entirely within RDDL.
This works assuming that one can define the entire instance as a sampling procedure, namely, specifies how to sample objects of each type, how to sample both non fluents and initial values of fluents within some predefined range. To handle the first task, we could replace the

x-pos : { x1, x2, ... }

declaration of objects with a randomizer block:

x-pos : {
prefix = x,
lower = 1,
upper = {
Uniform(2, 10)
}
}

Just an idea... The other ideas are in principle easy by just replacing the init-fluents and non-fluent blocks with expression support.

[mike-gimelfarb]

But, I don't know that this would cover all the possible cases of interest. Maybe instead of modifying the instance block as it already exists, we introduce an instance-generator file as described above, which follows the instance.rddl except for the required syntax changes. So it will be possible to swap the fixed instance file with an instance generator file for the given domain and it should run as it already does (just there is an additional sampling layer right after parsing, and before the domain and instance are stored internally as ASTs, that generates the instance file).

[ssanner]

I see, leverage the expressive RDDL syntax to avoid the onerous problem of generating all of the boilerplate RDDL instance file syntax. Indeed, this discussion has come up before. I like the idea of a separate instance-generator file in a RDDL syntax that could ideally generate instances that are stored as explicit instance files. Generating explicit instance files facilitates instance inspection during failure analysis and reproducibility. We would need distributions for sampling objects (shouldn't be too hard, though new), and we would need to explicitly sample not just fluent initial states but also potentially nonfluent configurations (e.g. network topologies). Let's put this on the wishlist.

…

On Sun, May 12, 2024 at 11:29 PM Mike Gimelfarb ***@***.***> wrote: But, I don't know that this would cover all the possible cases of interest. Maybe instead of modifying the instance block as it already exists, we introduce an instance-generator file as described above, which follows the instance.rddl except for the required syntax changes. So it will be possible to swap the fixed instance file with an instance generator file for the given domain and it should run as it already does (just there is an additional sampling layer right after parsing, and before the domain and instance are stored internally as ASTs, that generates the instance file). — Reply to this email directly, view it on GitHub <#256 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABRA3JFIYGYAPKCV7HFZF23ZCAXSDAVCNFSM6AAAAABG74YC5GVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TIMJTGA3TC> . You are receiving this because you commented.Message ID: ***@***.*** com>

[haz]

• Some nice way to wrap existing generators
• Hash of sample file + RDDL version + random seed that gets placed in the instance file as a comment
• Maybe even the exact command (w/ random seed) to reproduce the instance file
• Function defined to indicate "difficulty level" based on parameters so that can be scaled with a benchmark

So many fun possibilities ;)

[vshlroot]

Hello everyone,

I seem to be a bit late to this discussion but the discussion is too interesting to skip,

In our recent paper we had to generate new instances for certain domains of IPPC variants. Here are my thoughts,

1. We thought about generating new instances automatically using RL, but it soon turned out to be an active area of research in itself (See Procgen Benchmark and following papers).
2. It is relatively easy to write a script to generate random instances of a domain, but while it may turn out to be a valid instance, it may not be meaningful enough, thus needing some definition of meaningfulness/difficulty of the instance. Since this was not primary focus of our work, we left it at "how to define difficulty of an instance?"
3. Eventually we resorted to writing script with constrainted instance generation. You can find the hyper-parameters we used to generate instances in the section 6 of the supplementary of our paper. Paper | Supplement