-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Derive generators from type definitions #128
Conversation
This commit adds a macro to derive generators automatically from type definitions `@type`. Certain restrictions apply, such as not being able to generate a pid or reference. The macro can be used to create generators for all public types within a module: ```elixir use PropCheck.DeriveGenerators ``` It can also be used to create generators for other modules, as long as they are already compiled: ```elixir use PropCheck.DeriveGenerators, module: OtherModule ``` The macro creates a new module `module.Generate`, which contains the generators. For each `@type t`, a function `Generate.t()` is created, which can then be used as a generator in `forall`. For types `@type t(var)` taking a type variable, the function expects a generator as an argument for each type variable. To allow adding types which are cannot be turned into a generator, the option `:only` can be used. ```elixir use PropCheck.DeriveGenerators, only: [t: 0] ``` With this, only the type `t` will be turned into a generator.
@alfert I present this PR in an early stage to get some feedback. Most of the needed functionality is implemented (see the unit test for a comprehensive example), but besides the documentation, I think this needs a second set of eyes. Note that I did not implement deriving test functions from |
Specifically, this pr currently targets #3. |
For non-recursive types, this is rather simple (as we both showed by example), it is only lengthy and tedious code. Generatring recursive types are where the meat is. They are complicated to code and it is questionable whether this is an effective approach for testing. Typically, an automatically derived generator can only be the start of the test engineering process, see the lengthy stack example in the tests: you need to define a random distribution between the various recursive branches to generate all variants. This distribution decide massively on the shape and size of the generated data structures. In my experience, there is no one-size-fits-all generator for complicated stuff. Thus, I only started the adventure of generating recursive generators, but never came to an end. From the perspective: are non-recursive type still a useful feature? Or are there other development usability or feature enhancements for PropCheck we should follow instead (or additionally)? |
In my experience, most types are non-recursive. At least most custom types I usually use in my code are non-recursive. Hence I would argue that those simple cases can indeed be a nice usability feature (see for example https://github.com/alfert/propcheck/pull/128/files#diff-50c824f0c4ab83bd425c63d90fe2aa7bR10 in this PR; it feels kind-of right to be able to get a simple generator almost for free). Of course you are right w.r.t. recursive types, they are complicated. I wonder if it would still be nice to have automatic generators which work "most of the time" and clearly communicate the restrictions when it comes to custom recursive types. Note that I wouldn't be angry if this PR here does not lead to a merged commit on master. I found it already worthwhile to play around and get a feeling were the actual problems with this approach are :) |
I liked this comment by Kostis (mailing list):
This captures what I mean above. Maybe "good enough" is what is needed most of the time. |
This is true indeed, if you are writing application code with a (relational) database in mind. All functional data structures are usually recursive, but with lists, structs and maps at hand, it could be possible to avoid recursive data structure in most practical places. If we follow you approach, then there is the obligation to document it properly, since it contradicts my statements in the README. Also the limits of the approach needs to be documented clearly. The technical problem of recursive types is that they may be mutually recursive in an indirect manner while spawning across several modules at the same time. You need to detect and parse these structures properly and you need to store them somehow while creating the generators. That is what the type server in PropEr is for, but this nice beast is not intended to be used from Elixir or from outside of the internals from PropEr. If I remember right, this was the unpleasant experience that drove me to abandon the automatic generators. |
So, at the very least, I would like to detect problematic recursive types with cycle detection and throw an error. Otherwise, users would probably run into problems which are very hard to debug.
Yes. You are right that this needs some kind of type server. I don't have an intuitive understanding yet how that would look like. The All in all, this thing is a very interesting problem indeed. I got a better understanding now where some of the complexity in PropEr comes from :) |
There is the |
Thanks, I will take a look at it. |
I spent some time since November in implementing this. After some time, I decided to decouple this from PropCheck to simplify playing around with it. The result is an application I made a lot of progress and extended the simple derive generators with a type server, which checks if recursive types are present. The implementation can be found here. I haven't released this yet on hex.pm, as there is still some fleshing out to do. Nevertheless, I was able to create generators for all types in Elixir's standard lib, where the types do not have cycles and no non-implemented types such as |
This merge request adds a macro to derive generators automatically from type
definitions
@type
. Certain restrictions apply, such as not being ableto generate a pid or reference. The macro can be used to create
generators for all public types within a module:
It can also be used to create generators for other modules, as long as
they are already compiled:
The macro creates a new module
module.Generate
, which contains thegenerators. For each
@type t
, a functionGenerate.t()
is created,which can then be used as a generator in
forall
. For types@type t(var)
taking a type variable, the function expects a generatoras an argument for each type variable.
To allow adding types which are cannot be turned into a generator, the
option
:only
can be used.With this, only the type
t
will be turned into a generator.Fix #69