-
-
Notifications
You must be signed in to change notification settings - Fork 550
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
canonical-data.json: Shared data among test cases #699
Comments
Doh. And I know that I was at least partially involved in both: Let me pretend for a moment that we have decided to accept duplication in pov/canonical-data.json, and see what that looks like.
So we won't care if deeply_nested and kids go away, and we probably can survive repeating
In general I was having some doubts about whether the extra indirection of Given this, I could support removing the indirection in "pov" (using smaller trees instead of grep... what would someone think about just placing the three files (iliad.txt, midsummer-night.txt, paradise-lost.txt) as normal .txt files in this repo rather than embedding them in JSON? That would remove the need for the Alternatively, perhaps we think it is acceptable to embed smaller files within each individual case. It could be that we don't need to use those exact three files for all our tests. Being able to tailor the files to each test might help each test be a better check of the respective behaviours. Even if we think we can solve the problem with these two exercises today, what if another exercise comes along that does in fact need the shared data? Will we accept then that the schema will have to have it, and the test generators will have to deal with it? I also see that it will be pretty rare to have exercises that need this, which means that a test generator that simply doesn't understand this section will only miss out on a handful of exercises, and maybe that is acceptable for them. However, it seems the meaning of such a section will be very exercise-dependent. That is surely a negative. |
Good analysis, @petertseng.
Seems a good solution for this exercise.
I'm trying to find an example where this is indeed needed. Maybe that is a canonical-data smell. It could be the case that, instead of carefully designing data to test a specific thing, we are saving work by utilizing a bigger, shared data structure. Makes sense? |
Seems like a very reasonable solution.
I think smaller trees could actually be beneficial, as they can help people better focus on what the actual problem is instead of figuring out the structure. All in all, I think the benefit of reducing duplication by introducing a new, shared data concept does not outweight its advantages. With the existing tests, I feel that we are better off just accepting the duplication, and when possible, make the duplication less by cutting away unnecessary bits of data. |
If we think that it is not worthy to patch the schema, we have
I just checked and there is no exercise with additional files in Edit: The second option generates a huge file. It seems to make more sense to change as little as possible now and leave the problem for a future redesign of this exercise. Is anyone using a test generator for it?! |
@rbasso I'm in favor of the third option. This is a bit of an odd one, but I don't think we can fully encode all edge-cases without getting into trouble somewhere. Better to have one exception, than to make things more complex for maintaining (2) or less obvious (1). |
Is there something else to be decide here or can we close it? |
I think we can close. there is a possibility for more focused cases in grep, but those can be a separate issue. |
Reformat 'about' documentation for nextercism
While working in #625, we discovered that some
canonical-data.json
files have data shared among test cases:grep
- Has afiles
object that encodes files to be "grepped" in multiples test cases.pov
- Has atrees
object shared by multiple test cases, referenced by their names.The current JSON Schema doesn't allow that, so I plan to sidestep this issue putting the offending structures in comments, temporarily, until we decide how deal with that.
I see a few option:
resources
/shared
structure: This would avoid repetition, but also increase the complexity of test generators significantly.Any other option?
The text was updated successfully, but these errors were encountered: