-
Notifications
You must be signed in to change notification settings - Fork 525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tests: relocate 'ingredient_groups' from mandatory-tests to optional-tests. #1184
Tests: relocate 'ingredient_groups' from mandatory-tests to optional-tests. #1184
Conversation
…expectations where the corresponding scraper does not provide a custom implementation.
… when "ingredient_groups" exists as a key in the JSON test data.
… created by `generate.py`.
This a great cleanup! What do you think about removing the ingredient_group JSON key for the non-ingredient_group test of the sites that do contain the functionality? All sites that contain grouping have 2 test cases one covering the grouping functionality & another not, this could be another substantial clean up but would need to be done manually. Alternatively, this currently passes if |
Both good ideas, yep! Just to check I understand them correctly:
|
(and it took a few moments, but I understand what you mean about the first idea being more difficult to do if we implement the second idea) |
I'd like to avoid this route if possible - our tests aren't strictly isolated from the library code at the moment, so there can be some behavioural interference between test code and library code -- but generally I think we should write tests that are not allowed to gain much (or ideally any) information about the corresponding scraper and its output before deciding what to assert on. |
yep, exactly!
& yep again! I was thinking just about
I agree, this is definitely a last resort kind of idea. |
Ok, so, ignoring the last-resort situation for a moment, I think what we're talking about is whether test coverage should be enforced for overridden methods (case 2), or whether it is optional and we can omit the expectations if so (case 1). Arguably test coverage reporting tools could already do the check from case 2 for us -- they could tell us that an overridden method hasn't been covered ( If we did use continuous integration to detect this using coverage tools, that still leaves a question about whether to remove the expectations in cases where we have multiple tests. Whatever the scraper -- even for ones that don't override |
After thinking about it more, I’m leaning towards case 1. Removing the test data from non-ingredient_groups tests offers the most cleanup. Additionally, if the developer went through the process of working out the ingredient_groups scraping they are likely to include test coverage for it. We can always track this approach for a while and revisit if it causes confusion. note: are there any doc updates required as a result of this change?
Thanks for the tip! Had no idea this existed, lots interesting info here.
This sounds worth looking into but that may be a separate discussion. I've written code in the past to generate the missing coverages that could possibly be repurposed to do this.. i'll have to look around to see if i hung on to it. |
That makes sense to me - I'll go ahead with cleaning up those files.
Mm, good question - it seems quite likely. I'll check that too. |
…erated test JSON content.
…hen no expectation is presented.
…ingle-group-unpurposed copies of `ingredients` expectations.
…) as purpose in preference to empty-string.
…ing_utils.group_ingredients`.
Ok, I think that's more-or-less ready now; glad for any more suggestions/review. |
Just one last thing, in the In addition to the usual tests a scraper requires, the tests also needs to check the groups and the ingredients in each group are correct for the recipe. For the test cases where there are no ingredient groups, this should check for a single IngredientGroup object with purpose=None and ingredients set to the output from the scraper's ingredients() function. For the test case with ingredient groups, the output should match the groups in the recipe.
Each test case will automatically inherit a test that checks to make sure the same ingredients are found in .ingredients() and in the groups returned from .ingredient_groups(), so there is no need to write this test in the scraper test cases. Otherwise, looks good to go to me! |
Good catch, thank you! |
…re required for many ingredient-groupings scraper implementations.
…ocs with data-driven-test (JSON) example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me!
… existing documentation style uses long-lines.
…ause the existing documentation style uses long-lines." This reverts commit 9a20e6f.
… existing documentation style uses long-lines.
d5f9006
to
0e4b446
Compare
…is time try an integer `line_length` instead of `false`.
4102f53
to
9066da3
Compare
…ted-ingredient-groups
...and, at the same time, remove the
ingredient_groups
key from any data-driven test JSON expectation files for scrapers that don't implement custom retrieval of grouped ingredients.This requires a small change to the test suite so that test expectations are not created when not required.
Resolves #1181.