V1.0.0 Roadmap

[mmiller-max]

I think we're reasonably close to the TestReports being ready for V1.0.0. Here is an initial list of tasks for discussion.

Functionality

• TestSets that use the for syntax result in multiple testsuites with the same name (will be fixed by testsuite/id below) Missing/incorrect attributes in report #56
• Ensure all "required" fields in schema are present in report - main aim is that anything reading there report should have the fields that it expects
  • testcase/time (? this is tricky to fill accurately, could be defaulted) Add timing information #49
  • testsuite/time (This one we can do accurately by adding functionality to ReportingTestSets) Add timing information #49
  • testsuite/hostname (can default to localhost) Missing/incorrect attributes in report #56
  • testsuite/timestamp Add timing information #49
• Other schema updates Missing/incorrect attributes in report #56
  • testsuite/id should be an int that increments for each test suite Missing/incorrect attributes in report #56
  • testsuites doesn't need id Missing/incorrect attributes in report #56
• Fix compatibility with CustomTestSets CustomTestSets may be broken #61

Docs

• Update document examples when all schema updates done Documentation tidy/improve #60
• Review docs for grammar/clarity etc Documentation tidy/improve #60

CI

• Only test PRs and master, rather than all branches (completed by Move CI to GitHub Actions #52)

Code Hygiene

• Ensure all internal functions have docstrings Refactor runner.jl #58
• Refactor the larger functions in runner, e.g. test! Refactor runner.jl #58
• Split out runtests.jl into different files for different bits of functionality Refactor tests #59
• Move import/using to the TestRepots.jl file Tidy imports #69

Not for V1.0.0

• pass information, relies on updates to Julia
• It would be nice to rely less on Pkg internals, but I'm not sure how possible this is without re-writing lots of it ourselves

[oxinabox]

We have now tagged 1.0