Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Test Runner #4

Open
ErikSchierboom opened this issue Jan 19, 2021 · 12 comments
Open

Build Test Runner #4

ErikSchierboom opened this issue Jan 19, 2021 · 12 comments

Comments

@ErikSchierboom
Copy link
Member

ErikSchierboom commented Jan 19, 2021

In Exercism v3, one of the biggest changes is that we'll automatically check if a submitted solution passes all the tests.

We'll check this via a new, track-specific tool: the Test Runner. Each test runner is track-specific. When a new solution is submitted, we run the track's test runner, which outputs a JSON file that describes the test results.

The test runner must be able to run the tests suites of both Concept Exercises and Practice Exercises. Depending on the test runner implementation, this could mean having to update the Practice Exercises to the format expected by the test runner.

Each track must build a test runner according to the spec. For tracks building a test runner from scratch, we have a starting guide and a generic test runner that can be used as the base for the new test runner.

@neenjaw
Copy link

neenjaw commented Apr 25, 2021

@ErikSchierboom I'm playing around with making a crystal test-runner. :)

@ErikSchierboom
Copy link
Member Author

@neenjaw Brilliant! Let me know if you need any help.

@neenjaw
Copy link

neenjaw commented Apr 28, 2021

Will do, it's going alright so far!

@ErikSchierboom ErikSchierboom added x:action/create Work on something from scratch x:knowledge/advanced Comprehensive Exercism knowledge required x:module/test-runner Work on Test Runners x:size/massive Massive amount of work x:type/coding Write code that is not student-facing content (e.g. test-runners, generators, but not exercises) x:action/improve Improve existing functionality/content x:knowledge/none No existing Exercism knowledge required x:size/large Large amount of work x:knowledge/intermediate Quite a bit of Exercism knowledge required x:module/practice-exercise Work on Practice Exercises x:size/small Small amount of work x:size/tiny Tiny amount of work x:type/content Work on content (e.g. exercises, concepts) and removed x:action/improve Improve existing functionality/content x:size/massive Massive amount of work x:type/coding Write code that is not student-facing content (e.g. test-runners, generators, but not exercises) x:knowledge/none No existing Exercism knowledge required x:size/large Large amount of work x:knowledge/intermediate Quite a bit of Exercism knowledge required x:module/practice-exercise Work on Practice Exercises x:size/small Small amount of work x:size/tiny Tiny amount of work x:type/content Work on content (e.g. exercises, concepts) x:module/test-runner Work on Test Runners x:action/create Work on something from scratch x:knowledge/advanced Comprehensive Exercism knowledge required labels Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants