Out of sync practice exercises

[github-actions]

Out of sync practice exercises have been detected:

Command: configlet sync --tests --docs --metadata --filepaths

Output:

Checking exercises...
[warn] docs: instructions unsynced: scrabble-score
[warn] docs: introduction missing: scrabble-score
[warn] anagram: missing 2 test cases
       - handles case of greek letters (a6854f66-eec1-4afd-a137-62ef2870c051)
       - different characters may have the same bytes (fd3509e5-e3ba-409d-ac3d-a9ac84d13296)
[warn] reverse-string: missing 3 test cases
       - wide characters (1bed0f8a-13b0-4bd3-9d59-3d0593326fa2)
       - grapheme cluster with pre-combined form (93d7e1b8-f60f-4f3c-9559-4056e10d2ead)
       - grapheme clusters (1028b2c1-6763-4459-8540-2da47ca512d9)
[warn] some exercises have unsynced docs
[warn] some exercises are missing test cases

[kmarker1101]

So, what is the plan, or do we have one yet? When I run this command currently I get:

Updating cached 'problem-specifications' data... Checking exercises... [warn] docs: instructions unsynced: bank-account [warn] docs: instructions unsynced: resistor-color-duo [warn] docs: instructions unsynced: scrabble-score [warn] docs: introduction missing: scrabble-score [warn] docs: instructions unsynced: spiral-matrix [warn] docs: introduction missing: spiral-matrix [warn] anagram: missing 2 test cases - handles case of greek letters (a6854f66-eec1-4afd-a137-62ef2870c051) - different characters may have the same bytes (fd3509e5-e3ba-409d-ac3d-a9ac84d13296) [warn] reverse-string: missing 3 test cases - wide characters (1bed0f8a-13b0-4bd3-9d59-3d0593326fa2) - grapheme cluster with pre-combined form (93d7e1b8-f60f-4f3c-9559-4056e10d2ead) - grapheme clusters (1028b2c1-6763-4459-8540-2da47ca512d9) [warn] some exercises have unsynced docs [warn] some exercises are missing test cases

[BNAndras]

So configlet is a CLI tool for managing Exercism tracks. There's a script to download the latest version at bin/fetch-configlet, and it can do a few things (format config files, create stub files for a new practice exercise, and sync content from our problem-specifications repo. For practice exercises (we don't have concept exercises), the docs, metadata, and tests typically come from that repository so configlet is used to grab the latest data.

For docs, it's pretty straightforward. configlet sync --docs is a dry run to see what docs don't match the problem-specifications copy. configlet sync --docs -uy updates all the docs for you, and that's largely good enough. If there was some track-specific we wanted to keep, those should go in a separate file like instructions.append.md which gets appended to the instructions markdown when display on Exercism. Configlet doesn't touch those append files when syncing.

For tests, it's a little different. configlet sync --tests is a dry run of what tests are missing from the .meta/tests.toml file for a given exercise. Configlet can't actually tell what tests are implemented or not so you'll need to check that yourself and add missing tests to the exercise's test suite. This functionality was added three years ago but some of the exercises were ported years before that so the tests.toml might say something is implemented when it's not because the tests.toml files were created in bulk. So you may want to double-check that all the listed tests are present. We have a separate issue for tracking those exercises, and I've fixed most of them.

So, what I generally recommend is one PR for all the docs and then individual PRs for each exercise missing tests so they're easier to review. The configlet sync --tests workflow has a few different switches you can take a look at in the terminal, but generally I do configlet sync --tests include -uye <exercise-slug> which automatically updates the tests.toml for that specific exercise. Then I go and add the missing tests to the test suite. If a test doesn't make sense for how an exercise is implemented, add a line "include = false" underneath its description in the tests.toml, making sure there's an empty line between it and the next test. That tells configlet we're purposefully not including it so it's aware. Otherwise, it'll think the test was implemented, and a maintainer checking the tests.toml in the future will also think it's implemented.

[kmarker1101]

Might want to leave this open, tying to add missing tests to reverse-string but unicode is not straight forward in elisp.

[BNAndras]

Yeah the PR was autoclosed from the fix #364 reference in that commit. I’d avoid using close or fix and then an issue number.

[kmarker1101]

I think we can close this one now.
Updating cached 'problem-specifications' data... Checking exercises... Every exercise has up-to-date docs, filepaths, metadata, and tests