Clean up the Emoji tooling

[macchiati]

The emoji tooling needs some cleanup.

1. Right now, the emoji data files are built into a target directory, as per docs/... emoji/index.md. They should instead be built directly into the right location in unicodetools.
2. For new versions, all of the old files should be copied into the new directory and checked in, before generating new files over them. That way it is easy to capture and diff the new version's differences.
3. Check the PRs used for emoji 15.0 for more areas to fix to make it easier to update

[markusicu]

Right now, the emoji data files are built into a target directory, as per docs/... emoji/index.md. They should instead be built directly into the right location in unicodetools.

Really? For pretty much all of the other files, the tools write them to .../Generated/... folders; then we manually compare them with the ones in the input data folder, and then we copy the ones that have significant changes (not just in the time stamp) back into the input data folder and commit.

I don't like what GenerateConfusables does, always modifying the input data folder, because it creates churn when nothing really changes.

If a tool can be as clever as UCD.Main and only change an input file if there is a significant change, then we can discuss whether to write such files directly to the input data folder. However, that should still be optional, because it should be possible to run tools without changing the tree of files checked out from the repo.

For new versions, all of the old files should be copied into the new directory and checked in, before generating new files over them. That way it is easy to capture and diff the new version's differences.

We have been doing that for Unicode 15. In 2021-nov, in PR #156, I copied all of the .../15.0(.0) folders to "dev" folders, and we have been updating all data in those "dev" folders for Unicode 15. The only time that was not done was PR #232 for emoji files, but I fixed that in PR #248. See https://github.com/unicode-org/unicodetools/commits/main/unicodetools/data/emoji/dev

[nedley]

I think the first issue mentioned is covered by #310.

[markusicu]

I think the first issue mentioned is covered by #310.

I think not. That issue is about moving the .../dev/internal/ files to a different repo location.

This issue's item 1 is about where the generator tool(s) write output files. Currently the files are written to something like .../Generated/emoji/15.1/. The proposal is to write them to {repo}/unicodetools/data/emoji/dev if there are changes other than the time stamp.

[nedley]

I think the remaining issue mentioned by Markus was resolved by #397, see for example 39ea4ea#diff-6370a3c361dd76136ec7a286a34a855295cb186c4046b577c2b6e6c423f82377

[markusicu]

Emoji files with more than timestamp changes are written straight back into the input folder since PR #397 see changes in unicodetools/src/main/java/org/unicode/tools/emoji/GenerateEmojiData.java.