IDNA support

[valenting]

I wrote a UTS 46 implementation based on http://intertwingly.net/projects/pegurl/idna.js
It needs a fast path for ascii codepoints, but I wanted to get some feedback on it first.

[SimonSapin]

Is idna_table.txt provided by Unicode? Where can it be found?

[valenting]

http://www.unicode.org/Public/idna/latest/IdnaMappingTable.txt

[SimonSapin]

Please add this URL in a code comment.

[SimonSapin]

Could you also integrate the tests form http://www.unicode.org/Public/idna/latest/IdnaTest.txt ?

---

I’m considering rewriting the Python scripts to Rust and running them as Cargo build scripts to avoid having generated files in the repository. However in Gecko it’s not as easy if you’re not using Cargo. It would be a separate .rs file to be compiled for the host (does Gecko do cross-compilation?), run with an OUT_DIR environment variable, and have that run be a dependency of building the url crate, which should be done with the same environment variable.

How painful is it to do in Gecko? If it would make your like more difficult I can stick to Python and generated files in the repository.

[valenting]

It would be a bit harder to have the Gecko build system run the rust files at build time, but at the moment that's not a blocker. We can include the the generated rust-generated files just as we do with the python generated ones at the moment.

[bors-servo]

☔ The latest upstream changes (presumably #125) made this pull request unmergeable. Please resolve the merge conflicts.

[bors-servo]

☔ The latest upstream changes (presumably #129) made this pull request unmergeable. Please resolve the merge conflicts.

[SimonSapin]

See #152

[valenting]

I've implemented the BIDI checks, and now all of the tests pass (except the contextJ tests, which client software isn't required to pass).
For combining_mark I call into unicode_normalization::char::canonical_combining_class - which I fear might not map correctly to General_Category=Mark. The unicode-normalization table generator has this data, but doesn't use it:
https://github.com/unicode-rs/unicode-normalization/blob/master/scripts/unicode.py#L92

[SimonSapin]

Other than a few nits I think this is ready to land. Thanks again!

[SimonSapin]

@bors-servo r+

[SimonSapin]

Not sure what’s happening with homu, but the Travis build for the PR push is green.