Fix match_str_case_mismatch on uncased chars

[Herschel]

False positives would result because char::is_lowercase and friends will return false for non-alphabetic chars and alphabetic chars lacking case (such as CJK scripts). Care also has to be taken for handling titlecase characters (Dz) and lowercased chars with no uppercase equivalent (ʁ).

For example, when verifying lowercase:

• Check !any(char::is_ascii_uppercase) instead of all(char::is_ascii_lowercase) for ASCII.
• Check that all(|c| c.to_lowercase() == c) instead of all(char::is_lowercase) for non-ASCII

Fixes #7863.

changelog: Fix false positives in [match_str_case_mismatch] on uncased characters

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @xFrednet (or someone else) soon.

Please see the contribution instructions for more information.

[bachue]

👍 it can resolve my problem

[xFrednet]

Thank you for the PR! Just one small NIT where I would also like to get some feedback from a second team member, then it should be ready to be merged. You did a fantastic job with the tests, those should hopefully cover everything 👍

[xFrednet]

My understanding is that to_lowercase / to_uppercase may allocate a new struct in some cases. I think we can replace this by testing for the negation of what the code was previously testing.

        CaseMethod::LowerCase => |input: &str| -> bool { !input.chars().any(char::is_uppercase) },
        CaseMethod::AsciiLowerCase => |input: &str| -> bool { !input.chars().any(car::is_ascii_uppercase) },
        CaseMethod::UpperCase => |input: &str| -> bool { !input.chars().any(char::is_lowercase) },
        CaseMethod::AsciiUppercase => |input: &str| -> bool { !input.chars().any(char::is_ascii_uppercase) },

@Manishearth What are your thoughts on this? (Since you've worked extensively with Unicode, from what I know 🙃 )

[Herschel]

The issue is with characters like ʁ which have no opposite-case-equivalent, i.e. 'ʁ'.is_lowercase() is true, but 'ʁ'.to_uppercase() == 'ʁ'. This would cause the above UpperCase case to fail, for example. char::ToUppercase struct is 16 bytes, so this seemed better than doing input.to_uppercase() == input to allocate a whole new whole string.

I'm also worried about any cases where, say, char.to_lowercase() != char.to_lowercase().to_lowercase(), but I think(?) these fns are idempotent. Also paging @Manishearth because I'm sure he'll be of much help :-)

[Manishearth]

I'd rather not allocate: I think we can use the Changes_When_Uppercased/Lowercased properties instead.

I wonder if we can add an unstable internal function to core::unicode for this so we can continue using Rust's unicode data. To do this you'd need to modify the unicode generator tool in rustc

https://github.com/rust-lang/rust/blob/6b0b41729939c3f7520e9ed86b36fba2524c7970/src/tools/unicode-table-generator/src/main.rs#L129

[xFrednet]

That would be nice to have 👍. Side node regarding the allocation. The documentation states:

Returns an iterator that yields the lowercase mapping of this char as one or more chars.

If this char does not have a lowercase mapping, the iterator yields the same char.

Meaning that this would only allocate if the lint should actually be triggered and then only once, if we use any(|c| c.to_lowercase().next() != c)

[Manishearth]

Ah, that makes it slightly better

[xFrednet]

So, how should we continue? I think we can merge this as it is and create a new issue for the possible enhancement. Does that sound line a plan? 🙃

[Manishearth]

Yes. I don't even think using changes_when_uppercased/etc matters that much for perf so we may not even need a followup

[xFrednet]

Alright 👍 Thank you for the feedback on this stuff 🙃

[xFrednet]

Thank you for the changes! I hope you also had fun working on Clippy 🙃

@bors r+

[bors]

📌 Commit e953dff has been approved by xFrednet

[bors]

⌛ Testing commit e953dff with merge cb0132d...

[bors]

☀️ Test successful - checks-action_dev_test, checks-action_remark_test, checks-action_test
Approved by: xFrednet
Pushing cb0132d to master...