Check for typos.

[Eh2406]

Hi,

So in the recent discussion of the problems on the npm-malicious-packages and the problems on Python typosquatting I bilt a quick script for looking for typos. I think the python community added tools for checking if the packages you are using are similar to much more popular packages.

I was thinking of making a cargo-utill somehow based on my quick script. I started by looking at cargo-edit for info on how to read the toml, handle workspaces, do testing, etc.. But at this point it may as well be a part of cargo-edit.

Thoughts?

[killercup]

That's a very cool idea! It shouldn't be too difficult to have cargo-add do. some more checks before adding a crate to the Cargo.toml and alert the user/ask "is that really what you meant". Contributions welcome! Jacob Finkelman <notifications@github.com> schrieb am Do. 12. Okt. 2017 um 02:37:

…

Hi, So in the recent discussion of the problems on the npm-malicious-packages <https://iamakulov.com/notes/npm-malicious-packages/> and the problems on Python typosquatting <https://www.reddit.com/r/rust/comments/70aq3b/attackers_are_typosquatting_package_names_in_the/> I bilt a quick script for looking for typos <https://internals.rust-lang.org/t/cargo-package-aliases/5870/6>. I think the python community added tools for checking if the packages you are using are similar to much more popular packages. I was thinking of making a cargo-utill somehow based on my quick script. I started by looking at cargo-edit for info on how to read the toml, handle workspaces, do testing, etc.. But at this point it may as well be a part of cargo-edit. Thoughts? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#172>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABOX85f9VtL0Ux7PSBszk-gB6MTSYftks5srV8tgaJpZM4P2T9X> .

[Eh2406]

Thanks for the +1, just some notes as I dig in.

Looks to me like get_latest_dependency is where add talks to the internet and I'd prefer to add as few trips to the internet as possible. If we add this functionality there then it becomes part of both add and upgrade.

I want an example to test with. I think request should suggest reqwest. Unfortunately the search on the website for request does not find reqwest. So just checking the search api won't help. If we are going to need to get a list of all the names than using the web api's are not going to work. (Unless we get them to add one, but that seems like a big job.)

My original script used a checkout of crates.io-index to get all the names. But that is somewhat wasteful as cargo keeps a copy at "%CARGO_HOME%\registry\index\github.com-0somehash4` so maybe we can read it from there, no trips to the internet required. We would need to determine the exact path and sutch. edit: looks like the code is in https://github.com/rust-lang/cargo/tree/master/src/cargo/sources/registry

[killercup]

Let's get a general implementation plan in place before writing a lot of code :) I'd hoped crates.io would have the right kind of fuzzy search for that in place. Maybe it makes sense to also open an issue on their tracker as well? The crates.io search could benefit from this as well. But let's assume we need to do this ourselves. Is there a Cargo subcommand to query the internal index? Or at least, to get its path? This way, we could at least get a list of crate names (using the file names), normalize them (- -> _ and stuff like that), and then do a fuzzy match to get a list of potential candidates. We would then need to query crates.io's API for stats on each of the matches, right? Assuming we can do that in parallel, it shouldn't take more than a second. Then we can either add the crate if its not totally obscure (assuming we have an algorithm to determine that), or prompt the user to confirm or choose the most popular version, or just display a warning (there should an option that suppresses all interactive prompt from cargo-add, on by default in non-tty envs). I would really like cargo-add to feel snappy. Do you think this will slow it down a noticeable amount? Jacob Finkelman <notifications@github.com> schrieb am Fr. 13. Okt. 2017 um 00:43:

…

Thanks for the +1, just some notes as I dig in. Looks to me like get_latest_dependency is where add talks to the internet and I'd prefer to add as few trips to the internet as possible. If we add this functionality there then it becomes part of both add and upgrade. I want an example to test with. I think request <https://crates.io/search?q=request> should suggest reqwest <https://crates.io/search?q=reqwest>. Unfortunately the search on the website for request does not find reqwest. So just checking the search api won't help. If we are going to need to get a list of all the names than using the web api's are not going to work. (Unless we get them to add one, but that seems like a big job.) My original script used a checkout of crates.io-index to get all the names. But that is somewhat wasteful as cargo keeps a copy at "%CARGO_HOME%\registry\index\github.com-0somehash4` so maybe we can read it from there, no trips to the internet required. We would need to determine the exact path and sutch. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#172 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABOX3a8z6bOk5qhTvoanCHTZkTRJ-h-ks5srpYXgaJpZM4P2T9X> .

[ordian]

I'd hoped crates.io would have the right kind of fuzzy search for that in
place. Maybe it makes sense to also open an issue on their tracker as well?
The crates.io search could benefit from this as well.

Maybe this issue is relevant rust-lang/crates.io#1074?

[Eh2406]

@ordian that looks very relevant thanks for the link! If they want to add an API we should use it!

@killercup

Let's get a general implementation plan in place before writing a lot of
code :)

No worries, no code has been harmed written in my digging in. From you response we are on smaller wavelengths. I to want to keep this code snappy.

As to doing it ourselves, that is what I was thinking. I don't know of a way to get cargo to give the path to the index. (there is a library for querying it, but it is maintained separately and somewhat out of date.) Maybe we should talk to the cargo people, about adding a sub command or moving there code to a library? Updating cargos index is not a snappy operation, but if you just used add than it is the next thing cargo is going to do anyway. I think it should just be a warning like, Looking up request, but reqwest is far more popular. Please double check that you are using the library you intended.

Also if we get access to the index, then we do not need the internet to find what version to add.

[epage]

Moved to rust-lang/cargo#10655