newly updated crates are not built if there are too many of them #89
Comments
Byron
changed the title
|
I am actually working on a PR to fix this. The idea is to clone the crates index and keep track of the last commit we have seen. I will also setup a WIP PR in a moment so you can stay informed on the progress. |
|
Thanks for reporting. This is similar to #84. Since docs.rs started getting new crate list from crates.io/summary this issue also occurred. docs.rs is checking new crates every 5 minutes and crates.io/summary is only returning last updated 10 crates. If crates.io gets more than 10 uploads in 5 minutes, docs.rs is skipping them. I will revert this changes and start getting new crates from crates.io-index repository again. Leaving this commit id for reference: 712dcb6 #77 fixed link. |
|
Thanks for elaborating on this! |
|
Ah, I see there is code for that already. However, I wouldn't mind giving it a try too. Can you give me an advise on how you would want me to proceed? |
|
Thanks for offering help! I changed old code because it wasn't working properly. You can see the old code from here: queue.rs. I have been trying to parse DiffLine with Diff::print but this seems really wrong. Maybe you can figure out how to get new crates properly by using git2 crate and crates.io-index. There must be a better way or approach to do this but I have no idea. |
|
Alright :)! I happen to have been working with git internals quite a bit in the past, and I always wanted to work with it once again from Rust. |
|
Here is the first version of the crates-index-diff crate. It's already providing all the functionality you would need. |
|
@Byron this seems really promising, thank you very much. There is only one issue. Docs.rs dependencies (cargo, hyper and iron especially) are using openssl 0.7 and latest git2 crate started using openssl 0.9. Currently we can't use both of them together. We have to use git2 0.5 until hyper starts using openssl 0.9. There is an ongoing PR for hyper: hyperium/hyper#975. But updating iron to 0.4 in docs.rs will require some work. So basically your crate must depend on git2 0.5 for now, we can update it later when all dependencies supports openssl 0.9. BTW your crate is fully compatible with git2 0.5. |
|
Version 1.0 of the crate has now been released. It's dependency to git was already adjusted to match the requirements of docs.rs. Now I am working on a quick CLI to show how it is used, and to have it ready in case you would like to keep the git-related accesses out of process (and also because I want it :)). |
|
Alright, the crates-io-cli is ready, and allows to test manually use the functionality of the Next up is a PR to actually use the new functionality in |
Otherwise docs.rs cannot use use. This is due to the issue described here: https://github.com/onur/docs.rs/issues/89#issuecomment-269139132
I have recently published a great amount of crates in an automated fashion, and noticed that docs-rs didn't really catch up with them.
An example would be the google-logging2 crate, which exists in version 1.0.1, whereas docs.rs still shows it as version 1.0.0.
A glimpse into the code indicates that docs.rs detects newly updated crates by looking at the
.just_updatedfield of the crates.io summary endpoint (see code), which appears to be a problem if there are many new releases in an interval shorter than the one used to query the endpoint.As I just now moved all my links to point to docs.rs, it would be great if we could find out what could be done to alleviate the issue :).
Thank you.
The text was updated successfully, but these errors were encountered: