Publishing workspaces with large numbers of crates hits the rate limit #1643
Comments
|
I'm really sorry about that! We recently added some rate limiting on the publish endpoint, and 429 is too many requests. We don't want to prevent publishing multiple crates in a workspace, but we were unsure of what folks doing legitimate publishing would need the limit to be. Now we have a data point! There are many solutions; we've discussed making the rate limit apply to new crates only rather than new versions, we could slow down requests rather than rejecting them, and the most straightforward, we could increase the rate limit. I'm going to increase the limit to 30 so that you can publish, and I'll put this on our agenda for our next meeting to discuss a better long term solution. |
|
Oh also Cargo not displaying a nice error with a clear explanation is a bug that I'll make sure to file in cargo's repo. |
|
Thanks, @carols10cents, for the prompt fix! Yeah, making sure Cargo shows a nice message would be great. Besides that, I think we're good to close it here. Side note: we're aware of the fact that re-publishing all UNIC components at once is not ideal. Trying to find a better solution to be able to guarantee certain code and data compatibility without mass-publishing. That said, I think there are still legitimate cases for us to mass-publish, and this fix unblocks those future cases, as well. Thanks! :) |
|
@behnam Please try publishing again; the rate limit should now be 30 publishes per minute. I am going to leave this open because I bet you're not the only project out there, so we do need to figure out some sort of change here. I also filed a bug with cargo for improving the error display and added this issue to the meeting agenda for next week. |
carols10cents
changed the title
|
Thanks, Carol! All sounds good. About UNIC, I was able to publish the few last components with running the commands manually after a while. So, nothing block us at the moment. |
In rust-lang/crates.io#1596 it was added a rate limit for crate publishing endpoint connections (10 components/min). As is said in the PR. The strategy used allows to upload 10 components first and then the ratio starts to be applied. Now, if the rate is surpassed, crates.io returns a 429 HTTP response. Cargo, tries to publish all of the workspace/crate components as fast as possible (as we should expect) but when we have more than 10 components and we try to publish the 11th, usually a minute hasn't passed yet and therefore we get a HTTP-429. This made some users to experience confusing/not well explained error messages as seen in rust-lang/crates.io#1643. The limit got increased to 30 components/min. But who knows.. So this closes rust-lang#6714 while a better solution is not found.
|
as suggested in #3229 I'll go ahead and close this issue. if you feel like this is not solved sufficiently yet please let us know. |
|
This would be less of an issue if publishing an entire workspace was an option. Is this something coming to cargo? |
|
We're dealing with this issue now with the 30-crate rate limit. What came of the discussion regarding how to address this rate-limiting issue long-term? |
Contact help@crates.io with details on which crates you are looking to publish from a workspace and which user(s) will be performing the publish, and we can adjust the limit on a per-user basis. |
When publishing UNIC 0.9.0, there's a sequence of
cargo updateandcargo publishfor all the components (28 of them).Near the end of publish, at
cargo publish --verbose --manifest-path unic/emoji/char/Cargo.toml, I got the following error:The client,
cargo, seems to not understand the response. Not sure if the server is at fault (with a bad response) or the client (with not handling the response). Anyways, reporting to investigate.The text was updated successfully, but these errors were encountered: