Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hides crates on index calls unless they have at least one available v… #985

Closed
wants to merge 1 commit into from

Conversation

mauricio
Copy link

@mauricio mauricio commented Aug 20, 2017

…ersion

This makes the index call only show crates that have at least one available version. Crates that have all versions yanked will not show up anymore.

Main problem for the implementation here is that diesel doesn't allow you to build a subquery that references a field from the parent query, so the only way to get this done is through pure SQL.

Fixes #958

@carols10cents
Copy link
Member

This excludes yanked crates from ALL functionality that is powered by the index api route, which unfortunately is a lot:

  • search results
  • crates in a category
  • crates with a keyword
  • the "browse all crates" list
  • user pages
  • team pages
  • crates you're following

Issue #985 was intended to be just excluding yanked crates from user pages and team pages. This comment of mine on #145 explains why I don't think we should just hide yanked crates everywhere all the time without a way to opt-out of doing that.

If you'd like to just fix #958, could you change this PR to only include this filter if there's a user or team in the query params please?

Thinking about the query, could this be done with a LEFT JOIN that has a condition that the version is not yanked, and then the version must not be null? I haven't tried this yet, so it might not work, let me know if what I'm saying makes sense at all and I can elaborate.

Copy link
Member

@carols10cents carols10cents left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change to only filter yanked from user and team pages, possibly change query ❤️

@mauricio
Copy link
Author

Definitely!

Gonna fix and send it again later today!

src/krate.rs Outdated
@@ -564,6 +564,10 @@ pub fn index(req: &mut Request) -> CargoResult<Response> {

let recent_downloads = sql::<Nullable<BigInt>>("SUM(crate_downloads.downloads)");

let not_yanked_versions = sql::<Bool>(
"crates.id = ANY (SELECT vs.crate_id FROM versions vs WHERE vs.crate_id = crates.id AND vs.yanked IS FALSE)",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be left_join(crates_versions.on(crates::id.eq(crates_versions::crate_id).and(crates_versions::yanked.eq(false)))).filter(crates_versions::id.is_null())

@mauricio mauricio force-pushed the issue-958 branch 2 times, most recently from d7f9385 to 059b87b Compare August 25, 2017 05:00
@mauricio
Copy link
Author

@carols10cents updated and included a new test for the no user_id or team_id case.

@sgrif couldn't get that to compile, couldn't find a table/class crates_versions anywhere. There's no link table between crates and versions as well, it's a belongs to relationship.

…ersion

This makes the index call only show crates that have at least
one available version. Crates that have all versions yanked
will not show up anymore.

Fixes rust-lang#958
@mauricio
Copy link
Author

@carols10cents so I tried to add a new join here but changing this changed the whole return type and then it doesn't compile anymore.

Couldn't find any other examples on diesel's documentation on how to do it. Do you have any tips?

@carols10cents
Copy link
Member

@mauricio could you show me what you tried, explain what you're trying to do, and the error you were getting please?

@sgrif
Copy link
Contributor

sgrif commented Oct 9, 2017

Main problem for the implementation here is that diesel doesn't allow you to build a subquery that references a field from the parent query, so the only way to get this done is through pure SQL.

Perhaps we should defer this until we're pointing at Diesel master (soon)? Subselects can reference the outer table on master.

.any(|s| s.is_some())
{
let not_yanked_versions = sql::<Bool>(
"crates.id = ANY (SELECT vs.crate_id FROM versions vs WHERE vs.crate_id = crates.id AND vs.yanked IS FALSE)",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this just be EXISTS (SELECT 1 FROM versions WHERE versions.crate_id = crates.id AND versions.yanked IS FALSE)?

@@ -696,6 +696,17 @@ pub fn index(req: &mut Request) -> CargoResult<Response> {
));
}

if vec![params.get("user_id"), params.get("team_id")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think this is worth the complexity of sticking it in a vec and iterating over it, vs params.get("user_id").is_some() || params.get("team_id").is_some()?

@mauricio
Copy link
Author

mauricio commented Oct 9, 2017

@sgrif that would be much better! Much less magic to handle here.

@sgrif
Copy link
Contributor

sgrif commented May 4, 2018

This has gotten quite stale so I'm going to go ahead and close it. Feel free to open a new PR if you're interested in reviving this. Implementing this feature should no longer require dropping down to raw SQL

@sgrif sgrif closed this May 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants