issue search on my related repositories #9758

bhalbright · 2020-01-14T04:44:17Z

Added search box on dashboard issues page; searches across user's repositories as appropriate.

Ref issue: #2434

…l-issue-search

…tions

… also trying to fix issue of searching the right repos based on the filter

…l-issue-search

…pagination; using shown issue status for open/close count; removed some debugging

…l-issue-search

codecov-io · 2020-01-14T05:14:29Z

Codecov Report

Merging #9758 into master will increase coverage by 0.07%.
The diff coverage is 80.35%.

@@            Coverage Diff             @@
##           master    #9758      +/-   ##
==========================================
+ Coverage   43.71%   43.78%   +0.07%     
==========================================
  Files         586      586              
  Lines       81652    81764     +112     
==========================================
+ Hits        35691    35802     +111     
+ Misses      41535    41527       -8     
- Partials     4426     4435       +9

Impacted Files	Coverage Δ
modules/structs/user_app.go	`0% <ø> (ø)`	⬆️
routers/api/v1/api.go	`76.13% <100%> (+0.77%)`	⬆️
modules/convert/convert.go	`75.88% <100%> (+0.71%)`	⬆️
models/oauth2_application.go	`61.01% <53.84%> (+2.86%)`	⬆️
routers/api/v1/user/app.go	`68.55% <80.95%> (+9.46%)`	⬆️
services/pull/temp_repo.go	`31.62% <0%> (-2.57%)`	⬇️
modules/git/repo.go	`47.7% <0%> (+0.91%)`	⬆️
modules/process/manager.go	`78.31% <0%> (+3.61%)`	⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4667efc...b2998a9. Read the comment docs.

lunny · 2020-01-14T05:25:00Z

I think this should be issue search on my related repositories but not a generic global issue search on all the public repositories.

…l-issue-search

… so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats

bhalbright · 2020-01-16T04:05:36Z

Here's an animated gif showing the basics of the change

bhalbright · 2020-01-16T04:10:17Z

This PR is ready to review, thanks!

routers/user/home.go

6543 · 2020-01-16T13:25:51Z

@bhalbright you need to add &q={{$.Keyword}} to Line: 8, 13, 17, 21 too

bhalbright · 2020-01-16T16:15:03Z

@bhalbright you need to add &q={{$.Keyword}} to Line: 8, 13, 17, 21 too

@6543 I actually left those off on purpose but I can see it is debatable. My reasoning is that the numbers next to those links (for example "In your repositories") is not affected by the search, so I thought clicking on it should clear the search and show you the number of issues next to the link.

lunny · 2020-01-17T02:27:46Z

~~@bhalbright I like your previous design to put the search box on the second line and align right side. And I think that's also easier to optimize for mobile UI?~~

bhalbright · 2020-01-17T03:55:45Z

@bhalbright I like your previous design to put the search box on the second line and align right side. And I think that's also easier to optimize for mobile UI?

@lunny are you referring to this design that was proposed by @scullhead?

I started down that path but I didn't see an easy way to make it work in the organization view, there isn't quite enough room available depending on resolution with the existing links there (well, not enough room if aligning vertically with the list of issues):

…l-issue-search

models/issue.go

guillep2k · 2020-01-22T22:24:55Z

routers/user/home.go

+	var forceEmpty bool
+	var issueIDsFromSearch []int64
+	var keyword string
+	if !isPullList {


Why not pull requests too? Are they not indexed? 🤔

I think this is just because the scope of the change is for issues #2434

But I could see that this functionality would be useful for pull requests. I'm not sure the typical way things are done here...I guess some projects are strict about not expanding scope whereas others don't mind a bit. Any thoughts?

Oh, and I don't know if PRs are indexed or not

I'm just going to mark this resolved. I limited the scope to issues based on my understanding of #2434

In the database, pull requests are issues as well (only with is_pull = true), they are stored in the issue table and they are indexed indeed (the conversations, not the code itself). This page is used for issues and PRs equally, so I think there's no reason to skip this block when isPullList is true. The distinction is already taken care of when the IsPull flag is set a few lines above.

I agree. I updated and it now shows on the pulls page and seems to work just fine

…l-issue-search

…repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to

… issues ids that goes over 1000

…l-issue-search # Conflicts: # routers/user/home.go

…l-issue-search

…l-issue-search # Conflicts: # templates/user/dashboard/issues.tmpl

guillep2k

I'm so sorry it took me so long to review your PR. Here's some comments.

guillep2k · 2020-02-17T03:53:21Z

models/issue.go

@@ -76,6 +76,7 @@ var (
 const issueTasksRegexpStr = `(^\s*[-*]\s\[[\sx]\]\s.)|(\n\s*[-*]\s\[[\sx]\]\s.)`
 const issueTasksDoneRegexpStr = `(^\s*[-*]\s\[[x]\]\s.)|(\n\s*[-*]\s\[[x]\]\s.)`
 const issueMaxDupIndexAttempts = 3
+const maxIssueIDs = 1000


I think the actual limit on SQLite might be a little lower (997 or something), but you need to account for other parameters as well. For example, if the query targets repositories with a specific topic, the topic ID will be a parameter. I'd reduce this limit to say... 950? to be on the safe side.

thanks, I updated to 950

models/issue.go

routers/user/home.go

guillep2k · 2020-02-17T04:19:16Z

routers/user/home.go

+	var forceEmpty bool
+	var issueIDsFromSearch []int64
+	var keyword string
+	if !isPullList {


In the database, pull requests are issues as well (only with is_pull = true), they are stored in the issue table and they are indexed indeed (the conversations, not the code itself). This page is used for issues and PRs equally, so I think there's no reason to skip this block when isPullList is true. The distinction is already taken care of when the IsPull flag is set a few lines above.

guillep2k · 2020-02-17T04:22:11Z

routers/user/home.go

+	var keyword string
+	if !isPullList {
+		keyword = strings.Trim(ctx.Query("q"), " ")
+		if bytes.Contains([]byte(keyword), []byte{0x00}) {


Go supports the null character inside strings, so I don't see how this could be a problem.

Thanks, I removed the check. I had copied and pasted this from the issue search code inside a specific repository

guillep2k · 2020-02-17T04:30:04Z

templates/user/dashboard/issues.tmpl

+						</div>
+					</div>
+					<div class="column center aligned">
+					{{if .PageIsIssues}}


PRs could be supported too just by removing this if.

done, thanks

…l-issue-search

… GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues)

…l-issue-search

bhalbright added 13 commits December 26, 2019 14:37

adding search capability to user's issues dashboard

Loading
Loading status checks…

f7427fd

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

Loading
Loading status checks…

8295dc3

…l-issue-search

global issue search

Loading
Loading status checks…

9ff3405

placement of search bar on issues dashboard

50d21bb

fixed some bugs in the issue dashboard search

af34459

added unit test because IssueIDs option was added to UserIssueStatsOp…

a041a67

…tions

some renaming of fields in the issue dashboard code to be more clear;…

9f99cad

… also trying to fix issue of searching the right repos based on the filter

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

2a65245

…l-issue-search

added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on …

0c946bc

…pagination; using shown issue status for open/close count; removed some debugging

fix issue with all count showing incorrectly

9176642

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

85decd3

…l-issue-search

removed todo comment left in by mistake

babe113

typo pulling wrong count

1491e24

GiteaBot added the lgtm/need 2 label Jan 14, 2020

bhalbright changed the title ~~WIP Global issue search~~ WIP issue search on my related repositories Jan 14, 2020

bhalbright added 4 commits January 14, 2020 21:52

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

f3c4710

…l-issue-search

fxied all count being off when selecting repositories

15a681a

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

5742aae

…l-issue-search

setting the opts.IsClosed after pulling repos to search, this is done…

d7e34d2

… so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats

bhalbright changed the title ~~WIP issue search on my related repositories~~ issue search on my related repositories Jan 16, 2020

bhalbright marked this pull request as ready for review January 16, 2020 04:09

6543 reviewed Jan 16, 2020

View reviewed changes

routers/user/home.go Show resolved Hide resolved

6543 approved these changes Jan 16, 2020

View reviewed changes

GiteaBot removed the lgtm/need 2 label Jan 16, 2020

GiteaBot added the lgtm/need 1 label Jan 16, 2020

lafriks added the type/enhancement label Jan 16, 2020

lafriks added this to the 1.12.0 milestone Jan 16, 2020

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

455e71a

…l-issue-search

guillep2k reviewed Jan 22, 2020

View reviewed changes

bhalbright added 9 commits February 2, 2020 14:25

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

7781200

…l-issue-search

added "accessibleRepositoryCondition" check on the query to pull the …

d41ba39

…repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to

added code so that in the issues search, we won't use an in clause of…

15e9df6

… issues ids that goes over 1000

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

7d7b617

…l-issue-search # Conflicts: # routers/user/home.go

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

26819ce

…l-issue-search

fixed unit test

ee6e523

fix unit test

9c4e7c3

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

8bbaaa1

…l-issue-search

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

ab5799f

…l-issue-search # Conflicts: # templates/user/dashboard/issues.tmpl

guillep2k reviewed Feb 17, 2020

View reviewed changes

bhalbright added 4 commits February 17, 2020 21:56

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

b1c5807

…l-issue-search

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

bd39a5b

…l-issue-search

using 950 as the limit for issue search, removed unneeded group by in…

5080e4c

… GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues)

Merge branch 'master' of https://github.com/go-gitea/gitea into globa…

05b35e0

…l-issue-search

guillep2k approved these changes Feb 29, 2020

View reviewed changes

GiteaBot added lgtm/done and removed lgtm/need 1 labels Feb 29, 2020

guillep2k added 2 commits February 29, 2020 02:53

Merge branch 'master' into global-issue-search

4667efc

Merge branch 'master' into global-issue-search

b2998a9

guillep2k merged commit 82be59e into go-gitea:master Feb 29, 2020

bhalbright deleted the global-issue-search branch May 30, 2020 02:29

CirnoT mentioned this pull request May 30, 2020

Increase maximum SQLite variables count to 32766 #11696

Merged

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

issue search on my related repositories #9758

issue search on my related repositories #9758

bhalbright commented Jan 14, 2020

codecov-io commented Jan 14, 2020 •

edited

Loading

lunny commented Jan 14, 2020

bhalbright commented Jan 16, 2020

bhalbright commented Jan 16, 2020

6543 commented Jan 16, 2020

bhalbright commented Jan 16, 2020

lunny commented Jan 17, 2020 •

edited

Loading

bhalbright commented Jan 17, 2020 •

edited

Loading

guillep2k Jan 22, 2020

bhalbright Jan 23, 2020

bhalbright Jan 23, 2020

bhalbright Feb 3, 2020

guillep2k Feb 17, 2020

bhalbright Feb 20, 2020

guillep2k left a comment

guillep2k Feb 17, 2020

bhalbright Feb 20, 2020

guillep2k Feb 17, 2020

guillep2k Feb 17, 2020

bhalbright Feb 20, 2020

guillep2k Feb 17, 2020

bhalbright Feb 20, 2020

issue search on my related repositories #9758

issue search on my related repositories #9758

Conversation

bhalbright commented Jan 14, 2020

codecov-io commented Jan 14, 2020 • edited Loading

Codecov Report

lunny commented Jan 14, 2020

bhalbright commented Jan 16, 2020

bhalbright commented Jan 16, 2020

6543 commented Jan 16, 2020

bhalbright commented Jan 16, 2020

lunny commented Jan 17, 2020 • edited Loading

bhalbright commented Jan 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guillep2k left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Jan 14, 2020 •

edited

Loading

lunny commented Jan 17, 2020 •

edited

Loading

bhalbright commented Jan 17, 2020 •

edited

Loading