/explore/code rendering slowly when there are many repositories (> 2000) and a logged in user

[jnlin]

• Gitea version (or commit ref): 6a1a633
• Git version: 2.11.0
• Operating system: Docker
• Database (use [x]):
  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite
• Can you reproduce the bug at https://try.gitea.io:
  • Yes (provide example URL)
  • No
  • Not relevant
• Log gist: N/A

Description

We have a lot of repositories (>2000), and the rendering of /explore/code is slow (see the attached screenshot, > 17 seconds) with a logged in user.

We tried to remove some permission checking feature and the speed becomes good, however we are not sure which is the slowest part. The removed code is: master...jnlin:jnlin_test

The possible parts under suspicion are:
https://github.com/go-gitea/gitea/blob/master/routers/home.go#L330
https://github.com/go-gitea/gitea/blob/master/routers/home.go#L338-L343

Screenshots

[lunny]

The differences are GetRepositoriesMapByIDs and ctx.Data["RepoMaps"] = rightRepoMap.

[jnlin]

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L330

This spends < 0.5 secs

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L338-L343

This spends 17 secs, there may be some N+1 queries in CheckUnitUser()

[a1012112796]

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L330

This spends < 0.5 secs

https://github.com/go-gitea/gitea/blob/master/routers/home.go#L338-L343

This spends 17 secs, there may be some N+1 queries in CheckUnitUser()

How about #12854 ?

[jnlin]

How about #12854 ?

Still needs 15 secs

[lunny]

We need more PRs here, but let's merge this one at first.

[a1012112796]

Now the main issue is how to quickly find out all repos that the user has code read permission?

[jnlin]

@a1012112796 yes

[stale]

This issue has been automatically marked as stale because it has not had recent activity. I am here to help clear issues left open even if solved or waiting for more insight. This issue will be closed if no further activity occurs during the next 2 weeks. If the issue is still valid just add a comment to keep it alive. Thank you for your contributions.

[lafriks]

Up