User dashboard takes a long time to load since 1.17 #21611
Comments
|
#21045 may be related, but also maybe not as I didn't see such massive load times, maybe an 200-400ms extra for the heatmap data. Do you also see it on https://yourhost/youruser?tab=activity? |
|
No, this page loaded pretty quickly, along with the heatmap. |
|
Check your Maybe related to: |
|
Do you mean the mentioned database table in the database server, or somewhere else? Also, this is about the section marked on the following picture, right?
Oh I wanted to ask about it for some time, but always forgot it. I would really like if this could be filtered, for example so that commit sync events are not shown, because they happen so frequently (this is not a problem) that they hide everything else (this is a problem). So well, yeah, I don't need most of these. But then also, on 1.16.5 it did not take much time to load this page. |
|
It's very strange there is no router log in your file. |
|
That is because the log I made was made by a non-default logger, developer_debug here:
Would having router logs help? If so, let me know and I'll make a new log file that also includes the router log. |
Yes, router log will be helpful |
|
I'm having a hard time in figuring out how to set up the router logger to log into the developer_debug file logger. I'll try to describe what did I try. After every change, I used the "Log Configuration" section of the
This is what I had when creating the log gist linked in the issue: The following is how I tried to change the above. I tried to configure the router logger of the developer_debug sublogger, so I added this new section, and restarted Gitea: According to Gitea (https://myinstance.lan/admin/config, Log Configuration) this did not change or add a new router logger. Then I thought maybe the problem is that I didn't define the target (where should this be output to), so I added Later I started to suspect this is the same ROUTER as what can be found directly in [log]. Still no change, so I thought maybe this logger isn't detected by Gitea at all, and remembered that I had to add even my developer_debug sublogger to [log].MODE. With this, now I see that a new Router logger is visible in the log configuration (https://myinstance.lan/admin/config, Log Configuration), but I still don't see router logs in this log file. Then went back to this: https://docs.gitea.io/en-us/config-cheat-sheet/#router-log-log I had a hard time in setting up the developer_debug logger too, but I did manage that. This is all of the configuration that I now have related to logging: |
|
I think I got it.
@lunny here is a new log gist with router logs. It includes logs from recent startup of Gitea. Last event was made after the user dashboard finished loading. |
|
|
Could you test v1.18.1 resolved the problem? |
|
It doesn't yet. Page load time of the main page was 157949ms according to the footer.
|
|
How many records in your |
|
I have around 257.000, according to the summary on the admin dashboard at
|
|
If there is anything more I can help with (like additional statistics, logs, etc), please let me know |
|
And could you paste your logs when visit dashaboard? Include xorm log and router log. |
|
Sure, here is the log: https://gist.github.com/mpeter50/4fa95c0d7a575b273f0a9369c0073929 I have cut off what was happening while Gitea was starting up, but by timestamp I made sure to include everything from when I started loading the dashboard. |
So this is a database index problem. |
|
add new index to action table |
We need a new index for this. |
|
@mpeter50 I have added about 300000 records to the
Could you please add this index to your |
sillyguodong
added
the
issue/needs-feedback
|
|
So far, I don't think the effect is very obvious |
The database server runs on an ARMv7 device (a Raspberry Pi 4), and is backed by portable hard drive as storage. These may be necessary for this to occur. But, other pages dont take nearly as long to load.
I'll try to do so, but it might take some more time for me to get to that.
I'm not sure I understand you, but as far as I know a fix has not yet been implemented. |
@mpeter50 Thanks! Please let us know the result if you get the work done. |
Close #21611 Add the index mentioned in #21611 (comment) . Since we already have an index for `("created_unix", "user_id", "is_deleted")` columns on PostgreSQL, I removed the database type check to apply this index to all types of databases.
|
I'm sorry for the late reply! Today I have updated my instance to 1.19.0, and the latency is still very similar as before:
According to the database server, I had the following indexes on the action table at this time: So then I created the suggested index: And the list of indexes now look like this: However, the dashboard takes around the same amount of time to load as before:
|
|
I think you can look at gitea.log and find out the longest time SQL AND explain "SQL" |
|
I have re-enabled sql logging, and uploaded the logs here that were gathered the next time I tried loading the dashboard. The longest SQL statement was this one, it is on the 10th line in the gist: I'm sorry but I did not understand the second part of your comment. |
|
What's your Gitea instance version and how many records on your Gitea database table |
I guess these information is in #21611 (comment)
SELECT `action`.* FROM `action`
INNER JOIN `repository` ON `repository`.id = `action`.repo_id
WHERE act_user_id IN (
SELECT `user`.id FROM `user`
WHERE (keep_activity_private=? AND visibility IN (?,?)) OR id=?
)
AND repo_id IN (
SELECT id FROM repository
WHERE (`repository`.is_private=?
AND `repository`.owner_id NOT IN (SELECT id FROM `user` WHERE type=? AND visibility IN (?))
)
OR `repository`.id IN (SELECT repo_id FROM `access` WHERE `access`.user_id=? AND `access`.mode>?)
OR `repository`.id IN (
SELECT `team_repo`.repo_id FROM team_repo
INNER JOIN team_user ON `team_user`.team_id = `team_repo`.team_id WHERE `team_user`.uid=?
)
OR `repository`.owner_id=?
OR (`repository`.is_private=? AND `repository`.owner_id IN (
SELECT `org_user`.org_id FROM org_user WHERE `org_user`.uid=?)
)
)
AND user_id=? AND is_deleted=? ORDER BY `action`.`created_unix` DESC
LIMIT 20
--- args: [false public limited 4 false 1 private 4 0 4 4 false 4 4 false]If it's a MySQL index problem, it's possible try to use Otherwise, if the MySQL server is slow (eg: hardware limitation), at the moment it's difficult to optimize this SQL (the "activity" mechanism would be totally rewritten). I used to truncate my |
|
Thats right, Gitea 1.19.0 and ~459340 action records according to the admin dashboard. The
This is probably the case, while waiting for the dashboard to load I can see in a resource monitor that the mysqld process is waiting for io most of its time. Also, a related question: is there currently a mechanism for ignoring (as in not saving to db) certain types of actions? |
|
Update: the admin dashboard now counts ~547767 actions, probably the analyze table command recounted them. |
I guess you are using HDD/Slow storage and/or MySQL's memory is pretty low.
That's
IIRC, no ...... unfortunately |
|
Or you can enable |
Yes, it is a portable HDD that is shared with other services and even the OS.
Thanks a lot, this will be helpful!
I'll consider that too, thanks! |
|
For anyone (and future me) reading this in the future: BEFORE EXECUTING THESE COMMANDS you probably want to shut down Gitea and make a backup of the database. According to If this delay is still too much, you can check what other types of actions are amassed in the table with Be aware though that you will need to repeat this regularly, as "synced from" actions are still created at every automatic mirror sync when changes are pulled. You can also set up automatic deletion of old actions as Lunny said above, but this will probably also mess with your and other users' activity graph (on the profile page, as seen below) if you set it to a value too low.
|
Description
When I updated from 1.16.5 to 1.17.0, the main page of gitea (the user dashboard, hosting the contribution activity thing, some recent events and the repositories list) started to take a very long time to load.
It only takes a long time (80 - 100 s) if I didn't load it recently. If I waited for it to load, and then reloaded it, it would load quickly.
While I'm waiting for the dashboard to load, I see both increased database process CPU load and increased iowait.
I did not update the database server at the time of updating Gitea.
The logs were made slightly after restarting Gitea. I've waited with opening the dashboard until repository indexing finished.
Gitea Version
v1.17.3
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
https://gist.github.com/mpeter50/1fa49a4e9c9f05536baedef0668e8c92
Screenshots
No response
Git Version
Operating System
Raspbian
How are you running Gitea?
In self-built Docker container, built according to the Docker.rootless Dockerfile.
Database
MySQL
The text was updated successfully, but these errors were encountered: