Add PProf to admin pages and to `gitea manager` #22742

zeripath · 2023-02-03T22:21:06Z

The PProf endpoint is too difficult for many users to use - this PR will attempt to create a nicer UI to help in the collection PProf data from Gitea. It also adds pprof endpoints to the gitea manager command.

Signed-off-by: Andrew Thornton art27@cantab.net

The PProf endpoint is too difficult for many users to use - this PR will attempt to create a nicer UI to help in the collection PProf data from Gitea. Signed-off-by: Andrew Thornton <art27@cantab.net>

codecov-commenter · 2023-02-03T23:04:50Z

Codecov Report

❗ No coverage uploaded for pull request base (main@c07199f). Click here to learn what that means.
The diff coverage is 3.29%.

❗ Current head 1400ab2 differs from pull request most recent head edebd86. Consider uploading reports for the commit edebd86 to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##             main   #22742   +/-   ##
=======================================
  Coverage        ?   47.21%           
=======================================
  Files           ?     1113           
  Lines           ?   149657           
  Branches        ?        0           
=======================================
  Hits            ?    70658           
  Misses          ?    70617           
  Partials        ?     8382

Impacted Files	Coverage Δ
cmd/manager.go	`0.00% <0.00%> (ø)`
modules/private/manager.go	`0.00% <0.00%> (ø)`
modules/process/stacktraces_processlist.go	`0.00% <0.00%> (ø)`
routers/private/manager_process.go	`0.00% <0.00%> (ø)`
routers/web/admin/admin.go	`0.00% <0.00%> (ø)`
routers/web/admin/pprof.go	`0.00% <0.00%> (ø)`
modules/context/private.go	`100.00% <100.00%> (ø)`
routers/private/internal.go	`84.21% <100.00%> (ø)`
routers/web/web.go	`84.90% <100.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

wxiaoguang · 2023-02-04T11:44:35Z

templates/admin/stacktrace.tmpl

+		{{.locale.Tr "admin.monitor.stacktrace.download_stacktrace"}}
+	</div>
+	{{template "base/delete_modal_actions" .}}
+	<div class="hide" id="stacktrace-to-download">


Why not just:

User click the download button

Download from a HTTP URL directly

?

Then no need that complex dialog nor the JS code

No. Then they get a different stacktrace.

I mean , could there be no stacktrace on the UI? Just some links to help to collect problems.

The complex UI doesn't help users. As an end user, they should just download and report the stacktrace file.

The end users could do nothing even if they see the stacktrace on the UI.....

FYI, when I was using GitLab, they have an all-in-one tool. It collects everything into a file, and what I need to do is just using the tool and sending the generated file to them, then they can help to resolve problems.

Making the diagnosis system too complex doesn't benefit end users IMO.

I mean , could there be no stacktrace on the UI? Just some links to help to collect problems.

I'm going to add a direct download stacktrace profile below to get the stacktraces without seeing them.

The complex UI doesn't help users. As an end user, they should just download and report the stacktrace file.

The pretty stacktrace helps me and SEVERAL bugs have been solved using it. It has helped me a large number of times already.

The end users could do nothing even if they see the stacktrace on the UI.....

Not every user is incapable and whilst the purpose of this UI is to help us to help users we should enable users and developers to help themselves. A pretty UI can be helpful for us to solve issues in a way that the opaque stacktrace format is not.

I mean , could there be no stacktrace on the UI? Just some links to help to collect problems.

I'm going to add a direct download stacktrace profile below to get the stacktraces without seeing them.

If it means more complex, then it's not necessary IMO. My initial idea is about keeping the system simple but complete and useful.

While I do not think keeping the downloaded stacktrace file exactly the same as the UI list is meaningful -- everytime you refresh the page, you get a different stacktrace ..... Even if they are different, they are all helpful for resolving problem equally, and maybe sometimes the UI shown list is not helpful but the downloaded is helpful in case the downloaded one catches the problem, everything is possible.

Fundamentally it's really important to be able to download the stacktrace that you have seen. Given these are dynamic it's possible that you may actually only see the issue on the stacktrace that you have and if you try get another the issue will be gone.

We need the download.

FYI, when I was using GitLab, they have an all-in-one tool. It collects everything into a file, and what I need to do is just using the tool and sending the generated file to them, then they can help to resolve problems.

Making the diagnosis system too complex doesn't benefit end users IMO.

If I knew what we generally needed I would do that - but in general we don't need cpu-profiles etc.

We do need a way of reading the logs and I will get to that but ... let's get general routes in and then we can have pared back routes to get the common things. Right now we have no easy way of getting pprof profiles from users and that fundamentally limits our ability to use them for diagnostics. This PR makes that a hell of a lot easier.

Signed-off-by: Andrew Thornton <art27@cantab.net>

delvh

Hmm, I hope it is by design that the vast majority of generated reports is absolutely unreadable?
Perhaps we should add a "warning" at the top of the section stating something like Many of the generated files won't be understandable for you. Don't worry, the only thing that matters is that WE understand it. (Hopefully you find a better wording 😅)

cmd/manager.go

options/locale/locale_en-US.ini

routers/web/admin/admin.go

modules/process/stacktraces_processlist.go

templates/admin/pprof.tmpl

routers/web/admin/pprof.go

routers/web/admin/admin.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

templates/admin/pprof.tmpl

Signed-off-by: Andrew Thornton <art27@cantab.net>

silverwind · 2023-05-15T22:49:29Z

Any UI screenshots?

lunny · 2023-05-16T01:40:46Z

I think this has been replaced by #24636 ?

wxiaoguang · 2023-05-16T02:24:57Z

Yup, not needed any more. #24636 is much simpler and more friendly to users.

We can add more diagnosis report based on #24636 in the future if necessary

lunny · 2023-05-16T02:56:10Z

Maybe additional gitea manager could become another PR if it's still necessary and I will close this one.

Add PProf to admin pages

5093d7d

The PProf endpoint is too difficult for many users to use - this PR will attempt to create a nicer UI to help in the collection PProf data from Gitea. Signed-off-by: Andrew Thornton <art27@cantab.net>

zeripath added type/enhancement An improvement of existing functionality pr/wip This PR is not ready for review labels Feb 3, 2023

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Feb 3, 2023

zeripath added 4 commits February 4, 2023 11:19

Update pprof.tmpl

9cfcbb4

Update locale_en-US.ini

f22aa21

Add FGProf handler

b9989b1

wire in PProfFGProfile

6451d01

wxiaoguang reviewed Feb 4, 2023

View reviewed changes

zeripath added 14 commits February 4, 2023 11:56

Wire in FGProf

351149b

Update locale_en-US.ini

7941fbb

Update pprof.tmpl

0ef07f6

Update pprof.go

8d0d93e

Update pprof.go

768a960

Update web.go

82169ca

Update pprof.go

ba894a1

Update pprof.tmpl

75d96e8

Update pprof.tmpl

381f12a

Update locale_en-US.ini

2fc5d72

placate lint

79756cd

Signed-off-by: Andrew Thornton <art27@cantab.net>

Add pprof endpoints to manager too

6247e72

Signed-off-by: Andrew Thornton <art27@cantab.net>

add documentation

d9eac18

Signed-off-by: Andrew Thornton <art27@cantab.net>

Add output option

7decd93

Signed-off-by: Andrew Thornton <art27@cantab.net>

zeripath changed the title ~~WIP: Add PProf to admin pages~~ Add PProf to admin pages and to gitea manager Feb 4, 2023

zeripath removed the pr/wip This PR is not ready for review label Feb 4, 2023

zeripath added 4 commits February 4, 2023 21:20

fix missing name

4d916b4

Signed-off-by: Andrew Thornton <art27@cantab.net>

add trace and fix format

b9942cb

Signed-off-by: Andrew Thornton <art27@cantab.net>

fix format on stacktraces

7c4be9c

Signed-off-by: Andrew Thornton <art27@cantab.net>

fix trace

6c2017d

Signed-off-by: Andrew Thornton <art27@cantab.net>

delvh reviewed Feb 5, 2023

View reviewed changes

zeripath commented Feb 5, 2023

View reviewed changes

routers/web/admin/admin.go Outdated Show resolved Hide resolved

zeripath commented Feb 5, 2023

View reviewed changes

routers/web/admin/admin.go Outdated Show resolved Hide resolved

zeripath added 9 commits February 5, 2023 06:56

Apply suggestions from code review

d37978e

Merge remote-tracking branch 'origin/main' into add-pprof-to-admin-pages

950c474

as per delvh

2dbab58

Signed-off-by: Andrew Thornton <art27@cantab.net>

include indent in WriteProcess

2b523f1

Signed-off-by: Andrew Thornton <art27@cantab.net>

as per delvh

5925f47

Signed-off-by: Andrew Thornton <art27@cantab.net>

remove space

91bc97a

Signed-off-by: Andrew Thornton <art27@cantab.net>

as per delvh

61755a7

Signed-off-by: Andrew Thornton <art27@cantab.net>

as per delvh

1400ab2

Signed-off-by: Andrew Thornton <art27@cantab.net>

add some comments

edebd86

Signed-off-by: Andrew Thornton <art27@cantab.net>

delvh approved these changes Feb 19, 2023

View reviewed changes

templates/admin/pprof.tmpl Show resolved Hide resolved

templates/admin/pprof.tmpl Show resolved Hide resolved

GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Feb 19, 2023

zeripath added 2 commits February 19, 2023 12:09

Merge remote-tracking branch 'origin/main' into add-pprof-to-admin-pages

1ffe696

move trace help in to the ui form

92ac295

Signed-off-by: Andrew Thornton <art27@cantab.net>

lunny closed this May 16, 2023

go-gitea locked as resolved and limited conversation to collaborators Aug 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PProf to admin pages and to `gitea manager` #22742

Add PProf to admin pages and to `gitea manager` #22742

zeripath commented Feb 3, 2023 •

edited

Loading

codecov-commenter commented Feb 3, 2023 •

edited

Loading

wxiaoguang Feb 4, 2023

zeripath Feb 4, 2023

wxiaoguang Feb 4, 2023

wxiaoguang Feb 4, 2023 •

edited

Loading

zeripath Feb 4, 2023

wxiaoguang Feb 4, 2023 •

edited

Loading

zeripath Feb 4, 2023 •

edited

Loading

delvh left a comment

silverwind commented May 15, 2023

lunny commented May 16, 2023

wxiaoguang commented May 16, 2023

lunny commented May 16, 2023

Add PProf to admin pages and to gitea manager #22742

Add PProf to admin pages and to gitea manager #22742

Conversation

zeripath commented Feb 3, 2023 • edited Loading

codecov-commenter commented Feb 3, 2023 • edited Loading

Codecov Report

wxiaoguang Feb 4, 2023

Choose a reason for hiding this comment

zeripath Feb 4, 2023

Choose a reason for hiding this comment

wxiaoguang Feb 4, 2023

Choose a reason for hiding this comment

wxiaoguang Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

zeripath Feb 4, 2023

Choose a reason for hiding this comment

wxiaoguang Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

zeripath Feb 4, 2023 • edited Loading

Choose a reason for hiding this comment

delvh left a comment

Choose a reason for hiding this comment

silverwind commented May 15, 2023

lunny commented May 16, 2023

wxiaoguang commented May 16, 2023

lunny commented May 16, 2023

Add PProf to admin pages and to `gitea manager` #22742

Add PProf to admin pages and to `gitea manager` #22742

zeripath commented Feb 3, 2023 •

edited

Loading

codecov-commenter commented Feb 3, 2023 •

edited

Loading

wxiaoguang Feb 4, 2023 •

edited

Loading

wxiaoguang Feb 4, 2023 •

edited

Loading

zeripath Feb 4, 2023 •

edited

Loading