Slow page loads with a large repo

[deanpcmad]

Possibly linked/related to #490

I like to keep some mirrors of popular projects such as Rails on my Gitea server however whenever I go to view that repo, it can take 10 seconds plus (sometimes causing an nginx 502 timeout error) to load the page

[lunny]

I tested this on mac OS in a lowest power MacBook air, loading rails spent 3500ms ~ 4200ms. I think it's enough for v1.1.

[clandmeter]

@lunny could you please give alpinelinux aports a try?

Then try to browse the main directory.

It takes couple of minutes on a powerful server mainly because of git log....

I remember this issue has also been reported on gogs before but was never taken care of. Some have suggested to use a caching system. A simpler approach would be to fetch a directory list (like github does) and if needed a-sync fetch commit messages via javascript. Cgit just only shows the directory list which is pretty fast (if possible add this as an option to disable fetching of commit messages, if thats possible with current implementation).

[lunny]

I will try it. @clandmeter

[lunny]

Which page do you want to test? @clandmeter In my machine, main page is 1763ms and first release page is 6662ms .

[clandmeter]

@lunny can you check the main directory like this one at github:

https://github.com/alpinelinux/aports/tree/master/main

[clandmeter]

@lunny btw, im using 1.0.1 i believe the performance commits for tags page has landed after the 1.0.1, or in another branch.

[clandmeter]

@lunny I think #502 is related?

[lunny]

@clandmeter Yes, I tested in master. I think v1.0.1 maybe slower than master. Yes. it's related with #502

[clandmeter]

@lunny I tried master today both on Linux (Alpine Linux) and win10. Both crash at startup so i cannot verify if its faster.

[lunny]

Where is the crash log?

[clandmeter]

C:\Users\carlo\Desktop\gitea>gitea.exe web
2017/01/20 12:47:02 [W] Custom config 'C:/Users/carlo/Desktop/gitea/custom/conf/app.ini' not found, ignore this if you're running first time
2017/01/20 12:47:02 [T] Custom path: C:/Users/carlo/Desktop/gitea/custom
2017/01/20 12:47:02 [T] Log path: C:/Users/carlo/Desktop/gitea/log
2017/01/20 12:47:02 [I] Gitea v1.0.0+137-g1610b9f
2017/01/20 12:47:02 [I] Log Mode: Console(Trace)
2017/01/20 12:47:02 [I] Cache Service Enabled
2017/01/20 12:47:02 [I] Session Service Enabled
2017/01/20 12:47:02 [I] SQLite3 Supported
2017/01/20 12:47:02 [I] Run Mode: Development
panic: Macaron handler must be a callable function

goroutine 1 [running]:
panic(0xeec4e0, 0xc0434b7400)
        /usr/local/go/src/runtime/panic.go:500 +0x1af
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.validateHandler(0xeec4e0, 0xc0434b73e0)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/macaron.go:50 +0xbf
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.validateHandlers(0xc0434bfc80, 0x6, 0x8)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/macaron.go:58 +0x54
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Handle(0xc04200f360, 0x100f580, 0x4, 0xc0434c4160, 0x1b, 0xc0434c2f90, 0x6, 0x8, 0x0)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:176 +0x417
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Post(0xc04200f360, 0x1011b19, 0x6, 0xc0434c2f90, 0x3, 0x3, 0x10)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:210 +0x7c
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Post-fm(0x1011b19, 0x6, 0xc0434c2f90, 0x3, 0x3, 0x3)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:335 +0x63
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*ComboRouter).route(0xc0434bd100, 0xc0434a6bb8, 0x100f580, 0x4, 0xc0434a6cc8, 0x3, 0x3, 0xeec4e0)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:322 +0x12e
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*ComboRouter).Post(0xc0434bd100, 0xc0434a6cc8, 0x3, 0x3, 0xc0434b73e0)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:335 +0x99
code.gitea.io/gitea/routers/api/v1.RegisterRoutes.func1.6()
        /srv/app/src/code.gitea.io/gitea/routers/api/v1/api.go:409 +0x4d9
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Group(0xc04200f360, 0x1020d48, 0xe, 0xc0434a6f58, 0xc0434b70c0, 0x1, 0x1)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:190 +0x112
code.gitea.io/gitea/routers/api/v1.RegisterRoutes.func1()
        /srv/app/src/code.gitea.io/gitea/routers/api/v1/api.go:417 +0xc42
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Group(0xc04200f360, 0x100e74a, 0x3, 0xc0434a71a0, 0xc04348cfc0, 0x1, 0x1)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:190 +0x112
code.gitea.io/gitea/routers/api/v1.RegisterRoutes(0xc0422c6580)
        /srv/app/src/code.gitea.io/gitea/routers/api/v1/api.go:450 +0xdf
code.gitea.io/gitea/cmd.runWeb.func17()
        /srv/app/src/code.gitea.io/gitea/cmd/web.go:609 +0x31
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Group(0xc04200f360, 0x100f074, 0x4, 0xc0434a74a8, 0xc04348cfb0, 0x1, 0x1)
        /srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:190 +0x112
code.gitea.io/gitea/cmd.runWeb(0xc042184140, 0x0, 0xc042184100)
        /srv/app/src/code.gitea.io/gitea/cmd/web.go:610 +0x1506
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0xf07f80, 0x113f1c8, 0xc042184140, 0xc0421a2200, 0x0)
        /srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xc0
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x100edc8, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x10318a9, 0x16, 0x0, ...)
        /srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xcce
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc04246e340, 0xc04203e3a0, 0x2, 0x2, 0x0, 0x0)
        /srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x6aa
main.main()
        /srv/app/src/code.gitea.io/gitea/main.go:39 +0x35b

[drsect0r]

I am stopped by the same panic message as @clandmeter (I don't know if it is the same issue, I was trying to update my Gitea installation - running on Docker)

bash-4.3$ /app/gitea/gitea web 
2017/01/20 11:54:36 [T] Custom path: /data/gitea
2017/01/20 11:54:36 [T] Log path: /data/gitea/log
panic: Macaron handler must be a callable function

goroutine 1 [running]:
panic(0x7ffa36d24140, 0xc42152b720)
	/usr/lib/go/src/runtime/panic.go:500 +0x1a5
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.validateHandler(0x7ffa36d24140, 0xc42152b700)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/macaron.go:50 +0xba
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.validateHandlers(0xc421555400, 0x6, 0x8)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/macaron.go:58 +0x4f
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Handle(0xc4205cc460, 0x7ffa3661ce90, 0x4, 0xc42155e7c0, 0x1b, 0xc421568300, 0x6, 0x8, 0x7ffa37b93020)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:176 +0x412
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Post(0xc4205cc460, 0x7ffa3661f447, 0x6, 0xc421568300, 0x3, 0x3, 0x10)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:210 +0x77
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Post-fm(0x7ffa3661f447, 0x6, 0xc421568300, 0x3, 0x3, 0x3)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:335 +0x5e
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*ComboRouter).route(0xc4215430c0, 0xc4214d6bb8, 0x7ffa3661ce90, 0x4, 0xc4214d6cc8, 0x3, 0x3, 0x7ffa36d24140)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:322 +0x129
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*ComboRouter).Post(0xc4215430c0, 0xc4214d6cc8, 0x3, 0x3, 0xc42152b700)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:335 +0x94
code.gitea.io/gitea/routers/api/v1.RegisterRoutes.func1.6()
	/srv/app/src/code.gitea.io/gitea/routers/api/v1/api.go:409 +0x4d4
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Group(0xc4205cc460, 0x7ffa3662e69d, 0xe, 0xc4214d6f58, 0xc42152b3e0, 0x1, 0x1)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:190 +0x10d
code.gitea.io/gitea/routers/api/v1.RegisterRoutes.func1()
	/srv/app/src/code.gitea.io/gitea/routers/api/v1/api.go:417 +0xc3d
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Group(0xc4205cc460, 0x7ffa3661c13f, 0x3, 0xc4214d71a0, 0xc4214912e0, 0x1, 0x1)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:190 +0x10d
code.gitea.io/gitea/routers/api/v1.RegisterRoutes(0xc420473980)
	/srv/app/src/code.gitea.io/gitea/routers/api/v1/api.go:450 +0xda
code.gitea.io/gitea/cmd.runWeb.func17()
	/srv/app/src/code.gitea.io/gitea/cmd/web.go:609 +0x2c
code.gitea.io/gitea/vendor/gopkg.in/macaron%2ev1.(*Router).Group(0xc4205cc460, 0x7ffa3661c9d4, 0x4, 0xc4214d74a8, 0xc4214912d0, 0x1, 0x1)
	/srv/app/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:190 +0x10d
code.gitea.io/gitea/cmd.runWeb(0xc4201c17c0, 0x0, 0xc4201c1700)
	/srv/app/src/code.gitea.io/gitea/cmd/web.go:610 +0x1501
code.gitea.io/gitea/vendor/github.com/urfave/cli.HandleAction(0x7ffa36d3fcc0, 0x7ffa36e476e8, 0xc4201c17c0, 0xc420058d00, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:471 +0xbb
code.gitea.io/gitea/vendor/github.com/urfave/cli.Command.Run(0x7ffa3661c730, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffa3663ebbf, 0x16, 0x0, ...)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/command.go:191 +0xcc9
code.gitea.io/gitea/vendor/github.com/urfave/cli.(*App).Run(0xc42024b520, 0xc42000c140, 0x2, 0x2, 0x0, 0x0)
	/srv/app/src/code.gitea.io/gitea/vendor/github.com/urfave/cli/app.go:241 +0x6a5
main.main()
	/srv/app/src/code.gitea.io/gitea/main.go:39 +0x356