Attachment upload crashes Mattermost - Out of Memory #8741
Comments
|
Thanks @alecisec, Please see this issue that our devs are working on with community that might improve this issue... |
|
What is the 'fix' currently? i.e. how much memory does a docker container running mattermost need to upload 5 x 1mb images? I'm currently crashing on ~1.5gb of memory. |
|
I've done further testing and can observe that a server with 7132MB free memory drops down to 4800MB free when uploading 4 Images of sizes (54kb, 3kb, 3kb, 49kb). A container with 4GB available memory cannot be stable under these conditions meaning that an EC2 'Large' is required purely to have enough memory for small file uploads. It's extremely unclear why ~100kb of images being uploaded would require ~2.5GB of memory to process. |
|
Hi @alecisec, Thanks for your feedback, Quoting from the issue above, the reason so much memory is used while uploading images is because:
The fix being worked on in the above-linked issue will improve the file upload API to stream files instead of loading them entirely into memory. |
|
I can understand that, but the ratio of memory consumed to filesize makes no sense. How can 200kb of files use over 2GB of memory? are they being loaded into memory 10,000 times at once? |
|
@alecisec, you're right that the root of this problem can't just be the duplicate in-memory copies being uploaded. There's "something else" that's contributing to this problem. Thanks for the detailed numbers. We do have customers running with 1-2GB of memory and uploading files without issue, so I'm assuming a configuration difference of some kind. Are you able to point us at the Docker images you're using with Mattermost? We can replicate the RDS/S3 setup, but I want to make sure we're testing comparable images. |
|
Not the images but I can put the build files here - Dockerfile entrypoint.sh |
|
Hi @alecisec! Just wanted to give an update and let you know that I have let our engineers know about this and they will get back on this in the next few days. |
|
I've finally been able to reproduce this -- thanks, @alecisec for all the help in getting here! The root cause appears to be our use of the minio-go library, and this related issue: minio/minio-go#848. Since we aren't giving the size to library up front, it ends up allocating a I've filed https://mattermost.atlassian.net/browse/MM-10677 to fix the root of this. |
|
This makes perfect sense! Well done, do you know when this bug was introduced? I don't remember it ever being an issue prior to around version ~4.6 |
|
Hi @alecisec! We have created a Help Wanted issue here and are looking for community's help. If you are interested in helping with a pull request, please let us know. I'll close this issue for now as we can track this through the help wanted ticket. |
This is similar to the issue in #8450 however that issue was closed and this only started occurring for me in version 4.8
I'm running Mattermost in Docker, in a Ubuntu 16.04 container (previously 14.04), using S3 for attachment storage and MySQL on RDS.
4.7 is stable it seems, however updating the platform to 4.8 I began having platform crash each time a user tried to upload multiple attachments.
Debug log below - note this goes on for several pages.
The system does actually appear to run out of memory (monitoring memory usage I can see stable 500mb of 2GB then uploading 4x1mb image files at once consumes all available memory and crashes the container)
I've also replicated this with 4x 3kb files on a system with 4GB of memory. There is definitly a fault here with running mattermost in a docker container, seems OK outside of a container however.
I've also been applying all minor/major patches since with no change. (problem persists up to current 4.9.2)
`mattermost | [2018/05/08 06:11:15 UTC] [DEBG] GET - /api/v4/channels/osx8mnxyobbqpftemjk8yq95ur/posts
mattermost | [2018/05/08 06:11:17 UTC] [DEBG] POST - /api/v4/channels/members/me/view
mattermost | [2018/05/08 06:11:19 UTC] [DEBG] POST - /api/v4/channels/members/me/view
mattermost | [2018/05/08 06:11:19 UTC] [DEBG] POST - /api/v4/files
mattermost | [2018/05/08 06:11:19 UTC] [DEBG] POST - /api/v4/files
mattermost | [2018/05/08 06:11:19 UTC] [DEBG] POST - /api/v4/files
mattermost | [2018/05/08 06:11:19 UTC] [DEBG] POST - /api/v4/files
mattermost | [2018/05/08 06:11:20 UTC] [DEBG] POST - /api/v4/channels/members/me/view
mattermost | [2018/05/08 06:11:22 UTC] [DEBG] POST - /api/v4/channels/members/me/view
mattermost | [2018/05/08 06:11:24 UTC] [DEBG] POST - /api/v4/channels/members/me/view
mattermost | fatal error: runtime: out of memory
mattermost |
mattermost | runtime stack:
mattermost | runtime.throw(0x140f883, 0x16)
mattermost | /usr/local/go/src/runtime/panic.go:605 +0x95
mattermost | runtime.sysMap(0xc48fec0000, 0x24000000, 0x100, 0x24bbfb8)
mattermost | /usr/local/go/src/runtime/mem_linux.go:216 +0x1d0
mattermost | runtime.(*mheap).sysAlloc(0x24a2600, 0x24000000, 0x200)
mattermost | /usr/local/go/src/runtime/malloc.go:470 +0xd7
mattermost | runtime.(*mheap).grow(0x24a2600, 0x12000, 0x0)
mattermost | /usr/local/go/src/runtime/mheap.go:887 +0x60
mattermost | runtime.(*mheap).allocSpanLocked(0x24a2600, 0x12000, 0x24bbfc8, 0x100000001)
mattermost | /usr/local/go/src/runtime/mheap.go:800 +0x334
mattermost | runtime.(*mheap).alloc_m(0x24a2600, 0x12000, 0xffffffffffff0101, 0x7f87ceffcdb8)
mattermost | /usr/local/go/src/runtime/mheap.go:666 +0x118
mattermost | runtime.(*mheap).alloc.func1()
mattermost | /usr/local/go/src/runtime/mheap.go:733 +0x4d
mattermost | runtime.systemstack(0x7f87ceffcdd0)
mattermost | /usr/local/go/src/runtime/asm_amd64.s:360 +0xab
mattermost | runtime.(*mheap).alloc(0x24a2600, 0x12000, 0x2010101, 0x249a450)
mattermost | /usr/local/go/src/runtime/mheap.go:732 +0xa1
mattermost | runtime.largeAlloc(0x24000000, 0x101, 0x2499e30)
mattermost | /usr/local/go/src/runtime/malloc.go:827 +0x98
mattermost | runtime.mallocgc.func1()
mattermost | /usr/local/go/src/runtime/malloc.go:722 +0x46
mattermost | runtime.systemstack(0xc420022000)
mattermost | /usr/local/go/src/runtime/asm_amd64.s:344 +0x79
mattermost | runtime.mstart()
mattermost | /usr/local/go/src/runtime/proc.go:1135
mattermost |
mattermost | goroutine 3063 [running]:
mattermost | runtime.systemstack_switch()
mattermost | /usr/local/go/src/runtime/asm_amd64.s:298 fp=0xc4212eed60 sp=0xc4212eed58 pc=0x45be30
mattermost | runtime.mallocgc(0x24000000, 0x11e8ba0, 0xc4212ef201, 0xc4212ef090)
mattermost | /usr/local/go/src/runtime/malloc.go:721 +0x7ae fp=0xc4212eee08 sp=0xc4212eed60 pc=0x412a1e
mattermost | runtime.makeslice(0x11e8ba0, 0x24000000, 0x24000000, 0xc4212ef298, 0xc4212ef078, 0xc42001c060)
mattermost | /usr/local/go/src/runtime/slice.go:54 +0x77 fp=0xc4212eee38 sp=0xc4212eee08 pc=0x4455f7
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.putObjectMultipartStreamNoLength(0xc422027800, 0xc422f61260, 0 x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc421918ed0, 0xc422a689e0, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:202 +0x38a fp=0xc4212ef4b8 sp=0xc4212eee38 pc=0x96b58a
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.putObjectCommon(0xc422027800, 0xc422f61260, 0x0, 0x0, 0x0, 0x0 , 0x0, 0x1, 0xc421918ed0, 0xc422a689e0, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:152 +0x642 fp=0xc4212ef740 sp=0xc4212ef4b8 pc=0x96afd2
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.PutObjectWithContext(0xc422027800, 0xc422f61260, 0x0, 0x0, 0x0 , 0x0, 0x0, 0x1, 0xc421918ed0, 0xc422a689e0, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object-context.go:38 +0x302 fp=0xc4212ef8c8 sp=0xc4212ef740 pc=0x9620b2
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.PutObject(0xc422027800, 0xc422f61260, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc421918ed0, 0xc422a689e0, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:130 +0x114 fp=0xc4212efa50 sp=0xc4212ef8c8 pc=0x96a944
mattermost | github.com/mattermost/mattermost-server/utils.(*S3FileBackend).WriteFile(0xc422f61200, 0xc422862000, 0x10be, 0x20be, 0xc42263a2d0, 0x81, 0xc42 23a00c0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/file_backend_s3.g o:153 +0x2a4 fp=0xc4212efe10 sp=0xc4212efa50 pc=0x9bce34
mattermost | github.com/mattermost/mattermost-server/app.(*App).WriteFile(0xc420087d40, 0xc422862000, 0x10be, 0x20be, 0xc42263a2d0, 0x81, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:86 +0x8e fp =0xc4212efe58 sp=0xc4212efe10 pc=0xcbedce
mattermost | github.com/mattermost/mattermost-server/app.(*App).generateThumbnailImage(0xc420087d40, 0x1d6df00, 0xc422620000, 0xc42263a2d0, 0x81, 0xfc0, 0x bd0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:499 +0x163 fp=0xc4212eff60 sp=0xc4212efe58 pc=0xcc4ec3
mattermost | github.com/mattermost/mattermost-server/app.(*App).HandleImages.func1(0xc421afe1e0, 0xc420087d40, 0xc4220f2120, 0xc42263a2d0, 0x81, 0xfc0, 0xb d0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:402 +0x93 f p=0xc4212effa8 sp=0xc4212eff60 pc=0xd51653
mattermost | runtime.goexit()
mattermost | /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc4212effb0 sp=0xc4212effa8 pc=0x45ea61
mattermost | created by github.com/mattermost/mattermost-server/app.(*App).HandleImages
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:400 +0x154
mattermost |
mattermost | goroutine 1 [chan receive, 10 minutes]:
mattermost | github.com/mattermost/mattermost-server/cmd/commands.runServer(0x13f6bb4, 0xb, 0xc420056e00, 0xc420056e40, 0x0, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/cmd/commands/server.go: 191 +0xe68
mattermost | github.com/mattermost/mattermost-server/cmd/commands.serverCmdF(0x24800c0, 0x24ba100, 0x0, 0x0, 0x0, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/cmd/commands/server.go: 53 +0x10d
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/spf13/cobra.(*Command).execute(0x24800c0, 0xc420010180, 0x0, 0x0, 0x24800c0, 0xc4200 10180)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/spf13 /cobra/command.go:753 +0x475
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x24800c0, 0x24ba100, 0xc420109f40, 0x10f874c)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/spf13 /cobra/command.go:843 +0x334
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/spf13/cobra.(*Command).Execute(0x24800c0, 0x0, 0xc420109f40)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/spf13 /cobra/command.go:791 +0x2b
mattermost | github.com/mattermost/mattermost-server/cmd.Run(0xc420010180, 0x0, 0x0, 0xc420109f70, 0x1109829)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/cmd/cmd.go:14 +0x6d
mattermost | main.main()
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/main.go:33 +0x63
mattermost |
mattermost | goroutine 8 [syscall, 11 minutes]:
mattermost | os/signal.signal_recv(0x0)
mattermost | /usr/local/go/src/runtime/sigqueue.go:131 +0xa6
mattermost | os/signal.loop()
mattermost | /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
mattermost | created by os/signal.init.0
mattermost | /usr/local/go/src/os/signal/signal_unix.go:28 +0x41
mattermost |
mattermost | goroutine 2486 [select, 10 minutes]:
mattermost | github.com/mattermost/mattermost-server/model.createTask.func1(0xc421d46340, 0x34630b8a000, 0xc4229fa4b0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :49 +0x15a
mattermost | created by github.com/mattermost/mattermost-server/model.createTask
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :40 +0x10c
mattermost |
mattermost | goroutine 2673 [select]:
mattermost | net/http.(*persistConn).roundTrip(0xc4213d4b40, 0xc421918090, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/transport.go:1970 +0x608
mattermost | net/http.(*Transport).RoundTrip(0x247ed20, 0xc421458500, 0x247ed20, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/transport.go:413 +0x992
mattermost | net/http.send(0xc421458500, 0x1d64cc0, 0x247ed20, 0x0, 0x0, 0x0, 0xc4223ca1c0, 0x1d6b400, 0xc421c8fea8, 0x1)
mattermost | /usr/local/go/src/net/http/client.go:249 +0x1a9
mattermost | net/http.(*Client).send(0xc4219187b0, 0xc421458500, 0x0, 0x0, 0x0, 0xc4223ca1c0, 0x0, 0x1, 0x0)
mattermost | /usr/local/go/src/net/http/client.go:173 +0xfd
mattermost | net/http.(*Client).Do(0xc4219187b0, 0xc421458500, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/client.go:602 +0x28d
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.do(0xc422026c00, 0xc422f60f00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0 xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api.go:496 +0x50
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.executeMethod(0xc422026c00, 0xc422f60f00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api.go:601 +0x382
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.uploadPart(0xc422026c00, 0xc422f60f00, 0x0, 0x0, 0x0, 0x0, 0x0 , 0x1, 0xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object-multipart.go:283 +0x494
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.putObjectMultipartStreamNoLength(0xc422026c00, 0xc422f60f00, 0 x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:219 +0x71e
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.putObjectCommon(0xc422026c00, 0xc422f60f00, 0x0, 0x0, 0x0, 0x0 , 0x0, 0x1, 0xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:152 +0x642
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.PutObjectWithContext(0xc422026c00, 0xc422f60f00, 0x0, 0x0, 0x0 , 0x0, 0x0, 0x1, 0xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object-context.go:38 +0x302
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.PutObject(0xc422026c00, 0xc422f60f00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc4219187b0, 0xc422a68620, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:130 +0x114
mattermost | github.com/mattermost/mattermost-server/utils.(*S3FileBackend).WriteFile(0xc422f60ea0, 0xc423b1e000, 0x15616f, 0x1ffe00, 0xc421ed8500, 0x7b, 0 x81)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/file_backend_s3.g o:153 +0x2a4
mattermost | github.com/mattermost/mattermost-server/app.(*App).WriteFile(0xc420087d40, 0xc423b1e000, 0x15616f, 0x1ffe00, 0xc421ed8500, 0x7b, 0xa)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:86 +0x8e
mattermost | github.com/mattermost/mattermost-server/app.(*App).DoUploadFile(0xc420087d40, 0xbeb46dc30b5ac0de, 0x9975869e96, 0x249a060, 0x13ebcc3, 0x6, 0xc 4220c6780, 0x1a, 0xc422a1baa0, 0x1a, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:382 +0x690
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc4220c6780, 0x1a, 0xc422a1baa0, 0x1a, 0xc4220f2 3e0, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:329 +0x505
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadMultipartFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc4220c6780, 0x1a, 0xc422a1baa0, 0x1a, 0xc4223ca1a8, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:300 +0x41b
mattermost | github.com/mattermost/mattermost-server/api4.uploadFile(0xc422a77e00, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3600)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/file.go:120 +0x562
mattermost | github.com/mattermost/mattermost-server/api4.handler.ServeHTTP(0xc420087d40, 0x148a680, 0x10001, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3600)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/context.go:157 +0x b65
mattermost | github.com/mattermost/mattermost-server/api4.(*handler).ServeHTTP(0xc4221c5be0, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3600)
mattermost | :1 +0x63
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc42068a0e0, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3600)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/mux/mux.go:159 +0xed
mattermost | github.com/mattermost/mattermost-server/app.(*CorsWrapper).ServeHTTP(0xc4211e73e0, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3400)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:83 +0xa2
mattermost | github.com/mattermost/mattermost-server/app.(*RateLimiter).RateLimitHandler.func1(0x1d6e340, 0xc4216aa2a0, 0xc4215f3400)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/ratelimit.go:106 +0 xca
mattermost | net/http.HandlerFunc.ServeHTTP(0xc4201e2a40, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3400)
mattermost | /usr/local/go/src/net/http/server.go:1918 +0x44
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1d67540, 0xc4201e2a40, 0x1d631c0, 0x24b a100, 0x1, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3400)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/handlers/recovery.go:78 +0xb9
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4213a5f50, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3400)
mattermost | :1 +0x76
mattermost | net/http.serverHandler.ServeHTTP(0xc422014750, 0x1d6e340, 0xc4216aa2a0, 0xc4215f3400)
mattermost | /usr/local/go/src/net/http/server.go:2619 +0xb4
mattermost | net/http.(*conn).serve(0xc420178780, 0x1d6f0c0, 0xc4220bd680)
mattermost | /usr/local/go/src/net/http/server.go:1801 +0x71d
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 2876 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422a322a0, 0xc4222b06c0, 0xc4 21d6e060)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 3041 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb47f0, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc422027298, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc422027298, 0xc42210dd00, 0x1, 0x1)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc422027280, 0xc42210ddb1, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc422027280, 0xc42210ddb1, 0x1, 0x1, 0x7f87d701001c, 0x0, 0xc421918360)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc4223ca0a8, 0xc42210ddb1, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | net/http.(*connReader).backgroundRead(0xc42210dda0)
mattermost | /usr/local/go/src/net/http/server.go:660 +0x62
mattermost | created by net/http.(*connReader).startBackgroundRead
mattermost | /usr/local/go/src/net/http/server.go:656 +0xd8
mattermost |
mattermost | goroutine 2480 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422f60780, 0xc4223a0480, 0xc4 21e346c0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 2478 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422f606c0, 0xc4223a0240, 0xc4 21e34600)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 2477 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422f60660, 0xc4223a0a80, 0xc4 21e345a0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 2504 [select, 10 minutes]:
mattermost | github.com/mattermost/mattermost-server/model.createTask.func1(0xc421d21b40, 0x4e94914f0000, 0xc4229fb0c0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :49 +0x15a
mattermost | created by github.com/mattermost/mattermost-server/model.createTask
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :40 +0x10c
mattermost |
mattermost | goroutine 2878 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422a32de0, 0xc4222b09c0, 0xc4 21d6e660)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 2292 [syscall, 11 minutes]:
mattermost | syscall.Syscall6(0xe8, 0xa, 0xc42251fc24, 0x7, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
mattermost | github.com/mattermost/mattermost-server/vendor/golang.org/x/sys/unix.EpollWait(0xa, 0xc42251fc24, 0x7, 0x7, 0xffffffffffffffff, 0xc421504900, 0xc42249d980, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/golang.org/x/sys /unix/zsyscall_linux_amd64.go:1544 +0x77
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/fsnotify/fsnotify.(*fdPoller).wait(0xc422436860, 0x148ac00, 0xc422436860, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/fsnot ify/fsnotify/inotify_poller.go:86 +0x91
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc42241aa50)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/fsnot ify/fsnotify/inotify.go:192 +0x194
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/fsnotify/fsnotify.NewWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/fsnot ify/fsnotify/inotify.go:59 +0x23a
mattermost |
mattermost | goroutine 2980 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb4430, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc42200e318, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc42200e318, 0xc4211d8000, 0x2000, 0x2000)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc42200e300, 0xc4211d8000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc42200e300, 0xc4211d8000, 0x2000, 0x2000, 0x4cff9c, 0xc42122802d, 0x13e6e52)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc421f17660, 0xc4211d8000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | bufio.(*Reader).fill(0xc4220cfa40)
mattermost | /usr/local/go/src/bufio/bufio.go:97 +0x11a
mattermost | bufio.(*Reader).Peek(0xc4220cfa40, 0x2, 0x7622307501, 0x557f6bc7a01b, 0x2b, 0xbeb46dd9cb032f95, 0xc422412a00)
mattermost | /usr/local/go/src/bufio/bufio.go:129 +0x3a
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/websocket.(*Conn).read(0xc422ab5900, 0x2, 0xc422b748b8, 0xc421a8afc0, 0x1, 0 x33, 0xc422b748c8)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/websocket/conn_read.go:12 +0x40
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/websocket.(*Conn).advanceFrame(0xc422ab5900, 0x132e1e0, 0x0, 0x1c0000000000)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/websocket/conn.go:780 +0x5c
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/websocket.(*Conn).NextReader(0xc422ab5900, 0x44ded8, 0x55682350b618, 0xc422b 74a58, 0x81a911, 0xc422b74a58)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/websocket/conn.go:940 +0xa3
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/websocket.(*Conn).ReadJSON(0xc422ab5900, 0x1241b20, 0xc4217c05b0, 0x249a060, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/websocket/json.go:50 +0x2f
mattermost | github.com/mattermost/mattermost-server/app.(*WebConn).readPump(0xc42264e5a0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_conn.go:138 +0x 17e
mattermost | github.com/mattermost/mattermost-server/app.(*WebConn).Pump(0xc42264e5a0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_conn.go:113 +0x 79
mattermost | github.com/mattermost/mattermost-server/api4.connectWebSocket(0xc422a771d0, 0x1d6e340, 0xc422a86e00, 0xc421458f00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/websocket.go:39 +0 x320
mattermost | github.com/mattermost/mattermost-server/api4.handler.ServeHTTP(0xc420087d40, 0x148a028, 0x100, 0x1d6e340, 0xc422a86e00, 0xc421458f00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/context.go:157 +0x b65
mattermost | github.com/mattermost/mattermost-server/api4.(*handler).ServeHTTP(0xc422c4ca60, 0x1d6e340, 0xc422a86e00, 0xc421458f00)
mattermost | :1 +0x63
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc42068a0e0, 0x1d6e340, 0xc422a86e00, 0xc421458f00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/mux/mux.go:159 +0xed
mattermost | github.com/mattermost/mattermost-server/app.(*CorsWrapper).ServeHTTP(0xc4211e73e0, 0x1d6e340, 0xc422a86e00, 0xc421458d00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:83 +0xa2
mattermost | github.com/mattermost/mattermost-server/app.(*RateLimiter).RateLimitHandler.func1(0x1d6e340, 0xc422a86e00, 0xc421458d00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/ratelimit.go:106 +0 xca
mattermost | net/http.HandlerFunc.ServeHTTP(0xc4201e2a40, 0x1d6e340, 0xc422a86e00, 0xc421458d00)
mattermost | /usr/local/go/src/net/http/server.go:1918 +0x44
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1d67540, 0xc4201e2a40, 0x1d631c0, 0x24b a100, 0x1, 0x1d6e340, 0xc422a86e00, 0xc421458d00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/handlers/recovery.go:78 +0xb9
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4213a5f50, 0x1d6e340, 0xc422a86e00, 0xc421458d00)
mattermost | :1 +0x76
mattermost | net/http.serverHandler.ServeHTTP(0xc422014750, 0x1d6e340, 0xc422a86e00, 0xc421458d00)
mattermost | /usr/local/go/src/net/http/server.go:2619 +0xb4
mattermost | net/http.(*conn).serve(0xc4229d05a0, 0x1d6f0c0, 0xc422656b80)
mattermost | /usr/local/go/src/net/http/server.go:1801 +0x71d
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 2622 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb4df0, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc422026218, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc422026218, 0xc4224eb000, 0x1000, 0x1000)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc422026200, 0xc4224eb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc422026200, 0xc4224eb000, 0x1000, 0x1000, 0xc422b71c50, 0x6f063a, 0xc4220f4128)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc422cc9048, 0xc4224eb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | net/http.(*connReader).Read(0xc4220f4120, 0xc4224eb000, 0x1000, 0x1000, 0xc422b71d38, 0x52410c, 0xc422026200)
mattermost | /usr/local/go/src/net/http/server.go:753 +0x105
mattermost | bufio.(*Reader).fill(0xc422f614a0)
mattermost | /usr/local/go/src/bufio/bufio.go:97 +0x11a
mattermost | bufio.(*Reader).Peek(0xc422f614a0, 0x4, 0xdf5e242e9d, 0x249a060, 0x0, 0x0, 0x249a060)
mattermost | /usr/local/go/src/bufio/bufio.go:129 +0x3a
mattermost | net/http.(*conn).serve(0xc4228ad7c0, 0x1d6f0c0, 0xc4229a4c40)
mattermost | /usr/local/go/src/net/http/server.go:1826 +0x88f
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 2322 [select, 11 minutes]:
mattermost | database/sql.(*DB).connectionCleaner(0xc4201781e0, 0x34630b8a000)
mattermost | /usr/local/go/src/database/sql/sql.go:793 +0x446
mattermost | created by database/sql.(*DB).startCleanerLocked
mattermost | /usr/local/go/src/database/sql/sql.go:780 +0xba
mattermost |
mattermost | goroutine 2468 [select, 10 minutes, locked to thread]:
mattermost | runtime.gopark(0x148cdf0, 0x0, 0x13ec209, 0x6, 0x18, 0x1)
mattermost | /usr/local/go/src/runtime/proc.go:287 +0x12c
mattermost | runtime.selectgo(0xc421508f50, 0xc421e07c20)
mattermost | /usr/local/go/src/runtime/select.go:395 +0x1149
mattermost | runtime.ensureSigM.func1()
mattermost | /usr/local/go/src/runtime/signal_unix.go:511 +0x220
mattermost | runtime.goexit()
mattermost | /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1
mattermost |
mattermost | goroutine 2470 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/segmentio/analytics-go.(*Client).loop(0xc422355a00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/segme ntio/analytics-go/analytics.go:338 +0x1b5
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/segmentio/analytics-go.(*Client).startLoop
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/segme ntio/analytics-go/analytics.go:232 +0x3f
mattermost |
mattermost | goroutine 3029 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb4670, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc42239e418, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc42239e418, 0xc42178a000, 0x1, 0x1)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc42239e400, 0xc42178a0d1, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc42239e400, 0xc42178a0d1, 0x1, 0x1, 0xc4212f4540, 0x0, 0x1d64601)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc4223ca620, 0xc42178a0d1, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | net/http.(*connReader).backgroundRead(0xc42178a0c0)
mattermost | /usr/local/go/src/net/http/server.go:660 +0x62
mattermost | created by net/http.(*connReader).startBackgroundRead
mattermost | /usr/local/go/src/net/http/server.go:656 +0xd8
mattermost |
mattermost | goroutine 3069 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb48b0, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc42174a618, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc42174a618, 0xc42215b500, 0x1, 0x1)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc42174a600, 0xc42215b541, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc42174a600, 0xc42215b541, 0x1, 0x1, 0xc422a961e8, 0x14554fa, 0x13e41ae)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc4223ca258, 0xc42215b541, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | net/http.(*connReader).backgroundRead(0xc42215b530)
mattermost | /usr/local/go/src/net/http/server.go:660 +0x62
mattermost | created by net/http.(*connReader).startBackgroundRead
mattermost | /usr/local/go/src/net/http/server.go:656 +0xd8
mattermost |
mattermost | goroutine 3065 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb45b0, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc422653e98, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc422653e98, 0xc4223e8f00, 0x1, 0x1)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc422653e80, 0xc4223e8f71, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc422653e80, 0xc4223e8f71, 0x1, 0x1, 0xc422620900, 0x2, 0xd51603)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc4223ca8f8, 0xc4223e8f71, 0x1, 0x1, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | net/http.(*connReader).backgroundRead(0xc4223e8f60)
mattermost | /usr/local/go/src/net/http/server.go:660 +0x62
mattermost | created by net/http.(*connReader).startBackgroundRead
mattermost | /usr/local/go/src/net/http/server.go:656 +0xd8
mattermost |
mattermost | goroutine 3052 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb4130, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc422027898, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc422027898, 0xc422bb6000, 0x1000, 0x1000)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc422027880, 0xc422bb6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc422027880, 0xc422bb6000, 0x1000, 0x1000, 0xc42240c380, 0x0, 0x413576)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc4223ca0f8, 0xc422bb6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | crypto/tls.(*block).readFromUntil(0xc421918d50, 0x7f87d6f76468, 0xc4223ca0f8, 0x5, 0xc4223ca0f8, 0x413576)
mattermost | /usr/local/go/src/crypto/tls/conn.go:488 +0x95
mattermost | crypto/tls.(*Conn).readRecord(0xc42240ca80, 0x148cf17, 0xc42240cba0, 0x42f57b)
mattermost | /usr/local/go/src/crypto/tls/conn.go:590 +0xe0
mattermost | crypto/tls.(*Conn).Read(0xc42240ca80, 0xc421272000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/crypto/tls/conn.go:1134 +0x110
mattermost | net/http.(*persistConn).Read(0xc421490120, 0xc421272000, 0x1000, 0x1000, 0xc4228fa3c0, 0xc422019198, 0x458ad0)
mattermost | /usr/local/go/src/net/http/transport.go:1391 +0x140
mattermost | bufio.(*Reader).fill(0xc422f615c0)
mattermost | /usr/local/go/src/bufio/bufio.go:97 +0x11a
mattermost | bufio.(*Reader).Peek(0xc422f615c0, 0x1, 0x0, 0x0, 0x0, 0xc422018a20, 0x0)
mattermost | /usr/local/go/src/bufio/bufio.go:129 +0x3a
mattermost | net/http.(*persistConn).readLoop(0xc421490120)
mattermost | /usr/local/go/src/net/http/transport.go:1539 +0x185
mattermost | created by net/http.(*Transport).dialConn
mattermost | /usr/local/go/src/net/http/transport.go:1186 +0xa2e
mattermost |
mattermost | goroutine 2505 [select, 10 minutes]:
mattermost | github.com/mattermost/mattermost-server/model.createTask.func1(0xc421bb0040, 0x4e94914f0000, 0xc4229fb0e0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :49 +0x15a
mattermost | created by github.com/mattermost/mattermost-server/model.createTask
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :40 +0x10c
mattermost |
mattermost | goroutine 2464 [sleep, 10 minutes]:
mattermost | time.Sleep(0x34630b8a000)
mattermost | /usr/local/go/src/runtime/time.go:65 +0x130
mattermost | github.com/mattermost/mattermost-server/enterprise/compliance.(*ComplianceInterfaceImpl).StartComplianceDailyJob.func1(0xc4223ca938)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/enterprise/compliance/c ompliance.go:276 +0x97
mattermost | created by github.com/mattermost/mattermost-server/enterprise/compliance.(*ComplianceInterfaceImpl).StartComplianceDailyJob
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/enterprise/compliance/c ompliance.go:221 +0x3f
mattermost |
mattermost | goroutine 2479 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422f60720, 0xc4223a0900, 0xc4 21e34660)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 3040 [runnable]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb41f0, 0x77, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc421e0a698, 0x77, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitWrite(0xc421e0a698, 0xc4211de400, 0xb44, 0x1b0f)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d
mattermost | internal/poll.(*FD).Write(0xc421e0a680, 0xc4211de000, 0x1035, 0x2000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:227 +0x244
mattermost | net.(*netFD).Write(0xc421e0a680, 0xc4211de000, 0x1035, 0x2000, 0x14, 0x20, 0x10)
mattermost | /usr/local/go/src/net/fd_unix.go:220 +0x52
mattermost | net.(*conn).Write(0xc421f17850, 0xc4211de000, 0x1035, 0x2000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:188 +0x6d
mattermost | crypto/tls.(*Conn).write(0xc420110700, 0xc4211de000, 0x1035, 0x2000, 0x1ffb, 0x10, 0x0)
mattermost | /usr/local/go/src/crypto/tls/conn.go:832 +0x1a7
mattermost | crypto/tls.(*Conn).writeRecordLocked(0xc420110700, 0x148cf17, 0xc421481000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/crypto/tls/conn.go:909 +0x360
mattermost | crypto/tls.(*Conn).Write(0xc420110700, 0xc421481000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/crypto/tls/conn.go:1068 +0x1bb
mattermost | net/http.persistConnWriter.Write(0xc4213d4b40, 0xc421481000, 0x1000, 0x1000, 0xc422a31360, 0xc421481000, 0x1000)
mattermost | /usr/local/go/src/net/http/transport.go:1202 +0x52
mattermost | bufio.(*Writer).Flush(0xc422621ec0, 0xc421481000, 0x1000)
mattermost | /usr/local/go/src/bufio/bufio.go:567 +0x7e
mattermost | bufio.(*Writer).ReadFrom(0xc422621ec0, 0x1d64640, 0xc422a68300, 0x7f87d6f0f088, 0xc422621ec0, 0x1)
mattermost | /usr/local/go/src/bufio/bufio.go:691 +0x26c
mattermost | io.copyBuffer(0x1d62940, 0xc422621ec0, 0x1d64640, 0xc422a68300, 0x0, 0x0, 0x0, 0x12b2e00, 0x7f87d6f8ab01, 0xc422a68300)
mattermost | /usr/local/go/src/io/io.go:386 +0x2bb
mattermost | io.Copy(0x1d62940, 0xc422621ec0, 0x1d64640, 0xc422a68300, 0x13f371f, 0x5483cd, 0xc4214812fc)
mattermost | /usr/local/go/src/io/io.go:362 +0x68
mattermost | net/http.(*transferWriter).WriteBody(0xc422a31360, 0x1d62940, 0xc422621ec0, 0x2, 0x2)
mattermost | /usr/local/go/src/net/http/transfer.go:337 +0x62c
mattermost | net/http.(*Request).write(0xc421458500, 0x1d62940, 0xc422621ec0, 0x0, 0x0, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/request.go:618 +0x6eb
mattermost | net/http.(*persistConn).writeLoop(0xc4213d4b40)
mattermost | /usr/local/go/src/net/http/transport.go:1762 +0x201
mattermost | created by net/http.(*Transport).dialConn
mattermost | /usr/local/go/src/net/http/transport.go:1187 +0xa53
mattermost |
mattermost | goroutine 2659 [runnable]:
mattermost | image/jpeg.(*decoder).reconstructBlock(0xc421d56000, 0xc423057688, 0xc0, 0x63, 0x2, 0x0, 0x0)
mattermost | /usr/local/go/src/image/jpeg/scan.go:464 +0x34a
mattermost | image/jpeg.(*decoder).processSOS(0xc421d56000, 0xa, 0x2, 0x80)
mattermost | /usr/local/go/src/image/jpeg/scan.go:301 +0x85b
mattermost | image/jpeg.(*decoder).decode(0xc421d56000, 0x1d62900, 0xc422f600c0, 0x410900, 0x1276480, 0x1379e20, 0x148c400, 0x1d62900)
mattermost | /usr/local/go/src/image/jpeg/reader.go:629 +0x32f
mattermost | image/jpeg.Decode(0x1d62900, 0xc422f600c0, 0xc422f600c0, 0x1d62900, 0xc422f600c0, 0x2)
mattermost | /usr/local/go/src/image/jpeg/reader.go:778 +0x54
mattermost | image.Decode(0x1d62a00, 0xc421b06600, 0xc422f60300, 0xc423057a48, 0xc422f60300, 0x1ffe00, 0xc421ed8880, 0x400)
mattermost | /usr/local/go/src/image/format.go:84 +0xfc
mattermost | github.com/mattermost/mattermost-server/app.prepareImage(0xc42315c000, 0x15917f, 0x1ffe00, 0x445864, 0xc4220f2020, 0x24ba100)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:416 +0xba
mattermost | github.com/mattermost/mattermost-server/app.(*App).HandleImages(0xc420087d40, 0xc4220f2000, 0x1, 0x1, 0xc4220f2010, 0x1, 0x1, 0xc422a68020, 0x 1, 0x1)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:397 +0x93
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc4211c6e80, 0x1a, 0xc422a1baa0, 0x1a, 0xc4223f2 660, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:347 +0xd45
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadMultipartFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc4211c6e80, 0x1a, 0xc422a1baa0, 0x1a, 0xc421f17870, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:300 +0x41b
mattermost | github.com/mattermost/mattermost-server/api4.uploadFile(0xc422a77c20, 0x1d6e340, 0xc4216aa0e0, 0xc4215f3000)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/file.go:120 +0x562
mattermost | github.com/mattermost/mattermost-server/api4.handler.ServeHTTP(0xc420087d40, 0x148a680, 0x10001, 0x1d6e340, 0xc4216aa0e0, 0xc4215f3000)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/context.go:157 +0x b65
mattermost | github.com/mattermost/mattermost-server/api4.(*handler).ServeHTTP(0xc4221c5be0, 0x1d6e340, 0xc4216aa0e0, 0xc4215f3000)
mattermost | :1 +0x63
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc42068a0e0, 0x1d6e340, 0xc4216aa0e0, 0xc4215f3000)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/mux/mux.go:159 +0xed
mattermost | github.com/mattermost/mattermost-server/app.(*CorsWrapper).ServeHTTP(0xc4211e73e0, 0x1d6e340, 0xc4216aa0e0, 0xc4215f2e00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:83 +0xa2
mattermost | github.com/mattermost/mattermost-server/app.(*RateLimiter).RateLimitHandler.func1(0x1d6e340, 0xc4216aa0e0, 0xc4215f2e00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/ratelimit.go:106 +0 xca
mattermost | net/http.HandlerFunc.ServeHTTP(0xc4201e2a40, 0x1d6e340, 0xc4216aa0e0, 0xc4215f2e00)
mattermost | /usr/local/go/src/net/http/server.go:1918 +0x44
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1d67540, 0xc4201e2a40, 0x1d631c0, 0x24b a100, 0x1, 0x1d6e340, 0xc4216aa0e0, 0xc4215f2e00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/handlers/recovery.go:78 +0xb9
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4213a5f50, 0x1d6e340, 0xc4216aa0e0, 0xc4215f2e00)
mattermost | :1 +0x76
mattermost | net/http.serverHandler.ServeHTTP(0xc422014750, 0x1d6e340, 0xc4216aa0e0, 0xc4215f2e00)
mattermost | /usr/local/go/src/net/http/server.go:2619 +0xb4
mattermost | net/http.(*conn).serve(0xc4228ad9a0, 0x1d6f0c0, 0xc4229a5880)
mattermost | /usr/local/go/src/net/http/server.go:1801 +0x71d
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 2706 [semacquire]:
mattermost | sync.runtime_Semacquire(0xc421afe1ec)
mattermost | /usr/local/go/src/runtime/sema.go:56 +0x39
mattermost | sync.(*WaitGroup).Wait(0xc421afe1e0)
mattermost | /usr/local/go/src/sync/waitgroup.go:131 +0x72
mattermost | github.com/mattermost/mattermost-server/app.(*App).HandleImages(0xc420087d40, 0xc4220f20e0, 0x1, 0x1, 0xc4220f20f0, 0x1, 0x1, 0xc422a68140, 0x 1, 0x1)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:411 +0x1d7
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc42120dba0, 0x1a, 0xc422a1baa0, 0x1a, 0xc4223f9 2d0, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:347 +0xd45
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadMultipartFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc42120dba0, 0x1a, 0xc422a1baa0, 0x1a, 0xc421f17808, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:300 +0x41b
mattermost | github.com/mattermost/mattermost-server/api4.uploadFile(0xc422a77d10, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3300)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/file.go:120 +0x562
mattermost | github.com/mattermost/mattermost-server/api4.handler.ServeHTTP(0xc420087d40, 0x148a680, 0x10001, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3300)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/context.go:157 +0x b65
mattermost | github.com/mattermost/mattermost-server/api4.(*handler).ServeHTTP(0xc4221c5be0, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3300)
mattermost | :1 +0x63
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc42068a0e0, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3300)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/mux/mux.go:159 +0xed
mattermost | github.com/mattermost/mattermost-server/app.(*CorsWrapper).ServeHTTP(0xc4211e73e0, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3100)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:83 +0xa2
mattermost | github.com/mattermost/mattermost-server/app.(*RateLimiter).RateLimitHandler.func1(0x1d6e340, 0xc4216aa1c0, 0xc4215f3100)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/ratelimit.go:106 +0 xca
mattermost | net/http.HandlerFunc.ServeHTTP(0xc4201e2a40, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3100)
mattermost | /usr/local/go/src/net/http/server.go:1918 +0x44
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1d67540, 0xc4201e2a40, 0x1d631c0, 0x24b a100, 0x1, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3100)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/handlers/recovery.go:78 +0xb9
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4213a5f50, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3100)
mattermost | :1 +0x76
mattermost | net/http.serverHandler.ServeHTTP(0xc422014750, 0x1d6e340, 0xc4216aa1c0, 0xc4215f3100)
mattermost | /usr/local/go/src/net/http/server.go:2619 +0xb4
mattermost | net/http.(*conn).serve(0xc4200832c0, 0x1d6f0c0, 0xc422656dc0)
mattermost | /usr/local/go/src/net/http/server.go:1801 +0x71d
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 3053 [select]:
mattermost | net/http.(*persistConn).writeLoop(0xc421490120)
mattermost | /usr/local/go/src/net/http/transport.go:1759 +0x165
mattermost | created by net/http.(*Transport).dialConn
mattermost | /usr/local/go/src/net/http/transport.go:1187 +0xa53
mattermost |
mattermost | goroutine 3064 [runnable]:
mattermost | image/jpeg.(*encoder).emitHuffRLE(0xc4223a0c00, 0x2, 0xffffffef00000000)
mattermost | /usr/local/go/src/image/jpeg/writer.go:278 +0x100
mattermost | image/jpeg.(*encoder).writeBlock(0xc4223a0c00, 0xc422b6b4b0, 0x1, 0x56, 0xc4ffffffad)
mattermost | /usr/local/go/src/image/jpeg/writer.go:366 +0xc3
mattermost | image/jpeg.(*encoder).writeSOS(0xc4223a0c00, 0x1d6df00, 0xc422620600)
mattermost | /usr/local/go/src/image/jpeg/writer.go:556 +0x4c4
mattermost | image/jpeg.Encode(0x1d629c0, 0xc421569180, 0x1d6df00, 0xc422620600, 0xc422b8aea0, 0x148ac78, 0xc422620600)
mattermost | /usr/local/go/src/image/jpeg/writer.go:634 +0x2c6
mattermost | github.com/mattermost/mattermost-server/app.(*App).generatePreviewImage(0xc420087d40, 0x1d6df00, 0xc422620000, 0xc42263a240, 0x83, 0xfc0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:516 +0xb3
mattermost | github.com/mattermost/mattermost-server/app.(*App).HandleImages.func2(0xc421afe1e0, 0xc420087d40, 0xc4220f2120, 0xc42263a240, 0x83, 0xfc0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:407 +0x89
mattermost | created by github.com/mattermost/mattermost-server/app.(*App).HandleImages
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:405 +0x1c9
mattermost |
mattermost | goroutine 3068 [sleep]:
mattermost | time.Sleep(0x3791aa58)
mattermost | /usr/local/go/src/runtime/time.go:65 +0x130
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.newRetryTimer.func2(0xc4220183c0, 0xa, 0xc422018360, 0xc421918 330)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/retry.go:82 +0x84
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.newRetryTimer
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/retry.go:72 +0x168
mattermost |
mattermost | goroutine 2877 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422a32d80, 0xc4222b0540, 0xc4 21d6e1e0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 2260 [chan receive]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/alecthomas/log4go.(*ConsoleLogWriter).run(0xc422407860, 0x1d653c0, 0xc42000e018)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/alect homas/log4go/termlog.go:33 +0x116
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/alecthomas/log4go.NewConsoleLogWriter
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/alect homas/log4go/termlog.go:26 +0xcf
mattermost |
mattermost | goroutine 2777 [runnable]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.putObjectMultipartStreamNoLength(0xc422026300, 0xc422f60360, 0 x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc4219181e0, 0xc422a68240, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:202 +0x38a
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.putObjectCommon(0xc422026300, 0xc422f60360, 0x0, 0x0, 0x0, 0x0 , 0x0, 0x1, 0xc4219181e0, 0xc422a68240, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:152 +0x642
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.PutObjectWithContext(0xc422026300, 0xc422f60360, 0x0, 0x0, 0x0 , 0x0, 0x0, 0x1, 0xc4219181e0, 0xc422a68240, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object-context.go:38 +0x302
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.PutObject(0xc422026300, 0xc422f60360, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc4219181e0, 0xc422a68240, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/api-put-object.go:130 +0x114
mattermost | github.com/mattermost/mattermost-server/utils.(*S3FileBackend).WriteFile(0xc422f60300, 0xc42d082000, 0x150d65, 0x1ffe00, 0xc421aa8b80, 0x7b, 0 x81)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/file_backend_s3.g o:153 +0x2a4
mattermost | github.com/mattermost/mattermost-server/app.(*App).WriteFile(0xc420087d40, 0xc42d082000, 0x150d65, 0x1ffe00, 0xc421aa8b80, 0x7b, 0xa)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:86 +0x8e
mattermost | github.com/mattermost/mattermost-server/app.(*App).DoUploadFile(0xc420087d40, 0xbeb46dc30b3cc433, 0x997568a187, 0x249a060, 0x13ebcc3, 0x6, 0xc 4220c60c0, 0x1a, 0xc422a1baa0, 0x1a, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:382 +0x690
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc4220c60c0, 0x1a, 0xc422a1baa0, 0x1a, 0xc4220f2 0f0, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:329 +0x505
mattermost | github.com/mattermost/mattermost-server/app.(*App).UploadMultipartFiles(0xc420087d40, 0x13ebcc3, 0x6, 0xc4220c60c0, 0x1a, 0xc422a1baa0, 0x1a, 0xc4223ca078, 0x1, 0x1, ...)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/file.go:300 +0x41b
mattermost | github.com/mattermost/mattermost-server/api4.uploadFile(0xc422a77ef0, 0x1d6e340, 0xc4216aa380, 0xc4215f3900)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/file.go:120 +0x562
mattermost | github.com/mattermost/mattermost-server/api4.handler.ServeHTTP(0xc420087d40, 0x148a680, 0x10001, 0x1d6e340, 0xc4216aa380, 0xc4215f3900)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/api4/context.go:157 +0x b65
mattermost | github.com/mattermost/mattermost-server/api4.(*handler).ServeHTTP(0xc4221c5be0, 0x1d6e340, 0xc4216aa380, 0xc4215f3900)
mattermost | :1 +0x63
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc42068a0e0, 0x1d6e340, 0xc4216aa380, 0xc4215f3900)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/mux/mux.go:159 +0xed
mattermost | github.com/mattermost/mattermost-server/app.(*CorsWrapper).ServeHTTP(0xc4211e73e0, 0x1d6e340, 0xc4216aa380, 0xc4215f3700)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:83 +0xa2
mattermost | github.com/mattermost/mattermost-server/app.(*RateLimiter).RateLimitHandler.func1(0x1d6e340, 0xc4216aa380, 0xc4215f3700)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/ratelimit.go:106 +0 xca
mattermost | net/http.HandlerFunc.ServeHTTP(0xc4201e2a40, 0x1d6e340, 0xc4216aa380, 0xc4215f3700)
mattermost | /usr/local/go/src/net/http/server.go:1918 +0x44
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.recoveryHandler.ServeHTTP(0x1d67540, 0xc4201e2a40, 0x1d631c0, 0x24b a100, 0x1, 0x1d6e340, 0xc4216aa380, 0xc4215f3700)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/goril la/handlers/recovery.go:78 +0xb9
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/gorilla/handlers.(*recoveryHandler).ServeHTTP(0xc4213a5f50, 0x1d6e340, 0xc4216aa380, 0xc4215f3700)
mattermost | :1 +0x76
mattermost | net/http.serverHandler.ServeHTTP(0xc422014750, 0x1d6e340, 0xc4216aa380, 0xc4215f3700)
mattermost | /usr/local/go/src/net/http/server.go:2619 +0xb4
mattermost | net/http.(*conn).serve(0xc422a31cc0, 0x1d6f0c0, 0xc42249dc00)
mattermost | /usr/local/go/src/net/http/server.go:1801 +0x71d
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 2483 [select, 10 minutes]:
mattermost | github.com/mattermost/mattermost-server/model.createTask.func1(0xc421da5900, 0x34630b8a000, 0xc4229fa3c0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :49 +0x15a
mattermost | created by github.com/mattermost/mattermost-server/model.createTask
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :40 +0x10c
mattermost |
mattermost | goroutine 2450 [select]:
mattermost | github.com/mattermost/mattermost-server/app.(*App).HubStart.func1(0xc420087d40)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:88 +0x5d 6
mattermost | created by github.com/mattermost/mattermost-server/app.(*App).HubStart
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:80 +0x51 6
mattermost |
mattermost | goroutine 2449 [select]:
mattermost | github.com/mattermost/mattermost-server/app.(*Hub).Start.func1()
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:375 +0x4 2a
mattermost | github.com/mattermost/mattermost-server/app.(*Hub).Start.func2()
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:434 +0x4 c
mattermost | created by github.com/mattermost/mattermost-server/app.(*Hub).Start
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:451 +0x1 28
mattermost |
mattermost | goroutine 2447 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb4f70, 0x72, 0xffffffffffffffff)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc422026498, 0x72, 0xc4212f0e00, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc422026498, 0xffffffffffffff00, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Accept(0xc422026480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:335 +0x1e2
mattermost | net.(*netFD).accept(0xc422026480, 0xc4212f0fd8, 0x401c27, 0xc4229d0620)
mattermost | /usr/local/go/src/net/fd_unix.go:238 +0x42
mattermost | net.(*TCPListener).accept(0xc4223cb378, 0x6fa448, 0x45b1a0, 0xc4212f1020)
mattermost | /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e
mattermost | net.(*TCPListener).Accept(0xc4223cb378, 0x148c700, 0xc4229d05a0, 0x1d6f180, 0xc422f69020)
mattermost | /usr/local/go/src/net/tcpsock.go:247 +0x49
mattermost | net/http.(*Server).Serve(0xc422014750, 0x1d6e0c0, 0xc4223cb378, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/server.go:2695 +0x1b2
mattermost | github.com/mattermost/mattermost-server/app.(*App).StartServer.func2(0xc420087d40, 0xc421ec6c60, 0x1d6e0c0, 0xc4223cb378)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:197 +0x77 4
mattermost | created by github.com/mattermost/mattermost-server/app.(*App).StartServer
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/server.go:180 +0xcb 7
mattermost |
mattermost | goroutine 2448 [select]:
mattermost | github.com/mattermost/mattermost-server/app.(*Hub).Start.func1()
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:375 +0x4 2a
mattermost | github.com/mattermost/mattermost-server/app.(*Hub).Start.func2()
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:434 +0x4 c
mattermost | created by github.com/mattermost/mattermost-server/app.(*Hub).Start
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_hub.go:451 +0x1 28
mattermost |
mattermost | goroutine 2466 [select]:
mattermost | github.com/mattermost/mattermost-server/jobs.(*Watcher).Start(0xc4223e8f00)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/jobs/jobs_watcher.go:51 +0x294
mattermost | created by github.com/mattermost/mattermost-server/jobs.(*Workers).Start.func1
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/jobs/workers.go:80 +0x8 8
mattermost |
mattermost | goroutine 2467 [select]:
mattermost | github.com/mattermost/mattermost-server/jobs.(*Schedulers).Start.func1.1()
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/jobs/schedulers.go:79 + 0x4ad
mattermost | sync.(*Once).Do(0xc4201be798, 0xc42177b7b8)
mattermost | /usr/local/go/src/sync/once.go:44 +0xbe
mattermost | github.com/mattermost/mattermost-server/jobs.(*Schedulers).Start.func1(0xc4201be770)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/jobs/schedulers.go:61 + 0x4c
mattermost | created by github.com/mattermost/mattermost-server/jobs.(*Schedulers).Start
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/jobs/schedulers.go:60 + 0xb9
mattermost |
mattermost | goroutine 2321 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc4223efe60, 0xc4223a0540, 0xc4 2247aba0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 2970 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb4730, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc421e46718, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc421e46718, 0xc4223f0000, 0x1000, 0x1000)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc421e46700, 0xc4223f0000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc421e46700, 0xc4223f0000, 0x1000, 0x1000, 0xc4201f38c8, 0x6f066b, 0x130d0c0)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc421f17608, 0xc4223f0000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | net/http.(*connReader).Read(0xc42215bbc0, 0xc4223f0000, 0x1000, 0x1000, 0xc4228fcea0, 0xc421661980, 0x168)
mattermost | /usr/local/go/src/net/http/server.go:753 +0x105
mattermost | bufio.(*Reader).fill(0xc4228fd320)
mattermost | /usr/local/go/src/bufio/bufio.go:97 +0x11a
mattermost | bufio.(*Reader).ReadSlice(0xc4228fd320, 0xc4201f3a0a, 0x4120b6, 0x7f87d6fc94f8, 0x0, 0x7f87d6fb4760, 0xfc)
mattermost | /usr/local/go/src/bufio/bufio.go:338 +0x2c
mattermost | bufio.(*Reader).ReadLine(0xc4228fd320, 0x100, 0xf8, 0x13a7f80, 0xc4201f3a01, 0x1220000402d99, 0xf8)
mattermost | /usr/local/go/src/bufio/bufio.go:367 +0x34
mattermost | net/textproto.(*Reader).readLineSlice(0xc42231c510, 0xc4201f3b20, 0xc4201f3b20, 0x412ca8, 0x100, 0x13a7f80)
mattermost | /usr/local/go/src/net/textproto/reader.go:55 +0x70
mattermost | net/textproto.(*Reader).ReadLine(0xc42231c510, 0xc421458000, 0x249a060, 0x3, 0xc421788000)
mattermost | /usr/local/go/src/net/textproto/reader.go:36 +0x2b
mattermost | net/http.readRequest(0xc4228fd320, 0x148c600, 0xc421458000, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/request.go:925 +0x99
mattermost | net/http.(*conn).readRequest(0xc422906c80, 0x1d6f0c0, 0xc4226560c0, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/http/server.go:933 +0x17c
mattermost | net/http.(*conn).serve(0xc422906c80, 0x1d6f0c0, 0xc4226560c0)
mattermost | /usr/local/go/src/net/http/server.go:1739 +0x50e
mattermost | created by net/http.(*Server).Serve
mattermost | /usr/local/go/src/net/http/server.go:2720 +0x288
mattermost |
mattermost | goroutine 2261 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/alecthomas/log4go.NewFileLogWriter.func1(0xc4201adf40)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/alect homas/log4go/filelog.go:87 +0x140
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/alecthomas/log4go.NewFileLogWriter
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/alect homas/log4go/filelog.go:77 +0x26a
mattermost |
mattermost | goroutine 2895 [syscall]:
mattermost | syscall.Syscall6(0xe8, 0x6, 0xc4224ffc24, 0x7, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
mattermost | github.com/mattermost/mattermost-server/vendor/golang.org/x/sys/unix.EpollWait(0x6, 0xc4224ffc24, 0x7, 0x7, 0xffffffffffffffff, 0xc421788300, 0xc42249df40, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/golang.org/x/sys /unix/zsyscall_linux_amd64.go:1544 +0x77
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/fsnotify/fsnotify.(*fdPoller).wait(0xc422307560, 0x148ac00, 0xc422307560, 0x0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/fsnot ify/fsnotify/inotify_poller.go:86 +0x91
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc421da8730)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/fsnot ify/fsnotify/inotify.go:192 +0x194
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/fsnotify/fsnotify.NewWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/fsnot ify/fsnotify/inotify.go:59 +0x23a
mattermost |
mattermost | goroutine 2476 [select, 10 minutes]:
mattermost | github.com/mattermost/mattermost-server/model.createTask.func1(0xc421db4780, 0xd18c2e28000, 0xc4229fa180)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :49 +0x15a
mattermost | created by github.com/mattermost/mattermost-server/model.createTask
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/model/scheduled_task.go :40 +0x10c
mattermost |
mattermost | goroutine 2316 [select, 11 minutes]:
mattermost | github.com/mattermost/mattermost-server/utils.NewHTMLTemplateWatcher.func1(0xc422457d00, 0xc42241aa50, 0xc422436820, 0x15)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/html.go:55 +0x1ef
mattermost | created by github.com/mattermost/mattermost-server/utils.NewHTMLTemplateWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/html.go:50 +0x2d4
mattermost |
mattermost | goroutine 2317 [chan receive, 11 minutes]:
mattermost | database/sql.(*DB).connectionOpener(0xc4201781e0)
mattermost | /usr/local/go/src/database/sql/sql.go:871 +0x53
mattermost | created by database/sql.Open
mattermost | /usr/local/go/src/database/sql/sql.go:609 +0x1ee
mattermost |
mattermost | goroutine 2983 [select]:
mattermost | github.com/mattermost/mattermost-server/app.(*WebConn).writePump(0xc42264e5a0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_conn.go:164 +0x 243
mattermost | github.com/mattermost/mattermost-server/app.(*WebConn).Pump.func1(0xc42264e5a0, 0xc421603020)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_conn.go:110 +0x 2b
mattermost | created by github.com/mattermost/mattermost-server/app.(*WebConn).Pump
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/app/web_conn.go:109 +0x 6b
mattermost |
mattermost | goroutine 2879 [select]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher.func1(0xc422a32e40, 0xc4222b0840, 0xc4 21d6e6c0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:182 +0xf8
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/go-sql-driver/mysql.(*mysqlConn).startWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/go-sq l-driver/mysql/connection_go18.go:179 +0xce
mattermost |
mattermost | goroutine 3039 [IO wait]:
mattermost | internal/poll.runtime_pollWait(0x7f87d6fb41f0, 0x72, 0x0)
mattermost | /usr/local/go/src/runtime/netpoll.go:173 +0x57
mattermost | internal/poll.(*pollDesc).wait(0xc421e0a698, 0x72, 0xffffffffffffff00, 0x1d68080, 0x1d60bf0)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
mattermost | internal/poll.(*pollDesc).waitRead(0xc421e0a698, 0xc422bb7000, 0x1000, 0x1000)
mattermost | /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
mattermost | internal/poll.(*FD).Read(0xc421e0a680, 0xc422bb7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
mattermost | net.(*netFD).Read(0xc421e0a680, 0xc422bb7000, 0x1000, 0x1000, 0xc42240d880, 0x7f87d7010000, 0x0)
mattermost | /usr/local/go/src/net/fd_unix.go:202 +0x52
mattermost | net.(*conn).Read(0xc421f17850, 0xc422bb7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/net/net.go:176 +0x6d
mattermost | crypto/tls.(*block).readFromUntil(0xc421918a50, 0x7f87d6f76468, 0xc421f17850, 0x5, 0xc421f17850, 0xc4214ee4b0)
mattermost | /usr/local/go/src/crypto/tls/conn.go:488 +0x95
mattermost | crypto/tls.(*Conn).readRecord(0xc420110700, 0x148cf17, 0xc420110820, 0x42f57b)
mattermost | /usr/local/go/src/crypto/tls/conn.go:590 +0xe0
mattermost | crypto/tls.(*Conn).Read(0xc420110700, 0xc421480000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
mattermost | /usr/local/go/src/crypto/tls/conn.go:1134 +0x110
mattermost | net/http.(*persistConn).Read(0xc4213d4b40, 0xc421480000, 0x1000, 0x1000, 0xc4228fafa0, 0xc4216555b8, 0x458ad0)
mattermost | /usr/local/go/src/net/http/transport.go:1391 +0x140
mattermost | bufio.(*Reader).fill(0xc423052780)
mattermost | /usr/local/go/src/bufio/bufio.go:97 +0x11a
mattermost | bufio.(*Reader).Peek(0xc423052780, 0x1, 0x0, 0x0, 0x0, 0xc422018780, 0x0)
mattermost | /usr/local/go/src/bufio/bufio.go:129 +0x3a
mattermost | net/http.(*persistConn).readLoop(0xc4213d4b40)
mattermost | /usr/local/go/src/net/http/transport.go:1539 +0x185
mattermost | created by net/http.(*Transport).dialConn
mattermost | /usr/local/go/src/net/http/transport.go:1186 +0xa2e
mattermost |
mattermost | goroutine 2896 [select]:
mattermost | github.com/mattermost/mattermost-server/utils.NewConfigWatcher.func1(0xc421e55340, 0xc421da8730, 0xc4223069c0, 0x1e, 0xc4223069c0, 0x1e, 0xc42 2a37bd0)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/config.go:184 +0x 1e5
mattermost | created by github.com/mattermost/mattermost-server/utils.NewConfigWatcher
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/utils/config.go:179 +0x 1bb
mattermost |
mattermost | goroutine 3074 [runnable]:
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.newRetryTimer.func2(0xc4220188a0, 0xa, 0xc422018840, 0xc421919 050)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/retry.go:75 +0x147
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.newRetryTimer
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/retry.go:72 +0x168
mattermost |
mattermost | goroutine 3075 [sleep]:
mattermost | time.Sleep(0x29c6c17a)
mattermost | /usr/local/go/src/runtime/time.go:65 +0x130
mattermost | github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.newRetryTimer.func2(0xc422018420, 0xa, 0xc422018000, 0xc421b07 980)
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/retry.go:82 +0x84
mattermost | created by github.com/mattermost/mattermost-server/vendor/github.com/minio/minio-go.Client.newRetryTimer
mattermost | /var/lib/jenkins/jobs/msr/jobs/d/jobs/enterprise-release/workspace/src/github.com/mattermost/mattermost-server/vendor/github.com/minio /minio-go/retry.go:72 +0x168
mattermost exited with code 2
`
The text was updated successfully, but these errors were encountered: