Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #21957

sebastianertz · 2022-11-27T16:07:25Z

Description

On an i386 system gitea 1.17.3 runs perfectly.
Version 1.18.0-rc0 and 1.18.0-rc1 crashes when indexing code.

Gitea Version

1.18.0-rc0 and 1.18.0-rc1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

panic: unaligned 64-bit atomic operation

goroutine 2044 [running]:
runtime/internal/atomic.panicUnaligned()
        /usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0xf9ede74, 0x3b)
        /usr/local/go/src/runtime/internal/atomic/atomic_386.s:125 +0x11
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).incrementBytesWritten(...)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/contentcoder.go:114
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).flushContents(0xf9ede00)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/contentcoder.go:143 +0x24a
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).Close(...)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/contentcoder.go:110
github.com/blevesearch/zapx/v15.(*interim).writeDicts(0xe334a50)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:803 +0x1443
github.com/blevesearch/zapx/v15.(*interim).convert(0xe334a50)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:277 +0x2b5
github.com/blevesearch/zapx/v15.(*ZapPlugin).newWithChunkMode(0xd1bbf88, {0xe434ac0, 0x8, 0x8}, 0x402)
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:72 +0x18b
github.com/blevesearch/zapx/v15.(*ZapPlugin).New(0xd1bbf88, {0xe434ac0, 0x8, 0x8})
        /go/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.5/new.go:47 +0x43
github.com/blevesearch/bleve/v2/index/scorch.(*Scorch).Batch(0x10dfa000, 0xf5459c0)
        /go/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.4/index/scorch/scorch.go:411 +0x5ff
github.com/blevesearch/bleve/v2.(*indexImpl).Batch(0x10de1720, 0x10e15060)
        /go/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.4/index_impl.go:309 +0xa0
code.gitea.io/gitea/modules/indexer/bleve.(*FlushingBatch).Flush(0xf861da0)
        /source/modules/indexer/bleve/batch.go:53 +0x32
code.gitea.io/gitea/modules/indexer/code.(*BleveIndexer).Index(0xe64df10, {0xba44760, 0xe666740}, 0xe31cd80, {0x12299400, 0x28}, 0xfb440a8)
        /source/modules/indexer/code/bleve.go:311 +0x407
code.gitea.io/gitea/modules/indexer/code.(*wrappedIndexer).Index(0xe82a570, {0xba44760, 0xe666740}, 0xe31cd80, {0x12299400, 0x28}, 0xfb440a8)
        /source/modules/indexer/code/wrapped.go:85 +0x7d
code.gitea.io/gitea/modules/indexer/code.index({0xba44760, 0xe666740}, {0xba476ec, 0xe82a570}, 0x81)
        /source/modules/indexer/code/indexer.go:107 +0x11f
code.gitea.io/gitea/modules/indexer/code.Init.func2({0xf37c6e0, 0x1, 0x14})
        /source/modules/indexer/code/indexer.go:157 +0x1e0
code.gitea.io/gitea/modules/queue.NewByteFIFOUniqueQueue.func1({0xf37c6e0, 0x1, 0x14})
        /source/modules/queue/queue_bytefifo.go:399 +0x3e
code.gitea.io/gitea/modules/queue.(*WorkerPool).doWork(0x10e61080, {0xba44820, 0xe3fcbc0})
        /source/modules/queue/workerpool.go:564 +0x69f
code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers.func1()
        /source/modules/queue/workerpool.go:312 +0x5d
created by code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers
        /source/modules/queue/workerpool.go:310 +0x25

Git Version

2.35.1

Operating System

eisfair.org

How are you running Gitea?

Self hosted, binary from dl.gitea.org

Database

SQLite

The text was updated successfully, but these errors were encountered:

zeripath · 2022-11-27T19:47:37Z

Duplicate of #21869

zeripath · 2022-11-27T19:49:12Z

As stated in #21869 are you really using a 32bit OS and not a 64bit OS?

sebastianertz · 2022-11-27T21:14:09Z

It's a 32bit OS

uname -a
Linux eis 5.15.75-eisfair-1-VIRT #1 SMP Wed Oct 26 23:56:22 CEST 2022 i686 i686 i386 GNU/Linux

uname -m
i686

zeripath · 2022-12-04T18:08:41Z

OK now that zapx have merged my PR and updated the tag we can pull the new tag and rebuild.

@sebastianertz would it be possible for you to try #22031?

There is an unaligned atomic field in zapx 15.3.5 which should have been fixed in blevesearch/zapx#147 This bug causes issues on 32bit builds. Update bleve and zapx to account for this. Fix go-gitea#21957 Signed-off-by: Andrew Thornton <art27@cantab.net>

sebastianertz · 2022-12-05T11:06:49Z

@zeripath Same issue:

panic: unaligned 64-bit atomic operation

goroutine 3397 [running]:
runtime/internal/atomic.panicUnaligned()
	/home/se/go/src/runtime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0x10125574, 0x3b)
	/home/se/go/src/runtime/internal/atomic/atomic_386.s:125 +0x11
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).incrementBytesWritten(...)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/contentcoder.go:114
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).flushContents(0x10125500)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/contentcoder.go:143 +0x24a
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).Close(...)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/contentcoder.go:110
github.com/blevesearch/zapx/v15.(*interim).writeDicts(0xf2caff0)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:803 +0x1443
github.com/blevesearch/zapx/v15.(*interim).convert(0xf2caff0)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:277 +0x2b5
github.com/blevesearch/zapx/v15.(*ZapPlugin).newWithChunkMode(0xd11a688, {0x10338080, 0x8, 0x8}, 0x402)
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:72 +0x18b
github.com/blevesearch/zapx/v15.(*ZapPlugin).New(0xd11a688, {0x10338080, 0x8, 0x8})
	/home/se/gopath/pkg/mod/github.com/blevesearch/zapx/v15@v15.3.6/new.go:47 +0x43
github.com/blevesearch/bleve/v2/index/scorch.(*Scorch).Batch(0xe36bc00, 0xfb28dd0)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index/scorch/scorch.go:412 +0x5ff
github.com/blevesearch/bleve/v2.(*indexImpl).Batch(0x10234190, 0xfafcb80)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index_impl.go:310 +0xa0
code.gitea.io/gitea/modules/indexer/bleve.(*FlushingBatch).Flush(0x10375da0)
	/home/se/gitea/modules/indexer/bleve/batch.go:53 +0x32
code.gitea.io/gitea/modules/indexer/code.(*BleveIndexer).Index(0xd4a59f0, {0xb9c70e0, 0xe365790}, 0xf725320, {0xfaa4800, 0x28}, 0x117fbe60)
	/home/se/gitea/modules/indexer/code/bleve.go:311 +0x407
code.gitea.io/gitea/modules/indexer/code.(*wrappedIndexer).Index(0xe706750, {0xb9c70e0, 0xe365790}, 0xf725320, {0xfaa4800, 0x28}, 0x117fbe60)
	/home/se/gitea/modules/indexer/code/wrapped.go:85 +0x7d
code.gitea.io/gitea/modules/indexer/code.index({0xb9c70e0, 0xe365790}, {0xb9ca06c, 0xe706750}, 0x81)
	/home/se/gitea/modules/indexer/code/indexer.go:107 +0x11f
code.gitea.io/gitea/modules/indexer/code.Init.func2({0x11c79540, 0x1, 0x14})
	/home/se/gitea/modules/indexer/code/indexer.go:157 +0x1e0
code.gitea.io/gitea/modules/queue.NewByteFIFOUniqueQueue.func1({0x11c79540, 0x1, 0x14})
	/home/se/gitea/modules/queue/queue_bytefifo.go:399 +0x3e
code.gitea.io/gitea/modules/queue.(*WorkerPool).doWork(0x104a2ae0, {0xb9c71a0, 0x10304e80})
	/home/se/gitea/modules/queue/workerpool.go:564 +0x69f
code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers.func1()
	/home/se/gitea/modules/queue/workerpool.go:312 +0x5d
created by code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers
	/home/se/gitea/modules/queue/workerpool.go:310 +0x25

sebastianertz · 2022-12-05T13:01:25Z

@zeripath Your zapx PR is in version v15.3.8

zeripath · 2022-12-05T21:58:25Z

Damn. That brings incompatible changes with it.

zeripath · 2022-12-05T22:15:20Z

Please try again. I've managed to make a dedicated backport with the alignment fix

sebastianertz · 2022-12-06T09:48:45Z

Same:

panic: unaligned 64-bit atomic operation

goroutine 2534 [running]:
runtime/internal/atomic.panicUnaligned()
	/home/se/go/src/runtime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0x1048873c, 0x3b)
	/home/se/go/src/runtime/internal/atomic/atomic_386.s:125 +0x11
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).incrementBytesWritten(...)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/contentcoder.go:114
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).flushContents(0x10488700)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/contentcoder.go:143 +0x24a
github.com/blevesearch/zapx/v15.(*chunkedContentCoder).Close(...)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/contentcoder.go:110
github.com/blevesearch/zapx/v15.(*interim).writeDicts(0xf525ef0)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:803 +0x1443
github.com/blevesearch/zapx/v15.(*interim).convert(0xf525ef0)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:277 +0x2b5
github.com/blevesearch/zapx/v15.(*ZapPlugin).newWithChunkMode(0xd1b337c, {0xe65fb80, 0x8, 0x8}, 0x402)
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:72 +0x18b
github.com/blevesearch/zapx/v15.(*ZapPlugin).New(0xd1b337c, {0xe65fb80, 0x8, 0x8})
	/home/se/gopath/pkg/mod/github.com/zeripath/zapx/v15@v15.3.6-alignment-fix/new.go:47 +0x43
github.com/blevesearch/bleve/v2/index/scorch.(*Scorch).Batch(0x10306c00, 0x10c07f90)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index/scorch/scorch.go:412 +0x5ff
github.com/blevesearch/bleve/v2.(*indexImpl).Batch(0x1036f680, 0xf499980)
	/home/se/gopath/pkg/mod/github.com/blevesearch/bleve/v2@v2.3.5/index_impl.go:310 +0xa0
code.gitea.io/gitea/modules/indexer/bleve.(*FlushingBatch).Flush(0x100f7da0)
	/home/se/gitea-zeripath/modules/indexer/bleve/batch.go:52 +0x32
code.gitea.io/gitea/modules/indexer/code.(*BleveIndexer).Index(0xf0d9540, {0xba537c0, 0xf0c7fa0}, 0xf88e7e0, {0xf7abc00, 0x28}, 0x11755e30)
	/home/se/gitea-zeripath/modules/indexer/code/bleve.go:310 +0x407
code.gitea.io/gitea/modules/indexer/code.(*wrappedIndexer).Index(0xf26d920, {0xba537c0, 0xf0c7fa0}, 0xf88e7e0, {0xf7abc00, 0x28}, 0x11755e30)
	/home/se/gitea-zeripath/modules/indexer/code/wrapped.go:84 +0x7d
code.gitea.io/gitea/modules/indexer/code.index({0xba537c0, 0xf0c7fa0}, {0xba5674c, 0xf26d920}, 0x81)
	/home/se/gitea-zeripath/modules/indexer/code/indexer.go:106 +0x11b
code.gitea.io/gitea/modules/indexer/code.Init.func2({0x10185ea0, 0x1, 0x14})
	/home/se/gitea-zeripath/modules/indexer/code/indexer.go:156 +0x1e0
code.gitea.io/gitea/modules/queue.NewByteFIFOUniqueQueue.func1({0x10185ea0, 0x1, 0x14})
	/home/se/gitea-zeripath/modules/queue/queue_bytefifo.go:398 +0x3e
code.gitea.io/gitea/modules/queue.(*WorkerPool).doWork(0x10280840, {0xba53880, 0xe621500})
	/home/se/gitea-zeripath/modules/queue/workerpool.go:563 +0x69f
code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers.func1()
	/home/se/gitea-zeripath/modules/queue/workerpool.go:311 +0x5d
created by code.gitea.io/gitea/modules/queue.(*WorkerPool).addWorkers
	/home/se/gitea-zeripath/modules/queue/workerpool.go:309 +0x25

There is an unaligned atomic field in zapx 15.3.5 which should have been fixed in a subsequent patch This bug causes issues on 32bit builds. Update bleve and zapx to account for this. Fix #21957 Signed-off-by: Andrew Thornton <art27@cantab.net>

Backport go-gitea#22031 There is an unaligned atomic field in zapx 15.3.5 which should have been fixed in a subsequent patch This bug causes issues on 32bit builds. Update bleve and zapx to account for this. Fix go-gitea#21957 Signed-off-by: Andrew Thornton <art27@cantab.net>

Backport #22031 There is an unaligned atomic field in zapx 15.3.5 which should have been fixed in a subsequent patch This bug causes issues on 32bit builds. Update bleve and zapx to account for this. Fix #21957 Signed-off-by: Andrew Thornton <art27@cantab.net>

sebastianertz added the type/bug label Nov 27, 2022

zeripath marked this as a duplicate of #21869 Nov 27, 2022

zeripath mentioned this issue Dec 4, 2022

Update bleve and zapx to fix unaligned atomic #22031

Merged

jolheiser closed this as completed in #22031 Dec 22, 2022

zeripath mentioned this issue Dec 22, 2022

Update bleve and zapx to fix unaligned atomic (#22031) #22218

Merged

lunny mentioned this issue Dec 30, 2022

zapx: Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #22279

Closed

go-gitea locked and limited conversation to collaborators May 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #21957

Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #21957

sebastianertz commented Nov 27, 2022

zeripath commented Nov 27, 2022

zeripath commented Nov 27, 2022

sebastianertz commented Nov 27, 2022

zeripath commented Dec 4, 2022

sebastianertz commented Dec 5, 2022

sebastianertz commented Dec 5, 2022

zeripath commented Dec 5, 2022

zeripath commented Dec 5, 2022

sebastianertz commented Dec 6, 2022

Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #21957

Code indexer crashes gitea on i386/x86 with "panic: unaligned 64-bit atomic operation" #21957

Comments

sebastianertz commented Nov 27, 2022

Description

Gitea Version

Can you reproduce the bug on the Gitea demo site?

Log Gist

Screenshots

Git Version

Operating System

How are you running Gitea?

Database

zeripath commented Nov 27, 2022

zeripath commented Nov 27, 2022

sebastianertz commented Nov 27, 2022

zeripath commented Dec 4, 2022

sebastianertz commented Dec 5, 2022

sebastianertz commented Dec 5, 2022

zeripath commented Dec 5, 2022

zeripath commented Dec 5, 2022

sebastianertz commented Dec 6, 2022