Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Badger Error: Got error: While opening memtable, == Recovering from initIndex crash == #17

Closed
Akhilesh53 opened this issue Oct 3, 2023 · 1 comment
Closed

Badger Error: Got error: While opening memtable, == Recovering from initIndex crash == #17

Akhilesh53 opened this issue Oct 3, 2023 · 1 comment

Comments

@Akhilesh53
Copy link

GitHub Issues are deprecated. Use Discuss Issues for reporting issues about this repository.

Hi Team, I am continuously getting this error. Please check it out once.

  1. Error 1

badger 2023/09/21 12:38:37 ERROR: Got error: While opening memtable: /data/dedupe/data/table_50/Pentagram/00067.mem error: while opening file: /data/dedupe/data/table_50/Pentagram/00067.mem error: open /data/dedupe/data/table_50/Pentagram/00067.mem: no such file or directory
unable to open: /data/dedupe/data/table_50/Pentagram/00067.mem
github.com/dgraph-io/ristretto/z.OpenMmapFile
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/ristretto/z/file.go:80
github.com/dgraph-io/badger/v4.(*logFile).open
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:554
github.com/dgraph-io/badger/v4.(*DB).openMemTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:130
github.com/dgraph-io/badger/v4.(*DB).newMemTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:151
github.com/dgraph-io/badger/v4.(*DB).ensureRoomForWrite
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:995
github.com/dgraph-io/badger/v4.(*DB).writeRequests
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:829
github.com/dgraph-io/badger/v4.(*DB).doWrites.func1
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:884
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1598 for id: 67
badger 2023/09/21 12:38:37 ERROR: writeRequests: writeRequests err: cannot create new mem table error: newMemTable error: While opening memtable: /data/dedupe/data/table_50/Pentagram/00067.mem error: while opening file: /data/dedupe/data/table_50/Pentagram/00067.mem error: open /data/dedupe/data/table_50/Pentagram/00067.mem: no such file or directory
unable to open: /data/dedupe/data/table_50/Pentagram/00067.mem
github.com/dgraph-io/ristretto/z.OpenMmapFile
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/ristretto/z/file.go:80
github.com/dgraph-io/badger/v4.(*logFile).open
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:554
github.com/dgraph-io/badger/v4.(*DB).openMemTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:130
github.com/dgraph-io/badger/v4.(*DB).newMemTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:151
github.com/dgraph-io/badger/v4.(*DB).ensureRoomForWrite
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:995
github.com/dgraph-io/badger/v4.(*DB).writeRequests
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:829
github.com/dgraph-io/badger/v4.(*DB).doWrites.func1
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:884
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1598
badger 2023/09/21 12:38:38 ERROR: Failure while flushing memtable to disk: error while creating table err: while creating table: /data/dedupe/data/table_50/Pentagram/000133.sst error: open /data/dedupe/data/table_50/Pentagram/000133.sst: no such file or directory
unable to open: /data/dedupe/data/table_50/Pentagram/000133.sst
github.com/dgraph-io/ristretto/z.OpenMmapFile
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/ristretto/z/file.go:80
github.com/dgraph-io/badger/v4/table.CreateTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/table/table.go:259
github.com/dgraph-io/badger/v4.(*DB).handleFlushTask
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:1061
github.com/dgraph-io/badger/v4.(*DB).flushMemtable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:1083
github.com/dgraph-io/badger/v4.Open.func5
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:354
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1598. Retrying...
total time: cluster time for 50 took 13m33.967768745s



panic: cannot create new mem table error: newMemTable error: While opening memtable: /data/dedupe/data/table_50/Pentagram/00067.mem error: while opening file: /data/dedupe/data/table_50/Pentagram/00067.mem error: open /data/dedupe/data/table_50/Pentagram/00067.mem: no such file or directory
unable to open: /data/dedupe/data/table_50/Pentagram/00067.mem
github.com/dgraph-io/ristretto/z.OpenMmapFile
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/ristretto/z/file.go:80
github.com/dgraph-io/badger/v4.(*logFile).open
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:554
github.com/dgraph-io/badger/v4.(*DB).openMemTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:130
github.com/dgraph-io/badger/v4.(*DB).newMemTable
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/memtable.go:151
github.com/dgraph-io/badger/v4.(*DB).ensureRoomForWrite
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:995
github.com/dgraph-io/badger/v4.(*DB).writeRequests
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:829
github.com/dgraph-io/badger/v4.(*DB).doWrites.func1
    /data/badger_broadband/badger-cluster/vendor/github.com/dgraph-io/badger/v4/db.go:884
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1598


goroutine 3487 [running]:
cluster/badger_cluster.(*Cluster).AddMemberAddress(0xc490f28df0, {​​​​​​​​0xbf7d98, 0xc000194000}​​​​​​​​​​​​​​​, 0x32?, 0x32, {​​​​​​​​​​​​​​​0xc48e35f800, 0x2f, 0x7f521b3908f0?}​​​​​​​​​​​​​​​)
    /data/badger_broadband/badger-cluster/badger_cluster/cluster.go:69 +0x276
main.makeCluster({​​​​​​​​​​​​​​​0xbf7d98, 0xc000194000}​​​​​​​​​​​​​​​, 0x32)
    /data/badger_broadband/badger-cluster/app/create_cluster.go:99 +0x3ea
main.worker(...)
    /data/badger_broadband/badger-cluster/app/create_cluster.go:59
main.main.func1()
    /data/badger_broadband/badger-cluster/app/create_cluster.go:45 +0x8e
created by main.main
    /data/badger_broadband/badger-cluster/app/create_cluster.go:42 +0x7e
  1. Error 2

panic: runtime error: slice bounds out of range [-4169358568:] [recovered]
    panic: runtime error: slice bounds out of range [-4169358568:]
    panic: 
== Recovering from initIndex crash ==
File Info: [ID: 95, Size: 69121215, Zeros: 0]
isEnrypted: false checksumLen: 4238479779 
== Recovered ==



goroutine 6932842 [running]:
github.com/dgraph-io/badger/v4/table.(*Table).initBiggestAndSmallest.func1.1()
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:353 +0xa8
panic({​​​​​​​​​​​​​​​0xba9420, 0xc4324db2d8}​​​​​​​​​​​​​​​)
    /usr/local/go/src/runtime/panic.go:884 +0x212
github.com/dgraph-io/ristretto/z.(*MmapFile).Bytes(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/ristretto/z/file.go:116
github.com/dgraph-io/badger/v4/table.(*Table).read(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:420
github.com/dgraph-io/badger/v4/table.(*Table).readNoFail(0xd0a740?, 0xc4a7b68810?, 0xbfe74d?)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:424 +0xd0
github.com/dgraph-io/badger/v4/table.(*Table).initBiggestAndSmallest.func1()
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:383 +0x2ff
panic({​​​​​​​​​​​​​​​0xba9420, 0xc4324db2c0}​​​​​​​​​​​​​​​)
    /usr/local/go/src/runtime/panic.go:884 +0x212
github.com/dgraph-io/ristretto/z.(*MmapFile).Bytes(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/ristretto/z/file.go:116
github.com/dgraph-io/badger/v4/table.(*Table).read(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:420
github.com/dgraph-io/badger/v4/table.(*Table).readNoFail(0x6?, 0x0?, 0x9?)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:424 +0xd0
github.com/dgraph-io/badger/v4/table.(*Table).initIndex(0xc47e194180)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:445 +0x109
github.com/dgraph-io/badger/v4/table.(*Table).initBiggestAndSmallest(0xc47e194180)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:403 +0x7f
github.com/dgraph-io/badger/v4/table.OpenTable(0xc31e8f4160, {​​​​​​​​​​​​​​​0x0, 0x1, 0x1000000, 0xf33333, 0x0, 0x3f847ae147ae147b, 0x1000, 0x0, 0x1, ...}​​​​​​​​​​​​​​​)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:309 +0x278
github.com/dgraph-io/badger/v4/table.CreateTable({​​​​​​​​​​​​​​​0xc167146b80, 0x33}​​​​​​​​​​​​​​​, 0xccef5a4090)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:273 +0x305
github.com/dgraph-io/badger/v4.(*levelsController).subcompact.func4(0xccef5a4090, 0xc025c8b6e0?)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:846 +0x11b
created by github.com/dgraph-io/badger/v4.(*levelsController).subcompact
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:836 +0x707
  1. Error 3

panic: runtime error: invalid memory address or nil pointer dereference
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x941247]


goroutine 2551 [running]:
github.com/dgraph-io/badger/v4/table.(*Table).offsetsLength(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:133
github.com/dgraph-io/badger/v4/table.(*Table).DecrRef(0xc67b5ca9c0)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:169 +0x87
github.com/dgraph-io/badger/v4.decrRefs(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/level_handler.go:178
github.com/dgraph-io/badger/v4.(*levelsController).compactBuildTables.func6()
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:955 +0x56
github.com/dgraph-io/badger/v4.(*levelsController).runCompactDef.func1()
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:1431 +0x25
panic({​​​​​​​​​​​0xb11c60, 0x11587a0}​​​​​​​​​​​)
    /usr/local/go/src/runtime/panic.go:884 +0x213
github.com/dgraph-io/badger/v4/fb.(*TableIndex).StaleDataSize(0x0)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/fb/TableIndex.go:132 +0x1d
github.com/dgraph-io/badger/v4/table.(*Table).StaleDataSize(0x0?)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/table/table.go:667 +0x1e
github.com/dgraph-io/badger/v4.(*levelHandler).addSize(...)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/level_handler.go:214
github.com/dgraph-io/badger/v4.(*levelHandler).replaceTables(0xc242f49140, {​​​​​​​​​​​0xc48d01a248, 0x1, 0x4208767?}​​​​​​​​​​​, {​​​​​​​​​​​0xc6fd7f5a98, 0x1, 0xc48d02b800?}​​​​​​​​​​​)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/level_handler.go:137 +0x2b3
github.com/dgraph-io/badger/v4.(*levelsController).runCompactDef(_, _, _, {​​​​​​​​​​​0xc48bea9250, 0x1, {​​​​​​​​​​​0x4, {​​​​​​​​​​​0xc48d02b7c0, 0x7, 0x7}​​​​​​​​​​​, {​​​​​​​​​​​0xc48d02b800, ...}​​​​​​​​​​​}​​​​​​​​​​​, ...}​​​​​​​​​​​)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:1460 +0x43b
github.com/dgraph-io/badger/v4.(*levelsController).doCompact(0xc116de2a80, 0x1, {​​​​​​​​​​​0x5, 0x3ffd1ba93016f62f, 0x3ffd1ba93016f62f, {​​​​​​​​​​​0x0, 0x0, 0x0}​​​​​​​​​​​, {​​​​​​​​​​​0x4, {​​​​​​​​​​​0xc48d02b7c0, ...}​​​​​​​​​​​, ...}​​​​​​​​​​​}​​​​​​​​​​​)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:1544 +0x538
github.com/dgraph-io/badger/v4.(*levelsController).runCompactor.func2({​​​​​​​​​​​0x5, 0x3ffd1ba93016f62f, 0x3ffd1ba93016f62f, {​​​​​​​​​​​0x0, 0x0, 0x0}​​​​​​​​​​​, {​​​​​​​​​​​0x4, {​​​​​​​​​​​0xc48d02b7c0, 0x7, 0x7}​​​​​​​​​​​, ...}​​​​​​​​​​​}​​​​​​​​​​​)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:465 +0x78
github.com/dgraph-io/badger/v4.(*levelsController).runCompactor.func3()
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:488 +0x158
github.com/dgraph-io/badger/v4.(*levelsController).runCompactor(0xc116de2a80, 0x1, 0xc2026ffef0)
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:517 +0x382
created by github.com/dgraph-io/badger/v4.(*levelsController).startCompact
    /data/badger_broadband/badger_grpc/vendor/github.com/dgraph-io/badger/v4/levels.go:354 +0x4e
@github-actions
Copy link

github-actions bot commented Oct 3, 2023

Use Discuss Issues for reporting issues about this repository.

@github-actions github-actions bot closed this as completed Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Akhilesh53