Windows: default options limit database to 64MB. #504
Comments
|
Thanks for the workaround, I also ran into this issue yesterday (Windows 10, go1.5.1 windows/amd64) and had a hard time finding the fix. How should this work for cross-compilation? The docs indicate that |
|
We have been using bolt for a while and our users haven't reported this issue. Our database right now gets to about 500MB in size, well beyond that 64MB that you are reporting returns an error. There must be other conditions that are causing the problem? We are not changing the default values for |
|
I don't know what other conditions may be causing the problem. It happens 100% of the time for me from more than one set of code doing different sized transactions. The following code reproduces the problem for me: package main
import (
"encoding/binary"
"log"
"github.com/boltdb/bolt"
)
func panicOnErr(err error) {
if err != nil {
panic(err)
}
}
var bkt = []byte("bucket1")
var data = []byte("01234567890123456789012345678901")
type ID uint64
func (id ID) Bytes() []byte {
buf := make([]byte, binary.Size(id))
binary.BigEndian.PutUint64(buf, uint64(id))
return buf
}
func main() {
db, err := bolt.Open("test.db", 0644, nil)
panicOnErr(err)
defer db.Close()
db.Update(func(tx *bolt.Tx) error {
_, err := tx.CreateBucketIfNotExists(bkt)
panicOnErr(err)
return nil
})
for i := 0; ; i++ {
err := db.Update(func(tx *bolt.Tx) error {
b := tx.Bucket(bkt)
for i := 0; i < 1000; i++ {
ns, _ := b.NextSequence()
k := ID(ns)
err := b.Put(k.Bytes(), data)
panicOnErr(err)
if i == 0 {
log.Println(k)
}
}
return nil
})
panicOnErr(err)
}
}Trimmed output: |
|
@ChrisHines Can you try with this commit: 852d302 I believe the issue is caused by this commit: #453 I need to get better CI running so that non-Unix platforms can be tested regularly (Windows, ARM, etc). I don't have access to a Windows machine so it's difficult for me to test every pull request. |
|
@benbjohnson That commit does not exhibit the problem. I modified the code I posted above to drive a Update: Ignore this bisect log, it was buggy.$ git bisect log I have manually confirmed this result. |
|
@benbjohnson after a bit more digging, I found a bug in my bisect driver. Fixing the bug and re-running the bisect, I now get this log: git bisect start Sorry for the confusion. |
|
@ChrisHines Did you run the bisect multiple times to make sure the result is reliable? |
|
@benbjohnson You are probably right. I see windows has separate logic for handling mmap+grow. I do not have access to windows machines at the moment unfortunately. |
|
@benbjohnson @xiang90 Yes, I can reproduce the problem every time on a122e1c and it does not reproduce on 694a82a. |
|
After looking at the commits in question more carefully, I noticed that the first bad commit (a122e1c) added a (previously missing) error check to the call to grow in |
|
Researching the OS calls in play
|
|
This bug seems to be caught by the current test suite. |
|
I earlier stated that our users had not been running into this issue, although it appears that I have spoken too soon. They have started reporting this issue. Sorry for the confusion. |
|
I confirm this issue is fixed on master. |
|
@ChrisHines Thanks for checking on it and closing the issue. |
Environment:
OS: Windows 7
Go: go version go1.5.1 windows/amd64
Steps to reproduce:
niloptions.What happens:
The database file quickly grows to 64MB and then bolt returns an error:
Workarounds: The error does not occur if the database is opened with the
NoGrowSyncoption enabled.What was expected: The default options should work better on all platforms.
The text was updated successfully, but these errors were encountered: