storage/localstore: add subscriptions wait group before closing leveldb

[janos]

This PR add a protection against creating a LevelDB iterator after it has been closed. It is possible that a subscription goroutine can be scheduled after localstore DB Close method is called, resulting a panic:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x787fa8]
goroutine 3267 [running]:
github.com/syndtr/goleveldb/leveldb.(*DB).newRawIterator(0xc00025a340, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc0001149f0, 0x54)
    /home/travis/gopath/src/github.com/ethersphere/swarm/vendor/github.com/syndtr/goleveldb/leveldb/db_iter.go:54 +0x2a8
github.com/syndtr/goleveldb/leveldb.(*DB).newIterator(0xc00025a340, 0x0, 0x0, 0x0, 0x0, 0x54, 0x0, 0x0, 0x8b2a40)
    /home/travis/gopath/src/github.com/ethersphere/swarm/vendor/github.com/syndtr/goleveldb/leveldb/db_iter.go:79 +0x99
github.com/syndtr/goleveldb/leveldb.(*DB).NewIterator(0xc00025a340, 0x0, 0x0, 0x0, 0x0)
    /home/travis/gopath/src/github.com/ethersphere/swarm/vendor/github.com/syndtr/goleveldb/leveldb/db.go:891 +0x189
github.com/ethersphere/swarm/shed.(*DB).NewIterator(0xc000041680, 0xc000548001, 0x1)
    /home/travis/gopath/src/github.com/ethersphere/swarm/shed/db.go:140 +0xc0
github.com/ethersphere/swarm/shed.Index.Iterate(0xc000041680, 0xc00002c8e9, 0x1, 0x1, 0xc00304c640, 0xc0020c5800, 0x9210e8, 0x9210f0, 0xc0000c5d70, 0xc0000c5d08, ...)
    /home/travis/gopath/src/github.com/ethersphere/swarm/shed/index.go:304 +0x107
github.com/ethersphere/swarm/storage/localstore.(*DB).SubscribePull.func1(0x90aca5, 0x18, 0xc001095080, 0x0, 0xc000f32000, 0x5, 0xc0010950e0, 0x0, 0x99dca0, 0xc0007a0640, ...)
    /home/travis/gopath/src/github.com/ethersphere/swarm/storage/localstore/subscription_pull.go:90 +0x676
created by github.com/ethersphere/swarm/storage/localstore.(*DB).SubscribePull
    /home/travis/gopath/src/github.com/ethersphere/swarm/storage/localstore/subscription_pull.go:64 +0x3c4
FAIL    github.com/ethersphere/swarm/storage/localstore    31.341s

The issue was found by @acud.

[holisticode]

Not introduced by this PR only, but is it ok to do Wait() on an actual Close method, which is supposed to be for shutting down something? I mean, is it ensured that Wait() will not wait forever, e.g. because of blocked go routines?

[acud]

well if you want a clean shutdown you'd probably want to wait until all of the goroutines that use a certain resource exit no?
if they don't exit it means that there's a deadlock and that should be fixed

[janos]

As we talked, I've added a deadline of 5 seconds, that should be reasonable. And also I have added by @acud suggestion a goroutine profile dump, which we should improve int he future to control where it is written, currently it is in stdout.