boltdb for no-cluster setups

[fopina]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I really like dkron, but the bump in resources from v1 (using etcd) to v2 (with badger and raft) is huge.
I use dkron in cluster in one project, but in the other I just use a single node.
Maybe it doesn't make much sense to use dkron for non-cluster setup, but I'm used to the API and some of the internals..
In those cases, all the overhead from badgerdb/raft/cluster management is useless and still there...

Describe the solution you'd like
A clear and concise description of what you want to happen.

Would it be easy to support boltdb along badgerdb? so using some flag (instead of --bootstrap-expect 1) would disable all the cluster overhead and just use boltdb instead of the whole in-memory badgerdb.

[vcastellm]

@fopina is a perfectly valid use case and one that dkron should support, to run a single node setups.
Actually Badger is not in-memory, it's disk based.

I don't have metrics on memory usage of v1 vs v2, but the goal was to eliminate, external kvs because they are really resource intensive and an external piece to maintain. An etcd cluster is expensive, a Dkron cluster should be lightweight. Do your observations differ from this?

On the other hand in single node sure it uses more resources, though in theory it could be tuned to not consume more than using BoltDB alone.

Could you provide some metrics of single node instances?

[fopina]

thanks on the quick feedback @Victorcoder
I opened this suggestion before seeing #660 where I commented the little/quick metrics I took (that actually represent my issue).

So it wasn't really related from moving from dkron v1 to dkron v2, but more about moving from badger v1 to badger v2.

I've used dkron v1 and dkron v2.0.0-rc7 for quite some time now in a very low resource scenario and never had issues. Only after upgrading to dkron v2.0.4 I started running out of memory in that VPS and noticed the differences mentioned in #660

I've updated my old dkron fork with 2.0.4 and downgraded badger to v1 this morning and it compiled. I'll try to use it like that for a few days and maybe look into "asking" badger v2 to trade some memory for disk swapping operations then

[vcastellm]

Thanks, I saw the other issue after this.

Great to know where to look at, I'll check badger settings and memory consumption m

[vcastellm]

@fopina after checking this see the related fix, back to ~40Mb memory on startup. Responding to your question, yes Badger 2 brings lots of improvements and bug fixes, at the end memory usage was my fault.

[fopina]

awesome speed!!! 🔝 🙇

the commit seems really small, is the change basically about that FileIO instead of the default MemoryMap? So was it doing all badgerdb in-memory with MemoryMap?

[vcastellm]

No, the FileIO just will help to bring everything to disk. The problem was the DropAll operation 🤷‍♀️