Distributed key-value store written in Go.
- Simple API for managing data in key-value format
- Rebalancing data across volume servers
- Index build based on volumes data
To run tinykv with Docker on a single host:
$ make docker-up HOST=localhost VOLUME=3
HOSTis the host address, by default it is set tolocalhost, but it will only work for Linux machines; for Windows/MacOS, change it to a real IP address.
VOLUMEis the number of volume servers to run, by default is set to3.
To build and run master server from the source code:
- Requirements: go, make & nginx
- Install dependencies:
$ make setup- Build:
$ make build- Setup volume instances:
$ PORT=3001 VOLUME=tmp/vol1 ./volume/setup.sh
$ PORT=3002 VOLUME=tmp/vol2 ./volume/setup.sh
$ PORT=3003 VOLUME=tmp/vol3 ./volume/setup.sh- Run the master server binary:
$ ./bin/master -db ./tmp/indexdb/ -p 3000 -volumes localhost:3001,localhost:3002,localhost:3003Put "bigswag" in "wehave" key:
$ curl -L -X PUT -d bigswag localhost:3000/wehaveGet "wehave" key:
$ curl -L localhost:3000/wehaveDelete "wehave" key:
$ curl -L -X DELETE localhost:3000/wehaveList keys starting with "we":
$ curl -L "localhost:3000/we?list"
$ curl -L "localhost:3000/we?list&limit=100"
$ curl -L "localhost:3000?list&start=/we&limit=100"Change the amount of volume servers:
$ ./bin/master -cmd rebalance -db ./tmp/indexdb/ -volumes localhost:3001,localhost:3002Before rebalancing, make sure the master server is down, as LevelDB can only be accessed by one process.
Regenerate the LevelDB:
$ ./bin/master -cmd rebuild -db ./tmp/indexdb-alt/ -volumes localhost:3001,localhost:3002,localhost:3003Fetching non-existent key: ~10325 req/sec
$ wrk -t2 -c100 -d10s http://localhost:3000/keyFetching existent key: ~9800 req/sec
$ wrk -t2 -c100 -d10s http://localhost:3000/wehaveAll my code is MIT licensed. Libraries follow their respective licenses.