Skip to content
forked from speedb-io/speedb

Speedb is a library that provides persistent key-value stores.

License

Notifications You must be signed in to change notification settings

mrambacher/speedb

This branch is 214 commits ahead of, 970 commits behind speedb-io/speedb:main.

Folders and files

NameName
Last commit message
Last commit date
Sep 12, 2022
Dec 21, 2022
Jun 17, 2022
Feb 7, 2023
Feb 14, 2023
Nov 25, 2022
Apr 4, 2022
Feb 14, 2023
Feb 11, 2023
Nov 27, 2022
Feb 9, 2023
Feb 5, 2023
Jan 23, 2023
Nov 25, 2022
Feb 14, 2023
Jan 23, 2023
Nov 25, 2022
Nov 25, 2022
Dec 25, 2022
Aug 4, 2022
Jan 23, 2023
Feb 11, 2023
Feb 14, 2023
Feb 7, 2023
Dec 22, 2022
Feb 14, 2023
Feb 9, 2023
Nov 25, 2022
Feb 11, 2023
Nov 25, 2022
Feb 14, 2023
Feb 14, 2023
Jan 14, 2014
Nov 25, 2022
Jun 26, 2018
Jul 19, 2019
Nov 25, 2022
Feb 9, 2023
Nov 25, 2022
Nov 25, 2022
Jan 19, 2022
Jun 19, 2015
Jan 23, 2023
Nov 25, 2022
Aug 23, 2022
Nov 25, 2022
Jul 17, 2017
Feb 7, 2023
May 13, 2022
Jan 11, 2023
Apr 5, 2022
Feb 9, 2023
Feb 16, 2022
Feb 26, 2018
Sep 2, 2021
Nov 25, 2022
Nov 22, 2022
Jan 28, 2020
Nov 27, 2022
Feb 9, 2023
Jun 25, 2019

GitHub GitHub contributors GitHub pull requests GitHub closed pull requests

Speedb

A first-of-its-kind, community-led storage engine designed to support modern data sets. It focuses on high performance, optimized for modern storage hardware and scale, on-premise and in the cloud. We strive to simplify the usability of complex data engines as well as stabilize and improve performance.

We are building an open source community where RocksDB and Speedb users and developers can interact, improve, share knowledge, and learn best practices. You are welcome to join our community, contribute, and participate in the development of the next generation storage engine. We welcome any questions or comments you may have. Please use issues to submit them, and pull requests to make contributions.

This project is maintained by Speedb and is based on Rocksdb, developed by Facebook.

Join us to build the next generation data engine!

Benchmarks

Below is a graph comparing Speedb's paired bloom filter with the default bloom filter.

Screen Shot 2022-10-31 at 15 15 42

This test simulates a large number of non-existing keys, with the disk as the bottleneck. The test was running with 1 billion objects, value size of 256 bytes and 4 threads on a system with 8 CPU cores.

According to the graph, random reads produce significant improvements of up to 140% in this type of workload.

Another test using the same bits per key, memory consumption was reduced by 23%, while performance remained unchanged. The results are presented in the graph below. The test was running with 1 billion objects, value size of 256 bytes and 4 threads on a system with 16 CPU cores.

Blueberry performance results  (4)

You can read more about our new paired bloom filter algorithm in the documentation.

Usage

  • If speedb is in your default library path:

    In your CMakeLists.txt add:

    target_link_libraries(${PROJECT_NAME} speedb)
    

    where PROJECT_NAME is the name of your target application which uses speedb

  • Otherwise, you have to include the path to the folder the library is in like so:

    target_link_libraries(${PROJECT_NAME} /path/to/speedb/library/folder)
    

Usage of the library in your code is the same regardless of whether you statically linked the library or dynamically linked it, and examples can be found under the examples directory. The public interface is in include. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Build dependencies

Please refer to the file INSTALL.md for a list of all the dependencies and how to install them across different platforms.

πŸ”¨ Building Speedb

Debug:

mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug [cmake options]
make speedb

By default the build type is Debug.

Release:

mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release [cmake options]
make speedb

This will build the static library. If you want to build the dynamic library, use:

make speedb-shared

If you want make to increase the number of cores used for building, simply use the -j option.

If you want to build a specific target:

make [target name]

For development and functional testing, go with the debug version which includes more assertions and debug prints. Otherwise, for production or performance testing, we recommend building a release version which is more optimized.

πŸ“ˆ Performance

We are using DBbench to test performance and progress between the versions. It is available under tools and also in the artifact for direct download. In there you can also find a readme with the commands we are using to get you started.

Documentation

You can find a detailed description of all Speedb features here.

Speedb's documentation repository allows you to enhance, add content and fix issues.

πŸ›£οΈ Roadmap

The product roadmap provides a snapshot of the features we are currently developing, what we are planning for the future, and the items that have already been delivered.

We have added a column with items that are awaiting community feedback. We invite you to participate in our polls inside, share your thoughts about topics that are important to you, and let us know if there is anything else you would like to see on the list.

❔ Questions

  • For live discussion with the community you can use our official Discord channel.
  • For technical questions and discussions you can use our official Discourse forum.

🌎 Join us

Speedb is committed to a welcoming and inclusive environment where everyone can contribute.

Contributing code

See the contributing guide.

License

Speedb is open-source and licensed under the Apache 2.0 License.

About

Speedb is a library that provides persistent key-value stores.

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C++ 84.0%
  • Java 9.4%
  • C 2.6%
  • Python 1.8%
  • Shell 0.9%
  • Makefile 0.7%
  • Other 0.6%