The Spectrum Dispatch News

technology

TurboKV: Async Embedded Key-Value Store for Rust

The library offers atomic batches, ordered range scans, configurable durability, compression, and background compaction.

TurboKV: Async Embedded Key-Value Store for Rust

TurboKV is an async embedded key‑value store written in Rust that provides atomic batches, ordered range scans, configurable durability, compression, and background compaction, according to the source documentation. The library can be added to a Cargo project with cargo add turbokv and requires the Tokio runtime with the full feature set. To open a database, developers call Db::open_with_options or the convenience Db::open method, which exclusively owns the data directory until the handle is closed. The source shows three durability presets: DbOptions::fast() provides in‑memory visibility without a write‑ahead log (WAL), suitable for caches; DbOptions::durable() appends mutations to the WAL without a per‑write sync, offering process‑crash recovery; and DbOptions::paranoid() waits for a WAL group sync before acknowledging a write, delivering the strongest guarantee subject to filesystem limits. All presets start with a 64 MiB memtable, a 64 MiB block cache, and LZ4 compression. These values can be tuned via fields such as memtable_size, block_cache_size, and the compression option, which accepts LZ4, Snappy, Zstd, or None. The documentation notes that enabling AES instructions via RUSTFLAGS='-C target-feature=+aes,+sse2' (x86) or +aes,+neon (ARM) allows the persisted Bloom‑filter format to use hardware acceleration, or alternatively -C target-cpu=native when the binary runs on the same CPU model. Mutation APIs include insert, insert_many, remove, and write_batch. The write_batch method publishes a group of puts and deletes atomically; readers see either the state before the batch or the complete batch. Point reads (get) return an owned Vec<u8> or None for missing/deleted keys, while an empty value is represented as Some(vec![]). Range queries are available through range, scan_prefix, and their streaming iterator counterparts (range_iter, scan_prefix_iter). Iterators provide methods such as collect_pairs, keys, and paginate for lazy traversal. Guard APIs let a borrow of the key be obtained without loading the value, and guard.value() or guard.value_len() borrows the value or reports its length, copying a memtable value only on first request. The source also lists example projects demonstrating basic operations, batch writes, range queries, concurrent access, persistence with WAL recovery, and configuration of cache, memtable, and compression options. A clean shutdown requires calling close() or close_with_status(); dropping the handle does not guarantee a clean shutdown. Additionally, with the WAL enabled, one record or complete batch must fit in the WAL’s u32 payload length; a failed or cancelled mutation may already have reached the WAL, so the key should be inspected or the database reopened before retrying a non‑idempotent operation. WriteBatch owns copies of every key and value, and batch.ops() borrows the ordered operation list for inspection.

TurboKV: Async Embedded Key-Value Store for Rust

Key facts

Sources

← All posts