EdgeStore's public API surface is intentionally small. The core type is Engine — everything else is configuration, traits, or return types.
For full rustdoc, see docs.rs/edgestore.
The single-writer KV engine. All mutations go through one Engine instance per database directory.
Engine::open(config: EdgestoreConfig) -> Result<Engine, EdgestoreError>
Open or create a database at the path specified in config. Acquires a file lock to enforce single-writer semantics.
engine.put(ns, key, val) -> Result<Lsn, EdgestoreError>
Write a key-value pair in the given namespace. Appends to WAL, then memtable. Returns the assigned LSN.
engine.put_with_ttl(ns, key, val, ttl_seconds) -> Result<Lsn, EdgestoreError>
Write with a time-to-live. Expired via deathtime-cohort compaction, not per-read checks.
engine.get(ns, key) -> Result<Option<Vec<u8>>, EdgestoreError>
Point lookup. Checks memtable first, then segments in LSN order (newest wins).
engine.delete(ns, key) -> Result<Lsn, EdgestoreError>
Write a delete tombstone. Visible to subsequent reads; removed at compaction.
engine.range(ns, start, end) -> Result<Vec<(Vec<u8>, Vec<u8>)>, EdgestoreError>
Range scan with exclusive end bound. Returns latest LSN values across memtable and all segments.
engine.prefix(ns, prefix) -> Result<Vec<(Vec<u8>, Vec<u8>)>, EdgestoreError>
Prefix scan. Efficient for time-series keys, hierarchical data, etc.
engine.flush() -> Result<(), EdgestoreError>
Force WAL fsync. Does not flush memtable to segments.
engine.flush_to_segments() -> Result<SegmentMeta, EdgestoreError>
Flush memtable to an immutable segment on disk. Creates .dat, .idx, .xf, and .meta files. On AsyncEngine, uses the 3-phase lock-free protocol — the write lock is released during disk I/O so reads are not blocked.
engine.begin_flush() -> Option<FlushWork>
v2.2.0 Phase 1 of the 3-phase flush. Snapshots memtable entries without clearing them. Returns None if empty. The caller must call finalize_flush with the returned FlushWork::flushed_max_lsn.
engine.finalize_flush(meta, flushed_max_lsn) -> Result<(), EdgestoreError>
v2.2.0 Phase 3 of the 3-phase flush. Registers the completed segment, rebuilds the memtable (retaining only entries written after the snapshot), and retires old WAL files.
engine.compact_once() -> Result<CompactionStats, EdgestoreError>
Run one deathtime-cohort compaction cycle. Budget-bounded; stops after first partial cohort if write budget exhausted.
engine.snapshot() -> Result<Snapshot, EdgestoreError>
Capture a point-in-time view of current segments. RAII release on Drop.
engine.metrics() -> MetricsSnapshot
Return current engine metrics: segment count, memtable entries, WAL files, total bytes.
use edgestore::EdgestoreConfig;
let config = EdgestoreConfig::new("/tmp/mydb")
.with_wal_max_bytes(64 * 1024 * 1024) // 64 MB WAL rotation
.with_wal_max_seconds(60) // or 60 seconds
.with_segment_target_size(16 * 1024 * 1024) // 16 MB segments
.with_cohort_window_secs(3600) // 1 hour cohorts
.with_compression_wal(edgestore::Compression::Lz4)
.with_compression_segment(edgestore::Compression::Zstd { level: 1 });engine.begin() -> Transaction
Start a new multi-record transaction batch.
tx.put(ns, key, val, ttl, timestamp) -> Result<(), EdgestoreError>
Add a put to the transaction. Accumulated in memory until commit.
engine.commit_transaction(tx) -> Result<Lsn, EdgestoreError>
Atomically commit all transaction records to WAL with a single fsync.
engine.rollback_transaction(tx)
Discard the transaction without writing anything.
Every read has a _with_stats twin that returns a QueryStats alongside the result. Bounded variants stop early when a ScanBudget is exceeded.
engine.get_with_stats(ns, key) -> Result<(Option<Vec<u8>>, QueryStats), EdgestoreError>
Point lookup with cost reporting. Returns zero stats on cache miss, 1 item + byte count on hit.
engine.range_with_stats(ns, start, end) -> Result<(Vec<(Vec<u8>, Vec<u8>)>, QueryStats), EdgestoreError>
Range scan that accumulates bytes_scanned and items_examined as it iterates.
engine.prefix_with_stats(ns, prefix) -> Result<(Vec<(Vec<u8>, Vec<u8>)>, QueryStats), EdgestoreError>
Prefix scan with cost accounting.
engine.range_budgeted(ns, start, end, budget: ScanBudget) -> Result<BudgetedScan, EdgestoreError>
Range scan that stops when max_items or max_bytes is reached. BudgetedScan.truncated is true when cut short. For cursor-based pagination use range_page instead.
engine.prefix_budgeted(ns, prefix, budget: ScanBudget) -> Result<BudgetedScan, EdgestoreError>
Prefix scan with early termination on budget exhaustion.
engine.vector_search_with_stats(ns, query, k, metric) -> Result<(Vec<VectorSearchResult>, QueryStats), EdgestoreError>
ANN / flat-scan search with byte and segment accounting.
text.search(&engine, ns, query, results_limit) -> Result<Vec<TextSearchResult>, EdgestoreError>
BM25 text search via edgestore-text::TextIndex (text must be let mut). For cost accounting on the text path, combine with engine-level stats (segments scanned, bytes read) from surrounding KV operations.
Bounded page-by-page iteration over a key range. Each call reads only the segment blocks that overlap the current page window and stops at the I/O level once page_size live items are collected. Pass next_key from the result as cursor on the next call; next_key = None means the scan is exhausted.
engine.range_page(ns, start, end, cursor: Option<&[u8]>, page_size) -> Result<RangePage, EdgestoreError>
Forward cursor scan. Items returned in ascending key order. First call: cursor = None. Subsequent calls: cursor = page.next_key.as_deref().
engine.range_rev_page(ns, start, end, cursor: Option<&[u8]>, page_size) -> Result<RangePage, EdgestoreError>
Reverse cursor scan. Items returned in descending key order. next_key is the smallest key on the current page; pass it as cursor to continue scanning leftward. Efficient for latest-N queries when keys encode time.
RangePage.items: Vec<(Vec<u8>, Vec<u8>)>
Decoded (key, value) pairs. Ascending for range_page, descending for range_rev_page.
RangePage.next_key: Option<Vec<u8>>
Cursor for the next call. None when all items in the range have been returned.
// Forward pagination — ascending, page_size=100
let mut cursor: Option<Vec<u8>> = None;
loop {
let page = db.range_page(b"events", b"", b"\xff", cursor.as_deref(), 100)?;
for (key, val) in &page.items { process(key, val); }
cursor = page.next_key;
if cursor.is_none() { break; }
}
// Reverse pagination — latest-N, descending
let mut cursor: Option<Vec<u8>> = None;
loop {
let page = db.range_rev_page(b"logs", b"", b"\xff", cursor.as_deref(), 50)?;
for (key, val) in &page.items { process_newest_first(key, val); }
cursor = page.next_key;
if cursor.is_none() { break; }
}VectorEngine::vector_put(ns, key, dims, dtype, data) -> Result<(), EdgestoreError>
Store a typed vector. Uses a synthetic __vec__{ns} namespace underneath.
VectorEngine::vector_get(ns, key) -> Result<Option<VectorRecord>, EdgestoreError>
Retrieve a vector by key.
VectorEngine::vector_delete(ns, key) -> Result<(), EdgestoreError>
Remove a vector from the collection.
engine.vector_search(ns, query, k, metric) -> Result<Vec<VectorSearchResult>, EdgestoreError> v1.8.0: &self
Search top-k nearest neighbors. Routes to HNSW if an index exists, otherwise flat scan. Takes &self — concurrent reads do not block each other. The HNSW cache uses interior RwLock for lazy loading so the outer engine lock stays shared.
engine.try_hnsw_search(ns, query, k) -> Result<Option<Vec<VectorSearchResult>>, EdgestoreError> v1.8.0
Attempt the HNSW path only. Returns None if no fresh index is available, Some(results) otherwise. Used by AsyncEngine to try the fast path under a read lock before falling back to a paged flat scan.
engine.build_vector_index(ns) -> Result<(), EdgestoreError>
Build an HNSW approximate index for the collection. Persists to a sidecar file. Takes &mut self — this is a write operation.
engine.preload_vector_index(ns) -> Result<bool, EdgestoreError> v1.8.0: &self
Load the HNSW index from disk into the interior cache. Returns true if loaded, false if no index exists. Now takes &self — safe to call from a shared reference.
engine.vector_count(ns) -> Option<u64> v1.6.0
Returns the number of vectors in the HNSW cache for the namespace if the index is loaded, None otherwise. Never triggers a disk scan. Call preload_vector_index first if you need a guaranteed count. Takes &self.
engine.strip_vector_index(segment_id: u64) -> Result<SegmentMeta, EdgestoreError> v1.5.0
Remove all __vec__ namespace records from the named segment, returning a new segment (with a new ID) that contains only KV data. The original segment is deleted. Idempotent when called on the returned segment ID.
Dtype
F32, F16, I8 — the vector element type. All distances computed in f32 (widening for f16/i8).
Metric
Cosine, L2 (Euclidean), DotProduct — three supported distance metrics.
Full-text search lives in the edgestore-text crate. Add edgestore-text = "2.1" to your Cargo.toml. Declare TextIndex as let mut text = TextIndex::new() — all search methods take &mut self to cache WAL-reconstructed indices on first use. persist() writes a sidecar for faster cold-start reads; WAL reconstruction provides crash safety even without it.
TextIndex::new() -> TextIndex
Create a new in-memory text index (English tokenizer). Use TextIndex::with_language(lang) for other Snowball-supported languages.
text.index_document(&mut engine, ns, key, text, facets) -> Result<u64, EdgestoreError>
Tokenize and index a document. Stores the inverted index sidecar in the __text__{ns} namespace. Pass HashMap::new() for no facets.
text.search(&engine, ns, query, results_limit) -> Result<Vec<TextSearchResult>, EdgestoreError>
BM25-ranked search; takes &mut self (caches WAL-reconstructed index on first call). Returns up to results_limit results with doc_id: Vec<u8> and score: f32. Tokenizes the query, intersects posting lists, scores with BM25(k1=1.2, b=0.75).
text.search_with_stats(&engine, ns, query, results_limit) -> Result<(Vec<TextSearchResult>, TextSearchStats), EdgestoreError>
BM25 search with scan statistics: total_docs_indexed, docs_examined (matched before cutoff), bytes_scanned. Useful for diagnosing empty results — if docs_examined == 0 and total_docs_indexed > 0, the query terms are simply absent from the corpus.
text.search_with_options(&engine, ns, query, &SearchOptions) -> Result<Vec<TextSearchResult>, EdgestoreError>
Full-control search. SearchOptions fields: limit: usize (max results), facet_filters: Vec<FacetFilter>, typo_tolerance: bool (edit-distance ≤ 1 expansion). Example: SearchOptions { limit: 10, typo_tolerance: true, facet_filters: vec![] }.
text.search_with_snippets(&engine, ns, query, results_limit, context_chars) -> Result<Vec<SnippetResult>, EdgestoreError>
BM25 search with context windows around matched terms; takes &mut self. Each SnippetResult contains doc_id, score, and snippets: Vec<Snippet>. Each Snippet has text (context window), byte_start, byte_end (match offsets within text).
text.delete_document(&mut engine, ns, key) -> Result<u64, EdgestoreError>
Remove a document from the text index. Writes a tombstone to the WAL immediately; the sidecar is updated on the next persist() call. WAL reconstruction skips tombstoned documents, so the deletion is crash-safe without an explicit persist().
text.persist(&mut engine) -> Result<(), EdgestoreError>
Optional: write the inverted-index sidecar to the engine WAL for faster cold-start reads. Call once after a batch of index_document calls. No-op when nothing changed. Without persist(), the index is rebuilt from raw WAL records on next cold start — correct but slightly slower (one engine.prefix() scan per namespace).
SearchOptions { limit, facet_filters, typo_tolerance }
Passed to search_with_options. All fields: limit: usize (max results to return), facet_filters: Vec<FacetFilter> (narrow by facet value), typo_tolerance: bool (match within edit distance 1). Use ..Default::default() to fill unset fields.
Posting { doc_id, term_freq, doc_len_tokens, facets, positions }
A posting in the inverted index. doc_len_tokens is the token count of the document (not bytes) — used by BM25 for length normalization. Exported for custom scoring via bm25_score() or score_document().
engine.export_manifest() -> Result<Vec<SegmentRef>, EdgestoreError>
Export the current live segment list with BLAKE3 hashes. Send this to a replica to compare.
engine.missing_segments(peer_segments) -> Vec<[u8; 32]>
Compare local segments against a peer's manifest. Returns BLAKE3 hashes of missing segments.
engine.import_segment(seg_ref, data) -> Result<ImportResult, EdgestoreError>
Import a segment from a peer. Applies LWW conflict resolution by wall-clock timestamp. Verifies BLAKE3 hash.
engine.range_merkle_root() -> Result<[u8; 32], EdgestoreError>
Compute the root hash of the range-level Merkle tree over all segments.
engine.compare_merkle(other_root) -> Result<bool, EdgestoreError>
Compare local Merkle root with a peer's. Returns true if identical.
engine.confirmed_lsn() -> u64 v1.9.0
Highest LSN whose segment is durably flushed. Replicas use this to know what is safe to serve. Returns 0 when no segments exist yet.
engine.write_token() -> u64 v1.9.0
Current write-fencing token. Monotonically increasing; persisted to WTOKEN. A replica promoted to primary increments this token so anti-entropy loops detect the failover.
engine.set_write_token(token: u64) -> Result<(), EdgestoreError> v1.9.0
Persist a new write token atomically. Use via ReplicatedEngine::promote_to_primary which enforces the monotonic invariant.
GET /watermark → WatermarkResponse { confirmed_lsn, wtoken } v1.9.0
New HTTP endpoint on HttpReplicationServer. Returns the current commit watermark and write token. Polled by AntiEntropyLoop each cycle.
ReplicatedEngine::promote_to_primary(config, bind_addr, old_primary_url) v1.9.0
Fence the old primary, increment write token, start the replication server. Pass None for old_primary_url when the old primary is already down.
remote_store.upload_if_absent(hash, data) -> Result<bool, EdgestoreError> v1.9.0
Idempotent-create on RemoteStore. Returns true on first write, false when already present. Filesystem: O_CREAT|O_EXCL. S3: If-None-Match: * (HTTP 412 → false).
ReplicationProtocol::watermark() -> Result<WatermarkResponse, EdgestoreError> v1.9.0
New method on the ReplicationProtocol trait. Default returns Err(InvalidOperation) for backward compatibility. HttpReplicationClient implements it against GET /watermark.
TieredEngine::new(local: Engine, remote: Box<dyn RemoteStore>) -> TieredEngine
Wrap a local engine with a remote store. Reads fall through to remote on local miss. Writes always go to local only.
tiered.get(ns, key) -> Result<Option<Vec<u8>>, EdgestoreError>
Local-first read. On miss, downloads the matching archived segment ephemerally from S3 and serves from it.
tiered.archive_segments(metas: &[SegmentMeta]) -> Result<(), EdgestoreError>
Upload the named segments to the remote store. Does not delete local copies — caller decides when to prune.
tiered.local() -> &Engine / tiered.local_mut() -> &mut Engine
Direct access to the underlying local engine for operations that don't need remote fallback.
tiered.get_needs_archived_fetch(ns, key) -> bool
Returns true if answering this get would require fetching an archived segment. Used by AsyncTieredEngine to decide whether to escalate from read lock to write lock.
AsyncTieredEngine::flush_notify() -> Arc<tokio::sync::Notify>
Returns a Notify handle wired to the engine's flush callback. Race this against a polling interval to trigger archiving immediately after a segment is written.
TieredEngine::with_vector_stripping(self) -> TieredEngine v1.5.0
Builder method that wires strip_vector_index into the archive_segments path. When enabled, every segment uploaded to the remote store automatically has its __vec__ records removed from the local copy, reclaiming disk proportional to vector data size.
EdgestoreError
All errors: Io, KeyNotFound, Corruption, CompactionError, InvalidArgument, etc.
Snapshot / SnapshotRegistry
Point-in-time reads with RAII segment pinning. Registry tracks active snapshots for compaction exclusion.
CompactionStats
cohorts_collected, segments_removed, segments_written, bytes_written, live_records_relocated.
StorageBackend / DefaultStorageBackend / MemoryStorageBackend
Pluggable I/O trait. Default uses pread/pwrite. Memory backend for tests.
HnswIndex
Standalone HNSW graph. Can be serialized/deserialized for persistence. Used by Engine for vector_search routing.
RangeMerkleTree
Merkle tree over key ranges for anti-entropy probes and delta sync.
QueryStats v1.5.0
segments_scanned: u64, bytes_scanned: u64, items_examined: u64. Returned by all _with_stats read variants. Zero-valued on a miss that short-circuited before touching segments.
ScanBudget v1.5.0
max_items: usize, max_bytes: u64. Passed to range_budgeted / prefix_budgeted. The scan stops when either limit is reached, whichever comes first.
BudgetedScan v1.5.0
items: Vec<(Vec<u8>, Vec<u8>)>, truncated: bool, next_key: Vec<u8>. When truncated is true, pass next_key as the start of the next call to continue pagination.
SnippetResult / Snippet v1.5.0
SnippetResult{doc_id, score, snippets: Vec<Snippet>}. Each Snippet{term, span, context_start, context_end} holds the matched text and its character-accurate context window in the original document.
use edgestore_tokio::AsyncEngine;
let engine = AsyncEngine::open(EdgestoreConfig::new("/tmp/async_db")).await?;
engine.put(b"default", b"key", b"val").await?;
let val = engine.get(b"default", b"key").await?;All Engine methods are wrapped with tokio::task::spawn_blocking. No async code in the core library — the wrapper is thin and optional.