Research & Papers

This paper is the single most important influence on EdgeStore's storage engine. The authors demonstrate that traditional LSM compaction (level-based, size-tiered) suffers from high write amplification because it repeatedly rewrites live data alongside tombstones and expired records.

EdgeStore implements their cohort-grouped approach directly:

• Every record is assigned a cohort bucket = death_time / cohort_window (default 1 hour)
• Records with TTL cluster with others expiring in the same window
• Records without TTL fall back to write-time cohorts (temporal locality)
• Compaction prioritizes fully-expired cohorts first — no live-record reads or writes
• Partially-expired cohorts get one merge pass; only live records are relocated

The paper measured a 7.8x reduction in write amplification on TTL workloads compared to RocksDB. EdgeStore's design targets the same asymptotic behavior.

Durner's analysis showed that the write amplification problem is worse than previously understood: it's not just the database's fault, but a compound effect between application WA and SSD firmware WA.

EdgeStore's design directly addresses both layers:

• Append-only writes — never rewrite a page in place. SSD firmware sees sequential, full-page writes instead of scattered updates.
• 4 KiB block alignment — segment blocks match SSD page boundaries, eliminating read-modify-write cycles.
• Cohort invalidation alignment — when a cohort expires, all pages in those segments become invalid simultaneously. The SSD GC can reclaim entire erase blocks without copying valid remnants.

The compound effect is dramatic:

RocksDB:    30x (app WA) × 3x (SSD GC) = 90x total device WA
EdgeStore:   1x (app WA) × 1.2x (SSD GC) = 1.2x total device WA

SlateDB is an open-source cloud-native LSM that influenced EdgeStore's segment format and manifest design. While EdgeStore uses deathtime-cohort compaction instead of SlateDB's level-based approach, the segment file structure and manifest patterns draw directly from SlateDB's clean separations:

• .dat file — sorted key-value blocks, ZSTD compressed, 4 KiB aligned
• .idx file — sparse index with stride-64 sampling for efficient seeks
• .xf file — xor filter (not Bloom) for fast negative lookup checks
• .meta file — JSON metadata with cohort_bucket, death_time, min/max keys, BLAKE3 hash
• manifest.mf — append-only, CRC32C checksummed, live segment registry

SlateDB's insight that "segments should be immutable, self-describing, and content-addressed" became a core EdgeStore principle.

EloqData's architecture separates hot NVMe storage from cold S3 archive using a unified object model. EdgeStore adopts the same separation principle in its StorageBackend trait:

• DefaultStorageBackend — local pread/pwrite on NVMe/SSD
• MemoryStorageBackend — in-memory for tests and ephemeral workloads
• FdpStorageBackend — NVMe 2.0 FDP placement hints for cohort-aligned writes
• RemoteStore trait — content-addressed upload/download for S3 or HTTP replication

The key insight: segments are immutable blobs that can live on any backend. A segment written to local NVMe can be uploaded to S3 unchanged, then downloaded and imported into another node. Content addressing (BLAKE3) guarantees integrity across transports.