A learning project · Go
MiniKV
I am building a small key/value storage engine to understand how an LSM tree works, from an in-memory skip list to persistence and recovery.
The implementation is handwritten and uses only Go’s standard library. AI writes tests and guides the learning through the Coach skill.
What works today
The first four phases are complete: an in-memory memtable, sorted files on disk, automatic flushing at a memory threshold, and persistent deletions.
- Put and Get. Store byte values under string keys, and replace existing values.
- Persistence. Flush sorted entries to immutable SSTables and read them after reopening.
- Automatic flushing. Estimate memory usage and flush synchronously when the threshold is exceeded. Keep the memtable if a flush fails.
- Deletes and scans. Preserve tombstones across restarts. Return sorted range results with the newest value for each key.
From memory to disk
Writes enter a skip list in memory. A flush writes its entries in key order to a temporary file, syncs the contents, renames the file, and syncs the directory.
Reads check memory first, then search SSTables from newest to oldest. A tombstone marks a deleted key and stops an older value from appearing again.
Each record stores a type byte, explicit key and value lengths, and the key and value bytes. There are no external dependencies.
Learning by building
I write the storage logic myself. The agent provides hints and tests that check behavior: reopen the database, force several flushes, preserve deletes, or fail a flush and recover.
The project is educational. It is single-threaded, and writes still in memory can be lost in a crash. The next phase adds a write-ahead log for recovery.
Run the checks
From the repository, run the tests or measure database operations:
go test ./...
go test ./minikv -run '^$' -bench . -benchmemBenchmarks cover memory reads, disk reads across multiple SSTables, missing keys, and flushing on close. Disk reads use the operating system’s normal file cache.