VectorForge: Vector Database in C++17 — Siddhesh More

A vector database written from scratch in C++17 to find out what a production one actually has to get right: not just fast nearest-neighbour search, but staying correct when the process is killed mid-write, when filters are selective, and when readers and writers run at the same time. FAISS and hnswlib are used only as baselines to measure against.

The Approach

The index is a hand-written HNSW graph. Around it sit memory-mapped on-disk segments, a write-ahead log, crash recovery, tombstone deletes with compaction, a metadata filter planner, and a BM25 text index. Python bindings (pybind11), a FastAPI REST server and a CLI sit on top. Not built: IVF-PQ, quantization, sharding, replication, snapshots. The largest test set is SIFT1M (1M vectors), not billion-scale.

How is the database layered?

Clients
Python bindings, REST server, CLI
Collection
CRUD, tombstones, filter planner
HNSW + BM25
hand-written vector and text indexes
Write-ahead log
replayed on recovery
mmap segments
on-disk format, compaction

The stack from the client down to disk. Everything below the client is C++17; FAISS and hnswlib are never used inside it.

The Results

Compared at matched recall, single thread, one query per call, on SIFT1M. At 0.95 recall VectorForge served 17,719 QPS against 13,002 for FAISS and 7,650 for hnswlib, and built the index in 134 s against 190 s and 296 s. On 1,024-d embeddings from my FilingLens project it ran 1.05-1.35x FAISS. At 4,096-d FAISS was ahead (0.83-0.99x).

Profiling-driven changes added up: reserving vector storage up front cut memory from 7,054 to 4,109 bytes per vector, and computing four distances per step gave 1.32x more QPS at 1,024-d and 1.39x at 4,096-d at the same recall.

Filtered search is where a naive design quietly breaks. Searching first and dropping non-matching results returned almost nothing useful when the filter was selective; a planner that chooses the strategy per query held recall at 0.999 or better.

Staying Correct

Durability was tested by killing the process with kill -9 across 1,400 cycles in three runs (the last two on the final storage code), some with the log truncated to simulate power loss, with zero failures. The log was also truncated at every byte offset and fuzzed with 20,000 cases, and the C++ tests (35 cases, 350,320 assertions) are clean under ASan, UBSan and TSan. Read throughput scaled from 11.8k to 43.1k QPS going from 1 to 8 threads.

The Honest Parts

The speed numbers are one laptop with background load, and the FAISS control itself moved 0.7% between runs. FAISS wins on 4,096-d. For exact search, NumPy and FAISS beat this implementation in batch (7,714 and 29,139 QPS against 747) because they use matrix routines. With a writer flat out at about 3.2k docs/s, readers fall to 1.7k QPS because there is a single shared lock. After correlated deletes, recall dropped from 0.998 to 0.992 at ef=64 and recovered to 0.997 at ef=128, and I did not establish why. LeakSanitizer is unavailable on macOS arm64, and the sanitizers cover the C++ tests, not the Python extension. A C++ BM25 matched the Python scores and top-10 results from FilingLens at 60 us per query versus 5,400 us, but it only handles ASCII and has no deletes. The project is built and tested but not deployed.