Build a Real-Time, Self-Healing Distributed Database from Scratch — No Existing Libraries

I need a distributed, strongly-consistent key-value database built entirely from scratch — no Redis, no etcd, no Raft libraries, no SQLite, no LevelDB, no RocksDB. You may only use a standard HTTP/TCP socket library and a language's standard library.

Core Requirements:

1. Custom Raft implementation — leader election, log replication, snapshotting, membership changes. Must survive network partitions, node crashes, and clock skew. No copying existing Raft code; I will ask you to explain every design decision live.
2. Custom storage engine — LSM-tree or B+tree with WAL, compaction, and crash recovery. Must handle 50k+ writes/sec on a single node with fsync enabled.
3. Linearizable reads — implement read-index or lease-based reads. Prove it with a Jepsen-style test harness you also write.
4. Multi-region replication — 3 regions, simulated 200ms latency, must maintain consistency under partition. Demonstrate with a chaos test.
5. Zero-downtime rolling upgrades — version compatibility protocol you design yourself.
6. Observability — Prometheus metrics, structured logs, and a CLI that can inspect cluster state.

Deliverables:

· Full source code with your own test suite (>80% coverage)
· A written design doc explaining tradeoffs
· A video demo of you killing nodes and the cluster recovering
· A 1-hour live technical defense of your architecture

You will NOT be selected if:

· You paste a GitHub repo you didn't write
· You can't explain CAP theorem tradeoffs in your specific design
· You use "we" when you mean "I"
· Your proposal is a template

To apply, answer these 3 questions in your proposal:

1. What's the hardest bug you've debugged in a distributed system, and how did you find it?
2. In your Raft implementation, how will you handle a leader that's partitioned but still thinks it's leader?
3. Why is fsync-per-write a lie on most cloud VMs, and how would you handle that?

Proposals that don't answer all three will be ignored.