Replace journal backend with key-value storage
What
Replace custom journal backend with some proven key-value storage as a prototype for further implementation.
Why
The current backend has several limitations
- several 'wontfix' bugs
- reinventing the wheel
- can't accept more than fixed number of changes
- can't clean up nodes after failed transactions
- very limited recovery
- not parallel
- not centralized, can't store things like timers
- linear lookup
- problematic range lookup
Proposal
Replace with proven key-value storage, which could also have central storage for persistent data like timers, DNSSEC-related stuff etc. The http://symas.com/mdb/ seems like a very good candidate, proven with OpenLDAP and supporting all the features we want - growing the storage, updates, transactions, parallel access, range lookup, direct write through, file size limits, small codebase and good license etc. without the overhead of database.
Performance noted here: http://symas.com/mdb/microbench/
Quick example usage: https://gitorious.org/mdb/mdb/blobs/mdb.master/libraries/liblmdb/sample-mdb.c
Notes and remarks.
This is open for discussion. I know how you can feel strongly against this, so please restrict to constructive arguments and have a look at it before writing.