Journal lmdb joint

Proposal of LMDB-based journal.

Key features:

• all journals for all zones are in a single LMDB database (defaults to storage/journal; 1G size)
• the occupied space is measured per zone
• old changesets get preserved after zone flush until we run out of space
• if zone flushing is disabled and journal gets full, it tries to free up space by merging older changesets
• all changes are done by transactions, resulting in always-consistent DB (but some mutexes still necessary for opening DB && for keeping zone contents consistent with journal)
• kjournalprint provides a way to list zones in journal
• old journal is automatically imported, but the configuration needs to be updated manually

Caveats:

• not tested enough, needs experience from some "real" env

added 1 commit

• eb68ae54 - little fixup - codestyle and zone-purge

Compare with previous version

We need to resolve Caveat#2 by add configuration parameter: allowed DB usage of journal per-zone. There are some ideas, but no definitive proposal. <-- TODO!

added 3 commits

• ccb00df3 - fixup: proper handling of mapsize
• 6afdaab1 - journal: added >>max usage<< conf parameter
• 4c4c64f8 - added max-journal-depth; fixed journal limit on 32bit

Compare with previous version

added 1 commit

• 9f3f4705 - some fixup

Compare with previous version

added 7 commits

• 3ddc5d86 - test: test_conf() can be called repeatedly w/o leak
• 7ca7c303 - Journal: re-implemented completely, LMDB, all zones in one db
• 0b810b88 - journal_lmdb: performance loading many zones
• 2c933b9f - journal: added max-journal-usage and max-journal-depth conf parameters
• 093d2243 - Kjournalprint utility adaptation to new journal
• d7352dab - Kjournalprint: added option to list zones in journal DB
• bc820074 - fixup 2B rebased

Compare with previous version

added 8 commits

• bc14882f - Journal: re-implemented completely, LMDB, all zones in one db
• 85cde961 - journal_lmdb: performance loading many zones
• 66195a39 - journal: added max-journal-usage and max-journal-depth conf parameters
• 456b47d4 - journal: unit test: valgrind (not squashable)
• bcedc028 - journal: mostly coding-style fixes, not squashable
• 44787ccf - Kjournalprint utility adaptation to new journal
• 667461c7 - Kjournalprint: added option to list zones in journal DB
• 32605001 - doc: update to track journal_lmdb refactoring

Compare with previous version

added 24 commits

• 32605001...d6fe0f9e - 14 commits from branch master
• da8fef03 - Revert "namedb/lmdb: make sure the freelist can always hold db tree index"
• b7a79089 - Journal: re-implemented completely, LMDB, all zones in one db
• a9a424a2 - journal_lmdb: performance loading many zones
• d0157203 - journal: added max-journal-usage and max-journal-depth conf parameters
• ef76fd6c - journal: unit test: valgrind (not squashable)
• 5ef0fb44 - journal: mostly coding-style fixes, not squashable
• 894871f3 - Kjournalprint utility adaptation to new journal
• fe697a25 - Kjournalprint: added option to list zones in journal DB
• f6a73a93 - doc: update to track journal_lmdb refactoring
• edbbae73 - journal: added obsolete journal import; definitive configuration for journal

Compare with previous version

added 11 commits

• 44f7f034 - 1 commit from branch master
• 422b1caf - Revert "namedb/lmdb: make sure the freelist can always hold db tree index"
• d4fbb46b - Journal: re-implemented completely, LMDB, all zones in one db
• 34a5e917 - journal_lmdb: performance loading many zones
• 11c93df3 - journal: added max-journal-usage and max-journal-depth conf parameters
• 9d0226b1 - journal: unit test: valgrind (not squashable)
• 01f6795c - journal: mostly coding-style fixes, not squashable; version metadata becomes string
• c04696cc - Kjournalprint utility adaptation to new journal
• 260373d6 - Kjournalprint: added option to list zones in journal DB
• f40d60de - doc: update to track journal_lmdb refactoring
• 96838ab2 - journal: added obsolete journal import; definitive configuration for journal

Compare with previous version

unmarked as a Work In Progress

added 1 commit

• 7be84a71 - journal: added unused 4bytes to DB headers for future use

Compare with previous version

added 1 commit

• f09661ed - journal: added unused 4bytes to DB headers for future use

Compare with previous version

added 19 commits

• f09661ed...dfbd99b7 - 8 commits from branch master
• 357ea586 - Revert "namedb/lmdb: make sure the freelist can always hold db tree index"
• 91076f0c - Journal: re-implemented completely, LMDB, all zones in one db
• 6c1c83f0 - journal_lmdb: performance loading many zones
• 8cf58d8b - journal: added max-journal-usage and max-journal-depth conf parameters
• 630842ae - journal: unit test: valgrind (not squashable)
• 1488f37e - journal: mostly coding-style fixes, not squashable; version metadata becomes string
• dec86da4 - Kjournalprint utility adaptation to new journal
• 80ae2f07 - Kjournalprint: added option to list zones in journal DB
• 2de91308 - doc: update to track journal_lmdb refactoring
• 0b8c079e - journal: added obsolete journal import; definitive configuration for journal
• 70efd47e - journal: added unused 4bytes to DB headers for future use

Compare with previous version

added 2 commits

• 0676b746 - fixup! journal: added obsolete journal import; definitive configuration for journal
• 7bd2b0c7 - WIP

Compare with previous version

added 1 commit

• 9dd46fd4 - conf: tiny scheme cleanup

Compare with previous version

added 3 commits

• d5736712 - conf: tiny schema cleanup
• c1293630 - WIP
• 09777558 - a

Compare with previous version

added 14 commits

• 6a3fa81f - Journal: re-implemented completely, LMDB, all zones in one db
• ee0e5dd8 - journal_lmdb: performance loading many zones
• 53c6f598 - journal: added max-journal-usage and max-journal-depth conf parameters
• 36f8cf65 - journal: unit test: valgrind (not squashable)
• db60adfc - journal: mostly coding-style fixes, not squashable; version metadata becomes string
• b514bd1e - Kjournalprint utility adaptation to new journal
• b019a788 - Kjournalprint: added option to list zones in journal DB
• 048a75a7 - doc: update to track journal_lmdb refactoring
• a373eebd - journal: added obsolete journal import; definitive configuration for journal
• a12807d7 - journal: added unused 4bytes to DB headers for future use
• bf8ddc30 - journal: serialization refactored; old_journal-related serialization moved
• 745066df - journal: large code cleanup; identifiers renamed etc
• fc29c744 - journal: serialization able to chunk split even rrset, just rdata needs to be compact
• 5ab287be - journal: allow DB path reconfigure if not open yet

Compare with previous version