cache garbage collector

Experimental code for cache GC.

I've set the defaults to values that seemed to work well (clear about ~10% of cache when 80% full, use 100 operations per transaction). Higher values for allowed cache usage didn't seem to properly prevent complete cache flush. High values of ops per transaction (or unlimited) caused issues as well.

Closes #257 (closed)

I should review this before merging. It certainly seems to be missing changes to consider aggressive NSEC3 in cache (probably merged after most of GC work being done).

assigned to @ljezek

Current WIP is also missing a systemd unit for the gc. It'd be easiest to go with a dedicated unit file (example), but since we might need a "maintenance" service (#459) eventually, I'm not sure it's the best solution.

Source codes seems good.

assigned to @tkrizek

changed milestone to %4.1.0

added 73 commits

• 360311a5...603a24fc - 54 commits from branch master
• 1d3c8b41 - kr_cache_gc: initial commit
• 1184b665 - kr_cache_gc: UI improvements
• ef0b1b73 - kr_cache_gc: separated main function
• d5caab14 - kr_cache_gc: added optional interval of infinite periodic action
• 6dbe9548 - kr_cache_gc: separated DB transactions to prevent blocking; config; bugfix
• 70cdcaaf - kr_cache_gc: little improvements
• 5153af5b - kr_cache_gc: increasing mapsize if the cache is full
• afc4b963 - kr_cache_gc: implemented categories, code split and cleanup
• 62f67e32 - kr_cache_gc: improvements, mostly categories
• e9bc3aee - kr_cache_gc: implemented dry-run (only occupation watching mode)
• 9498d19e - kr_cache_gc: randomized deleting of parts of huge categories
• addd8c68 - kr_cache_gc: build with meson
• f080afcf - utils/kr_cache_gc: skip invalid entries during iteration
• b72b0e88 - utils/kr_cache_gc: re-initialize lmdb txn in case of error
• 78c8ad0a - utils/kr_cache_gc: set default rw_txn_items to 100
• 3278d4d2 - utils/kr_cache_gc: add params for configuring cache
• e7ea3813 - distro: package kr_cache_gc
• 31c19bfc - utils/kr_cache_gc: fix lint:pedantic
• de2be793 - doc: document cache garbage collector

Compare with previous version

added 1 commit

• d64863c3 - doc: document cache garbage collector

Compare with previous version

@vcunat Would you please take a look regarding the aggressive NSEC3?

From my point of view, this is ready to merge. A systemd unit will be provided later, as a part of a proper maintenance service, #459.

unmarked as a Work In Progress

changed title from WIP: cache garbage colletor to cache garbage collector

changed the description

assigned to @vcunat

Yes, I'll have a look.

Nitpick idea: when we now have utils/ in source, it feels like client/ should move under it. WDYT?

I wonder whether using sbin/kr_cache_gc path is good for packaging. I'd probably prefer to reuse the prefix sbin/kres* again.

added 3 commits

• 4fe83f1a - lib/cache: make entry_h_consistent() generic
• 6d874231 - utils/kr_cache_gc: support NSEC3 (aggressive) RRs
• 55a2f5aa - utils/kr_cache_gc: minor nitpicks

Compare with previous version

added 1 commit

• 7d7c3ccf - fixup! lib/cache: make entry_h_consistent() generic

Compare with previous version