declarative policy module and other user-supplied DNS data

Current problem

Our current imperative policy module is using chain of Lua functions: This is quite slow and hard to use for non-programmers.

Proposal

Design a new method to configure "policies", preferably in a declarative way. By "policies" I mean a generic way to influence resolving and inject user-supplied data into DNS tree or block other stuff.

A declarative way should be more intuitive to use than writing Lua functions, and also faster if we design it right.

Here is incomplete list of stuff we might want to express.

• ability to also block sub-queries, e.g. when following CNAMEs (#217 (closed))
• ability to block RR data - e.g. rebinding protection, blacklist of NS names etc. (#523 (closed))
• ACLs (including negative ACLs, #370 (closed))
  • merge views with other policies (see also #445 (closed))?
• redirecting specific zones to user-configured servers (#428 (closed), !651 (closed))
  • beware that we need also port number, not just IP address
  • theoretical "helper" NS+glue records from kresd config should not be retrievable from outside
• FORWARDing
  • TLS forwarding has many knobs and might need even more: #481
  • do we still need STUB policy? If so see #218
  • FORWARDing might need exceptions for some subtrees; see e.g.
    • https://lists.nlnetlabs.nl/pipermail/unbound-users/2019-December/006560.html
    • https://github.com/CZ-NIC/knot-resolver/issues/110
• generally special EDNS tricks: #314, #303 (closed); also improve #657 (closed)
• special cache semantics (do not cache this sub-tree, limit TTL in this sub-tree)
• maybe DNS64 module should be merged with policies and ACLs: #368 (closed)
• maybe hints module should be merged in as well (see also #205 (closed), #349)
  • maybe also a way to provide other user-supplied data - #540 (closed)
    • (well, more ways can always be added)
• maybe prefill module should be merged as well (see also #417)
• think of interaction with daf module (beware of #183 (closed))
  • @vcunat would prefer to deprecate DAF, but theoretically we could think of translating DAF rules into the new policy rules
• design should be able to support full strength of RPZ (example of a problem: #194 (closed))
  • the most common features are in 6.0.x – CNAME redirection in particular, and interacting well with other rules (multiple rules of different kinds can trigger when jumping through CNAME chains)
• design needs to support efficient mechanism which mimicks RPZ with zone transfer including IXFR(!) (#195)
• build mechanism for better visibility into policies (#364)
• it needs to work with huge lists (apparently users want to have long block lists, see https://lists.nlnetlabs.nl/pipermail/unbound-users/2019-December/006559.html)
  • improved in 6.0.x: shared inside LMDB across all processes, but efficiency of restarts/reloads/updates could be significantly improved (as of 6.0.6)
• open question: at which stage should the module kick in? Can it be e.g. used to implement ignore-cd-flag policy as seen in Unbound?
  • the view: part can be used to set such options, though there's no ignore-cd in particular so far
• per-domain setting for rate-limits e.g. like ratelimit-below-domain, ratelimit-for-domain etc. like in Unbound
  • first per-user changes in rate-limits in views: (when we have any rate-limiting)
• special handling for reserved and local-only names: see #205 (closed) and think it through