lib/dnssec: validation is slow

Resolver uses ±2.2x more CPU time with validation enabled on Alexa data set. There are several reasons for it:

• libdnssec key setup and conversion from rdata->key is expensive and puts small allocations in hotpath, in resolver this is not unfortunately sunk cost
• the same for signing context setup
• signature verification itself is expensive, but that's expected

There are several high-level crypto operations we need:

• keytag from dnskey rr (this algo is described in dnssec rfc)
• dnskey rr has SEP (implemented)
• dnskey rr is revoked (implemented)
• dnskey rr matches another (compare algo + pubkey parts)
• nsec3 hash of name (not so critical, as most of the answers are positive)
• hasher init/put/finalize (for comparing DS/DNSKEY pairing, RRSIGs)
• verify(digest,signature,pubkey)

All of these should work on RR, as there is no metadata storage required.

Steps:

• Formalize described API and create a header
• Implement it using libdnssec current API
• Trim/cleanup current dnssec interface with only this API
• Implement it using mbed

assigned to @pspacek

For the record: Latest profiling done by @vcunat does not confirm that slowness is caused by validator. For this reason I propose to postpone this until a serious profiling shows problems in validation code.

Yes, at least on a general-use dataset that we got from an ISP. This certainly isn't high on my priority list, even if restricted to performance improvements.

Feel free to re-open when needed.

closed

added plan label

mentioned in commit 3df5dcbf

mentioned in commit d917e52f

mentioned in commit b436c5b3

mentioned in commit 30c36ca9