validator: avoid assertion in an edge-case

See merge request !1169

Case: NSEC3 with too many iterations used for a positive wildcard proof.

To really fix the answers, this also needed fixing the `any_rank` part
which I somehow forgot in commit 7107faeb :-(

GitLab doesn't show the <system-*> tags, so let's replace them.
For now it's just hacky sed; I can't use xsltproc or similar.

https://docs.gitlab.com/ce/ci/unit_test_reports.html
https://mesonbuild.com/Unit-tests.html#testlogjunitxml

Implemented fully: build, build-asan; partially: pytests, deckard.

cache: clear any stale readers when opening cache

See merge request !1172

dnstap: add TCP RTT collection (experimental, optional)

See merge request !1170

I see no potential use for `set`; it's not a suitable abstraction.
And for `get` we want to use a pointer to the public type
instead of the private one.

Overall, worker.h has way too many stuff, but this branch is
not a good place to clean it up.

Tomas Krizek authored 4 years ago and Vladimír Čunát committed 3 years ago

This can happen for example when we want to send an answer, but the
http stream (or the connection?) is already closed.

Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x7f5ad2445459 in __interceptor_malloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x55c0db3fc442 in http_write_pkt ../daemon/http.c:610
    #2 0x55c0db3fc882 in http_write ../daemon/http.c:651
    #3 0x55c0db3e9bb1 in qr_task_send ../daemon/worker.c:700
    #4 0x55c0db3ee86c in qr_task_finalize ../daemon/worker.c:1321
    #5 0x55c0db3f0123 in qr_task_step ../daemon/worker.c:1633
    #6 0x55c0db3f0982 in worker_submit ../daemon/worker.c:1755
    #7 0x55c0db3d992a in session_wirebuf_process ../daemon/session.c:759
    #8 0x55c0db3c5f01 in udp_recv ../daemon/io.c:89
    #9 0x7f5ad22b0e0e  (/usr/lib/libuv.so.1+0x20e0e)

Tomas Krizek authored 3 years ago and Lukas Jezek committed 3 years ago

The on_frame_recv() callback ins't guaranteed to be called by nghttp2.
This can happen e.g. in a case when nghttp2 issues a PROTOCOL_ERROR
RST_STREAM frame. Previously, it would leave the connection in a
stream-processing state, making it completely useless.

While this guarantees a cleanup will be called eventually, some streams
may still get ignored due to the order of various callbacks and data
processing procedures. Still, it's better than the previous
implementation.

various undefined-behavior fixes

Closes #426

See merge request !1167

It's spread over multiple commits.  The other undefined-behavior fixes
probably don't cause any issue in practice, so they aren't mentioned.

Really support 64-bit return value even there.
Currently we only use such large value in an unimportant case
(lua_Number seed).

Vladimír Čunát authored 3 years ago and Tomas Krizek committed 3 years ago

murmurhash3.c:43:40: runtime error: addition of unsigned offset
                     to 0x7ffce41c2014 overflowed to 0x7ffce41c2000
The `i` was used in a super-ugly way; I suspect the only reason was
to optimize that end-loop condition was zero comparison *vomit*

Vladimír Čunát authored 3 years ago and Tomas Krizek committed 3 years ago

The misaligned accesses were just throwing warnings for me (with gcc),
so I added the flag to stregthen what we get in CI.

Vladimír Čunát authored 3 years ago and Tomas Krizek committed 3 years ago

Some less common HW (not x86, usually ARM) doesn't tolerate unaligned
access to memory and it's breakage of C as well.

It's easiest to check by meson's -Db_sanitize=undefined (on any HW).
I pushed millions of real-life QNAME+QTYPE queries over UDP in default
mode and the sanitizer seems clear now.

Vladimír Čunát authored 3 years ago and Tomas Krizek committed 3 years ago

libknot >= 2.9 provides it and their version is less buggy :-)
In particular, it works with unaligned pointers.

cache: improve handling write errors from LMDB

See merge request !1159

Vladimír Čunát authored 4 years ago and Tomas Krizek committed 3 years ago

In particular, ignore ENOSPC from LMDB for a short time unless the
space-usage estimate is over 90%.  See code comments for details.

Vladimír Čunát authored 4 years ago and Tomas Krizek committed 3 years ago

I suspect there's an edge case where cache thinks it provided enough
data but iterator (or who) disagrees and resolution continues.
We observed (flags.CACHED == true) even when processing a reply from
internet, and that could be confusing and even trigger a segfault.
Clearing the flag sounds OK semantically; it never meant that no cached
data have been used within the kr_query (e.g. zone cut, DS/DNSKEY, ...)

fix SERVFAIL for some rare dynamic proofs

See merge request !1166

Our aggressive NSEC cache doesn't handle these well and the case
with only the end-label being like this was forgotten.
See the parent commit for a test case.

Also, larger NSEC* sets are now considered weird.

Example case: denying existence of ok.rdns.dev by
oj\255.rdns.dev. NSEC ok\000.rdns.dev.
This NSEC end was incorrectly ordered with the QNAME.
https://gitter.im/CZ-NIC/knot-resolver?at=606055b82beb1e1da3d73892

The code is Libor's :-)

dnstap: fix repeated configuration

See merge request !1168

In practice it can easily happen, as loading module and really
configuring it is often done separately.  Then we'd see two fstrm
threads, etc.

release 5.3.1

See merge request !1162

This change already took place in !1082, this just updates the files to
correctly reflect the current situation.

validator: downgrade NSEC3 records with too many iterations

See merge request !1160

(in aggressive cache part)  Also bump cache version, so that we clear
those that have been left by previous kresd releases.

It seems better to check RRSIGs before checking negative proofs,
in terms of reasoning, being less error-prone, etc.