Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

For the usual use cases, whole milliseconds seem to make more sense
than seconds with 10ms precision.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

TL;DR: tune the test - now it works quite reliably for me,
though it's perhaps not nice.

With 1 MiB cache it's not easy to avoid overflows, as the defaults are
meant for much larger sizes.  Normal GC target is to decrease usage
by 10% when above 80% in 100 records per transaction.  That just won't
work reliable due to 10% being only 25 pages.

This commit makes the test run GC with more suitable tuning and
frequently pauses kresd to give GC better chance to catch up.

GC should prevent cache from overflowing.

Resolvers must answer queries even if the shared cache overflown during query processing.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

... in case of usage from kresd (GC does it a bit differently).

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

This is important for GC - otherwise the usage computation would be
wrong after another process changed size (without replacing the file).

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

As the code has been so far, there's no usable cache in that case
and some code just can't handle that.  Up to now we were getting
SIGSEGV from inside LMDB on the next attempted operation.

We might consider loosening preallocation in that case or even
retrying after a short sleep.  Systemd's restart after hold-off
timeout has an effect similar to the short sleep.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

In the unlikely case that GC happens "too late", it could fail when
deleting, in which case it seems best to reopen the cache and try again,
as it will probably be deleted by a kresd instance by the next interval.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

Until now the analyzing pass over full DB was taking place
in a single RO transaction.  For an unknown reason this caused kresd
processes to get MDB_MAP_FULL from mdb_put(), even though clearly there
were plenty free pages at that point.

Basic experiments show that 1k steps are OK and 10k steps are not.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

though I've never seen it happening.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

This apparently gets rid of MDB_BAD_TXN failures that we were getting
when cache overflows.  Unfortunately LMDB docs don't mention that
after operation failures one should abort the corresponding transaction.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

Now it should keep working if the file has been replaced.

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

FIXME: review, testing, etc.

A couple functions got folded into cdb_open_env(), as the split was
complicating situation (mainly around error handling).

doc: upcoming changes

See merge request !1057

scripts, docs: specify lua version in `luarocks install`

Closes #601

See merge request !1052

Vladimír Čunát authored 4 years ago and Petr Špaček committed 4 years ago

On some systems luarocks defaults to other lua version (e.g. Fedora),
so the result would not be usable from kresd.  I didn't touch scripts
for older distro versions (Debian < 10, Ubuntu < 20.04, CentOS 7).

lib/dnssec: fix build against libdnssec 3.0

See merge request !1053

It hasn't been released yet, but this patch fixes build against
current Knot master already.

lib/generic/queue: fix a bug + minor changes

See merge request !1050

... and most importantly, review all the code.

Detail: queue_{head,tail}_impl() no longer "safely return" NULL
on an empty queue, as the API always dereferences the returned
pointer anyway.

It... feels better that way.

Emptying the queue and using it again... didn't work :-(
Fortunately, no use case in kresd so far could trigger this, I believe:
 - struct session::waiting is a list of tasks waiting
   while connection is being established
 - the temporary queue in session_tasklist_finalize_expired() is also
   only once filled and emptied

The trailing _t implies the type is a typedef like:

typedef struct tls_ctx {
...
} tls_ctx_t;

But it is a plain struct - so remove it to avoid confusion.

Petr Špaček authored 4 years ago and Vladimír Čunát committed 4 years ago

It was rather confusing:
- event.after(0, ...) executed function immediatelly
- event.recurrent(0, ...) executed function immediatelly (seemingly worked)
  but stoped after the first execution, i.e. no recurrence took place.

config test improvements

See merge request !1045

Petr Špaček authored 4 years ago and Tomas Krizek committed 4 years ago

In theory there should be no background task but we need cleanup in case
kresd gets into infinite loop or something like that.

Fixes #596.

distro/tests: fix file permissions set by ansible

See merge request !1044

Ansible switched default file perm to 600 from more permissive 666. Add
file mode explicitly to avoid issues.

docs: clarify policy.ANSWER

See merge request !1037