Tomas Krizek authored 5 years ago and Petr Špaček committed 5 years ago

cherry-picked from f0ca89ac, original author Vlada Cunat

TL;DR: I believe all lua_gc() calls stemmed from misunderstanding lua
documentation, and the current settings seem potentially dangerous.

First, let me rely on lua 5.1 docs, as luajit 2 is documented to have
done only minor changes in the GC.
http://www.lua.org/manual/5.1/manual.html#lua_gc
http://wiki.luajit.org/New-Garbage-Collector#rationale

Commit 5a709411 claims to have increased the speed of GC to 400 % of
speed of allocation, but LUA_GCSETSTEPMUL is the parameter that
controls that, and that one was lowered to 99 % and later in
0ee2d1d7 even to 50 %.  Documentation explicitly says that setting
the value under 100 % may cause problems.

The default values seem perfectly sane to me and currently I can't see
any particular reason to change them.  It's 200 % relative GC speed,
and waiting for allocated size to double before starting another cycle.

I assume the resulting possibility of GC being too slow caused the need
to explicitly force a non-incremental GC cycle once in a while, but
that seems not useful anymore and not good for latency.

Vladimír Čunát authored 5 years ago and Petr Špaček committed 5 years ago

and don't use SO_REUSEPORT on non-Linux.  (Free)BSD has a different
meaning for it, which only brings confusion - only the last instance
would be getting packets.

It's minimalistic: no change if in interactive or --verbose mode.

Vladimír Čunát authored 5 years ago and Tomas Krizek committed 5 years ago

- written relatively defensively - act OK even if the API
  isn't used in an ideal way
- CI lint:scan-build: bump the error count;
  It's only another instance of the mis-detected array_push().
- the removed stale note in modules/meson.build isn't really related

The accounting was just broken and overly messy anyway.

The new way of transitioning to layer callbacks - done because of
portability (mainly to aarch64) - is a bit expensive.  This is a simple
way of recovering that cost.  Merge 603a24fc regressed speed a bit.

Lukas Jezek authored 5 years ago and Petr Špaček committed 5 years ago

Directory with subdirectory "packaging" is called "component".

List all components: python3 tests/packaging-doc.py --list
Run all tests/compoments: python3 tests/packaging-doc.py
Run specific test/component: python3 tests/packaging-doc.py --test <component>

The file structure for 1 component:
	daemon - dependencies for 1 component "kresd daemon" (default component, must always be there)
	scripts/distros - dependencies for 1 component for specific distro (must always be there)
	scripts/dockerfile_gen.py - test Dockerfile generator, see below
	tests/packaging.py - script to generate and build all combinations
	                     of Docker files for all components
	[component] - directory of component/test, see below
		      (e.g. "client/packaging/", "modules/http/packaging/" etc.)

The file structure of each component:
[component]
	<distro>/<version> - package names
		- builddeps - list of build depedencies
		- rundeps - list of runtime depedencies
		- pre-build.sh - script called before build phase
		- post-build.sh - script called after build phase
		- pre-run.sh - script called before run phase
		- post-run.sh - script called after run phase
		- install.sh and build.sh script called during build phase
	test.config or test.sh - kresd config test or shell script
note: content of "scripts/distroos" is same as "<distro>/<version>" of component.

There are "build" and "run" phases. "build" phase precedes "run" phase.
All script are called in this order:
	1. pre-<phase>.sh
	2. install packages specifed in the file "<phase>deps"
	3a. for "build" phase: run build.sh and install.sh
	3b. for "run" phase: run 'kresd -c [component]/test.config' or config.sh
	4. remove packages specified in the file "<phase>deps"
	5. post-<phase>.sh

Each step above is combines base components with a component under test.
E.g. component "scripts/distros" always precedes component "daemon/packaging"
and it precedes the tested component e.g. "modules/http".

In long term we might migrate this to py.test or some other well known
framework.

It might detect some use-after-free cases even without ASAN.

I must admit I don't really understand why we had a rare case
of use-after-free in the sendmmsg call, but this change should avoid
that without affecting anything else.

Vladimír Čunát authored 5 years ago and Petr Špaček committed 5 years ago

This shouldn't change the operation in any way, it's just nicer.

Vladimír Čunát authored 5 years ago and Petr Špaček committed 5 years ago

Calling this on every incoming UDP request could cost us up to 5% time.

The watchdog module now can be loaded without systemd, has customisable
callbacks, and can do real DNS queries and check their results.

I'm really sorry; I didn't notice and it only hit parts that
*apparently* aren't tested normally.  Only 32-bit systems would be
affected, due to the structure only changing ABI on 32-bit systems.

Refusing to answer queries without RD bit makes it harder
to read what data is present in resolver's cache.

Somehow I didn't notice this field when adding ::add_selected.
We probably never put anything into answer's ADDITIONAL,
so noone's noticed a problem until now.

Tomas Krizek authored 5 years ago and Vladimír Čunát committed 5 years ago

The code incorrectly assumes GNUTLS_E_AGAIN can only be caused
by reading the entire libuv buffer. Legitime causes of GNUTLS_E_AGAIN
wouldn't be processed correctly. These could be caused by a new
session ticket sent by the server.

Fixes #489

Vladimír Čunát authored 5 years ago and Petr Špaček committed 5 years ago

- don't print a line in every checking cycle
- don't exit when cache isn't found
- reduce recommended interval to 10s (compromise)
- don't increase cache size (but keep the code for now)

This reverts commit c3a754dd, reversing
changes made to 06d01bf0.

I've have accidentally merged incorrect branch, let's try to fix that by
reverting incorrect version of GC and merging the correct one.

Vladimír Čunát authored 5 years ago and Petr Špaček committed 5 years ago

Don't stash a packet with mismatching QNAME+QTYPE.
When receiving an NXDOMAIN or NODATA packet in an insecure zone,
it would get cached with KR_RANK_INSECURE regardless of mismatch
in QNAME.  If the 0x20 pattern was preserved in the fake QNAME,
such packet would then be used to answer queries with matching QNAME,
even if there's no proof that this QNAME is insecure.

Also the log now uses the same format query UID format as elsewhere.

name of SOA record must be "."

Fixes: #474

Vladimír Čunát authored 5 years ago and Petr Špaček committed 5 years ago

- don't print a line in every checking cycle
- don't exit when cache isn't found
- reduce recommended interval to 10s (compromise)
- don't increase cache size (but keep the code for now)