daemon: decrease tcp backlog to 128

From my TCP benchmarks, TCP backlog size over 128 don't seem to have any measurable benefits, even with hundreds of thousands of connections.

On the contrary, during very high TCP and CPU load, smaller backlog seems to dramatically improve latency for clients that keep idle TCP connections.

During normal/low load, smaller backlog doesn't seem to have any benefits.

When measured against "aggressive" clients that immediately close the TCP connection once their query is answered, backlog smaller than 128 was measured to hurt performance.

The application's backlog size is ultimately limited by net.core.somaxconn, which has been set to 128 prior to Linux 5.4. Therefore, this change only affects newer kernels and those who have manually set this value to a higher size.

---

Following benchmarks are split into two periods - resolver startup (t < 15s) and normal operation (t > 15s). The overall load is higher during startup due to cold cache and more TCP handshakes (when clients use idle timeout). All traffic is over TCP. Every client attempts to establish at least one connection over the period of the test (45s).

The first couple of graphs show the motivation behind this change - smaller backlog improves latency during heavy load. Performance with backlog of size 511 (current default) was so bad I wasn't even able to measure it.

The following graph shows performance impact during low load when clients don't use idle timeout (are "aggressive"). Even though the "aggressive" behaviour probably shouldn't be the common, it shows the drawback of shrinking the backlog further.

(green(128) is the same as violet(511), see top left corner)

The last graph shows smaller values of backlog when clients use idle timeout. During low load, smaller acklog doesn't seem to affect performance.

assigned to @vcunat

mentioned in commit 6e0c68be

added 5 commits

• 5e3f7904...ae812d2c - 4 commits from branch master
• 6e0c68be - daemon: decrease tcp backlog to 128

Compare with previous version

Knot DNS: #define TCP_BACKLOG_SIZE 10 but their architecture is far too different anyway.

The deckard in CI is a bit broken at this moment, but with this MR it worked fine when first submitted and it clearly seems unrelated anyway.

merged

mentioned in commit 66d05b73