At some random time cache starts returning NXDOMAIN for valid addresses
Hi there, first, thank you for this project, it is really amazing.
I am with sort of bug which I cannot understand from the trace what is actually happening, I have a stub with suffix which resolves cluster.local
on the IP 10.43.0.10
which always resolve with dig on that IP:
dig transmission-server-2.transmission-server-statefulset.default.svc.cluster.local @10.43.0.10
; <<>> DiG 9.16.1-Ubuntu <<>> transmission-server-2.transmission-server-statefulset.default.svc.cluster.local @10.43.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22425
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: bc00b8ee2c0fb66e (echoed)
;; QUESTION SECTION:
;transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. IN A
;; ANSWER SECTION:
transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. 5 IN A 10.42.0.109
;; Query time: 51 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Wed Jan 05 10:39:01 UTC 2022
;; MSG SIZE rcvd: 215
but then, with kresd it never gets resolved
curl localhost:8053/trace/transmission-server-2.transmission-server-statefulset.default.svc.cluster.local
[iterat][66025.00] 'transmission-server-2.transmission-server-statefulset.default.svc.cluster.local.' type 'A' new uid was assigned .01, parent uid .00 [0/895][cache ][66025.01] => skipping exact packet: rank 021 (min. 020), new TTL -561
[cache ][66025.01] => trying zone: ., NSEC, hash 0
[cache ][66025.01] => NSEC sname: covered by: loans. -> locker., new TTL 85824
[cache ][66025.01] => NSEC wildcard: covered by: . -> aaa., new TTL 85824
[iterat][66025.01] <= answer received:
;; ->>HEADER<<- opcode: QUERY; status: NXDOMAIN; id: 8040
;; Flags: qr aa QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 0
;; QUESTION SECTION
transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. A
;; AUTHORITY SECTION
. 85714 SOA a.root-servers.net. nstld.verisign-grs.com. 2022010500 1800 900 604800 86400
. 85714 RRSIG SOA 8 0 86400 1642482000 1641355200 9799 . PBUfopj8CcHa5BSiFMPrxYmE4RXh0ychS2itywyQh53uDIt1SbekqCWWtCgzUCfPzX+EMa0fKIfGdFMdgICOrZjfWpvBb4jzPrxxtLtJIaaEL20iRLl0Q4Oh/sC7FVHnXgxNNvQBRLjTwjNcrVgwCWdOmS9DOJxqb4OAYI4EZbcqD1rWjjy0tqfeqrQyzsquVNJYxUDMxfAOx4Ki2hyxVig/SZUi2IvpI50oyceLpvr7qerqKYUipoAtnxWWPA+Ko4cKjXr8IpzdcENmToiEZmTVKilCbcfi/JLAO9M/CKu9Mt4UGJlsByGKB2ne2N5+IxQmKQR3HAaXknQk09YUUw==
loans. 85824 NSEC locker. NS DS RRSIG NSEC
loans. 85824 RRSIG NSEC 8 1 86400 1642482000 1641355200 9799 . e7fCMBhzNi5oqQ2qR0x91JOHisj/v+k+ekwEPNvtnhpqpA15kd6x+ZcNol5tewW9NKQv/hOidyWSGDB0X75fLjSvBah4+KWrzUMLt3X7XxXqwzoCOzgfGqcwI/pY5OlCCmnidrpALAv62QGiziMSiPwIvUwJwJ2ZjAtKramFyYTp+GJIf1TyLCyaQH7e7ATrn6ChIpWY3v6zGWuSVODiuYBvCtBdVB+ydddVAdYvAtPylaQ/tLBYyQYsX8P2s1GpSDo+WwFHJE0s8mpqDROz5/Q1taRCr+K98xt173iApdt/qfp2wSM4MY/Mnrw0ksFbUfo4Am+YAf9+8EST7/glfA==
. 85824 NSEC aaa. NS SOA RRSIG NSEC DNSKEY
. 85824 RRSIG NSEC 8 0 86400 1642482000 1641355200 9799 . OLjHvokrSTOIELcevP7HxUx9G+OIz1V8vUE5JnlXJHHrKxq68IsBmM07A7GzQlHADHp/cpcvsbkrxLTB5+t6E3wfMvxDPvdJkTtMSBFJjszhX+VEgNlGJYiv5RuhDVeVltZe8O2/5oMCfSQyl+CUtexmW4lWBlSzHN4Nlnuuu3N1+fTle/rrtb0/JZTA54guI359tPaFgwZn5F4WoOo723Ge4AH6O6pJdl9EZNUAeqGqRIBLFoSNBgkJ4Luo3dYe9oWtSb+/1JVvXUnq2wxE7octNja9TnupYxutGKjod6QrNMelt2PVxpfkG198GbrQkOv3Jaqlp0vChJVEPdGbMw==
[resolv][66025.01] AD: request NOT classified as SECURE
[resolv][66025.01] finished in state: 4, queries: 1, mempool: 163952 B
;; selected from AUTHORITY sections:
; ranked rrset to_wire true, rank 060 (secure auth), cached false, qry_uid 1, revalidations 0
. 85714 SOA a.root-servers.net. nstld.verisign-grs.com. 2022010500 1800 900 604800 86400
; ranked rrset to_wire true, rank 021 (omit auth), cached false, qry_uid 1, revalidations 0
. 85714 RRSIG SOA 8 0 86400 1642482000 1641355200 9799 . PBUfopj8CcHa5BSiFMPrxYmE4RXh0ychS2itywyQh53uDIt1SbekqCWWtCgzUCfPzX+EMa0fKIfGdFMdgICOrZjfWpvBb4jzPrxxtLtJIaaEL20iRLl0Q4Oh/sC7FVHnXgxNNvQBRLjTwjNcrVgwCWdOmS9DOJxqb4OAYI4EZbcqD1rWjjy0tqfeqrQyzsquVNJYxUDMxfAOx4Ki2hyxVig/SZUi2IvpI50oyceLpvr7qerqKYUipoAtnxWWPA+Ko4cKjXr8IpzdcENmToiEZmTVKilCbcfi/JLAO9M/CKu9Mt4UGJlsByGKB2ne2N5+IxQmKQR3HAaXknQk09YUUw==
; ranked rrset to_wire true, rank 060 (secure auth), cached false, qry_uid 1, revalidations 0
loans. 85824 NSEC locker. NS DS RRSIG NSEC
; ranked rrset to_wire true, rank 021 (omit auth), cached false, qry_uid 1, revalidations 0
loans. 85824 RRSIG NSEC 8 1 86400 1642482000 1641355200 9799 . e7fCMBhzNi5oqQ2qR0x91JOHisj/v+k+ekwEPNvtnhpqpA15kd6x+ZcNol5tewW9NKQv/hOidyWSGDB0X75fLjSvBah4+KWrzUMLt3X7XxXqwzoCOzgfGqcwI/pY5OlCCmnidrpALAv62QGiziMSiPwIvUwJwJ2ZjAtKramFyYTp+GJIf1TyLCyaQH7e7ATrn6ChIpWY3v6zGWuSVODiuYBvCtBdVB+ydddVAdYvAtPylaQ/tLBYyQYsX8P2s1GpSDo+WwFHJE0s8mpqDROz5/Q1taRCr+K98xt173iApdt/qfp2wSM4MY/Mnrw0ksFbUfo4Am+YAf9+8EST7/glfA==
; ranked rrset to_wire true, rank 060 (secure auth), cached false, qry_uid 1, revalidations 0
. 85824 NSEC aaa. NS SOA RRSIG NSEC DNSKEY
; ranked rrset to_wire true, rank 021 (omit auth), cached false, qry_uid 1, revalidations 0
. 85824 RRSIG NSEC 8 0 86400 1642482000 1641355200 9799 . OLjHvokrSTOIELcevP7HxUx9G+OIz1V8vUE5JnlXJHHrKxq68IsBmM07A7GzQlHADHp/cpcvsbkrxLTB5+t6E3wfMvxDPvdJkTtMSBFJjszhX+VEgNlGJYiv5RuhDVeVltZe8O2/5oMCfSQyl+CUtexmW4lWBlSzHN4Nlnuuu3N1+fTle/rrtb0/JZTA54guI359tPaFgwZn5F4WoOo723Ge4AH6O6pJdl9EZNUAeqGqRIBLFoSNBgkJ4Luo3dYe9oWtSb+/1JVvXUnq2wxE7octNja9TnupYxutGKjod6QrNMelt2PVxpfkG198GbrQkOv3Jaqlp0vChJVEPdGbMw==
unless I clear the whole cache with this:
echo 'cache.clear(".")' | sudo nc -U /run/knot-resolver/control/0 -N
> {
['count'] = 504,
}
then it starts resolving again
curl localhost:8053/trace/transmission-server-2.transmission-server-statefulset.default.svc.cluster.local
[iterat][65580.00] 'transmission-server-2.transmission-server-statefulset.default.svc.cluster.local.' type 'A' new uid was assigned .01, parent uid .00
[resolv][65580.01] => id: '43911' querying: '.'@'10.43.0.10#00053' zone cut: '.' qname: 'TranSMissIon-SERVeR-2.tRAnSmIsSIOn-sERver-staTEFUlSet.dEFaUlT.SVC.clUSter.locAL.' qtype: 'A' proto: 'udp'
[select][65580.01] => id: '43911' updating: '.'@'10.43.0.10#00053' zone cut: '.' with rtt 24 to srtt: 24 and variance: 12
[iterat][65580.01] <= answer received:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 43911
;; Flags: qr aa rd QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 1232 B; ext-rcode: Unused
;; QUESTION SECTION
transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. A
;; ANSWER SECTION
transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. 5 A 10.42.0.109
;; ADDITIONAL SECTION
[cache ][65580.01] => stashed packet: rank 021, TTL 5, A transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. (215 B)
[resolv][65580.01] AD: request NOT classified as SECURE
[resolv][65580.01] finished in state: 4, queries: 1, mempool: 180352 B
;; selected from ANSWER sections:
; ranked rrset to_wire true, rank 021 (omit auth), cached false, qry_uid 1, revalidations 0
transmission-server-2.transmission-server-statefulset.default.svc.cluster.local. 5 A 10.42.0.109
Funny thing is that it only works if I clear the whole cache, cluster.local
or local
never do the trick.
My configuration is the following:
-- Network interface configuration
net.listen('192.168.1.21', 53, { kind = 'dns' })
net.listen('192.168.1.22', 53, { kind = 'dns' })
net.listen('127.0.0.1', 53, { kind = 'dns' })
net.listen('127.0.0.1', 853, { kind = 'tls' })
net.listen('::1', 53, { kind = 'dns', freebind = true })
net.listen('::1', 853, { kind = 'tls', freebind = true })
--net.listen('::1', 443, { kind = 'doh2' })
net.listen('127.0.0.1', 8053, { kind = 'webmgmt' })
-- Load useful modules
modules = {
'hints > iterate', -- Load /etc/hosts and allow custom root hints
'stats', -- Track internal statistics
'predict', -- Prefetch expiring/frequent records
'serve_stale < cache',
http = {
host = 'localhost',
port = 8053,
--geoip = '/usr/share/GeoIP/GeoIP.dat',
}
}
-- Cache size
cache.size = 100 * MB
policy.add(policy.suffix(policy.STUB('10.43.0.10'), {todname('cluster.local')}))
--policy.add(policy.pattern(policy.DEBUG_ALWAYS, '.*?cluster'))
policy.add(policy.all(policy.TLS_FORWARD({
{'1.1.1.1', hostname='cloudflare-dns.com'},
})))
Versions: Ubuntu 20.04.3
cat /etc/apt/sources.list.d/knot-resolver-latest.list
deb http://download.opensuse.org/repositories/home:/CZ-NIC:/knot-resolver-latest/xUbuntu_20.04/ /
ii knot-resolver 5.4.3-cznic.1 amd64 caching, DNSSEC-validating DNS resolver
ii knot-resolver-module-http 5.4.3-cznic.1 all HTTP module for Knot Resolver
ii knot-resolver-release 1.9-1 all Knot Resolver official upstream repositories