respdiff issueshttps://gitlab.nic.cz/knot/respdiff/-/issues2021-03-04T11:53:17+01:00https://gitlab.nic.cz/knot/respdiff/-/issues/37blacklist: add other "meta" types2021-03-04T11:53:17+01:00Petr Špačekpspacek@isc.orgblacklist: add other "meta" typesAt least NSEC and NSEC3 have ill defined behavior when queried directly. See [dns-operations mailing list](https://lists.dns-oarc.net/pipermail/dns-operations/2021-February/020931.html).At least NSEC and NSEC3 have ill defined behavior when queried directly. See [dns-operations mailing list](https://lists.dns-oarc.net/pipermail/dns-operations/2021-February/020931.html).https://gitlab.nic.cz/knot/respdiff/-/issues/35contrib/ci: add info to artifacts to make it easier to analyze data2020-11-02T14:53:56+01:00Petr Špačekcontrib/ci: add info to artifacts to make it easier to analyze data- [ ] full path to job directory on isengard
- [ ] copy of respdiff.cfg - necessary for local runs like `diffsum -l0`
- [ ] URL to LMDB with respdiff queries (also necessary for diffsum)- [ ] full path to job directory on isengard
- [ ] copy of respdiff.cfg - necessary for local runs like `diffsum -l0`
- [ ] URL to LMDB with respdiff queries (also necessary for diffsum)https://gitlab.nic.cz/knot/respdiff/-/issues/33use binary format to store results2019-03-20T16:30:10+01:00Tomas Krizekuse binary format to store resultsUsing a binary format (such as CBOR) to store results instead of JSON would make it possible to capture the DNS query and its replies.
Benefits:
- exact responses could be used for extra analysis and easier debugging
- LMDB would no lon...Using a binary format (such as CBOR) to store results instead of JSON would make it possible to capture the DNS query and its replies.
Benefits:
- exact responses could be used for extra analysis and easier debugging
- LMDB would no longer be needed to generate textual reports
- diffrepro replies could be saved as well, allowing more precise difference analysishttps://gitlab.nic.cz/knot/respdiff/-/issues/31dnsviz: make the tool more robust2021-07-09T16:05:46+02:00Tomas Krizekdnsviz: make the tool more robustdnsviz itself is prone to random [tracebacks](https://gitlab.labs.nic.cz/knot/knot-resolver/-/jobs/219923) or even [deadlocks](https://github.com/dnsviz/dnsviz/issues/46).
When used to analyze multiple domains at once (`-f`), in multipl...dnsviz itself is prone to random [tracebacks](https://gitlab.labs.nic.cz/knot/knot-resolver/-/jobs/219923) or even [deadlocks](https://github.com/dnsviz/dnsviz/issues/46).
When used to analyze multiple domains at once (`-f`), in multiple threads (`-t`), even one such failure will cause the entire process to fail.
Even though it's slower and less efficient, the analysis will be more resilient if dnsviz is used to verify a single domain only, one at a time (and with a timeout for the command). The parallelization should take place in our code.https://gitlab.nic.cz/knot/respdiff/-/issues/30diffrepro: add parallel mode2020-09-14T14:57:08+02:00Tomas Krizekdiffrepro: add parallel modeWhen diffrepro is executed without a restart script, the queries should be pipelined, similar to orchestrator (instead of waiting for a given chunk to finish, and then restart).
While this mode might be less accurate, it could provide m...When diffrepro is executed without a restart script, the queries should be pipelined, similar to orchestrator (instead of waiting for a given chunk to finish, and then restart).
While this mode might be less accurate, it could provide major speedup for CI is used instead of the parallel-restart mode. Currently, diffrepro is a bottleneck: while it takes ~4m for 430k queries in orchestrator, mere 4k differences take almost the same time to verify with diffrepro in parallel-restart mode.https://gitlab.nic.cz/knot/respdiff/-/issues/27job_manager/respdiff: don't execute diffrepro when target_disagreements excee...2019-11-20T13:21:48+01:00Tomas Krizekjob_manager/respdiff: don't execute diffrepro when target_disagreements exceeds limitIn the number of target disagreements is much larger than usual, it doesn't make sense to execute diffrepro (which takes very long time to finish in such cases). Setting a hard limit like 10k should be good enough. It should cause the jo...In the number of target disagreements is much larger than usual, it doesn't make sense to execute diffrepro (which takes very long time to finish in such cases). Setting a hard limit like 10k should be good enough. It should cause the job to fail early and save execution time.https://gitlab.nic.cz/knot/respdiff/-/issues/25write proper docs2018-07-20T13:00:11+02:00Petr Špačekwrite proper docsJerry pointed out that respdiff.cfg needs more docs, especially when it comes to:
- which sections affect what tool
- what criterias and weights exists and what they do, how they match etc.
Match description can be mostly copied from
ht...Jerry pointed out that respdiff.cfg needs more docs, especially when it comes to:
- which sections affect what tool
- what criterias and weights exists and what they do, how they match etc.
Match description can be mostly copied from
https://gitlab.labs.nic.cz/knot/deckard/blob/master/doc/scenario_guide.rst#id1
Mapping config section to names is not straightforward as I thought because `[diff]` section configures `[msgdiff]` tool and `diffrepro` as well.https://gitlab.nic.cz/knot/respdiff/-/issues/21diffrepro: handle server timeouts2018-05-30T19:42:50+02:00Tomas Krizekdiffrepro: handle server timeoutsIf a server is unreachable when attempting to reproduce differences with `diffrepro`, the timeouts results in unreproducible results, which is probably not desirable in most cases.
Related to #8If a server is unreachable when attempting to reproduce differences with `diffrepro`, the timeouts results in unreproducible results, which is probably not desirable in most cases.
Related to #8https://gitlab.nic.cz/knot/respdiff/-/issues/18improve mismatch identifiers for RRsets2018-05-29T16:25:18+02:00Tomas Krizekimprove mismatch identifiers for RRsetsFields such as `question`, `answer`, `authority`, `additional` can have mismatching RR sets. It's not useful to attempt to group them together like mismatches in other fields. It's also difficult to represent this mismatch with a string ...Fields such as `question`, `answer`, `authority`, `additional` can have mismatching RR sets. It's not useful to attempt to group them together like mismatches in other fields. It's also difficult to represent this mismatch with a string representation. It currently contains the entire RR set, which is difficult to handle/format/read. Mismatches should have a short and readable identifier.
Mismatches of RRsets should be handled differently, perhaps only printing `qid` of the mismatch. The full RRset could then be displayed with a separate tool.https://gitlab.nic.cz/knot/respdiff/-/issues/14conditional message comparison2018-10-30T17:11:54+01:00Petr Špačekconditional message comparisonSome fields of DNS messages need to be compared (and match) only under certain conditions.
Open question: How to express these conditions / checks?
- [ ] add all-in-one check `authorityIfRelevant` with all the login under it
- [ ] more...Some fields of DNS messages need to be compared (and match) only under certain conditions.
Open question: How to express these conditions / checks?
- [ ] add all-in-one check `authorityIfRelevant` with all the login under it
- [ ] more granular checks like `authorityIfNXDOMAIN`, `authorityIfNODATA`, etc.
- [ ] support fancy conditions like `if (rcode == NXDOMAIN) {authority}` etc.
Example 1
=========
If DNS answer is a terminal answer with `RCODE = NOERROR` and no delegation, it does not make sense to insist on equality of `AUTHORITY` section because its content is not standardized for this case. E.g. Unbound will stuff in NS records even if they are not strictly required, and BIND with `minimal-responses: yes` (or version 9.12 and newer) will not add these NS records.
Example 2
=========
On the other hand if the answer has `RCODE = NXDOMAIN` then content of `AUTHORITY` section should match because it is mandatory to add SOA record in there, and possibly proof-of-nonexistence if the domain is signed.
Example 3
=========
Nodata response, i.e. `RCODE = NOERROR` accompanied with `ANSWER` section containing just CNAME/DNAME (where QNAME and QTYPE does not match the values in `ANSWER` section) should again contain proof proof-of-nonexistence if the domain is signed.https://gitlab.nic.cz/knot/respdiff/-/issues/11manual inspection of individual mismatches2021-07-09T15:41:12+02:00Petr Špačekmanual inspection of individual mismatchesRight now reports include only domain name and query type, like this:
```
== Field "rcode" mismatch ('NOERROR', 'SERVFAIL') query details
NS2.SPYROCOIN.BID. A 2 mismatches
```
This is kind of misleading because query flags like `DO`, `C...Right now reports include only domain name and query type, like this:
```
== Field "rcode" mismatch ('NOERROR', 'SERVFAIL') query details
NS2.SPYROCOIN.BID. A 2 mismatches
```
This is kind of misleading because query flags like `DO`, `CD`, `AD`, EDNS options etc. affect query processing. In other words, it might not be enough to copy&paste domain name and type to dig command line `dig NS2.SPYROCOIN.BID. A` to reproduce the issue.
We need a way to uniquely identify particular issue and allow extraction and display of captured DNS messages.
- [ ] Extract binary blobs with DNS messages into separate files for inspection in hexeditor
- [ ] Display text representation of individual messages
- [ ] Store text representation in a way suitable for existing diff tools
- [ ] Result of `msgdiff` algorithm should be available to user (possibly re-computed to diff all possible fields instead of subset configured in `respdiff.cfg`)https://gitlab.nic.cz/knot/respdiff/-/issues/3group results by core domain2018-05-29T16:07:24+02:00Petr Špačekgroup results by core domain"Core domain" is for our pusposes domain one level below domain in Public Suffix List. E.g. `com.` and `co.uk.` are both on PSL. Example core domains are then `example.com.` and `example.co.uk.`.
diffstat tool should have a level where ..."Core domain" is for our pusposes domain one level below domain in Public Suffix List. E.g. `com.` and `co.uk.` are both on PSL. Example core domains are then `example.com.` and `example.co.uk.`.
diffstat tool should have a level where statistics are grouped by core domain, so e.g. all three failures (in single result category) should be grouped together.
* 0120-0-ams30befd8225f0e083b76f6ab9226057c58b90501609.beacon.rum.dynapis.info.
* 0120-0-ams303452cd4c1fa8c712fdb0d2efee031c937f73d649.beacon.rum.dynapis.info.
* 0120-0-ams308b4c784031489e7cbf961d059e56f7074e2deb2f.beacon.rum.dynapis.info.
These three domain names should be grouped to `dynapis.info.`
Current result looks like this:
```
== Field "rcode" mismatch ('NOERROR', 'SERVFAIL') query details
iNcOmiNG.teLEMETRy.moZiLla.orG. AAAA 7 mismatches
iNcOmInG.TElEmetRY.MoZiLlA.orG. A 4 mismatches
nORmANDy-cLoudfROnT.CdN.mOziLLA.nET. AAAA 4 mismatches
NORmAnDY-cLoudFrONT.cdn.Mozilla.NET. A 4 mismatches
SNiPPeTs.CDN.MOzilla.neT. AAAA 3 mismatches
NS2.SPYROCOIN.BID. A 2 mismatches
ns1.spyrocoin.bid. AAAA 2 mismatches
ns1.spyrocoin.bid. A 2 mismatches
```
In the end it should look like this:
```
== Field "rcode" mismatch ('NOERROR', 'SERVFAIL') core domain statistics
moziLla.org. 11 mismatches
mozilla.net. 11 mismatches
spyrocoin.bid. 6 mismatches
[... other core domains statistics here ...]
== Field "rcode" mismatch ('NOERROR', 'SERVFAIL') query details
iNcOmiNG.teLEMETRy.moZiLla.orG. AAAA 7 mismatches
iNcOmInG.TElEmetRY.MoZiLlA.orG. A 4 mismatches
nORmANDy-cLoudfROnT.CdN.mOziLLA.nET. AAAA 4 mismatches
NORmAnDY-cLoudFrONT.cdn.Mozilla.NET. A 4 mismatches
SNiPPeTs.CDN.MOzilla.neT. AAAA 3 mismatches
NS2.SPYROCOIN.BID. A 2 mismatches
ns1.spyrocoin.bid. AAAA 2 mismatches
ns1.spyrocoin.bid. A 2 mismatches
```