Sphinx: Update ASN, Country Code and RTT fields in exported data schema

parent 07246395
Pipeline #75928 passed with stages
in 8 minutes and 3 seconds
......@@ -27,9 +27,11 @@ DNS Probe uses local file in YAML format to load configuration at startup. Its s
+--rw dnstap-socket-list?* <string>
+--rw dns-ports?* <uint16>
+--rw export
| +--rw asn-maxmind-db? <string>
| +--rw cdns-blocks-per-file? <uint64>
| +--rw cdns-fields?* <string>
| +--rw cdns-records-per-block? <uint64>
| +--rw country-maxmind-db? <string>
| +--rw export-dir? <string>
| +--rw export-format? <enumeration>
| +--rw file-compression? <boolean>
......@@ -106,6 +108,14 @@ anonymize-ip
If this flag is true, client IP addresses in exported data (Parquet or C-DNS, NOT optional PCAPs) will be anonymized using Crypto-PAn prefix-preserving algorithm.
asn-maxmind-db
^^^^^^^^^^^^^^
:data node: ``<instance-id>/export/asn-maxmind-db``
:default: empty
Path to Maxmind ASN database. If this option is set to a valid database file, the ``asn`` implementation field in exported Parquets or C-DNS will be filled with Autonomous System Number (ASN) based on client's IP address.
.. _cdns-blocks-per-file:
cdns-blocks-per-file
......@@ -158,6 +168,14 @@ Bitmask indicating which CPU cores should DNS Probe use. At least 3 CPU cores ar
The default value of 7 indicates that DNS Probe should use the first 3 CPU cores with IDs of 0, 1 and 2.
country-maxmind-db
^^^^^^^^^^^^^^^^^^
:data node: ``<instance-id>/export/country-maxmind-db``
:default: empty
Path to Maxmind Country database. If this option is set to a valid database file, the ``country`` field in exported Parquets or ``country-code`` implementation field in exported C-DNS will be filled with ISO 3166-1 country code based on client's IP address.
.. _dnstap-socket-list:
dnstap-socket-list
......
This diff is collapsed.
......@@ -91,6 +91,7 @@ distribution repositories:
- OpenSSL (libssl-dev)
- fstrm
- Protocol Buffers (libprotobuf-dev, protobuf-compiler)
- libmaxminddb
- DPDK (only for DPDK version)
Optionally, to build this user documentation (``make doc``) or manual pages (``make man``)
......
......@@ -16,6 +16,15 @@ For changes in software see `version descriptions <https://gitlab.nic.cz/adam/dn
- Segment
- Change description
* - **0.7.1**
- **1.1**
- :doc:`Configuration <Configuration>`, :doc:`Exported Data Schema <ExportedDataSchema>`,
:doc:`Default YAML file <YAMLfile>`
- Update ASN, Country Code and RTT fields in exported data schema
* -
-
- :doc:`Installation <Installation>`
- Add libmaxminddb as dependency
* -
- **1.0**
- :doc:`Architecture <Architecture>`, :doc:`Configuration <Configuration>`,
:doc:`Installation <Installation>`, :doc:`Overview <Overview>`,
......
......@@ -9,7 +9,7 @@ It is also included in the project repository (`data-model/dns-probe.yml <https:
.. code-block:: yaml
# Last revision: 2021-01-27
# Last revision: 2021-02-03
#
# Default instance configuration.
# This configuration is always loaded before other configuration specified by given instance's ID.
......@@ -88,6 +88,7 @@ It is also included in the project repository (`data-model/dns-probe.yml <https:
export-format: 'parquet'
# This sequence indicates which fields from the C-DNS standard schema are included in exported data.
# 3 implementation specific fields are also included (asn, country_code, round_trip_time).
# By default all fields available in DNS Probe are enabled as shown below.
cdns-fields:
- 'transaction_id'
......@@ -113,7 +114,9 @@ It is also included in the project repository (`data-model/dns-probe.yml <https:
- 'query_opt_rdata'
- 'response_additional_sections'
- 'response_size'
- 'response_delay' # TCP RTT
- 'asn' # asn-maxmind-db configuration option also needs to be set
- 'country_code' # country-maxmind-db configuration option also needs to be set
- 'round_trip_time' # TCP RTT
# Maximum number of DNS records in one exported C-DNS block.
cdns-records-per-block: 10000
......@@ -144,6 +147,16 @@ It is also included in the project repository (`data-model/dns-probe.yml <https:
# Valid values are 'all', 'invalid', 'disabled'.
pcap-export: 'disabled'
# Path to Maxmind Country database. If this option is set to a valid database file, the 'country'
# field in exported Parquets or 'country-code' implementation field in exported C-DNS will be
# filled with ISO 3166-1 country code based on client's IP address.
country-maxmind-db: ''
# Path to Maxmind ASN database. If this iption is set to a valid database file, the 'asn'
# implementation field in exported Parquets or C-DNS will be filled with Autonomous System
# Number (ASN) based on client's IP address.
asn-maxmind-db: ''
# [SECTION] Configuration of client IP anonymization in exported data (Parquet or C-DNS).
# The optional PCAP export does NOT get anonymized!!!
ip-anonymization:
......
......@@ -15,7 +15,7 @@ scanned requests and responses.
which means that it contains authoring/editing/managing notes
in addition to the actual content.
:Edition: :doc:`1.0 <RecordOfChanges>`
:Edition: :doc:`1.1 <RecordOfChanges>`
:Source: The source code of this documentation is open and can be found
on `GitLab <https://gitlab.nic.cz/adam/dns-probe/-/tree/master/doc>`_.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment