design-jetconf-zone-provisioning.md

+# Zone Provisioning with JetConf
+
+This text discusses the options for using
+[JetConf](https://gitlab.labs.nic.cz/labs/jetconf) along with a YANG
+data model for zone data
+([current version](https://gitlab.labs.nic.cz/llhotka/zone-data-yang))
+as a configuration tool for provisioning DNS zones served by Knot DNS.
+
+## Requirements
+
+* The system should be more reliable and easier to use than the
+   traditional way of editing zone files.
+
+* The impact on Knot should be minimal. Ideally, we'd like to use the
+   API that Knot already provides.
+
+Note that these requirements go somewhat in opposite directions, so some
+kind of a compromise is needed.
+
+## Configuration and State Data
+
+The
+[RESTCONF](https://tools.ietf.org/html/draft-ietf-netconf-restconf-13)
+protocol and
+[YANG](https://tools.ietf.org/html/draft-ietf-netmod-rfc6020bis-12)
+make a distinction between configuration and state data. These
+definitions are from [RFC 6241](https://tools.ietf.org/html/rfc6241):
+
+* *configuration data*: The set of writable data that is required to
+  transform a system from its initial default state into its current
+  state.
+
+* *state data*: The additional data on a system that is not
+  configuration data such as read-only status information and
+  collected statistics.
+
+We believe it is useful to make this distinction is our case, too,
+treating configuration data as the part of zone data that an operator
+has to create manually. We want to make such data editable through the
+RESTCONF protocol while, at the same time, make the complete contents
+of zones, including DDNS contributions or DNSSEC signatures, available
+to the JetConf client as read-only state data.
+
+If we take into account how Knot currently works and what operators
+expect, it might be useful to distinguish these categories of zone
+data:
+
+1. *Manual*: this is the data that has to be edited
+   manually. It is persistent in that it will survive the restart of
+   both Knot and JetConf.
+
+2. *Automatic*: This data is generated automatically and, but once
+   added to the zone it has to be retained across restarts. Regular
+   JetConf clients should not be able to directly edit this data, but
+   they might ba able to trigger its (re)generation, e.g. via an RPC
+   operation.
+
+3. *Ephemeral*: Added at run-time, e.g. via DDNS, this data
+   won't be present after the JetConf server is restarted.
+   
+The following table summarises the properties of each category of zone
+data from the viewpoint of a JetConf client (zone operator).
+
+|               | **persistent** | **transient** |
+| ------------- | -------------- | ------------- |
+| **writable**  | manual         |               |
+| **read-only** | automatic      | ephemeral     |
+
+## Possible Solutions
+
+We have identified the following potential solutions, and we believe
+the first is the best. Other suggestions are of course welcome.
+
+In any case, the complete current contents of a zone is supposed to be
+available to a JetConf client as state data - upon each request, it
+should be retrieved directly from Knot.
+
+### JetConf Manages Manual and Automatic Data
+
+JetConf keeps manual and automatic data in its datastore. However,
+automatic data can be read and written only by a special NACM (access
+control) group. A client belonging to this group may, for example, add
+DNSSEC signatures. Finer-grained classification of automatic data with
+corresponding NACM users/groups is also possible.
+
+A regular JetConf user without read access to the automatic data won't
+see it in the configuration.
+
+A set RPC operations may be provided so as to allow selected JetConf
+clients to operate on the automatic data (e.g. trigger re-signing the
+zone).
+
+Whenever the JetConf server starts, or is explicitly reinitialized, it
+wipes all zone data that Knot currently uses, and populates the
+managed zones from its configuration datastore. At this moment,
+ephemeral data (if any) will be removed.
+
+The advantage of this solution is that the JetConf server can use the
+YANG data model for a comprehensive syntactic, data type and semantic
+validation taking into account manual and automatic data
+together. Ephemeral data won't be taken into account for validation.
+
+A drawback of this solution is that some procedures for automatic data
+generation (e.g. zone signing) will have to be updated to pass the
+generated data to the JetConf server.
+
+### JetConf Manages only Manual Data
+
+In this case, JetConf configuration datastore contains only manual
+data. Every change to manual data will be recorded in the datastore,
+and also passed to Knot. Automatic data will be generated as before,
+and stored in classical zone files outside the reach of the JetConf
+server.
+
+If the JetConf server is restarted it *won't* send its configuration
+data to Knot. Knot will use the zone files for zone initialization.
+
+The advantage of this approach is that most existing procedures,
+e.g. for zone signing, need no change.
+
+Manual data can still be validated against the data model, but not
+together with automatic data. This is suboptimal because there may be
+semantic constraints involving both manual and automatic data.
+
+The biggest disadvantage of this approach is that the JetConf
+configuration datastore and zone files may get out of sync. Therefore,
+this solution could probably work only if DDNS is blocked for zones
+managed by JetConf.
+
+### JetConf Manages No Data
+
+This is most likely the simplest possible solution. The JetConf
+configuration datastore contains no zone data at all. JetConf clients
+can only issue RPC operations for manipulating zone data, and these
+requests will be passed directly to Knot.
+
+With this solution, JetConf clients will have to work with complete
+zones – manual, automatic and ephemeral data cannot be
+identified. Also, the YANG data model becomes pretty much useless.
+
+### Dominik's comments
+
+From performance perspective wiping and re-populating zone data as mentioned in the first solution seems wasteful. If we are to scale we have to avoid large data transfers. Secondly there are further changes planned in the area of how Knot manages the data - thus a simpler solution should be preferred. And finally DDNS changes should not be always discarded.
+
+* Knot always has the current version of the zone loaded in memory (obviously) so there's no problem in accessing that. RCU also provides a transactional access to the data. This offers easy access to the final zone that is accessible from the outside. 
+* Knot does not have a direct access to original zone configuration. But it should be fairly easy to read that file and dump it to a control channel to Jetconf. I am not aware of any locking, though.
+* Knot also has an easy access to journal which always provides all changes between what is in the zonefile and what is the current zone. But these changes ('changesets') consist of both normal DNS, human-entered records AND their digital signatures. There is no separation. If a DDNS change comes, it is applied to the zone and signed, then stored in the journal.
+
+So in practice the simple Jetconf implementation should be able to:
+1) Access the original zonefile that Knot reads on startup/reload. The zonefile should not be read every time Jetconf is started, but perhaps a hash should be computed at first and then decided whether a zone copy/transfer is required. When Jetconf writes a zonefile, Knot should be automatically reloaded.
+2) Access the current zone that is in memory. When written to we can simulate a DDNS change incoming - apply, sign and store in journal without the parsing and semantic checks.
+3) Access all changes in between those two, read-only. For reference only.
+
+In other words, I don't see how we can differentiate easily between "automatic" and "ephemeral" data.
+
+<br><br>
\ No newline at end of file