diff --git a/README.md b/README.md
index 799df3888e8d905c62417f003f8d178009f48093..29f32a6c1a1c616c033f3d379a69af6e74818be3 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,8 @@
# Knot DNS Resolver
-[](https://travis-ci.org/CZ-NIC/knot-resolver)
-[](https://coveralls.io/r/CZ-NIC/knot-resolver)
-[](https://scan.coverity.com/projects/3912)
+[](https://travis-ci.org/CZ-NIC/knot-resolver)
+[](https://coveralls.io/r/CZ-NIC/knot-resolver)
+[](https://scan.coverity.com/projects/3912)
The Knot DNS Resolver is a minimalistic caching resolver implementation. The project provides both a resolver
diff --git a/daemon/README.rst b/daemon/README.rst
new file mode 100644
index 0000000000000000000000000000000000000000..1c625877b0d43950806b26e1cc2492bfdc70c359
--- /dev/null
+++ b/daemon/README.rst
@@ -0,0 +1,326 @@
+************************
+Knot DNS Resolver daemon
+************************
+
+Requirements
+============
+
+* libuv_ 1.0+ (a multi-platform support library with a focus on asynchronous I/O)
+* Lua_ 5.1+ (embeddable scripting language, LuaJIT_ is preferred)
+
+Running
+=======
+
+There is a separate resolver library in the `lib` directory, and a minimalistic daemon in
+the `daemon` directory.
+
+.. code-block:: bash
+
+ $ ./daemon/kresolved -h
+
+Interacting with the daemon
+---------------------------
+
+The daemon features a CLI interface if launched interactively, type ``help`` to see the list of available commands.
+You can load modules this way and use their properties to get information about statistics and such.
+
+.. code-block:: bash
+
+ $ kresolved /var/run/knot-resolver
+ [system] started in interactive mode, type 'help()'
+ > cache.count()
+ 53
+
+.. role:: lua(code)
+ :language: lua
+
+Configuration
+=============
+
+.. contents::
+ :depth: 2
+ :local:
+
+In it's simplest form it requires just a working directory in which it can set up persistent files like
+cache and the process state. If you don't provide the working directory by parameter, it is going to make itself
+comfortable in the current working directory.
+
+.. code-block:: sh
+
+ $ kresolved /var/run/kresolved
+
+And you're good to go for most use cases! If you want to use modules or configure daemon behavior, read on.
+
+There are several choices on how you can configure the daemon, a RPC interface a CLI and a configuration file.
+Fortunately all share common syntax and are transparent to each other, e.g. changes made during the runtime are kept
+in the redo log and are immediately visible.
+
+.. warning:: Redo log is not yet implemented, changes are visible during the process lifetime only.
+
+Configuration example
+---------------------
+.. code-block:: lua
+
+ -- 10MB cache
+ cache.open(10*MB)
+ -- static hints
+ modules = {
+ hints = true,
+ cachectl = true
+ }
+ -- interfaces
+ net.listen('127.0.0.1')
+
+Configuration syntax
+--------------------
+
+The configuration is kept in the ``config`` file in the daemon working directory, and it's going to get loaded automatically.
+If there isn't one, the daemon is going to start with sane defaults, listening on `localhost`.
+The syntax for options is like follows: ``group.option = value`` or ``group.action(parameters)``.
+You can also comment using a ``--`` prefix.
+
+A simple example would be to load static hints.
+
+.. code-block:: lua
+
+ modules = {
+ cachectl = true -- no configuration
+ }
+
+If the module accepts accepts configuration, you can provide a table.
+The syntax for table is ``{ key1 = value, key2 = value }``, and it represents the unpacked `JSON-encoded`_ string, that
+the modules use as the :ref:`input configuration <mod-properties>`.
+
+.. code-block:: lua
+
+ modules = {
+ cachectl = true,
+ hints = {
+ file = '/etc/hosts'
+ }
+ }
+
+The possible simple data types are strings, integers or floats and boolean.
+
+.. tip:: The configuration and CLI syntax is Lua language, with which you may already be familiar with.
+ If not, you can read the `Learn Lua in 15 minutes`_ for a syntax overview. Spending just a few minutes
+ will allow you to break from static configuration, write more efficient configuration with iteration, and
+ leverage events and hooks. Lua is heavily used for scripting in applications ranging from embedded to game engines,
+ but in DNS world notably in `PowerDNS Recursor`_. Knot DNS Resolver does not simply use Lua modules, but it is
+ the heart of the daemon for everything from configuration, internal events and user interaction.
+
+Dynamic configuration
+^^^^^^^^^^^^^^^^^^^^^
+
+Knowing that the the configuration is a valid Lua script enables you to write dynamic rules, and also avoid
+additional configuration templating. One example is to differentiate between internal and external
+interfaces based on environment variable.
+
+.. code-block:: lua
+
+ if hostname() == 'hidden' then
+ net.listen(net.eth0)
+ else
+ net.listen(net.eth1.addr[1])
+ end
+
+Another example would show how it is possible to bind to all interfaces, using iteration.
+
+.. code-block:: lua
+
+ for name, addr_list in pairs(net.interfaces()) do
+ net.listen(addr_list)
+ end
+
+You can also use third-party packages (available for example through LuaRocks_) as on this example
+to download cache from parent, to avoid cold-cache start.
+
+.. code-block:: lua
+
+ local http = require('socket.http')
+ local ltn12 = require('ltn12')
+
+ if cache.count() == 0 then
+ -- download cache from parent
+ http.request {
+ url = 'http://parent/cache.mdb',
+ sink = ltn12.sink.file(io.open('cache.mdb', 'w'))
+ }
+ -- reopen cache with 100M limit
+ cache.open('.', 100*MB)
+ end
+
+Events and services
+^^^^^^^^^^^^^^^^^^^
+
+The Lua supports a concept called closures, this is extremely useful for scripting actions upon various events.
+
+.. note:: Work in progress, come back later!
+
+* Timers and events
+* File watchers
+* Data I/O
+
+
+Configuration reference
+-----------------------
+
+This is a reference for variables and functions available to both configuration file and CLI.
+
+Environment
+^^^^^^^^^^^
+
+.. envvar:: env (table)
+
+ Return environment variable.
+
+ .. code-block:: lua
+
+ env.USER -- equivalent to $USER in shell
+
+.. function:: hostname()
+
+ :return: Machine hostname.
+
+Network configuration
+^^^^^^^^^^^^^^^^^^^^^
+
+.. function:: net.listen(address, [port = 53])
+
+ :return: boolean
+
+ Listen on address, port is optional.
+
+.. function:: net.listen({address1, ...}, [port = 53])
+
+ :return: boolean
+
+ Listen on list of addresses.
+
+.. function:: net.listen(interface, [port = 53])
+
+ :return: boolean
+
+ Listen on all addresses belonging to an interface.
+
+ Example:
+
+ .. code-block:: lua
+
+ net.listen(net.eth0) -- listen on eth0
+
+.. function:: net.close(address, [port = 53])
+
+ :return: boolean
+
+ Close opened address/port pair, noop if not listening.
+
+.. function:: net.list()
+
+ :return: Table of bound interfaces.
+
+ Example output:
+
+ .. code-block:: lua
+
+ [127.0.0.1] => {
+ [port] => 53
+ [tcp] => true
+ [udp] => true
+ }
+
+.. function:: net.interfaces()
+
+ :return: Table of available interfaces and their addresses.
+
+ Example output:
+
+ .. code-block:: lua
+
+ [lo0] => {
+ [addr] => {
+ [1] => ::1
+ [2] => 127.0.0.1
+ }
+ [mac] => 00:00:00:00:00:00
+ }
+ [eth0] => {
+ [addr] => {
+ [1] => 192.168.0.1
+ }
+ [mac] => de:ad:be:ef:aa:bb
+ }
+
+ .. tip:: You can use ``net.<iface>`` as a shortcut for specific interface, e.g. ``net.eth0``
+
+Modules configuration
+^^^^^^^^^^^^^^^^^^^^^
+
+The daemon provides an interface for dynamic loading of :ref:`daemon modules <modules-implemented>`.
+
+.. tip:: Use syntactic sugar for module loading. Declaring a variable ``modules`` equals to loading a table of modules.
+
+ .. code-block:: lua
+
+ modules = { hints = {file = '/etc/hosts'} }
+
+ Equals to:
+
+ .. code-block:: lua
+
+ modules.load('cachectl')
+ cachectl.config({file = '/etc/hosts'})
+
+
+.. function:: modules.list()
+
+ :return: List of loaded modules.
+
+.. function:: modules.load(name)
+
+ :param string name: Module name, e.g. "hints"
+ :return: boolean
+
+ Load a module by name.
+
+.. function:: modules.unload(name)
+
+ :param string name: Module name
+ :return: boolean
+
+ Unload a module by name.
+
+Cache configuration
+^^^^^^^^^^^^^^^^^^^
+
+The cache in Knot DNS Resolver is persistent with LMDB backend, this means that the daemon doesn't lose
+the cached data on restart or crash to avoid cold-starts. Interestingly the cache may be reused between cache
+daemons or manipulated from other processes, making for example synchronisation between load-balanced recursors possible.
+
+.. function:: cache.open(max_size)
+
+ :param number max_size: Maximum cache size in bytes.
+ :return: boolean
+
+ Open cache with size limit. The cache will be reopened if already open.
+ Note that the max_size cannot be lowered, only increased due to how cache is implemented.
+
+ .. tip:: Use ``kB, MB, GB`` constants as a multiplier, e.g. ``100*MB``.
+
+.. function:: cache.count()
+
+ :return: Number of entries in the cache.
+
+.. function:: cache.close()
+
+ :return: boolean
+
+ Close the cache.
+
+.. _`JSON-encoded`: http://json.org/example
+.. _`Learn Lua in 15 minutes`: http://tylerneylon.com/a/learn-lua/
+.. _`PowerDNS Recursor`: https://doc.powerdns.com/md/recursor/scripting/
+.. _LuaRocks: https://rocks.moonscript.org/
+.. _libuv: https://github.com/libuv/libuv
+.. _Lua: http://www.lua.org/about.html
+.. _LuaJIT: http://luajit.org/luajit.html
\ No newline at end of file
diff --git a/daemon/bindings.c b/daemon/bindings.c
index e28aafb6e1935db582db00aa19b9797b2b378be7..b009814f84bf9b91c9344da498309d410643cbba 100644
--- a/daemon/bindings.c
+++ b/daemon/bindings.c
@@ -126,20 +126,61 @@ static int net_list(lua_State *L)
return 1;
}
+/** Listen on interface address list. */
+static int net_listen_iface(lua_State *L, int port)
+{
+ /* Expand 'addr' key if exists */
+ lua_getfield(L, 1, "addr");
+ if (lua_isnil(L, -1)) {
+ lua_pop(L, 1);
+ lua_pushvalue(L, 1);
+ }
+
+ /* Bind to address list */
+ struct engine *engine = engine_luaget(L);
+ size_t count = lua_rawlen(L, -1);
+ for (size_t i = 0; i < count; ++i) {
+ lua_rawgeti(L, -1, i + 1);
+ int ret = network_listen(&engine->net, lua_tostring(L, -1),
+ port, NET_TCP|NET_UDP);
+ if (ret != 0) {
+ lua_pushstring(L, kr_strerror(ret));
+ lua_error(L);
+ }
+ lua_pop(L, 1);
+ }
+
+ lua_pushboolean(L, true);
+ return 1;
+}
+
/** Listen on endpoint. */
static int net_listen(lua_State *L)
{
/* Check parameters */
int n = lua_gettop(L);
- if (n < 2) {
- lua_pushstring(L, "expected (string addr, int port)");
+ int port = KR_DNS_PORT;
+ if (n > 1 && lua_isnumber(L, 2)) {
+ port = lua_tointeger(L, 2);
+ }
+
+ /* Process interface or (address, port) pair. */
+ if (lua_istable(L, 1)) {
+ return net_listen_iface(L, port);
+ } else if (n < 1 || !lua_isstring(L, 1)) {
+ lua_pushstring(L, "expected (string addr, int port = 53)");
lua_error(L);
}
/* Open resolution context cache */
struct engine *engine = engine_luaget(L);
- int ret = network_listen(&engine->net, lua_tostring(L, 1), lua_tointeger(L, 2), NET_TCP|NET_UDP);
- lua_pushboolean(L, ret == 0);
+ int ret = network_listen(&engine->net, lua_tostring(L, 1), port, NET_TCP|NET_UDP);
+ if (ret != 0) {
+ lua_pushstring(L, kr_strerror(ret));
+ lua_error(L);
+ }
+
+ lua_pushboolean(L, true);
return 1;
}
@@ -160,8 +201,7 @@ static int net_close(lua_State *L)
return 1;
}
-/** List available interfaces.
- */
+/** List available interfaces. */
static int net_interfaces(lua_State *L)
{
/* Retrieve interface list */
@@ -227,23 +267,45 @@ int lib_net(lua_State *L)
return 1;
}
+/** Return number of cached records. */
+static int cache_count(lua_State *L)
+{
+ struct engine *engine = engine_luaget(L);
+ const namedb_api_t *storage = kr_cache_storage();
+
+ /* Fetch item count */
+ namedb_txn_t txn;
+ int ret = kr_cache_txn_begin(engine->resolver.cache, &txn, NAMEDB_RDONLY);
+ if (ret != 0) {
+ lua_pushstring(L, kr_strerror(ret));
+ lua_error(L);
+ }
+
+ lua_pushinteger(L, storage->count(&txn));
+ kr_cache_txn_abort(&txn);
+ return 1;
+}
+
/** Open cache */
static int cache_open(lua_State *L)
{
/* Check parameters */
int n = lua_gettop(L);
- if (n < 2) {
- lua_pushstring(L, "expected (string path, int size)");
+ if (n < 1) {
+ lua_pushstring(L, "expected (number max_size)");
lua_error(L);
}
- /* Open resolution context cache */
+ /* Close if already open */
struct engine *engine = engine_luaget(L);
- engine->resolver.cache = kr_cache_open(lua_tostring(L, 1), engine->pool, lua_tointeger(L, 2));
+ if (engine->resolver.cache != NULL) {
+ kr_cache_close(engine->resolver.cache);
+ }
+
+ /* Open resolution context cache */
+ engine->resolver.cache = kr_cache_open(".", engine->pool, lua_tointeger(L, 1));
if (engine->resolver.cache == NULL) {
- lua_pushstring(L, "invalid cache directory: ");
- lua_pushstring(L, lua_tostring(L, 1));
- lua_concat(L, 2);
+ lua_pushstring(L, "can't open cache in rundir");
lua_error(L);
}
@@ -255,8 +317,9 @@ static int cache_close(lua_State *L)
{
struct engine *engine = engine_luaget(L);
if (engine->resolver.cache != NULL) {
- kr_cache_close(engine->resolver.cache);
+ struct kr_cache *cache = engine->resolver.cache;
engine->resolver.cache = NULL;
+ kr_cache_close(cache);
}
lua_pushboolean(L, 1);
@@ -266,6 +329,7 @@ static int cache_close(lua_State *L)
int lib_cache(lua_State *L)
{
static const luaL_Reg lib[] = {
+ { "count", cache_count },
{ "open", cache_open },
{ "close", cache_close },
{ NULL, NULL }
diff --git a/daemon/lua/config.lua b/daemon/lua/config.lua
index 116bc66a6325c3ef2b770f64be539b2595847a5f..e23e0d3d6734f45b4f53d90b859076f2712ab813 100644
--- a/daemon/lua/config.lua
+++ b/daemon/lua/config.lua
@@ -1,2 +1,8 @@
-- Default configuration
-cache.open('.', 10485760)
+cache.open(10*MB)
+-- Listen on localhost
+if not next(net.list()) then
+ if not pcall(net.listen, '127.0.0.1') then
+ error('failed to bind to localhost#53')
+ end
+end
\ No newline at end of file
diff --git a/daemon/lua/sandbox.lua b/daemon/lua/sandbox.lua
index d64dd19344728d2e5c7990c396cfe43d538f69af..e8e7fe0b671b3a7b2eb1803fe09b20afc42efc1b 100644
--- a/daemon/lua/sandbox.lua
+++ b/daemon/lua/sandbox.lua
@@ -1,3 +1,26 @@
+-- Units
+kB = 1024
+MB = 1024*1024
+GB = 1024*1024
+
+-- Function aliases
+-- `env.VAR returns os.getenv(VAR)`
+env = {}
+setmetatable(env, {
+ __index = function (t, k) return os.getenv(k) end
+})
+
+-- Quick access to interfaces
+-- `net.<iface>` => `net.interfaces()[iface]`
+setmetatable(net, {
+ __index = function (t, k)
+ local v = rawget(t, k)
+ if v then return v
+ else return net.interfaces()[k]
+ end
+ end
+})
+
-- Syntactic sugar for module loading
-- `modules.<name> = <config>`
setmetatable(modules, {
diff --git a/doc/build.rst b/doc/build.rst
index 9bae20c7335f0f2a2383e426d761d279a050fb12..eb6c38f1f81d76f2ff2e631b4b8b8ce11c2e3508 100644
--- a/doc/build.rst
+++ b/doc/build.rst
@@ -49,8 +49,8 @@ There are also *optional* packages that enable specific functionality in Knot DN
.. [#] Requires C99, ``__attribute__((cleanup))`` and ``-MMD -MP`` for dependency file generation. GCC, Clang and ICC are supported.
.. [#] You can use variables ``<dependency>_CFLAGS`` and ``<dependency>_LIBS`` to configure dependencies manually (i.e. ``libknot_CFLAGS`` and ``libknot_LIBS``).
-Docker image
-~~~~~~~~~~~~
+Getting Docker image
+--------------------
Docker images require only either Linux or a Linux VM (see boot2docker_ on OS X).
@@ -65,8 +65,10 @@ You can hack on the container by changing the container entrypoint to shell like
$ docker run -it --entrypoint=/bin/bash cznic/knot-resolver
+.. tip:: You can build the Docker image yourself with ``docker build -t knot-resolver scripts``.
+
Building from sources
-~~~~~~~~~~~~~~~~~~~~~
+---------------------
The Knot DNS Resolver depends on the development version of the Knot DNS library, and a reasonably recent version of `libuv`.
Several dependencies may not be in the packages yet, the script pulls and installs all dependencies in a chroot.
@@ -93,18 +95,20 @@ Usually you only really need to rebuild `libknot`.
$ make check libknot_CFLAGS="-I/opt/include" libknot_LIBS="-L/opt/lib -lknot -lknot-int -ldnssec"
-.. note:: If the dependencies lie outside of library search path, you need to add them somehow.
- Try ``LD_LIBRARY_PATH`` on Linux/BSD, and ``DYLD_FALLBACK_LIBRARY_PATH`` on OS X. Otherwise you might
- need to add the locations to the linker search path.
+.. warning:: If the dependencies lie outside of library search path, you need to add them somehow.
+ Try ``LD_LIBRARY_PATH`` on Linux/BSD, and ``DYLD_FALLBACK_LIBRARY_PATH`` on OS X.
+ Otherwise you need to add the locations to linker search path.
When you have all the dependencies ready, you can build, test and install.
.. code-block:: bash
- $ make
+ $ make PREFIX="/usr/local"
$ make check
$ make install
+.. note:: Always build with ``PREFIX`` if you want to install, as it is hardcoded in the executable for module search path.
+
Alternatively you can build only specific parts of the project, i.e. ``library``.
.. code-block:: bash
diff --git a/doc/conf.py b/doc/conf.py
index 7fdba137532ad5938e85cafbdc9af0f1a2952bf2..25ca5624cbcb7480a83f15b3913e001f6bcf33c2 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -49,6 +49,8 @@ html_use_smartypants = True
# Output file base name for HTML help builder.
htmlhelp_basename = 'apidoc'
+# Theme
+html_theme = 'sphinx_rtd_theme'
# -- Options for LaTeX output --------------------------------------------------
diff --git a/doc/config.rst b/doc/config.rst
deleted file mode 100644
index 870950b2265b736d6f5fb641a86975a163c1e54f..0000000000000000000000000000000000000000
--- a/doc/config.rst
+++ /dev/null
@@ -1,51 +0,0 @@
-Daemon configuration
---------------------
-
-The Knot DNS Resolver daemon has no traditional concept of static configuration.
-In it's simplest form it requires just a working directory in which it can set up persistent files like
-cache and the process state.
-
-.. code-block:: sh
-
- $ kresolved /var/run/kresolved
-
-And you're good to go!
-
-Introduction
-~~~~~~~~~~~~
-
-There are several choices on how you can configure the daemon, a RPC interface a CLI or a configuration file,
-but fortunately all share a common syntax and are transparent to each other, e.g. if you change a knob, you're going to
-see it projected to other interfaces as well.
-
-.. note:: Expect this page to change a lot, as it's still just a proof of concept implementation.
-
-Configuration 101
-~~~~~~~~~~~~~~~~~
-
-If there is a `config` file in the daemon working directory, it's going to get loaded automatically, if there isn't one
-the daemon is going to start with sane defaults and listening on `localhost`. The syntax for options is like follows: ``group.option = value``
-or ``group.action(parameters)``. You can also comment using a ``--`` prefix.
-
-A simple example would be to increase the cache size.
-
-.. code-block:: lua
-
- -- increase the cache to 100MB
- cache.open(".", 100*1024*1024)
-
-Dynamic configuration
-~~~~~~~~~~~~~~~~~~~~~
-
-Packages and services
-~~~~~~~~~~~~~~~~~~~~~
-
-The Lua supports a concept called closures, this is extremely useful for scripting actions upon various events.
-
-.. note:: TODO, come back later!
-
-* Timers and events
-* File watchers
-* Serialization
-* Data I/O
-
diff --git a/doc/daemon.rst b/doc/daemon.rst
index 1cd4004a449c6f6ea86d52d0f40a1034a2c4ebab..dbeecabfa9755cac060255c1d7fd96987becad72 100644
--- a/doc/daemon.rst
+++ b/doc/daemon.rst
@@ -1,36 +1 @@
-Knot DNS Resolver daemon
-========================
-
-Requirements
-------------
-
-* libuv_ 1.0+ (a multi-platform support library with a focus on asynchronous I/O)
-
-Starting the daemon
--------------------
-
-There is a separate resolver library in the `lib` directory, and a minimalistic daemon in
-the `daemon` directory. The daemon accepts a few CLI parameters, and there's no support for configuration
-right now.
-
-.. code-block:: bash
-
- $ ./daemon/kresolved -h
- $ ./daemon/kresolved -a 127.0.0.1#53
-
-.. _libuv: https://github.com/libuv/libuv
-
-Interacting with the daemon
----------------------------
-
-The daemon features a CLI interface if launched interactively, type ``help`` to see the list of available commands.
-You can load modules this way and use their properties to get information about statistics and such.
-
-.. code-block:: bash
-
- $ kresolved /var/run/knot-resolver
- ...
- [system] started in interactive mode, type 'help()'
- > modules.load('cachectl')
- > return cachectl.size()
- { "size": 53 }
\ No newline at end of file
+.. include:: ../daemon/README.rst
\ No newline at end of file
diff --git a/doc/index.rst b/doc/index.rst
index 18f4d2e1f87862edb0badf1d4318305a0034594e..47527df9548322e2eedf762c829cbeee5eb6bbb6 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -1,17 +1,17 @@
+#################
Knot DNS Resolver
-=================
+#################
The Knot DNS Resolver is a minimalistic caching resolver implementation.
The project provides both a resolver library and a small daemon.
Modular architecture of the library keeps the core tiny and efficient, and provides a state-machine like API for extensions.
.. toctree::
- :maxdepth: 2
+ :maxdepth: 4
build
lib
daemon
- config
modules
diff --git a/doc/lib.rst b/doc/lib.rst
index 9091a70e2fbb9cb95a44764d396b81933ab4421a..e61cb206624d8f3f01d7af27443879fdabb0425c 100644
--- a/doc/lib.rst
+++ b/doc/lib.rst
@@ -2,43 +2,8 @@
.. include:: ../lib/README.rst
-Library layout
---------------
-
-The library as described provides basic services for name resolution, which should cover the usage.
-The following part is for those who are planning to hack on the library or develop modules, to give
-you an idea about the API and the library layout.
-
-Name resolution
-~~~~~~~~~~~~~~~
-
-.. _lib_rplan:
-
-Resolution plan
-~~~~~~~~~~~~~~~
-
-.. _lib_cache:
-
-Cache
-~~~~~
-
-.. _lib_nameservers:
-
-Nameservers
-~~~~~~~~~~~
-
-.. _lib_modules:
-
-Modules
-~~~~~~~
-
-Utilities
-~~~~~~~~~
-
-.. _lib_api:
-
API reference
--------------
+=============
.. doxygengroup:: resolution
:project: libkresolve
@@ -54,6 +19,8 @@ API reference
.. doxygengroup:: nameservers
:project: libkresolve
+.. _lib_api_modules:
+
.. doxygengroup:: modules
:project: libkresolve
diff --git a/doc/modules.rst b/doc/modules.rst
index 428e09530bf086240a421985f88bb2e87707ec5c..21a2d9e51685b31385717cf8a0f15f1c4809d9c6 100644
--- a/doc/modules.rst
+++ b/doc/modules.rst
@@ -1,7 +1,13 @@
.. include:: ../modules/README.rst
+.. _modules-implemented:
+
Implemented modules
--------------------
+===================
+
+.. contents::
+ :depth: 1
+ :local:
.. include:: ../modules/hints/README.rst
-.. include:: ../modules/cachectl/README.rst
+.. include:: ../modules/cachectl/README.rst
\ No newline at end of file
diff --git a/lib/README.rst b/lib/README.rst
index 8852320040cdb8536fd597f6aa084749785984e6..7af22e49201e25cfb159ca445d544498be1a322e 100644
--- a/lib/README.rst
+++ b/lib/README.rst
@@ -1,23 +1,128 @@
+*************************
Knot DNS Resolver library
-=========================
+*************************
Requirements
-------------
+============
* libknot_ 2.0 (Knot DNS high-performance DNS library.)
-Overview
---------
+Library layout
+==============
+
+.. contents::
+ :depth: 1
+ :local:
+
+The library as described provides basic services for name resolution, which should cover the usage.
+The following part is for those who are planning to hack on the library or develop modules, to give
+you an idea about the API and the library layout.
+
+Name resolution
+---------------
+
+.. _lib_rplan:
+
+Resolution plan
+---------------
+
+.. _lib_cache:
+
+Cache
+-----
+
+.. _lib_nameservers:
+
+Nameservers
+-----------
+
+Utilities
+~~~~~~~~~
+
+.. warning:: Work in progress.
Resolving a name
-~~~~~~~~~~~~~~~~
+----------------
.. note:: Migrating from ``getaddrinfo``
-Using cache
-~~~~~~~~~~~
+Using getdns API
+----------------
+
+.. note:: These are not the droids you're looking for, move along.
+
+.. _lib-layers:
+
+Writing layers
+==============
+
+The resolver :ref:`library <lib_index>` leverages the `processing API`_ from the libknot to separate packet processing code
+into layers. In order to keep the core library sane and coverable, there are only two built-in layers:
+the :c:func:`iterate_layer`, and the :c:func:`itercache_layer`.
-Loading modules
-~~~~~~~~~~~~~~~
+*Note* |---| This is only crash-course in the library internals, see the resolver :ref:`library <lib_index>` documentation for the complete overview of the services.
+
+The library offers following services:
+
+- :ref:`Cache <lib_cache>` - MVCC cache interface for retrieving/storing resource records.
+- :ref:`Resolution plan <lib_rplan>` - Query resolution plan, a list of partial queries (with hierarchy) sent in order to satisfy original query. This contains information about the queries, nameserver choice, timing information, answer and its class.
+- :ref:`Nameservers <lib_nameservers>` - Reputation database of nameservers, this serves as an aid for nameserver choice.
+
+A processing layer is going to be called by the query resolution driver for each query,
+so you're going to work with :ref:`struct kr_layer_param <lib_api_rplan>` as your per-query context. This structure contains pointers to
+resolution context, resolution plan and also the final answer. You're likely to retrieve currently solved query from the query plan:
+
+.. code-block:: c
+
+ int consume(knot_layer_t *ctx, knot_pkt_t *pkt)
+ {
+ struct kr_layer_param *param = ctx->data;
+ struct kr_query *query = kr_rplan_current(param->rplan);
+ }
+
+This is only passive processing of the incoming answer. If you want to change the course of resolution, say satisfy a query from a local cache before the library issues a query to the nameserver, you can use states (see the :ref:`Static hints <mod-hints>` for example).
+
+.. code-block:: c
+
+ int produce(knot_layer_t *ctx, knot_pkt_t *pkt)
+ {
+ struct kr_layer_param *param = ctx->data;
+ struct kr_query *cur = kr_rplan_current(param->rplan);
+
+ /* Query can be satisfied locally. */
+ if (can_satisfy(cur)) {
+ /* This flag makes the resolver move the query
+ * to the "resolved" list. */
+ query->resolved = true;
+ return KNOT_STATE_DONE;
+ }
+
+ /* Pass-through. */
+ return ctx->state;
+ }
+
+It is possible to not only act during the query resolution, but also to view the complete resolution plan afterwards.
+This is useful for analysis-type tasks, or *"on-resolution"* hooks.
+
+.. code-block:: c
+
+ int finish(knot_layer_t *ctx)
+ {
+ struct kr_layer_param *param = ctx->data;
+ struct kr_rplan *rplan = param->rplan;
+
+ /* Print the query sequence with start time. */
+ char qname_str[KNOT_DNAME_MAXLEN];
+ struct kr_query *qry = NULL
+ WALK_LIST(qry, rplan->resolved) {
+ knot_dname_to_str(qname_str, qry->sname, sizeof(qname_str));
+ printf("%s at %u\n", qname_str, qry->timestamp);
+ }
+
+ return ctx->state;
+ }
.. _libknot: https://gitlab.labs.nic.cz/labs/knot/tree/master/src/libknot
+.. _`processing API`: https://gitlab.labs.nic.cz/labs/knot/tree/master/src/libknot/processing
+
+.. |---| unicode:: U+02014 .. em dash
diff --git a/modules/README.rst b/modules/README.rst
index 325f17a258ac9b6e049359845e28030bfe4e6b06..92aba06a2f5d75fd36c372bb489055548a4c6e24 100644
--- a/modules/README.rst
+++ b/modules/README.rst
@@ -1,90 +1,28 @@
+****************************
Knot DNS Resolver extensions
-============================
+****************************
-The resolver :ref:`library <lib_index>` leverages the `processing API`_ from the libknot to separate packet processing code
-into layers. In order to keep the core library sane and coverable, there are only two built-in layers:
-the :c:func:`iterate_layer`, and the :c:func:`itercache_layer`. The resolver context however can
-load shared libraries on runtime, which allows us to build and register external modules as well.
+Writing extensions
+==================
+
+.. contents::
+ :depth: 2
+ :local:
Supported languages
-------------------
Currently modules written in C are supported.
-There is also a rudimentary support for writing modules in Go |---| ⑴ the library has no native Go bindings, library is accessible using CGO_, ⑵ gc doesn't support building shared libraries, GCCGO_ is required, ⑶ no coroutines and no garbage collecting thread, as the Go code is called from C threads.
-
-There is a plan for Lua scriptables, but it's not implemented yet.
-
-Available services
-------------------
-
-*Note* |---| This is only crash-course in the library internals, see the resolver :ref:`library <lib_index>` documentation for the complete overview of the services.
-
-The library offers following services:
-
-- :ref:`Cache <lib_cache>` - MVCC cache interface for retrieving/storing resource records.
-- :ref:`Resolution plan <lib_rplan>` - Query resolution plan, a list of partial queries (with hierarchy) sent in order to satisfy original query. This contains information about the queries, nameserver choice, timing information, answer and its class.
-- :ref:`Nameservers <lib_nameservers>` - Reputation database of nameservers, this serves as an aid for nameserver choice.
-
-If you're going to publish a layer in your module, it's going to be called by the query resolution driver for each query,
-so you're going to work with :ref:`struct kr_layer_param <lib_api_rplan>` as your per-query context. This structure contains pointers to
-resolution context, resolution plan and also the final answer. You're likely to retrieve currently solved query from the query plan:
-
-.. code-block:: c
-
- int consume(knot_layer_t *ctx, knot_pkt_t *pkt)
- {
- struct kr_layer_param *param = ctx->data;
- struct kr_query *query = kr_rplan_current(param->rplan);
- }
+There is also a rudimentary support for writing modules in Go |---| |(1)| the library has no native Go bindings, library is accessible using CGO_, |(2)| gc doesn't support building shared libraries, GCCGO_ is required, |(3)| no coroutines and no garbage collecting thread, as the Go code is called from C threads.
-This is only passive processing of the incoming answer. If you want to change the course of resolution, say satisfy a query from a local cache before the library issues a query to the nameserver, you can use states (see the `Static hints`_ for example).
-
-.. code-block:: c
-
- int produce(knot_layer_t *ctx, knot_pkt_t *pkt)
- {
- struct kr_layer_param *param = ctx->data;
- struct kr_query *cur = kr_rplan_current(param->rplan);
-
- /* Query can be satisfied locally. */
- if (can_satisfy(cur)) {
- /* This flag makes the resolver move the query
- * to the "resolved" list. */
- query->resolved = true;
- return KNOT_STATE_DONE;
- }
-
- /* Pass-through. */
- return ctx->state;
- }
-
-It is possible to not only act during the query resolution, but also to view the complete resolution plan afterwards.
-This is useful for analysis-type tasks, or *"on-resolution"* hooks.
-
-.. code-block:: c
-
- int finish(knot_layer_t *ctx)
- {
- struct kr_layer_param *param = ctx->data;
- struct kr_rplan *rplan = param->rplan;
-
- /* Print the query sequence with start time. */
- char qname_str[KNOT_DNAME_MAXLEN];
- struct kr_query *qry = NULL
- WALK_LIST(qry, rplan->resolved) {
- knot_dname_to_str(qname_str, qry->sname, sizeof(qname_str));
- printf("%s at %u\n", qname_str, qry->timestamp);
- }
-
- return ctx->state;
- }
+.. note:: There is a plan for Lua scriptables, but it's not implemented yet.
The anatomy of an extension
---------------------------
A module is a shared library defining specific functions, here's an overview of the functions.
-*Note* |---| the :ref:`Modules <lib_modules>` header documents the module loading and API.
+*Note* |---| the :ref:`Modules <lib_api_modules>` header documents the module loading and API.
.. csv-table::
:header: "C", "Go", "Params", "Comment"
@@ -93,7 +31,7 @@ A module is a shared library defining specific functions, here's an overview of
"``X_init()``", "``Init()``", "``module``", "Constructor"
"``X_deinit()``", "``Deinit()``", "``module, key``", "Destructor"
"``X_config()``", "``Config()``", "``module``", "Configuration"
- "``X_layer()``", "``Layer()``", "", "Module layer"
+ "``X_layer()``", "``Layer()``", "", ":ref:`Module layer <lib-layers>`"
"``X_props()``", "``Props()``", "", "NULL-terminated list of properties"
.. [#] Mandatory symbol.
@@ -101,12 +39,10 @@ A module is a shared library defining specific functions, here's an overview of
The ``X_`` corresponds to the module name, if the module name is ``hints``, then the prefix for constructor would be ``hints_init()``.
This doesn't apply for Go, as it for now always implements `main` and requires capitalized first letter in order to export its symbol.
-How does the module get loaded
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The resolution context :c:type:`struct kr_context` holds loaded modules for current context. A module can be registered with :c:func:`kr_context_register`, which triggers module constructor *immediately* after the load. Module destructor is automatically called when the resolution context closes.
-
-If the module exports a layer implementation, it is automatically discovered by :c:func:`kr_resolver` on resolution init and plugged in. The order in which the modules are registered corresponds to the call order of layers.
+.. note::
+ The resolution context :c:type:`struct kr_context` holds loaded modules for current context. A module can be registered with :c:func:`kr_context_register`, which triggers module constructor *immediately* after the load. Module destructor is automatically called when the resolution context closes.
+
+ If the module exports a layer implementation, it is automatically discovered by :c:func:`kr_resolver` on resolution init and plugged in. The order in which the modules are registered corresponds to the call order of layers.
Writing a module in C
---------------------
@@ -122,7 +58,7 @@ As almost all the functions are optional, the minimal module looks like this:
Let's define an observer thread for the module as well. It's going to be stub for the sake of brevity,
but you can for example create a condition, and notify the thread from query processing by declaring
-module layer (see the `Available services`_).
+module layer (see the :ref:`Writing layers <lib-layers>`).
.. code-block:: c
@@ -243,6 +179,8 @@ Configuring modules
There is a callback ``X_config()`` but it's NOOP for now, as the configuration is not yet implemented.
+.. _mod-properties:
+
Exposing module properties
--------------------------
@@ -285,7 +223,7 @@ Here's an example how a module can expose its property:
{
static struct kr_prop prop_list[] = {
/* Callback, Name, Description */
- {&get_size, "size", "Return number of records."},
+ {&get_size, "get_size", "Return number of records."},
{NULL, NULL, NULL}
};
return prop_list;
@@ -301,14 +239,16 @@ Once you load the module, you can call the module property from the interactive
...
[system] started in interactive mode, type 'help()'
> modules.load('cached')
- > return cached.size()
+ > cached.get_size()
{ "size": 53 }
*Note* |---| this relies on function pointers, so the same ``static inline`` trick as for the ``Layer()`` is required for C/Go.
-.. _`processing API`: https://gitlab.labs.nic.cz/labs/knot/tree/master/src/libknot/processing
.. _`not present in Go`: http://blog.golang.org/gos-declaration-syntax
.. _CGO: http://golang.org/cmd/cgo/
.. _GCCGO: https://golang.org/doc/install/gccgo
.. |---| unicode:: U+02014 .. em dash
+.. |(1)| unicode:: U+2474 .. (1)
+.. |(2)| unicode:: U+2475 .. (2)
+.. |(3)| unicode:: U+2476 .. (3)
diff --git a/modules/cachectl/README.rst b/modules/cachectl/README.rst
index ebf86466f7f5b53255143250e7e72fb7f845659c..448049cbad8016299220e751bf5f148b7dacb801 100644
--- a/modules/cachectl/README.rst
+++ b/modules/cachectl/README.rst
@@ -1,16 +1,11 @@
Cache control
-~~~~~~~~~~~~~
+-------------
Module providing an interface to cache database, for inspection, manipulation and purging.
Properties
-..........
+^^^^^^^^^^
-``get_size``
- Return number of cached records.
-
- :Input: N/A
- :Output: ``{ size: int }``
``prune``
Prune expired/invalid records.
diff --git a/modules/cachectl/cachectl.c b/modules/cachectl/cachectl.c
index 59290311e89363a1b55c2fac65ee51649ebc40ea..24a78e9f4894a9d8e6d26a28213bf7b7cd20e869 100644
--- a/modules/cachectl/cachectl.c
+++ b/modules/cachectl/cachectl.c
@@ -35,32 +35,6 @@
* Properties.
*/
-/**
- * Return number of cached records.
- *
- * Input: N/A
- * Output: { size: int }
- *
- */
-static char* get_size(void *env, struct kr_module *module, const char *args)
-{
- char *result = NULL;
- struct engine *engine = env;
- const namedb_api_t *storage = kr_cache_storage();
-
- /* Fetch item count */
- namedb_txn_t txn;
- int ret = kr_cache_txn_begin(engine->resolver.cache, &txn, NAMEDB_RDONLY);
- if (ret == 0) {
- asprintf(&result, "{ \"size\": %d }", storage->count(&txn));
- kr_cache_txn_abort(&txn);
- } else {
- asprintf(&result, "{ \"error\": \"%s\" }", knot_strerror(ret));
- }
-
- return result;
-}
-
/** Return boolean true if a record in the RR set is expired. */
static int is_expired(struct kr_cache_rrset *rr, uint32_t drift)
{
@@ -165,7 +139,6 @@ static char* clear(void *env, struct kr_module *module, const char *args)
struct kr_prop *cachectl_props(void)
{
static struct kr_prop prop_list[] = {
- { &get_size, "size", "Return number of cached records.", },
{ &prune, "prune", "Prune expired/invalid records.", },
{ &clear, "clear", "Clear all cache records.", },
{ NULL, NULL, NULL }
diff --git a/modules/hints/README.rst b/modules/hints/README.rst
index 6c6e9b966d9ffbfd7f650b0577e4c37427170756..627f79954250b2fe20011b0914906ba842cd288e 100644
--- a/modules/hints/README.rst
+++ b/modules/hints/README.rst
@@ -1,4 +1,6 @@
+.. _mod-hints:
+
Static hints
-~~~~~~~~~~~~
+------------
This is a module providing static hints from ``/etc/hosts``, document me.
\ No newline at end of file