* Provide function `get_turris_serial` returning serial number to be
  used inside Lua

Make sure that serial number is a number and divisible by 11 before
passing it back to lua.

This is a dependency for staged updates. We need to somehow group
various routers together and gradually add more and more routers to
every new release. To do that we need a way how to uniquely identify
them and serial number is here precisely for that. This way we will be
able to use it in lua scripts and point right batch of routers to the
right direction.

* Use escape sequences only on terminal
* Do not fail when `--state-log` or `--task-log` is provided. Output a warning and
  ignore them instead for a sake of backward compatibility.

Do not fail when `--state-log` or `--task-log` is provided. Output a
warning and ignore them instead for a sake of backward compatibility. We
encountered that in some conditions, updater might be run the old way
during migration from TOS 5.0 to TOS 6.0 and this should prevent
errors and half-installations in such cases.

When redirecting the output, do not use the escape sequences. Those are
nice and useful when running from console, but they clutter the log when
redirected somewhere else.

Fixed
* Invalid keyword in documentation stating that transaction ends with `STOP` but
  it is signaled by keyword `END` instead
* Using correct hashing function to detect changes to non-configuration files

Following error

[string "backend"]:1225: [string "backend"]:380: attempt to call global 'TRACE' (a nil value)

was introduced in 37960dd9
This should fix it.

We have a nice code that detects the hash used and chooses the
appropriate hashing function to use when detecting changes in
configuration. Let's split it to a separate function and use it to
detect changes in the rest of the package as well.

There was a mismatch between documentation and the implementation where
documentation stated `STOP` as the keyword for transaction end and the
code instead was using `END`. In the end it doesn't matter which we
choose and it is easier for use to just tweak the documentation instead
changing the code now and thus from now on it is `END` and not `STOP`.

It turns out that calculating hash can be pretty time consuming
specially if device the package is installed on is too slow. These trace
messages should give us a hint that updater is not just stuck that it
only waits for storage to calculate the hash of the files.

Added
* Change log that leaves minimal trace of changes updater performed
  stored in system.
* Support for `FilesSignature` field in packages. On mismatch it trigger
  reinstall.
* Support for `files-sha256sum` in packages as hash index

Fixed
* Subprocess call is now terminated way earlier thanks to `SIGCHLD`
  signal handling. This improves update time for any scripts spawning
  "daemon" processes that do not correctly redirect or close standard
  outputs.
* Reinstall of packages if some fields were formated differently in
  index than in package (updater no longer detects change by simple
  string comparison)

Changed
* Internal implementation of base64 replaced with base64c library.
* Utility opkg-create template and input format

Removed
* `--state-log` argument
* `--task-log` argument

This fixed issues that package is reinstalled if Depends field has a
different formating (spaces) than what is specified in the package. This
is caused by OPKG formating it automatically.
The solution here is to parse the dependency and compare parsed tree
instead. It is not exactly ideal solution as we have to parse
dependencies possibly multiple times but it is for now the easiest. The
ideal way would be to have here already parsed form we could compare
directly.

The hashes of modified files are now stored in file *.files-sha256sum
instead of *.files-sha256.

It turns out that we also were not reading the *.files-sha256 and thus
move of modified files actually did not work.

In reality base64c library is based on this code so it is pretty much
same code just moved to separate library. The reason for the move is
that we want to use it in other projects as well and having it as
dedicated library should give us less code to maintain and fix.

This updates C tests to run only as one binary.

The important change is that tests are enabled explicitly and thus in
default disabled. The same now also applies to linters.

This updates code to be better and in general just more sensible bash.
This also modifies the template by removing control directory. The files
that can go to control archive are strictly defined thus we can have
them in top level directory and just cherry pick them.

Karel Koci authored Nov 03, 2020 and Karel Koci committed Oct 04, 2021

The Changelog replaces previous logs that are right now pretty much no
longer relevant and in case of state log not even used.

This is replacement for task log. It is suppose to server as long term
log that is easy to parse and thus easy to use in scripts.

The primary use for this is in reporting changes done to system by
updater.

The subprocess now handles SIGCHLD for faster subprocess termination. We
care about the original child and nothild else. If that dies we consider
it as termination of subprocess and abandon reading from pipes even when
there could be still process writing to them.

This should prevent from scripts hanging.

The issue here was that invalid function was used to mask extra option
version. It was called before the correct call and thus message was
reported for version instead of ignore argument.

Changed
* Error generated from Lua when URI is being finished now includes error
  message from downloader if it was download failure.

Linters in general sometimes do invalid assumptions and try to rather
report possible issue over ignoring it. This makes them kind of
unreliable as tests required for success. The failed tests are still
manifested with alert but it clearly differentiates between just
possible error and truly hones error detected in tests.

The download error reported on URI finish was handled by else branch
that only printed error that download failed. This reaches to downloader
for error message in case of download fail. This should expand info
available to users on download fail.

We need this to cover case when files change in package without anything
other changing. The issue this tries to cover is conflicts between
packages because two packages provide same file but in reality that is
only because local version is older than the one in repository. This
forces reinstall and thus resolved such collision.

Fixed
- Immediate reboot not being performed when combined with replan that actually
  performs some changes in the system

The replan correctly passed flag to pkgupdate that reboot has to be
performed but if that run did any changes we unconditionally overwrote
it. This fixes that.

The test verifies that fix is really working by forcing installation and
removal of package while at the same time requesting reboot.

Fixed
* error on transaction recovery created by updater before version 69.0.1
  about `upgraded_packages` being nil

On update the old transaction created by updater version before 69.0.1
does not contain this field. We should not expect it. In worst case it
is going to be reported as not upgrade as it was previously.

Changed
* Package requests solver no longer tries to maximize number of selected
  requests but rahter follows strictly rules of requests priority
* `Install` requests with same priority as `Unistall` are resolved first
* `Install` and `Unistall` requests without condition as resolved before
  requests with condition given they have same priority specified.

Removed
* Error reported when request to install and unistall same package was
  specified

The original implementation tries to satisfy maximum number of requests
in given priority level. This has been seen as ideal solution because
this way most of the user's requests are satisfied.
This changes this behavior to rather be strictly defined. This means
that we do not care about number of requests satisfied. We care only
about priority of request and satisfy them in order.

This also removes error raised when there was Install and Uninstall
request for same package with same priority. Now Install request is
simply preferred.

This was required because of conditions. The demonstration of this issue
can be done on two packages. Let's have package 'a' and 'b'. There are
three requests of same priority:
* To install 'a'
* To unistall 'b'
* To install 'b' with condition 'a'
The intuitive resolution is that user clearly wants to install 'a' and
not to have 'b'. The third request need not apply as there is more
accurate request to not have 'b'.
The original implementation tries to solve this by maximizing number of
satisfied requests. This has three solution, which is on its own pretty
problematic. The first solution is the intuitive one. Second solution is
to install 'a' as well as 'b'. But the third solution is to not install
any package. This is because third request can be satisfied either by
installing 'b' or by not installing 'a'. All of these solutions are
equivalent in solvers conditions as they all result in only one of the
requests to not be satisfied, thus we have no direct way to hint solver
what to do. What is even worst we even don't know what should be the
correct solution without defining additional resolution rules.

Using the new algorithm the solution for problematic request combination
is simply that we order them using our new defined rules, that is also
the order they are ordered in example. Now we take and resolve one
request at the time. We can clearly satisfy installation of 'a' and thus
first request. We can also satisfy not-installation of 'b' and thus
second request. The third request now can't be satisfied and thus
resulting in just 'a' installed.

Full order rules are described in documentation. Reasoning for them is
also hinted but to be complete there are also alternative orders we
could choose. The reasoning is as follows:
* We need to resolve critical packages first to make sure that all of
  them can be satisfied (so we would not close some combination by
  regular requests).
* The user specified priority has to be abided.
* The requests without condition are preferred as they clearly state
  what should happen. Conditions can have side effect such as forcing
  some specific combination to satisfy given request. This is of course
  highly undesirable as condition should control only if request is
  satisfied not to influence packages specified in condition. Given the
  nature of satisfiability solver this backward injection is just going
  to happen so we instead simply prioritize requests that have no such
  effect.
* Another major change buried in these changes is that Install and
  Uninstall requests for same package no longer raise error. We prefer
  Install requests as a choice. The removal of error is because it was
  arbitrary anyway. There are simple ways Install and Unistall requests
  can be in direct collision without specifying same package name, such
  example being if package 'a' depends on 'b' and there is request to
  install 'a' and to unistall 'b'.
* The last resort order is order of definition is scripts. In the effect
  this is not that important. This affects only requests that have same
  priority, same type (install or uninstall) and both have or not have
  condition. Such requests effectively want the same thing so they
  hopefully should not collide that much in their requests.

Fixed
* environment variable `PKG_UPGRADE` were always set to `1` for package
  `postinst` script no matter if it was package upgrade or not.

Changed
* usage of `example.org` replaced with `application-test.turris.cz` in
  tests

This simply adds these script to dep-package as well as to test-package.
It is nice test for this as one package is newly installed and another
is only upgraded.

The issue here was that control fields in status were updated in
pkg_move function but same and thus invalid check was used in
pkg_scripts later on.
This adds new table with all upgraded packages. This way we check
original state before it is removed from status and pass this info to
subsequent pkg_scripts.

Web example.org now responds with valid HTML site on any URL thus we
have to use different web to get 404 response. We use our own server
that is used in other tests already.

It also turns out that curl changed error messages so instead of
matching exact error message this just tries to match the known minimum.

The usage of this was removed in commit
6e670a7a.