GitLab

The `lxc exec` expects `--` before the command. It seems to work even
without it but it behaves weirdly in cases when application relies on
its name (that is argv[0] and example of such application is Busybox).

This allows you to specify name for container. This name is used only in
logs and if not specified the image name is used. This should
primarily be used when you spawn multiple containers from same image
side by side. Then specifying name allows easy recognition of different
containers in the logs.

The '-' is invalid so this replaces it with 'x'. For consistency this
also changes it in such a way that it is always appended so the hidden
issue like this won't happen again. It also makes the naming scheme
consistent.

This tests txt_write, txt_read, bin_write and bin_read Shell methods.

There were also few issues with it that were discovered during the
testing. For example calling prompt twice causes match of new prompt.
The output of txt_read was incosistent with text prepresentation in
Python.

The new feature here is ability of txt_write to append instead of
overwrite.

This was causing issue when tests from multiple packages were run in one
session as serial console is not correctly closed on cleanup of board
fixture.

These should be session anyway as they do not cause any conflict with
anything else (compare to boot_serial that boots and reboots the
router). There is no reason why we would want the new instances of these
objects, rather the opposite.

replaced nonexisting debug mark
the test is now skipped due to some incompatibility
that will be investigated.

Jan Betik authored Nov 12, 2019 and Karel Koci committed Oct 13, 2021

Added pppoe tests and pppoe container
Be aware that isp-pppoe container requires mounted dev/ppp

Jan Miksik authored Aug 20, 2021 and Karel Koci committed Oct 13, 2021

Added tests:
tests/network/test_dhcp.py:
    - ip limit
    - ip range
    - ip duplicity
    - leasetime

This improves cleanup to not only remove old images but also a stale
containers.

There are two types of containers nsfarm spawns. Those are bootstrap
and regular containers. The bootstrap containers are spawned with
intention to create image while regular containers are just spawned to
be later removed. The difference is that while regular containers are
always tied to one specific process (the instance of nsfarm) the
bootstrap is intentionally global to serialize multiple instances on
single image creation.
The cleanup algorithm can remove any regular container that's parent is
no longer running. The bootstrap containers do not contain info about
parent process and thus we use time to remove just old containers.

Warning: This changes the template for regular container name. The
change is needed for reliable PID parsing from template name. The
unclear state is when image would end with dash and number.

This converts argument identification from 'target0' to real name of
target.

This reverts commit 532183a7.

ESR is really old and it seems that issues in Firefox were removed in
the end.

These are simple "click trough" tests of reForis guide.

It only ensures that we can correctly pass through guide. The tests of
dialogues we are passing trough should be implemented.

The scope of session reaches not only tests but also selftests and in
general we want to keep those two separate thus these fixtures should be
limited by package they are defined in.

In some cases this is rather extension because for some reason for
example client_board was defined with module scope. The effect of that
was that client was been reconnected for every module. Making that for
whole package saves a little bit of time. The original idea was to make
sure that we are connected but if ever test plays with local network it
on teardown it has to restore original settings and thus it should make
original connection available again.

This covers some mising types specifiers.

The original use for lazy-fixtures was pretty much just to test if they
help us to solve organization issue but they did not. Now they are
unused and we do not plan to use them right now so they should not be
part of requirements.

This switches to stable Firefox. There are some issues with bleeding
edge Firefox and we should ensure that we work with stable version more
than with newest one.

This is just for better orientation in logs. The idea is that anything
with > or < is text while the rest are just regular messages from NSFarm.

We do not run tests on multiple boards at the same time and it is
technically undesirable in terms of how current tests are setup. That
makes inclusion of target name just a duplicate value as target is
stated in meta-data for tests and thus directly visible at the start
test report.

This makes it much faster and reduces the hack. It also makes this
setting independent on the Internet connection.

The issue here is that we should not use full path to file never to
calculate hash as different location of NSFarm project result in
different hashes and thus we do not share them at all.
This makes one instance where we included full path just relative to
images directory. This way the path should be contant no matter where
the project is located.

This also does small tweak in form of removal of unnecessary
comprehension error. We can convert directly to list without creating
generator.

The previous commit added shell property exactly for this. This now
tweaks code to use it.

This implements dedicated Selenium container class as an extension to
LXD Container. This class provides easy access to Selenium drivers for
selected browser.
The special hack in Selenium extended Container class is that it is able
to run vncviewer on its start. This is controlled by class variable that
can be set for nsfasrm run. This way developer can select to see what is
happening on virtual desktop browsers are spawned to.

With all this there is also need to store some screenshots as part of
failed or even standard execution of tests. The screenshot fixture was
created for this purpose. It hides some ugliness of screenshot saving.

Minor but notable tweak here is addition of shell property to
container. The idea is to prevent spawning of multiple one-time use
shells we do for wait commands. The regular usage for testing is
discouraged.

This is image that can be used to test web interface.

It is based on client container but we do not want to put this as part
of it. It generates pretty big image and in most tests it is not
necessary to start web drivers.

The container itself provides driver for Chrome (Chromium) and Firefox
browsers.
There is also driver for WebKit but it does not work, it crashes right
now so it is there just for the future once something is fixed in
upstream or we found out what we are doing wrong.

This allows access of TCP and UDP services trough container. This can be
used not only to access services running locally in container but also
any service accessible from container network.

The simple use case of this is access of web interfaces of router.

The Alpine Linux creates /dev/shm using init script devfs but that is
not run in container. It won't even run correctly in container. That
makes /dev/shm unavailable but we need it in some cases and thus this is
hacky script that adds just very small init script that creates it.

The issue here is that some applications such as sshd are modifying its
own arguments to store readable info in there. This breaks check for
processes but they keep its name intact and so we can match that.

The -x is replaced with -a to show process info in logs so we can
identify if we match something that is not suppose to be matched.

This is to support HBD that right now boots way longer.
Waiting four minutes for boot is very long time but for testing it is
reasonable to margin for boot.

Jan Miksik authored May 25, 2021 and Karel Koci committed Jun 22, 2021

Added basic throughput tests for basic testing.
Test is run for 60 seconds and checks if the speed is at least
400 Mbps.
It also checks min/max of 10s chunks.
And checks if some of chunks is under the limit - raises warning.
Prints the speeds to output log.

added property of network as class to conatiner instances to be
more easily available for usage.

There was problem with reading and writing files in Shell class
using file_read and file_write.
Additionally added option to send string to write file, which
is converted to bytes and sent.

This allows to run selftests not on all available targets but only on
specified one. This way we do not pollute the results with test of any
other target.

This also improves marks initialization. Marks with limited effect
(those are lan1 and lan2) are defined in runtime. This makes their
definition and effect at the same level and thus we should be able to
better manage them.

This is simple test on OpenWrt as well as on Alpine of nsfarm's Shell
abstraction.

There is an issue that although software reports interface being up a
few initial frames won't pass. The original "fix" for it was to just
wait a moment but that brakes depending on hardware we are running tests
on. Although it is not ideal we instead use ping. Pinging our gateway
that is our ISP container should be enough to make sure that connection
is established.
At the same time this is not in any way required for DHCP. The reason is
because to get IP address we have to communicate with ISP and thus
connection is established for sure. This means that any sleep is nor any
other wait is required there.

This is causing issues because containers in some cases are not starting
fast enough. We should always make sure that container is prepared for
tests and for exactly that purpose wait scripts available.

The Omnia wan interface is eth2 not eth0.

One todo is about not working flush but that was resolved by
84e6788e and thus no longer  relevant.

The second one is about DNS to be specified to ISP but that is already
set to local ISP's address.

This test was written on old version of LXD API and wasn't updated.

This can potentially take a while thanks to slow connection or
repo.turris.cz load. Two minutes should be enough time to download
medkit and do preparation.