- Oct 13, 2021
-
-
Karel Koci authored
This converts argument identification from 'target0' to real name of target.
-
- Sep 24, 2021
-
-
Karel Koci authored
-
Karel Koci authored
This reverts commit 532183a7. ESR is really old and it seems that issues in Firefox were removed in the end.
-
Karel Koci authored
These are simple "click trough" tests of reForis guide. It only ensures that we can correctly pass through guide. The tests of dialogues we are passing trough should be implemented.
-
Karel Koci authored
The scope of session reaches not only tests but also selftests and in general we want to keep those two separate thus these fixtures should be limited by package they are defined in. In some cases this is rather extension because for some reason for example client_board was defined with module scope. The effect of that was that client was been reconnected for every module. Making that for whole package saves a little bit of time. The original idea was to make sure that we are connected but if ever test plays with local network it on teardown it has to restore original settings and thus it should make original connection available again.
-
Karel Koci authored
This covers some mising types specifiers.
-
Karel Koci authored
The original use for lazy-fixtures was pretty much just to test if they help us to solve organization issue but they did not. Now they are unused and we do not plan to use them right now so they should not be part of requirements.
-
Karel Koci authored
This switches to stable Firefox. There are some issues with bleeding edge Firefox and we should ensure that we work with stable version more than with newest one.
-
Karel Koci authored
This is just for better orientation in logs. The idea is that anything with > or < is text while the rest are just regular messages from NSFarm.
-
- Aug 18, 2021
-
-
Karel Koci authored
We do not run tests on multiple boards at the same time and it is technically undesirable in terms of how current tests are setup. That makes inclusion of target name just a duplicate value as target is stated in meta-data for tests and thus directly visible at the start test report.
-
Karel Koci authored
This makes it much faster and reduces the hack. It also makes this setting independent on the Internet connection.
-
- Aug 03, 2021
-
-
Karel Koci authored
The issue here is that we should not use full path to file never to calculate hash as different location of NSFarm project result in different hashes and thus we do not share them at all. This makes one instance where we included full path just relative to images directory. This way the path should be contant no matter where the project is located. This also does small tweak in form of removal of unnecessary comprehension error. We can convert directly to list without creating generator.
-
Karel Koci authored
The previous commit added shell property exactly for this. This now tweaks code to use it.
-
Karel Koci authored
This implements dedicated Selenium container class as an extension to LXD Container. This class provides easy access to Selenium drivers for selected browser. The special hack in Selenium extended Container class is that it is able to run vncviewer on its start. This is controlled by class variable that can be set for nsfasrm run. This way developer can select to see what is happening on virtual desktop browsers are spawned to. With all this there is also need to store some screenshots as part of failed or even standard execution of tests. The screenshot fixture was created for this purpose. It hides some ugliness of screenshot saving. Minor but notable tweak here is addition of shell property to container. The idea is to prevent spawning of multiple one-time use shells we do for wait commands. The regular usage for testing is discouraged.
-
Karel Koci authored
-
Karel Koci authored
This is image that can be used to test web interface. It is based on client container but we do not want to put this as part of it. It generates pretty big image and in most tests it is not necessary to start web drivers. The container itself provides driver for Chrome (Chromium) and Firefox browsers. There is also driver for WebKit but it does not work, it crashes right now so it is there just for the future once something is fixed in upstream or we found out what we are doing wrong.
-
Karel Koci authored
This allows access of TCP and UDP services trough container. This can be used not only to access services running locally in container but also any service accessible from container network. The simple use case of this is access of web interfaces of router.
-
Karel Koci authored
The Alpine Linux creates /dev/shm using init script devfs but that is not run in container. It won't even run correctly in container. That makes /dev/shm unavailable but we need it in some cases and thus this is hacky script that adds just very small init script that creates it.
-
- Jun 22, 2021
-
-
Karel Koci authored
The issue here is that some applications such as sshd are modifying its own arguments to store readable info in there. This breaks check for processes but they keep its name intact and so we can match that. The -x is replaced with -a to show process info in logs so we can identify if we match something that is not suppose to be matched.
-
Karel Koci authored
This is to support HBD that right now boots way longer. Waiting four minutes for boot is very long time but for testing it is reasonable to margin for boot.
-
Added basic throughput tests for basic testing. Test is run for 60 seconds and checks if the speed is at least 400 Mbps. It also checks min/max of 10s chunks. And checks if some of chunks is under the limit - raises warning. Prints the speeds to output log.
-
- Jun 21, 2021
-
-
Jan Miksik authored
added property of network as class to conatiner instances to be more easily available for usage.
-
Jan Miksik authored
There was problem with reading and writing files in Shell class using file_read and file_write. Additionally added option to send string to write file, which is converted to bytes and sent.
-
- Apr 15, 2021
-
-
Karel Koci authored
This allows to run selftests not on all available targets but only on specified one. This way we do not pollute the results with test of any other target. This also improves marks initialization. Marks with limited effect (those are lan1 and lan2) are defined in runtime. This makes their definition and effect at the same level and thus we should be able to better manage them.
-
Karel Koci authored
This is simple test on OpenWrt as well as on Alpine of nsfarm's Shell abstraction.
-
Karel Koci authored
There is an issue that although software reports interface being up a few initial frames won't pass. The original "fix" for it was to just wait a moment but that brakes depending on hardware we are running tests on. Although it is not ideal we instead use ping. Pinging our gateway that is our ISP container should be enough to make sure that connection is established. At the same time this is not in any way required for DHCP. The reason is because to get IP address we have to communicate with ISP and thus connection is established for sure. This means that any sleep is nor any other wait is required there.
-
Karel Koci authored
This is causing issues because containers in some cases are not starting fast enough. We should always make sure that container is prepared for tests and for exactly that purpose wait scripts available.
-
Karel Koci authored
The Omnia wan interface is eth2 not eth0.
-
Karel Koci authored
One todo is about not working flush but that was resolved by 84e6788e and thus no longer relevant. The second one is about DNS to be specified to ISP but that is already set to local ISP's address.
-
Karel Koci authored
This test was written on old version of LXD API and wasn't updated.
-
Karel Koci authored
This can potentially take a while thanks to slow connection or repo.turris.cz load. Two minutes should be enough time to download medkit and do preparation.
-
- Apr 13, 2021
-
-
Karel Koci authored
This changes check from carrier being up to just interface being up. That is actually the correct check we want to do. The carrier might be down if we are connected directly to board (not trough switch) and thus this test might fail even if everything is all right. In reality we want to check that interface is set to be up and thus is going to be up once carrier is up. This information is encoded in flags as lowest bit (this was discovered by checking flag value rather not from documentation). So this test now only checks if this bit is set or not.
-
This allows better logging control when we run nsfarm tool directly.
-
This changes the order of setup. The original pretty much relied on configuration change happening before service actually started and system booted. This of course can't be ensured and is pretty fragile. In general we can change file access or any file content without waiting for system boot but once we want to communicate with services or to access the Internet we need to wait for system to actually boot. The clean effect here is the need to reload networking service once we modify interfaces after boot. This should have always been there.
-
These tests aim to check if firewall is well configured in such a way attackers can be caught by Sentinel.
-
This way we can have easy access to ISP container as well as to WAN configuration. This is required as we have to know WAN IPv4 address in some generic way. We could define constant but this way it is prepared for standard ISP being based rather on DHCP in the future.
-
This is only alias for sending ^C but this way it is way less cryptic in tests them self.
-
The idea behind this is that we have more descriptive names of marks and thus it should be easier to understand why some tests are marked with given combination of boards as they are. At the same time it should be easier to go trough them at one location once we add new boards. The new board would have to be selectively added to appropriate marks and all tests using these marks would be automatically handled. Of course that does not solve it fully but still it reduces the burden on introducing new boards.
-
This removes concept of exclusive device. The only use of it was to pass exclusive access to network interface but that is not essentially required as it is even more versatile to use macvlan as thus we can easily spawn multiple containers to simulate network. The only known use for physical device pass trough and thus exclusive is Wi-Fi. It won't be possible to use macvlan for it. At the same time this is not an issue as it is not expected that we are going to be reusing this interface in single tests run multiple times over and over. In the end there is no need to automatically suspend containers to steal devices as it has been implemented (and in reality not finished). The introduced device management now required all devices to be defined in image as attributes. This gives image definition control over name of this device in container. It is up to container user to assign appropriate real device for it. This is done using device map that is simply pair of attribute and real device specifier. This concept can be in future expanded to even encode additional configuration if have need for it.
-
Karel Koci authored
-