1. 23 Nov, 2021 2 commits
  2. 18 Nov, 2021 6 commits
  3. 16 Nov, 2021 2 commits
    • Karel Koci's avatar
      selftests/cli: test and fix txt and bin write and read functions · eee7b93c
      Karel Koci authored
      This tests txt_write, txt_read, bin_write and bin_read Shell methods.
      
      There were also few issues with it that were discovered during the
      testing. For example calling prompt twice causes match of new prompt.
      The output of txt_read was incosistent with text prepresentation in
      Python.
      
      The new feature here is ability of txt_write to append instead of
      overwrite.
      eee7b93c
    • Karel Koci's avatar
      tests/conftest: change scope of top level fixtures to session · 03280967
      Karel Koci authored
      This was causing issue when tests from multiple packages were run in one
      session as serial console is not correctly closed on cleanup of board
      fixture.
      
      These should be session anyway as they do not cause any conflict with
      anything else (compare to boot_serial that boots and reboots the
      router). There is no reason why we would want the new instances of these
      objects, rather the opposite.
      03280967
  4. 04 Nov, 2021 1 commit
    • Jan Miksik's avatar
      bugfix: tests wan-pppoe · cd479b30
      Jan Miksik authored
      replaced nonexisting debug mark
      the test is now skipped due to some incompatibility
      that will be investigated.
      cd479b30
  5. 13 Oct, 2021 4 commits
    • Jan Betik's avatar
      tests: Added wan-pppoe tests · 32d2f461
      Jan Betik authored and Karel Koci's avatar Karel Koci committed
      Added pppoe tests and pppoe container
      Be aware that isp-pppoe container requires mounted dev/ppp
      32d2f461
    • Jan Miksik's avatar
      tests: Added DHCPv4 tests · 987b1333
      Jan Miksik authored and Karel Koci's avatar Karel Koci committed
      Added tests:
      tests/network/test_dhcp.py:
          - ip limit
          - ip range
          - ip duplicity
          - leasetime
      987b1333
    • Karel Koci's avatar
      nsfarm/lxd: improve cleanup to remove stale containers · 2341380f
      Karel Koci authored
      This improves cleanup to not only remove old images but also a stale
      containers.
      
      There are two types of containers nsfarm spawns. Those are bootstrap
      and regular containers. The bootstrap containers are spawned with
      intention to create image while regular containers are just spawned to
      be later removed. The difference is that while regular containers are
      always tied to one specific process (the instance of nsfarm) the
      bootstrap is intentionally global to serialize multiple instances on
      single image creation.
      The cleanup algorithm can remove any regular container that's parent is
      no longer running. The bootstrap containers do not contain info about
      parent process and thus we use time to remove just old containers.
      
      Warning: This changes the template for regular container name. The
      change is needed for reliable PID parsing from template name. The
      unclear state is when image would end with dash and number.
      2341380f
    • Karel Koci's avatar
      selftests/target: use ids for target · 79af6d5f
      Karel Koci authored
      This converts argument identification from 'target0' to real name of
      target.
      79af6d5f
  6. 24 Sep, 2021 8 commits
    • Karel Koci's avatar
      0add2f74
    • Karel Koci's avatar
      Revert "imgs/selenium: use firefox-esr instead of bleeding edge" · 388b3ba7
      Karel Koci authored
      This reverts commit 532183a7.
      
      ESR is really old and it seems that issues in Firefox were removed in
      the end.
      388b3ba7
    • Karel Koci's avatar
      tests/reforis: add guide tests · abd6b85f
      Karel Koci authored
      These are simple "click trough" tests of reForis guide.
      
      It only ensures that we can correctly pass through guide. The tests of
      dialogues we are passing trough should be implemented.
      abd6b85f
    • Karel Koci's avatar
      tests: decrease scope of fixture to just package · c4e16a31
      Karel Koci authored
      The scope of session reaches not only tests but also selftests and in
      general we want to keep those two separate thus these fixtures should be
      limited by package they are defined in.
      
      In some cases this is rather extension because for some reason for
      example client_board was defined with module scope. The effect of that
      was that client was been reconnected for every module. Making that for
      whole package saves a little bit of time. The original idea was to make
      sure that we are connected but if ever test plays with local network it
      on teardown it has to restore original settings and thus it should make
      original connection available again.
      c4e16a31
    • Karel Koci's avatar
      nsfarm/cli: improve typing · 98467dd4
      Karel Koci authored
      This covers some mising types specifiers.
      98467dd4
    • Karel Koci's avatar
      requirements: we do not use lazy-fixtres · 86f4bbe2
      Karel Koci authored
      The original use for lazy-fixtures was pretty much just to test if they
      help us to solve organization issue but they did not. Now they are
      unused and we do not plan to use them right now so they should not be
      part of requirements.
      86f4bbe2
    • Karel Koci's avatar
      imgs/selenium: use firefox-esr instead of bleeding edge · b6ba4e4a
      Karel Koci authored
      This switches to stable Firefox. There are some issues with bleeding
      edge Firefox and we should ensure that we work with stable version more
      than with newest one.
      b6ba4e4a
    • Karel Koci's avatar
      nsfarm/cli: visualize direction by prefix · 5222cc59
      Karel Koci authored
      This is just for better orientation in logs. The idea is that anything
      with > or < is text while the rest are just regular messages from NSFarm.
      5222cc59
  7. 18 Aug, 2021 2 commits
    • Karel Koci's avatar
      nsfarm/board: do not use target name in logging · 85e13f93
      Karel Koci authored
      We do not run tests on multiple boards at the same time and it is
      technically undesirable in terms of how current tests are setup. That
      makes inclusion of target name just a duplicate value as target is
      stated in meta-data for tests and thus directly visible at the start
      test report.
      85e13f93
    • Karel Koci's avatar
      tests: upload test key instead of installation · c8b9f442
      Karel Koci authored
      This makes it much faster and reduces the hack. It also makes this
      setting independent on the Internet connection.
      c8b9f442
  8. 03 Aug, 2021 7 commits
    • Karel Koci's avatar
      nsfarm/lxd: fix image hash calculation · c56829c8
      Karel Koci authored
      The issue here is that we should not use full path to file never to
      calculate hash as different location of NSFarm project result in
      different hashes and thus we do not share them at all.
      This makes one instance where we included full path just relative to
      images directory. This way the path should be contant no matter where
      the project is located.
      
      This also does small tweak in form of removal of unnecessary
      comprehension error. We can convert directly to list without creating
      generator.
      c56829c8
    • Karel Koci's avatar
      tests: use container.shell for wait scripts invocation · 8197146c
      Karel Koci authored
      The previous commit added shell property exactly for this. This now
      tweaks code to use it.
      8197146c
    • Karel Koci's avatar
      nsfarm/web: add Selenium container class · 335126c4
      Karel Koci authored
      This implements dedicated Selenium container class as an extension to
      LXD Container. This class provides easy access to Selenium drivers for
      selected browser.
      The special hack in Selenium extended Container class is that it is able
      to run vncviewer on its start. This is controlled by class variable that
      can be set for nsfasrm run. This way developer can select to see what is
      happening on virtual desktop browsers are spawned to.
      
      With all this there is also need to store some screenshots as part of
      failed or even standard execution of tests. The screenshot fixture was
      created for this purpose. It hides some ugliness of screenshot saving.
      
      Minor but notable tweak here is addition of shell property to
      container. The idea is to prevent spawning of multiple one-time use
      shells we do for wait commands. The regular usage for testing is
      discouraged.
      335126c4
    • Karel Koci's avatar
      0657f024
    • Karel Koci's avatar
      imgs/selenium: add new image · 32e311dd
      Karel Koci authored
      This is image that can be used to test web interface.
      
      It is based on client container but we do not want to put this as part
      of it. It generates pretty big image and in most tests it is not
      necessary to start web drivers.
      
      The container itself provides driver for Chrome (Chromium) and Firefox
      browsers.
      There is also driver for WebKit but it does not work, it crashes right
      now so it is there just for the future once something is fixed in
      upstream or we found out what we are doing wrong.
      32e311dd
    • Karel Koci's avatar
      nsfarm/lxd: add ability to proxy TCP and UDP ports · bc286a97
      Karel Koci authored
      This allows access of TCP and UDP services trough container. This can be
      used not only to access services running locally in container but also
      any service accessible from container network.
      
      The simple use case of this is access of web interfaces of router.
      bc286a97
    • Karel Koci's avatar
      imgs/base-alpine: fix mount of /dev/shm · a77aaddd
      Karel Koci authored
      The Alpine Linux creates /dev/shm using init script devfs but that is
      not run in container. It won't even run correctly in container. That
      makes /dev/shm unavailable but we need it in some cases and thus this is
      hacky script that adds just very small init script that creates it.
      a77aaddd
  9. 22 Jun, 2021 3 commits
    • Karel Koci's avatar
      tests/test_bootup: do not match exactly · 7a747bbf
      Karel Koci authored
      The issue here is that some applications such as sshd are modifying its
      own arguments to store readable info in there. This breaks check for
      processes but they keep its name intact and so we can match that.
      
      The -x is replaced with -a to show process info in logs so we can
      identify if we match something that is not suppose to be matched.
      7a747bbf
    • Karel Koci's avatar
      nsfarm/board: wait for boot longer · 55bcefc4
      Karel Koci authored
      This is to support HBD that right now boots way longer.
      Waiting four minutes for boot is very long time but for testing it is
      reasonable to margin for boot.
      55bcefc4
    • Jan Miksik's avatar
      tests: Added basic throughput tests · c385d63a
      Jan Miksik authored and Karel Koci's avatar Karel Koci committed
      Added basic throughput tests for basic testing.
      Test is run for 60 seconds and checks if the speed is at least
      400 Mbps.
      It also checks min/max of 10s chunks.
      And checks if some of chunks is under the limit - raises warning.
      Prints the speeds to output log.
      c385d63a
  10. 21 Jun, 2021 2 commits
    • Jan Miksik's avatar
      feature/lxd: added network to container · 05676899
      Jan Miksik authored
      added property of network as class to conatiner instances to be
      more easily available for usage.
      05676899
    • Jan Miksik's avatar
      nsfarm/cli: update of file handling · 118ade71
      Jan Miksik authored
      There was problem with reading and writing files in Shell class
      using file_read and file_write.
      Additionally added option to send string to write file, which
      is converted to bytes and sent.
      118ade71
  11. 15 Apr, 2021 3 commits
    • Karel Koci's avatar
      tests: move target specification to top level · 8cc886cb
      Karel Koci authored
      This allows to run selftests not on all available targets but only on
      specified one. This way we do not pollute the results with test of any
      other target.
      
      This also improves marks initialization. Marks with limited effect
      (those are lan1 and lan2) are defined in runtime. This makes their
      definition and effect at the same level and thus we should be able to
      better manage them.
      8cc886cb
    • Karel Koci's avatar
      selftests: add simple tests for cli shell · db1b9aab
      Karel Koci authored
      This is simple test on OpenWrt as well as on Alpine of nsfarm's Shell
      abstraction.
      db1b9aab
    • Karel Koci's avatar
      tests: use ping instead of sleep to check for network readiness · cae250c2
      Karel Koci authored
      There is an issue that although software reports interface being up a
      few initial frames won't pass. The original "fix" for it was to just
      wait a moment but that brakes depending on hardware we are running tests
      on. Although it is not ideal we instead use ping. Pinging our gateway
      that is our ISP container should be enough to make sure that connection
      is established.
      At the same time this is not in any way required for DHCP. The reason is
      because to get IP address we have to communicate with ISP and thus
      connection is established for sure. This means that any sleep is nor any
      other wait is required there.
      cae250c2