1. 13 Apr, 2021 4 commits
    • Karel Koci's avatar
      conftest: make pytest-html optional · e7360410
      Karel Koci authored
      We do not need pytest-html for normal functionality. It is just nice
      plugin we can use to generate human readable report to go alongside with
      pytest native xml and log output.
      We also support only pytest-html with minimal version 2.1.0 and thus all
      this is included only if pytest-html is at least of that version.
      
      The plugin is removed by this from requirements.txt as this file lists
      necessary Python packages and this is now only optional.
      e7360410
    • Karel Koci's avatar
      tests: start using deploy mark · 34197d04
      Karel Koci authored and Karel Koci's avatar Karel Koci committed
      The tests added in df0bf8d057735d316fc35f36928b2875b39e3f87 made it
      necessary to start using deploy mark. The deploy mark is used on tests
      needed to pass for deploy to proceed. These should be minimal necessary
      tests and should be fast enough.
      The reasoning why we need it here is because in most cases it should be
      enough the check in the test_no_wan that runs fast. The additional
      slower but more general test TestNoInternetAccess should cover only
      corner case and thus is not essentially required as a blocker for
      deploy.
      34197d04
    • Karel Koci's avatar
      test_bootup: check not only for config for disabled network on boot · b592db4b
      Karel Koci authored and Karel Koci's avatar Karel Koci committed
      This adds additional test that tries if router really does not try to
      receive address from DHCP even if not configured. This is even more
      powerful test but thanks to need to blindly wait it can take some
      considerable amount of time and thus does not replace test_no_wan but
      complements it.
      b592db4b
    • Karel Koci's avatar
      test-bootup: check if router does not specify WAN access after boot · 7888e362
      Karel Koci authored and Karel Koci's avatar Karel Koci committed
      This checks if there is no configuration fo WAN after clean boot. This
      is intended as default that foces user to go immediately trough first
      setup guide and configure router and primarily the administration
      password.
      7888e362
  2. 12 Apr, 2021 1 commit
    • Josef Schlehofer's avatar
      README: prefer pip3 instead of pip · 32a51b55
      Josef Schlehofer authored and Karel Koci's avatar Karel Koci committed
      I tried to run pip command on Debian Bullseye and it failed with
      following output:
      
      Collecting pytest>=5.0 (from -r requirements.txt (line 1))
        Could not find a version that satisfies the requirement pytest>=5.0 (from -r requirements.txt (line 1)) (from versions: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.2, 2.7.3, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6, 2.8.7, 2.9.0, 2.9.1, 2.9.2, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5, 3.0.6, 3.0.7, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4, 3.2.5, 3.3.0, 3.3.1, 3.3.2, 3.4.0, 3.4.1, 3.4.2, 3.5.0, 3.5.1, 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.8.0, 3.8.1, 3.8.2, 3.9.1, 3.9.2, 3.9.3, 3.10.0, 3.10.1, 4.0.0, 4.0.1, 4.0.2, 4.1.0, 4.1.1, 4.2.0, 4.2.1, 4.3.0, 4.3.1, 4.4.0, 4.4.1, 4.4.2, 4.5.0, 4.6.0, 4.6.1, 4.6.2, 4.6.3, 4.6.4, 4.6.5, 4.6.6, 4.6.7, 4.6.8, 4.6.9, 4.6.10, 4.6.11)
      No matching distribution found for pytest>=5.0 (from -r requirements.txt (line 1))
      32a51b55
  3. 10 Apr, 2021 1 commit
    • Josef Schlehofer's avatar
      requirements: bump pytest-html to 2.1.x version · 5d9d1ced
      Josef Schlehofer authored
      Version 2.0.x does not support hook - pytest_html_report_title.
      It fails with:
      
      INTERNALERROR> pluggy.manager.PluginValidationError: unknown hook 'pytest_html_report_title' in plugin <module 'conftest' from '/foo/nsfarm/conftest.py'>
      5d9d1ced
  4. 30 Mar, 2021 20 commits
    • Karel Koci's avatar
      imgs/isp-common: improve bind configuration · ac47a771
      Karel Koci authored and Karel Koci's avatar Karel Koci committed
      This explicitly enables DNSSEC validation and adds additional option to
      conform to RFC1035.
      It also uses named.ca file shipped as part of package as root hints.
      We can safely left out `listen-on` as `listen-on-v6` uses dual-stack and
      thus listens on IPv4 as well as on IPv6.
      
      And the last but the most important change is disable of IPv6. This
      solves issues on IPv4 only network but IPv6 once we begin to support
      IPv6 we should allow disable/removal of this line.
      ac47a771
    • Karel Koci's avatar
      imgs/isp-common: fix access to DNS · a93ff988
      Karel Koci authored and Karel Koci's avatar Karel Koci committed
      This fixes commit: 57f0b251
      It added Bind as ISP's DNS resolver but open in firewall was missing.
      a93ff988
    • Karel Koci's avatar
      tests/sentinel: add basic tests for sentinel services · 06d5ea25
      Karel Koci authored
      This only installs Sentinel components and checks if appropriate
      services started.
      06d5ea25
    • Karel Koci's avatar
      nsfarm: allow and disable input read for serial console · 3c7282dd
      Karel Koci authored
      This reason for this is that we use at some point serial console only to
      get logs. We do not process it further. There is no reader for serial
      console at most of time.
      This change allows disable of serial console input and serial console
      logs are thus only logged without further propagation.
      3c7282dd
    • Karel Koci's avatar
      tests/test_bootup: check if services we want are enabled · 748f9db2
      Karel Koci authored
      This checks not only if services we want are running but also if they
      are enabled.
      The real reason for this is not to cover issue of disabled but running
      service as that is highly unlikely. This rather covers services that
      spawn actually no process but still we have to check if they are enabled
      and thus executed in some manner. There is a lot of such services.
      748f9db2
    • Karel Koci's avatar
      tests/test_bootup: add check for accessbile web interface · 1912c20a
      Karel Koci authored
      It should be enough to just download index for now. It checks only if we
      are able to access HTTP, nothing more.
      1912c20a
    • Karel Koci's avatar
      nsfarm/cli: fix delayed logging of PexpectLogging · 4ab73125
      Karel Koci authored
      The issue was invalid condition as well as that if all input was
      consumed that linebuf stayed set to old (already consumed) value.
      4ab73125
    • Karel Koci's avatar
      tests: add not_board mark · cebecdc2
      Karel Koci authored
      This is negation of board mark. This should help us to limit tests not
      only to explicit list of boards but also do exclusion list.
      
      The immediate usage of this is in DNS resolver processes check.
      cebecdc2
    • Karel Koci's avatar
      nsfarm/toolbox: add check for running service · 5909ade1
      Karel Koci authored
      The idea of having one module called toolbox is simply to share common
      short but still little bit complicated functions in multiple tests.
      
      This also implements first test using this function. It checks if all
      services we expect to run after router start are actually running.
      5909ade1
    • Karel Koci's avatar
      nsfarm/mterm: suppress logging · 71785f37
      Karel Koci authored
      It is common that tests are run with logs printed to terminal. It helps
      developer to see what is actually happening.
      The problem is that log printing interferes with terminal output. It
      should be safe to set level of root logger such it prints no messages
      for time of mterm execution.
      71785f37
    • Karel Koci's avatar
      nsfarm/lxd: improve warning about image bootstrap · f3f0e88f
      Karel Koci authored
      The message was being printed almost right on prepare method enter. The
      problem is that it was called before parent image prepare method was
      called. In effect this generated messages in reverse order. It also made
      it pretty much impossible to identify which container exactly failed to
      prepare as warnings for all images were printed even before any of them
      started actually preparing. This moves it after parent prepare method
      call and thus makes it so prepare messages are printed in order and
      right before actual work is being performed.
      
      This also gives us the option to include full container name we use for
      bootstrapping.
      f3f0e88f
    • Karel Koci's avatar
      nsfarm/mterm: fix invalid semicolon command in Shell · 93573aef
      Karel Koci authored
      Originally mterm was using just semicolon but bare semicolon is invalid
      in shell so it prints error. This instead passes decision of command to
      be used to specific Cli implementations.
      93573aef
    • Karel Koci's avatar
      tool.sh: add script to access more easilly nsfarm library tools · de5188d5
      Karel Koci authored
      The nsfarm library tools can be invoked by: python3 -m nsfarm
      This is pretty simple but it is not directly visible, the documentation
      has to be investigated to found this out, as well as not exactly short.
      Having this short script should do us no harm.
      de5188d5
    • Karel Koci's avatar
      tests/network/test_lan: wait for network before continuing · b0828c9b
      Karel Koci authored
      This should prevent hopefully failures when network is not yet up in the
      client container.
      b0828c9b
    • Karel Koci's avatar
      test_cryptowrapper: add some basic tests for crypto-wrapper · 8cc46caa
      Karel Koci authored
      It tests only if hardware type is correctly reported and if appropriate
      serial number is returned.
      
      This adds new requirement for configuration and that is serial number.
      We need this to verify that crypto-wrapper correctly returns appropriate
      number.
      8cc46caa
    • Karel Koci's avatar
      tests: improve reports · 466e943b
      Karel Koci authored
      This improves XML and HTML reports. It is minor expansion with required
      info: tested branch and target.
      466e943b
    • Karel Koci's avatar
      tests: remove serial and wan markers · 65c8fea6
      Karel Koci authored
      The only possible execution right now is with serial and wan present.
      In reality we need lan1 as well but we have to get around that because
      CPU only Mox.
      This simply removes parameters and marks for appropriate fixtures.
      The effect is just less marks as well as better tests reports as they no
      longer report None parameters.
      65c8fea6
    • Karel Koci's avatar
      configtest: remove inclusion of pytest_html · d3fe6c6a
      Karel Koci authored
      It seems that pytest_html is now included automatically and including it
      second time here produces warning.
      d3fe6c6a
    • Karel Koci's avatar
      tests: implement target autoselection · 5bc88a7f
      Karel Koci authored
      This reduces user's overhead to launch tests. It uses targets filtering.
      This makes target specification optional.
      At the moment user can specify that he wants to run tests on specific
      board. In the future we should add filters for additional configured
      board features, such as presence of Wi-Fi cards or USB storage device.
      5bc88a7f
    • Karel Koci's avatar
      nsfarm/target: move target parsing to nsfarm library · cff8fe91
      Karel Koci authored
      This not only moves target config parsing to nsfarm library but also it
      changes it to be implementation defined not just access to configparser.
      cff8fe91
  5. 29 Mar, 2021 2 commits
  6. 18 Mar, 2021 1 commit
    • Karel Koci's avatar
      selftests/lxd: add simple ping test for LXD container · 74051ebe
      Karel Koci authored
      The current tests check if appropriate profiles exists but not if they
      actually work. The nsfarm-root has to work when container at least
      starts. The problem is with nsfarm-internet. It is required to test if
      it is possible to access the Internet or not with that profile. That is
      exactly what this test does.
      74051ebe
  7. 17 Mar, 2021 1 commit
  8. 16 Mar, 2021 1 commit
    • Karel Koci's avatar
      nsfarm/lxd: include files in hash calculations · 2ecf2196
      Karel Koci authored
      There was for some reason condition that only directories are listed
      when image hash is generated. In reality the code was prepared to handle
      any path and should include all paths. This removes this invalid
      condition and makes it so all files provided for image are included in
      hash calculation.
      
      This in effect makes it so new image build is triggered on any file
      modification.
      2ecf2196
  9. 25 Feb, 2021 9 commits