Karel Koci authored 3 years ago and Josef Schlehofer committed 2 years ago

This makes Gitlab CI docker image update part of the CI run and thus in
sync with Dockerfile in repository.

This change moves registry URL from registry.labs.nic.cz to
registry.nic.cz.

This fixes:

- WARNING: Failed to pull image with policy "always": Error response from daemon: Head https://registry.nic.cz/v2/turris/os/build/manifests/latest: unauthorized: HTTP Basic: Access denied (manager.go:205:0s)

-ERROR: Preparation failed: failed to pull image "registry.labs.nic.cz/turris/os/build" with specified policies [always]: Error response from daemon: Head https://registry.nic.cz/v2/turris/os/build/manifests/latest: unauthorized: HTTP Basic: Access denied (manager.go:205:0s)

Karel Koci authored 4 years ago and Karel Koci committed 3 years ago

There are two changes in this commit. First most noticable is removal of
all updater lists.

The second but more significant change is pretty much reformat of all
generate scripts. It extracts common git and configuration handling from
compile_pkgs to generate_common.
The advantage of this is of course code sharing but also that other
generate scripts now get features such as local git mirror.

The repository have been moved.

We build only primary branches and we need build packages to test medkit
generation. In most cases relying on default PUBLISH_BRANCH is exactly
what we want but in case of HBK branch the target is correctly set to
HBS. That is too far off branch and it makes more sense to just tests
against HBK. This overrides tests to be executed against HBK if publish
branch is HBS.

We are testing in Gitlab CI if patches apply and also that package
list can be generated and then we checked them. We should try to
generate and check for medkit as well.

Because someone on IRC (#turris on Freenode) complained that he could not
generate medkit.

It helped me to reproduce his issue with Gitlab CI and he didn't have installed pkg-config (dependency for Updater-ng, which is used to generate medkits), because it was working for me locally and also it works on our CI, but it does not provide any logs to public.
This can help users to know if medkits can be generated.

In some cases patches does not apply for some devices, let's try to
test Turris MOX as well.

There was an added test for package lists, but it runs twice.
If you check in the previous changes then there are two jobs, which
do practically the same. In generate pkglists there are generated
lists with debug parameter, but we don't work with that anymore, which
is wrong. And because of that, we had another job check generated
pkglists, where we are generating lists once again and then we are
checking if those have been generated.

Solution: Don't waste hardware resources and run it just once. When package lists
are generated then the artifact is uploaded and uses in the next job in
the next stage

Changes:
- add lua5.1 and m4 to Dockerfile
- try to generate pkglists
- try to check generated pkglists

This reduces run time because we dont need to update and install all
dependencies in each run and it will be stored in container image, which
is going to be pulled in each run.

OpenWrt master requires Python3 and it fixes this issue:
Build dependency: Please install Python >= 3.5

We are using Gitlab CI for OpenWrt 19.07 and OpenWrt master.

OpenWrt 19.07 requires to have Python2 as it shows.
Build dependency: Please install Python 2.x

It should check if we can apply patches to
openwrt or feeds branches before merging it to HB*