updater issueshttps://gitlab.nic.cz/turris/updater/updater/-/issues2024-03-25T15:28:43+01:00https://gitlab.nic.cz/turris/updater/updater/-/issues/330More detailed User-Agent header2024-03-25T15:28:43+01:00Lukas JelinekMore detailed User-Agent headerIt would be good to provide more information inside the `User-Agent` header. I suggest especially these ones:
- Precise specification of the given Turris OS version (Build ID) - it would be useful to distinguish between multiple release...It would be good to provide more information inside the `User-Agent` header. I suggest especially these ones:
- Precise specification of the given Turris OS version (Build ID) - it would be useful to distinguish between multiple release candidates, nightly builds etc.
- Device type including its revision and HW architecture - it can be partially detected from accessed paths but it generates additional overhead
### Current header
```
Turris Updater/70.0.0 (TurrisOS 6.0.3)
```
### Proposed header
```
Turris Updater/70.0.0 (TurrisOS 6.0.3; r16703+129-079ce0413a; Turris Omnia/v0; arm_cortex-a9_vfpv3-d16)
```https://gitlab.nic.cz/turris/updater/updater/-/issues/324Add another updater mode: approvals with good update confirmation2022-02-14T23:43:41+01:00Martin PeckaAdd another updater mode: approvals with good update confirmationHi, regarding to the occasional reports on forum from people with failed updates of routers they can't physically access, I think implementing another updater mode would help them a lot.
You’d basically install a cron job to revert the ...Hi, regarding to the occasional reports on forum from people with failed updates of routers they can't physically access, I think implementing another updater mode would help them a lot.
You’d basically install a cron job to revert the update before executing it, and the user would need to manually connect to the router and click a button in Reforis to confirm the update went well. If not, the router would automatically revert the update and reboot after some time. Of course, that’s not the best idea with MOX, but on Omnia, it could be pretty useful (I’m myself doing something similar manually).
This way, I could update the remote Omnias much quicker without fear that there will be undesired downtime. The downtime would be limited to tens of minutes max if the update fails.https://gitlab.nic.cz/turris/updater/updater/-/issues/323Configuration file -opkg should be created only if that is really a new versi...2022-06-06T13:59:04+02:00Karel KociConfiguration file -opkg should be created only if that is really a new version of that configuration fileIt is expected that advanced users are going to use `-opkg` files to merge configuration files. We have an original hash of the configuration file and we should not request a merge of configuration (by creating `-opkg` file) unless there...It is expected that advanced users are going to use `-opkg` files to merge configuration files. We have an original hash of the configuration file and we should not request a merge of configuration (by creating `-opkg` file) unless there is really some update. We just have to compare the current hash with the new one and create that file only if there is a difference.https://gitlab.nic.cz/turris/updater/updater/-/issues/314Take in to considerations priorities in plan ordering2022-06-06T14:11:02+02:00Karel KociTake in to considerations priorities in plan orderingWe have priority of packages and they should be installed in that priority. It is weak order requirement we should include. It also improves update in general.We have priority of packages and they should be installed in that priority. It is weak order requirement we should include. It also improves update in general.https://gitlab.nic.cz/turris/updater/updater/-/issues/312Reinstall check should also tak in to consideration other fields2022-06-06T14:15:04+02:00Karel KociReinstall check should also tak in to consideration other fieldsAt the moment we reinstall when one of these fields change: `"Version", "Architecture", "LinkSignature", "Depends", "Conflicts", "Provides"`. We should reinstall also on other fields.
These fields are used in OpenWrt and make sense to t...At the moment we reinstall when one of these fields change: `"Version", "Architecture", "LinkSignature", "Depends", "Conflicts", "Provides"`. We should reinstall also on other fields.
These fields are used in OpenWrt and make sense to trigger reinstall on them:
* Essential (although we do not use this)
* Conffiles
* Alternativeshttps://gitlab.nic.cz/turris/updater/updater/-/issues/306Preserve modified orphan config files for even installed packages2023-05-30T14:39:37+02:00Karel KociPreserve modified orphan config files for even installed packagesAt the moment when package is removed and it contains configuration file we keep it in database to track modified config. Problem is that we do not do the same if package is upgraded and removes configuration. We just remove modified con...At the moment when package is removed and it contains configuration file we keep it in database to track modified config. Problem is that we do not do the same if package is upgraded and removes configuration. We just remove modified configuration in such case.
The reason why we should not do it is to on one side preserve configuration modifications by users even if that might mean keeping around no longer used files. The second reason and more important one is that this might be just an issue in package and new version can revert to it as well as some other new package can take ownership of it. We should not reset config in such case.https://gitlab.nic.cz/turris/updater/updater/-/issues/302Allow (or check if possible) replan on virtual package2020-06-15T12:15:06+02:00Karel KociAllow (or check if possible) replan on virtual packageDoing replan on specially immediate one on virtual package turns out to be pretty useful. The point is that you can just line all package you want to update once before you reexecute updater without needing to select one of them to be th...Doing replan on specially immediate one on virtual package turns out to be pretty useful. The point is that you can just line all package you want to update once before you reexecute updater without needing to select one of them to be the root of three and inserting others as additional dependencies. The reason is that might not be always possible such as if that creates cycle that just can't be broken. With such cycle updater possibly won't allow update as updater is critical package.
I am convince that using virtual packages do not work at the moment with replan as those are ignored when plan is created. This means that with routine cutting away plan in case of replan won't cut it simply because there is not package to cat plan at.https://gitlab.nic.cz/turris/updater/updater/-/issues/289Change how approvals work to instead call hook2020-10-15T15:11:16+02:00Karel KociChange how approvals work to instead call hookCurrently approvals generate to provided path file that requests approval. Next run user can use hash from that file to approve those changes.
This is not ideal I feel and much less complicated and direct would be to just use hooks. We ...Currently approvals generate to provided path file that requests approval. Next run user can use hash from that file to approve those changes.
This is not ideal I feel and much less complicated and direct would be to just use hooks. We could have approval hooks that are called to decide if update is approved. There can be more than one approval script so approve can be done by more than now one fixed way.
Updater should provide hook with list of changes and expect it to exit in specific way to signal one of three states:
* undecided
* deny
* approve
When at least one script provides deny or approve than such action is performed. No other script should be executed after that. Undecided result forwards updater on to another hook.https://gitlab.nic.cz/turris/updater/updater/-/issues/286Check if updater is protected against multiple same names in provides field2019-12-02T16:26:55+01:00Karel KociCheck if updater is protected against multiple same names in provides fieldReferencing following issue https://gitlab.labs.nic.cz/turris/updater/updater/issues/285 we should check if we are also protected against multiple provides statements such as `Provides: foo foo`.Referencing following issue https://gitlab.labs.nic.cz/turris/updater/updater/issues/285 we should check if we are also protected against multiple provides statements such as `Provides: foo foo`.https://gitlab.nic.cz/turris/updater/updater/-/issues/281Suppress only immediate replan with `--no-replan` option2022-06-06T14:27:50+02:00Karel KociSuppress only immediate replan with `--no-replan` optionReplan makes sense to be used to not only update updater self (immediate) but also to apply changes to updater configuration introduced from packages. The problem with disabling replan because we don't want to immediate replan is that th...Replan makes sense to be used to not only update updater self (immediate) but also to apply changes to updater configuration introduced from packages. The problem with disabling replan because we don't want to immediate replan is that those configs are not considered and ignored.
We should either introduce new `--no-immediate-replan` and change `--out-of-root` to include that one. Or we can just replace behavior of `--no-replan` to disable only immediate replan.https://gitlab.nic.cz/turris/updater/updater/-/issues/261New transaction and journal implementation2020-03-09T14:21:12+01:00Karel KociNew transaction and journal implementationWe should rewrite transactions from Lua to C and we should redesign journal. This means that this is effectively two major steps/changes.
##### New journal design:
Current journal has few shortcomings such as that its content is give...We should rewrite transactions from Lua to C and we should redesign journal. This means that this is effectively two major steps/changes.
##### New journal design:
Current journal has few shortcomings such as that its content is given by program flow and not by some explicit data structure. Its content is not strictly defined and can contain unnecessary items. This is not only wasting storage write cycles (damaging mmc) it also means that it is harder to extend or change functionality.
New journal should contain all required information before it is even written to disk and later it should be only something like a checklist of steps that were already done and steps that were not. The transaction later should go trough this checklist and should just do the task and set it as completed.
All sanity checks should be done before journal is created and written to disk. That means that collision checking, packages extraction and others stuff should not be part of journal!
In transaction there is something called files stealing where package to be removed or updated releases some path that is introduced by some other package. This should be integrated to journal in form of remove exclusions but those exclusions should be calculated before journal is created/written.
If we should do reboot or replan then we should store all information we need to do so. In case of reboot it can be just separate step. In case of replan we should store all arguments to journal so we can run replan with correct executable and same arguments.
The preupdate and postupdate hooks of pkgupdate should be implemented as part of journal.
Journal itself might be some lightweight database (maybe? http://www.lmdb.tech/doc/starting.html) because what we need is effectively table of dictionaries with pointer going trough them. We could write it but if we can safe our self some work with database that solves serialization, consistency and other problems then it makes sense to use that instead of reinventing wheel.
##### New transaction:
Current transaction is "do all" spaghetti. We should take Lua code and rewrite it to C. The primary idea is that where ever now transaction step can be skipped thanks to journal the new design should be rather driven by journal. First we do sanity checks of packages (look for collisions between paths). Then we prepare journal and every step is later executed according to journal. When journal is finished then all operations are effectively done and no additional operations should be required.
There should effectively be these steps (not in execution order):
* Package script
* prerm
* preinst
* postrm
* postinst
* Package files merge (this step merges new files and removes old ones)
* System scripts (hooks)
* preudpdate
* postupdate
* reboot_required
* Immediate reboot
* Replan (terminate transaction/journal and exec given command)
We can also solve following issues as part of this: #202 #240 #260 https://gitlab.nic.cz/turris/updater/updater/-/issues/258Test that we don't left any garbage in tmpdir2019-05-24T09:36:45+02:00Karel KociTest that we don't left any garbage in tmpdirWe have ability to set tmpdir for all tests so we can do that and check if none of them left some garbage there. The idea is that we can detect if even on error we remove our temporally files.
Currently we are not verifying this at all.We have ability to set tmpdir for all tests so we can do that and check if none of them left some garbage there. The idea is that we can detect if even on error we remove our temporally files.
Currently we are not verifying this at all.https://gitlab.nic.cz/turris/updater/updater/-/issues/246Differenciate between failure in script from critical and non-critical packages2022-06-06T14:32:45+02:00Karel KociDifferenciate between failure in script from critical and non-critical packagesWhen some script (postinst, prerm,..) fails then we should (and I hope that we do) print error to user. But it's more or less warning in this sense. If some tiny package fails it's postinst then it should be warning. But that is not same...When some script (postinst, prerm,..) fails then we should (and I hope that we do) print error to user. But it's more or less warning in this sense. If some tiny package fails it's postinst then it should be warning. But that is not same if kernel fails its postinst. So all packages requested as critical should result to error to be reported to user but anything else should be warning.https://gitlab.nic.cz/turris/updater/updater/-/issues/240Split package managing to its own separate library2019-07-08T11:02:55+02:00Karel KociSplit package managing to its own separate libraryOperations like single package installation, removal and journal feature should be in its own library. Then having program like opkg-trans (maybe rename it) should help us with low level packages manipulations. At the moment becase opkg-...Operations like single package installation, removal and journal feature should be in its own library. Then having program like opkg-trans (maybe rename it) should help us with low level packages manipulations. At the moment becase opkg-trans depends on heavy libupdater when libupdater is broken because of one of its dependencies (libcurl is prime example) it won't start and won't allow you to fix it. Fixing this without opkg would be nightmare. If we have posix only library and executable that is for this low level packages manipulation it would be much much more easier. This way we could also add at least one more layer of apis to updater.https://gitlab.nic.cz/turris/updater/updater/-/issues/232Refactor sandbox to its own Lua context and simplify security levels (Updater...2022-06-06T14:34:59+02:00Karel KociRefactor sandbox to its own Lua context and simplify security levels (Updater language 2)Whole sandbox is complicated just because we need to ensure that configuration
scripts can't access some advanced features. But that is some what questionable
when it can install any arbitrary package with any arbitrary code in it. Yes t...Whole sandbox is complicated just because we need to ensure that configuration
scripts can't access some advanced features. But that is some what questionable
when it can install any arbitrary package with any arbitrary code in it. Yes true
is that installation has to be optionally confirmed so we probably should have at
least one protected level that denies access to uci, files and shell execution.
But having more than two makes probably no sense.
Proposition is to have single Lua instance, separate from primary one. This
instance is some what simple lua with possibility to switch to protected mode.
Also running sandbox in same instance as rest of the updater just complicates
things and adds nothing of value. Yes we can share variables (by context magic)
and functions but in the end there is no other reason. Just initializing new
interpreter with specific limitations should be enough.
As part of this we should also implement: #198https://gitlab.nic.cz/turris/updater/updater/-/issues/230OPKG replacement2019-07-08T11:02:56+02:00Karel KociOPKG replacementWe want to do complete opkg replacement. Current state some what works but is not systematical as we are using one tool for updates and another for explicit packages installation. But simply removing opkg is also not possible as user are...We want to do complete opkg replacement. Current state some what works but is not systematical as we are using one tool for updates and another for explicit packages installation. But simply removing opkg is also not possible as user are used on interactive package installation. The idea is to add new application that would replace opkg in most of the features and wrapper script to simulate opkg.
To support opkg the new command has to support following features:
* [ ] Package installation
* [ ] Package removal
* [ ] Package upgrade
* [ ] Package reconfiguration
* [ ] Listing available and installed packages
* [ ] Listing upgradable packages
* [ ] Listing files belonging to some package
* [ ] Searching packages content
* [ ] Displaying packages info
* [ ] Querying for package status
* [ ] Possibility to download some package
* [ ] Querying for package dependencies
To do that we need to resolve following issues in updater it self:
* [ ] Possibility of considering installed packages
* [ ] Resolve package removal (should we add Uninstall command and in what priority)
* [ ] Lua configuration management (automatically managed file with lua syntax)
* [ ] Updater should use index and userlists cache to make execution faster and less demanding on network connection
Features present in opkg but probably not required by us:
* Multiple architectures support
* Destination names
* `--autoremove` because we are doing that automatically
* Flags settinghttps://gitlab.nic.cz/turris/updater/updater/-/issues/229What package is installed as provided from possibilities is decided alphabeti...2019-07-08T12:48:59+02:00Karel KociWhat package is installed as provided from possibilities is decided alphabeticallyIf we have more than one possible provided candidate for package then we decide
which one we use simply alphabetically by their name. This makes decision stable
but gives us no free hands and only way how to change that is to add `Instal...If we have more than one possible provided candidate for package then we decide
which one we use simply alphabetically by their name. This makes decision stable
but gives us no free hands and only way how to change that is to add `Install`
command but that has side effect that it won't be removed as soon as other
provided candidate is selected. Solution would be to add possibility for packages
to specify their own preference (`Package("pkg", {prefer={"apkg", "bpkg"}})`.https://gitlab.nic.cz/turris/updater/updater/-/issues/228Request for conflicting packages is resolved randomly2019-07-08T11:02:57+02:00Karel KociRequest for conflicting packages is resolved randomlyIf we have requests of same priority for conflicting packages then we choose one
of those in random. In wost case scenario we could be constantly switching between
those packages.
The solution is to add one additional parameter and that...If we have requests of same priority for conflicting packages then we choose one
of those in random. In wost case scenario we could be constantly switching between
those packages.
The solution is to add one additional parameter and that is order of introductions
of requests. If request is introduced before some other request then we should
prefer it. The implementation can be done using binary search algorithm to always
assume first half of internal and if that is not possible then we can divide it
and do the same recursively.https://gitlab.nic.cz/turris/updater/updater/-/issues/224Errors from pacakges should not be reported as errors from updater it self2019-07-31T11:07:55+02:00Karel KociErrors from pacakges should not be reported as errors from updater it selfIf package's script fails then it's reported as Updater's error same as if updater
fails. This confuses users and elevates small in single package to system level
errors.
Also maybe don't report packages errors as errors but as warning...If package's script fails then it's reported as Updater's error same as if updater
fails. This confuses users and elevates small in single package to system level
errors.
Also maybe don't report packages errors as errors but as warning (exception should
probably be critically requested packages).https://gitlab.nic.cz/turris/updater/updater/-/issues/221When preinst of critically requested package fails then we should not continue2019-07-08T11:02:58+02:00Karel KociWhen preinst of critically requested package fails then we should not continueCurrently if critically requested package's preinst script fails we just ignore it
as any other package and repost error later on. We should abandon the transaction
in such case.
We really don't have too much critical package with prein...Currently if critically requested package's preinst script fails we just ignore it
as any other package and repost error later on. We should abandon the transaction
in such case.
We really don't have too much critical package with preinst scripts so it's not
exactly important. But we should do that some time in future nevertheless.