updater issueshttps://gitlab.nic.cz/turris/updater/updater/-/issues2023-08-16T15:01:28+02:00https://gitlab.nic.cz/turris/updater/updater/-/issues/328Make sure we run all postupdate hooks regardless of previous errors2023-08-16T15:01:28+02:00Michal HruseckyMake sure we run all postupdate hooks regardless of previous errorsRegardless of whatever failed before, we should run postupdate hooks all the time if we did at least something.Regardless of whatever failed before, we should run postupdate hooks all the time if we did at least something.Michal HruseckyMichal Hruseckyhttps://gitlab.nic.cz/turris/updater/updater/-/issues/268Replace build in busybox (and utility functions) with C code2023-05-30T14:40:20+02:00Karel KociReplace build in busybox (and utility functions) with C codeFor simplicity up now we were just calling shell and busybox to handle archives and files operations. We should replace that with C only code and drop this humongous dependency.
* [ ] implement files move, copy and remove (`cp`, `mv`, `...For simplicity up now we were just calling shell and busybox to handle archives and files operations. We should replace that with C only code and drop this humongous dependency.
* [ ] implement files move, copy and remove (`cp`, `mv`, `rm`)
* [x] implement gzip and tar replacement (`tar`, `gzip`)
* [x] check if still required and if so implement find replacement (`find`)
* [ ] drop events
We can drop events module from library as the only thing we are using it for is to handle these utility calls. Replacing them should allows us to drop events all to together.https://gitlab.nic.cz/turris/updater/updater/-/issues/232Refactor sandbox to its own Lua context and simplify security levels (Updater...2022-06-06T14:34:59+02:00Karel KociRefactor sandbox to its own Lua context and simplify security levels (Updater language 2)Whole sandbox is complicated just because we need to ensure that configuration
scripts can't access some advanced features. But that is some what questionable
when it can install any arbitrary package with any arbitrary code in it. Yes t...Whole sandbox is complicated just because we need to ensure that configuration
scripts can't access some advanced features. But that is some what questionable
when it can install any arbitrary package with any arbitrary code in it. Yes true
is that installation has to be optionally confirmed so we probably should have at
least one protected level that denies access to uci, files and shell execution.
But having more than two makes probably no sense.
Proposition is to have single Lua instance, separate from primary one. This
instance is some what simple lua with possibility to switch to protected mode.
Also running sandbox in same instance as rest of the updater just complicates
things and adds nothing of value. Yes we can share variables (by context magic)
and functions but in the end there is no other reason. Just initializing new
interpreter with specific limitations should be enough.
As part of this we should also implement: #198https://gitlab.nic.cz/turris/updater/updater/-/issues/261New transaction and journal implementation2020-03-09T14:21:12+01:00Karel KociNew transaction and journal implementationWe should rewrite transactions from Lua to C and we should redesign journal. This means that this is effectively two major steps/changes.
##### New journal design:
Current journal has few shortcomings such as that its content is give...We should rewrite transactions from Lua to C and we should redesign journal. This means that this is effectively two major steps/changes.
##### New journal design:
Current journal has few shortcomings such as that its content is given by program flow and not by some explicit data structure. Its content is not strictly defined and can contain unnecessary items. This is not only wasting storage write cycles (damaging mmc) it also means that it is harder to extend or change functionality.
New journal should contain all required information before it is even written to disk and later it should be only something like a checklist of steps that were already done and steps that were not. The transaction later should go trough this checklist and should just do the task and set it as completed.
All sanity checks should be done before journal is created and written to disk. That means that collision checking, packages extraction and others stuff should not be part of journal!
In transaction there is something called files stealing where package to be removed or updated releases some path that is introduced by some other package. This should be integrated to journal in form of remove exclusions but those exclusions should be calculated before journal is created/written.
If we should do reboot or replan then we should store all information we need to do so. In case of reboot it can be just separate step. In case of replan we should store all arguments to journal so we can run replan with correct executable and same arguments.
The preupdate and postupdate hooks of pkgupdate should be implemented as part of journal.
Journal itself might be some lightweight database (maybe? http://www.lmdb.tech/doc/starting.html) because what we need is effectively table of dictionaries with pointer going trough them. We could write it but if we can safe our self some work with database that solves serialization, consistency and other problems then it makes sense to use that instead of reinventing wheel.
##### New transaction:
Current transaction is "do all" spaghetti. We should take Lua code and rewrite it to C. The primary idea is that where ever now transaction step can be skipped thanks to journal the new design should be rather driven by journal. First we do sanity checks of packages (look for collisions between paths). Then we prepare journal and every step is later executed according to journal. When journal is finished then all operations are effectively done and no additional operations should be required.
There should effectively be these steps (not in execution order):
* Package script
* prerm
* preinst
* postrm
* postinst
* Package files merge (this step merges new files and removes old ones)
* System scripts (hooks)
* preudpdate
* postupdate
* reboot_required
* Immediate reboot
* Replan (terminate transaction/journal and exec given command)
We can also solve following issues as part of this: #202 #240 #260 https://gitlab.nic.cz/turris/updater/updater/-/issues/240Split package managing to its own separate library2019-07-08T11:02:55+02:00Karel KociSplit package managing to its own separate libraryOperations like single package installation, removal and journal feature should be in its own library. Then having program like opkg-trans (maybe rename it) should help us with low level packages manipulations. At the moment becase opkg-...Operations like single package installation, removal and journal feature should be in its own library. Then having program like opkg-trans (maybe rename it) should help us with low level packages manipulations. At the moment becase opkg-trans depends on heavy libupdater when libupdater is broken because of one of its dependencies (libcurl is prime example) it won't start and won't allow you to fix it. Fixing this without opkg would be nightmare. If we have posix only library and executable that is for this low level packages manipulation it would be much much more easier. This way we could also add at least one more layer of apis to updater.