New transaction and journal implementation
We should rewrite transactions from Lua to C and we should redesign journal. This means that this is effectively two major steps/changes.
New journal design:
Current journal has few shortcomings such as that its content is given by program flow and not by some explicit data structure. Its content is not strictly defined and can contain unnecessary items. This is not only wasting storage write cycles (damaging mmc) it also means that it is harder to extend or change functionality.
New journal should contain all required information before it is even written to disk and later it should be only something like a checklist of steps that were already done and steps that were not. The transaction later should go trough this checklist and should just do the task and set it as completed.
All sanity checks should be done before journal is created and written to disk. That means that collision checking, packages extraction and others stuff should not be part of journal!
In transaction there is something called files stealing where package to be removed or updated releases some path that is introduced by some other package. This should be integrated to journal in form of remove exclusions but those exclusions should be calculated before journal is created/written.
If we should do reboot or replan then we should store all information we need to do so. In case of reboot it can be just separate step. In case of replan we should store all arguments to journal so we can run replan with correct executable and same arguments.
The preupdate and postupdate hooks of pkgupdate should be implemented as part of journal.
Journal itself might be some lightweight database (maybe? http://www.lmdb.tech/doc/starting.html) because what we need is effectively table of dictionaries with pointer going trough them. We could write it but if we can safe our self some work with database that solves serialization, consistency and other problems then it makes sense to use that instead of reinventing wheel.
Current transaction is "do all" spaghetti. We should take Lua code and rewrite it to C. The primary idea is that where ever now transaction step can be skipped thanks to journal the new design should be rather driven by journal. First we do sanity checks of packages (look for collisions between paths). Then we prepare journal and every step is later executed according to journal. When journal is finished then all operations are effectively done and no additional operations should be required.
There should effectively be these steps (not in execution order):
- Package script
- Package files merge (this step merges new files and removes old ones)
- System scripts (hooks)
- Immediate reboot
- Replan (terminate transaction/journal and exec given command)