memory: the global issue tracking the effort
So, everybody knows what should have been done.
Pitfalls
The problem lies in the small allocations using malloc/sbrk, as the address space taken by the process can only shrink down to the highest used address. This is a problem with transfers because newer allocations (=new zone) take memory from higher addresses, and the old zone lies below that. When we free the old zone, we end up with an unused hole in the address space that can't be returned to the operating system. Is this bad? Not so much, the memory gets reused later, but it is annoying that the memory usage doesn't lower after transfer.
Questions
- Do we want zones loaded in the same process or move zone loading to separate process?
- Do we want shallow zone copies?
- Do we want to have full zones in memory?
- ... ?
Ideas
There are number of solutions to this, each requiring a substantial rewrite.
- Probably the easiest thing to do is to implement support for
mm_ctx_t
for each zone-related structure like nodes, rrsets and names.- Then have a memory pool for each zone instance.
- If the zone should be substantially changed, create new memory pool and zone from it, free the old pool.
- If there's only a small difference, we could omit the fact that the old data isn't really freed and just allocate new bits without full zone copy.
- Temporary data for zone loading should be allocated from the temporary memory pool.
Another stuff
For v1.4.1, I'm going to add malloc_trim(0)
after each large operation and turn off SLAB for events to see if there's any difference.