I missed some parts when finishing this.  I should've tested it better.
GC would hit assertions or NULL dereferences when removing entries,
and eventually that would lead to cache overflowing (and getting
cleared).