The implementation is now similar to set-associative caches
that x86 CPU use.  Also the API is changed a bit, leading to
slight simplification of our use patterns.