Skip to content

Least Recently Used (LRU) Eviction Strategy #57

Description

@Turnerj

Issue created from #53 (comment)

Currently the only eviction strategy is one of absolute time. If there was a method to define a "size" of cache, it would be worth exploring a "Least Recently Used" system to ensure that the hottest items stay in the cache.

Thoughts:

  • Implement an extension (eg. AutoEvict/FixedCapacity) that manages the more advanced eviction strategy
  • New extension tracks number of items in the cache, cache keys, expiry dates and last access dates
    • Could use a LinkedList to track most recently used or a timestamp directly
    • Probably easier with a ConcurrentDictionary with cache keys as the key and a custom type as the value
  • Once a criteria has been hit (eg. X number of items in the cache), use what it knows to evict locally

Challenges:

  • Keeping the extension up-to-date with cache data
    • Can use the various extension hooks but still is a bit fiddly
  • Only evicting locally
    • Can do what I already do for RedisRemoteEvictionExtension (pass the cache layers in directly) but it is pretty cumbersome

Notes:
To handle a distributed LRU would be excessively complex and likely not useful. Would mean that every access to a cache item (including local) would have to then broadcast that access back through the cache system and whatever distributed backends are used.

Activity

  1. added this to the Future milestone on May 16, 2020
  2. Turnerj commented on Sep 11, 2020

    @Turnerj
    MemberAuthor

    Might be worth investigating with or comparing against (when the time comes) this .NET caching solution which uses a LRU strategy: https://github.com/bitfaster/BitFaster.Caching

  3. prat0088 commented on Dec 10, 2020

    @prat0088

    That would be fantastic. LRU and bounded size are two requirements for my usage scenario.

  4. Turnerj commented on Jan 25, 2021

    @Turnerj
    MemberAuthor

    Thanks for your feedback - it won't make the next release (coming in the next few days) but it is definitely something I'm looking into!

  5. Turnerj commented on Oct 1, 2021

    @Turnerj
    MemberAuthor

    (For my own reference)

    An interesting LFU cache implementation: https://github.com/papers-we-love/papers-we-love/blob/master/caching/a-constant-algorithm-for-implementing-the-lfu-cache-eviction-scheme.pdf

    Not sure how applicable it is to this given its a LFU not an LRU plus it is a storage mechanism, not an eviction strategy. There still might be some useful pieces of information to extract from it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions