LMCache Controller#

LMCache Controller exposes a set of APIs for users and orchestrators to manage the KV cache.

Currently, the controller provides the following APIs:

  • Clear: Clear the KV caches.

  • Compress: Compress the KV cache.

  • Health: Check the health status of cache workers.

  • Lookup: Lookup the KV cache for a given list of tokens.

  • Move: Move the KV cache to a different location.

  • Pin: Persist the KV cache to prevent it from being evicted.

  • CheckFinish: Check whether a (non-blocking) control event has finished or not.