LMCache Integration

Subpackages

• lmcache.integration.vllm package
  • Submodules
  • lmcache.integration.vllm.utils module
    • lmcache_get_config()
  • lmcache.integration.vllm.vllm_adapter module
    • ModelInputSubset
      • ModelInputSubset.attn_layers
      • ModelInputSubset.end_layer
      • ModelInputSubset.model_layers
      • ModelInputSubset.start_layer
    • RetrieveStatus
      • RetrieveStatus.CHUNK_PREFILL
      • RetrieveStatus.CHUNK_PREFILL_LAST
      • RetrieveStatus.NONE
      • RetrieveStatus.PREFILL
    • StoreStatus
      • StoreStatus.CHUNK_PREFILL
      • StoreStatus.DECODE
      • StoreStatus.NONE
      • StoreStatus.PREFILL
      • StoreStatus.SUFFIX_PREFILL
    • broadcast_seq_group_list()
    • broadcast_seq_group_metadata()
    • build_partial_prefill_input()
    • close_lmcache_engine()
    • create_model_input_subset()
    • init_lmcache_engine()
    • lmcache_retrieve_kv()
    • lmcache_should_retrieve()
    • lmcache_should_store()
    • lmcache_store_kv()
  • Module contents

Module contents