Extending the CLI#

LMCache’s CLI is built on a plugin-style architecture that supports N-level nested subcommands with zero-registration auto-discovery. This guide explains how to add commands at each level.

Architecture Overview#

The CLI framework is composed of two core classes in lmcache/cli/commands/base.py:

  • BaseCommand — Abstract base class for leaf commands (commands that perform actual work).

  • CompositeCommand — A BaseCommand subclass for commands that only group sub-subcommands. It auto-discovers child commands by scanning the package where it is defined.

The discovery mechanism uses pkgutil.iter_modules to scan direct submodules of a package, then collects all concrete BaseCommand subclasses found in those modules. This means:

  1. Each command is a separate .py file (or a sub-package with an __init__.py).

  2. No manual registration is needed — just create the file and it is picked up automatically.

  3. Utility/helper modules should be prefixed with _ (e.g. _helpers.py) so they are excluded from the scan.

Directory Layout#

lmcache/cli/commands/
├── __init__.py            # Top-level discovery (scans this package)
├── base.py                # BaseCommand & CompositeCommand
├── ping.py                # Level-1 leaf command
├── server.py              # Level-1 leaf command
├── quota/                 # Level-2 composite command
│   ├── __init__.py        # QuotaCommand(CompositeCommand)
│   ├── _helpers.py        # Utility (excluded from scan by _ prefix)
│   ├── get_command.py     # Level-2 leaf: ``lmcache quota get``
│   ├── set_command.py     # Level-2 leaf: ``lmcache quota set``
│   └── ...
└── tool/                  # Level-2 composite command
    ├── __init__.py         # ToolCommand(CompositeCommand)
    └── cache_simulator/    # Level-3 composite command
        ├── __init__.py     # CacheSimulatorCommand(CompositeCommand)
        ├── simulate_command.py   # Level-3 leaf: ``lmcache tool cache-simulator simulate``
        └── sweep_command.py      # Level-3 leaf: ``lmcache tool cache-simulator sweep``

Key Rules#

  • A CompositeCommand subclass must be defined in the __init__.py of its package.

  • Modules starting with _ are excluded from auto-discovery (use this for helpers, utilities, internal logic).

  • Each leaf command file should contain exactly one concrete BaseCommand subclass.

  • The scan is non-recursive — each CompositeCommand only scans its own package’s direct submodules.

Level 1: Adding a Top-Level Command#

A top-level command appears directly under lmcache <command>.

Step 1: Create a new file under lmcache/cli/commands/:

# lmcache/cli/commands/hello.py
"""``lmcache hello`` — a simple greeting command."""

import argparse

from lmcache.cli.commands.base import BaseCommand


class HelloCommand(BaseCommand):
    """Print a greeting message."""

    def name(self) -> str:
        return "hello"

    def help(self) -> str:
        return "Print a greeting message."

    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
        parser.add_argument("--name", default="World", help="Who to greet.")

    def execute(self, args: argparse.Namespace) -> None:
        metrics = self.create_metrics("Hello", args)
        metrics.add("greeting", "Greeting", f"Hello, {args.name}!")
        metrics.emit()

Step 2: Done! The command is automatically discovered. Test it:

lmcache hello --name LMCache

Note

This works because the top-level lmcache/cli/commands/__init__.py calls discover_subclasses on its own package at import time. It uses pkgutil.iter_modules to find all direct submodules (files and sub-packages), imports each one, and collects every concrete BaseCommand subclass. The resulting list is stored in ALL_COMMANDS and registered with the argument parser in main.py. So adding a new .py file with a BaseCommand subclass is all that is needed — no edits to any other file.

Level 2: Adding a Subcommand Group#

A subcommand group appears as lmcache <group> <subcommand>.

Step 1: Create a package directory:

mkdir lmcache/cli/commands/mygroup/

Step 2: Define the CompositeCommand in __init__.py:

# lmcache/cli/commands/mygroup/__init__.py
"""``lmcache mygroup`` command group.

Sub-subcommands are auto-discovered from modules in this package.
"""

from lmcache.cli.commands.base import CompositeCommand


class MyGroupCommand(CompositeCommand):
    """Command group for my custom operations."""

    def name(self) -> str:
        return "mygroup"

    def help(self) -> str:
        return "My custom command group."

Step 3: Add leaf subcommands as separate files:

# lmcache/cli/commands/mygroup/foo_command.py
"""``lmcache mygroup foo`` — do something."""

import argparse

from lmcache.cli.commands.base import BaseCommand


class FooCommand(BaseCommand):
    """Execute the foo action."""

    def name(self) -> str:
        return "foo"

    def help(self) -> str:
        return "Execute the foo action."

    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
        parser.add_argument("--value", type=int, required=True)

    def execute(self, args: argparse.Namespace) -> None:
        metrics = self.create_metrics("Foo Result", args)
        metrics.add("result", "Result", args.value * 2)
        metrics.emit()

Step 4: (Optional) Add helper modules with _ prefix:

# lmcache/cli/commands/mygroup/_utils.py
"""Internal utilities for mygroup commands (not auto-discovered)."""

def compute_something(x: int) -> int:
    return x * 42

Result:

lmcache mygroup foo --value 5

Level 2: Adding a Subcommand to an Existing Group#

If a CompositeCommand group already exists (e.g. bench, quota, trace), you can extend it by simply adding one new file — no other changes are required.

For example, to add a new lmcache bench l2 subcommand under the existing bench group:

Step 1: Create a single file (or sub-package) in the existing group’s package directory:

# lmcache/cli/commands/bench/l2_adapter_bench/__init__.py
"""``lmcache bench l2`` subpackage."""

import argparse

from lmcache.cli.commands.base import BaseCommand


class L2AdapterBenchCommand(BaseCommand):
    """Benchmark an L2 adapter (store / lookup / load)."""

    def name(self) -> str:
        return "l2"

    def help(self) -> str:
        return "Benchmark an L2 adapter (store / lookup / load)."

    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
        from lmcache.cli.commands.bench.l2_adapter_bench.command import (
            add_l2_arguments,
        )
        add_l2_arguments(parser)

    def execute(self, args: argparse.Namespace) -> None:
        from lmcache.cli.commands.bench.l2_adapter_bench.command import (
            run_l2_adapter_bench,
        )
        run_l2_adapter_bench(self, args)

Step 2: Done! The parent CompositeCommand (BenchCommand) auto-discovers the new subcommand at startup. No registration code, no imports to add, no __init__.py edits in the parent.

Note

This works because CompositeCommand.register() scans all direct submodules of its package each time the CLI starts. A new file (or sub-package) is automatically picked up as long as:

  • It does not start with _.

  • It contains a concrete BaseCommand subclass.

Level N: Arbitrary Nesting#

The framework supports unlimited nesting depth. Each level follows the same pattern: a CompositeCommand in a package’s __init__.py auto-discovers its children.

Example: Adding a 3rd level under lmcache mygroup:

mkdir lmcache/cli/commands/mygroup/nested/

# lmcache/cli/commands/mygroup/nested/__init__.py
"""``lmcache mygroup nested`` — a nested command group."""

from lmcache.cli.commands.base import CompositeCommand


class NestedCommand(CompositeCommand):
    """Nested subcommand group."""

    def name(self) -> str:
        return "nested"

    def help(self) -> str:
        return "A nested command group under mygroup."

# lmcache/cli/commands/mygroup/nested/bar_command.py
"""``lmcache mygroup nested bar`` — a deeply nested command."""

import argparse

from lmcache.cli.commands.base import BaseCommand


class BarCommand(BaseCommand):
    """Execute the bar action at level 3."""

    def name(self) -> str:
        return "bar"

    def help(self) -> str:
        return "Execute the bar action."

    def add_arguments(self, parser: argparse.ArgumentParser) -> None:
        parser.add_argument("--msg", default="deep")

    def execute(self, args: argparse.Namespace) -> None:
        metrics = self.create_metrics("Bar Result", args)
        metrics.add("message", "Message", args.msg)
        metrics.emit()

Result:

lmcache mygroup nested bar --msg "hello from level 3"

You can continue nesting indefinitely by repeating this pattern.

Real-World Example#

The existing lmcache tool cache-simulator simulate command demonstrates 3-level nesting:

lmcache tool cache-simulator simulate
│       │    │               └── Level-3 leaf (SimulateCommand in simulate_command.py)
│       │    └── Level-2 composite (CacheSimulatorCommand in cache_simulator/__init__.py)
│       └── Level-1 composite (ToolCommand in tool/__init__.py)
└── CLI entry point

Summary#

Level

Pattern

How to add

1 (top)

Single .py file

Create lmcache/cli/commands/<name>.py with a BaseCommand subclass.

2+

Package directory

Create lmcache/cli/commands/<group>/__init__.py with a CompositeCommand subclass, then add leaf commands as sibling .py files.

N (any)

Nested package

Same as level 2, but inside an existing composite command’s package. Each CompositeCommand scans only its own direct submodules.

Tip

  • Prefix helper/utility modules with _ to exclude them from auto-discovery.

  • Each CompositeCommand must be defined in its package’s __init__.py.

  • The name() method determines the CLI token (e.g. "foo" becomes lmcache ... foo).