Skip to content

vllm.models.minimax_m3.amd.mono.config

Shapes the mono kernels are compiled for: one MiniMax-M3 TP4 rank.

Every value here is checked against the loaded model in weights.py; a model that does not match is never routed to the mono path.

Classes:

  • IndexHeads –

    The index q heads in a rank's fused projection, a build parameter:

  • MonoUnsupported –

    The loaded model or runtime configuration is outside what mono serves.

Functions:

  • qkv_rows –

    Rows of a rank's fused projection, q | k | v | index_q | index_k: 16 q

IndexHeads dataclass

The index q heads in a rank's fused projection, a build parameter: count of them (1, or all TP in head order under indexer context parallelism) and own, the one this rank's selection scores.

Source code in vllm/models/minimax_m3/amd/mono/config.py 
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
@dataclass(frozen=True)
class IndexHeads:
    """The index q heads in a rank's fused projection, a build parameter:
    ``count`` of them (1, or all TP in head order under indexer context
    parallelism) and ``own``, the one this rank's selection scores."""

    count: int = 1
    own: int = 0

    def __post_init__(self):
        assert self.count in (1, TP) and 0 <= self.own < self.count

    @property
    def rows(self) -> int:
        return qkv_rows(self.count)

    @property
    def iq_off(self) -> int:
        return (LOCAL_Q_HEADS + 2 + self.own) * HEAD_DIM

    @property
    def ik_off(self) -> int:
        return (LOCAL_Q_HEADS + 2 + self.count) * HEAD_DIM

MonoUnsupported

Bases: Exception

The loaded model or runtime configuration is outside what mono serves.

Source code in vllm/models/minimax_m3/amd/mono/config.py 
107
108
class MonoUnsupported(Exception):
    """The loaded model or runtime configuration is outside what mono serves."""

qkv_rows(idx_heads)

Rows of a rank's fused projection, q | k | v | index_q | index_k: 16 q heads, one k and v head, idx_heads index q heads (the rank's one, or every one under indexer context parallelism) and the index k head.

Source code in vllm/models/minimax_m3/amd/mono/config.py 
27
28
29
30
31
def qkv_rows(idx_heads: int) -> int:
    """Rows of a rank's fused projection, q | k | v | index_q | index_k: 16 q
    heads, one k and v head, ``idx_heads`` index q heads (the rank's one, or every
    one under indexer context parallelism) and the index k head."""
    return (LOCAL_Q_HEADS + 3 + idx_heads) * HEAD_DIM