Mellum2.1: Running JetBrains’ Open Coding-Agent Model Locally with vLLM

1. What JetBrains shipped

Mellum started life as a code-completion model. Mellum2, released in June 2026 with a technical report on arXiv, was the move to a 12B mixture-of-experts design that activates about 2.5B parameters per token. Mellum2.1 is the next version of that Thinking model: same architecture, new post-training.

The weights are on Hugging Face under JetBrains/Mellum2.1-12B-A2.5B-Thinking. The launch post positions it for coding agents and fast sub-agents on your own hardware. There is no hosted JetBrains API or per-token price that I could find, so self-hosting is the path.

2. What changed vs Mellum2

Same skeleton, different training. JetBrains says the architecture is unchanged and that reinforcement learning became the main part of training rather than a short final stage. The data widened to math, competitive programming, science, tool use and software engineering, and JetBrains built in-house infrastructure that launched millions of sandboxes.

The practical shift is stated plainly in the launch post: Mellum2 could not work inside a repository at the level JetBrains wanted, while Mellum2.1 is meant to explore a codebase, edit files and check its own changes. That is the difference between a completion-style model and one you can put in a loop.

The three jumps (model card, self-reported)

3. The numbers, with provenance

Every score below comes from the Hugging Face model card and was produced by JetBrains. Non-agentic benchmarks used greedy decoding. Agentic ones used a harness called Pi (v0.73.1) with shell and file tools, a 114K-token context, up to 16K tokens per turn and temperature 1.0. The comparison set is Mellum2, Gemma 4 (E4B) and Qwen3.5 (9B).

Speed is a separate claim from the launch post: on one H200, under heavy load, JetBrains says Mellum2.1 serves almost twice as many tokens as Qwen3.5-9B, and that multi-token prediction would make a single request about 1.6 times faster. The exact values appear only in a chart, and the MTP head for vLLM is listed as coming soon.

4. Quickstart: serve it with vLLM

The card’s serving recipe is two commands: one without tool calling, one with. For an agent you want the second. Both pass --max-model-len 131072 and the qwen3 reasoning parser, which splits the thinking trace from the final answer. vLLM’s supported-models page lists the MellumForCausalLM architecture; the card states no minimum vLLM version, so use a recent release.

Memory is the real question, and the card gives no VRAM guidance. The numbers below are my arithmetic from the checkpoint size and config.json, not measurements. The 2.5B active parameters cut compute per token, not memory: all 12B must be resident.

vllm serve JetBrains/Mellum2.1-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --reasoning-parser qwen3

vllm serve JetBrains/Mellum2.1-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

from openai import OpenAI
# Configured by environment variables
client = OpenAI()

messages = [
    {"role": "user", "content": "Find the bug in this function and explain the fix: def mean(xs): return sum(xs) / len(xs) - 1"},
]

chat_response = client.chat.completions.create(
    model="JetBrains/Mellum2.1-12B-A2.5B-Thinking",
    messages=messages,
    max_tokens=81920,
    temperature=0.6,
    top_p=0.95,
    extra_body={"top_k": 20},
)
print("Chat response:", chat_response)

# JetBrains GGUF repo, Q4_K_M (the quant its card marks as recommended)
ollama run hf.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M

# or llama.cpp
llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M --ctx-size 131072 --temp 0.6 --top-p 0.95 --top-k 20

5. Wire it into a coding-agent loop

vLLM exposes an OpenAI-compatible server, so any harness that accepts a base URL and speaks chat-completions with tools can point at it. The sketch below is the smallest loop I would use to test the model: one tool, a step cap, no memory tricks, the card’s sampling settings and a per-turn token cap.

Two details from the vLLM docs matter. Reasoning comes back in its own field (currently reasoning, formerly reasoning_content, so read both). And tool calls are parsed only from the content, not from inside the reasoning: if the model emits a call inside its think block, your loop sees nothing.

import json
import subprocess
from openai import OpenAI

# Sketch: not from the model card, not run by me.
client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
MODEL = "JetBrains/Mellum2.1-12B-A2.5B-Thinking"

TOOLS = [{
    "type": "function",
    "function": {
        "name": "run_shell",
        "description": "Run a shell command in the repo; returns stdout and stderr.",
        "parameters": {
            "type": "object",
            "properties": {"cmd": {"type": "string"}},
            "required": ["cmd"],
        },
    },
}]

def run_shell(cmd: str) -> str:
    # DANGER: model-written commands. Run inside a container, no secrets mounted.
    p = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=120)
    return (p.stdout + p.stderr)[-8000:]

messages = [
    {"role": "system", "content": "You are a coding agent. Explore with run_shell, edit files, run the tests, stop when they pass."},
    {"role": "user", "content": "tests/test_mean.py fails. Find the cause and fix it."},
]

for step in range(20):
    resp = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        tools=TOOLS,
        tool_choice="auto",
        max_tokens=16384,          # per-turn cap; the card quickstart uses 81920
        temperature=0.6,
        top_p=0.95,
        extra_body={"top_k": 20},  # same sampling as the card quickstart
    )
    choice = resp.choices[0]
    msg = choice.message
    reasoning = getattr(msg, "reasoning", None) or getattr(msg, "reasoning_content", None)
    print(f"[step {step}] finish={choice.finish_reason} reasoning_chars={len(reasoning or '')} tool_calls={len(msg.tool_calls or [])}")

    assistant = {"role": "assistant", "content": msg.content or ""}
    if msg.tool_calls:
        assistant["tool_calls"] = [
            {
                "id": c.id,
                "type": "function",
                "function": {"name": c.function.name, "arguments": c.function.arguments},
            }
            for c in msg.tool_calls
        ]
    messages.append(assistant)

    if not msg.tool_calls:
        print(msg.content)
        break

    for c in msg.tool_calls:
        args = json.loads(c.function.arguments)
        messages.append({"role": "tool", "tool_call_id": c.id, "content": run_shell(args["cmd"])})

6. Where it fits, and the honest limits

JetBrains frames Mellum2.1 as a model for coding agents and fast sub-agents on your own hardware, not as a frontier replacement. Its comparison set is small models only. I have no same-harness numbers for larger or closed models, so I will not invent a gap; that is the question your own eval has to answer.

For Mellum2, the launch post suggested sub-agent jobs such as planning, validation, transformation and context preparation. Mellum2.1 keeps that role and adds repository-level edits, but the card’s own table shows a 9B rival ahead on the hardest agentic evals.

Verdict

Mellum2.1 is a credible small-footprint model for local agent work: Apache 2.0, one GPU, and a self-reported jump from 2.0 to 47.0 on SWE-bench Verified that comes from RL alone. It is not a frontier replacement, and the card itself shows Qwen3.5 (9B) ahead on SWE-bench Verified, SWE-bench Pro and Terminal-Bench 2.1.

What I would do: pull the weights, serve with the tool-calling command, run 20 to 50 of your own tickets with tests as the judge, and compare resolve rate, wall-clock and GPU cost against whatever you run now. Until someone outside JetBrains reruns SWE-bench, treat 47.0 as a promising number, not a settled one.