Open Code Review: Alibaba’s Rule-Pipeline + LLM Review CLI

1. What Is Open Code Review

Open Code Review is a Go CLI (ocr) that reads a Git diff, sends the changed files to a configurable LLM through an agent with tool use, and returns structured review comments anchored to line numbers. According to its README it started as Alibaba Group’s internal AI code review assistant and was open-sourced after about two years of internal use. Beyond diffs, ocr scan reviews whole files for directories that have no meaningful diff.

Stars are attention, not proof. 47K in under five months says the pitch landed; it does not say the review quality holds on your stack. The sections below separate what the docs show from what you should test yourself.

2. Rule Pipeline + LLM vs. Agentic Review

The common way to build an AI reviewer is to hand a general-purpose coding agent a review skill and let it decide everything: which files to open, what to check, where to anchor each comment. The README names three failure modes it attributes to that approach:

OCR’s answer is to move the steps that must not go wrong out of the prompt and into code. The comparison below is my summary of the README’s design section:

3. The Architecture in 4 Steps

The architecture doc describes a linear pipeline: bootstrap, diff provider, filter and rules, grouping, sub-agents, output. Four ideas carry it:

4. What the Benchmark Says (and Doesn’t)

The README publishes a chart on AACR-Bench: 50 open-source repositories, 200 real pull requests, 10 languages and 1,505 annotated issues, cross-validated by 80+ senior engineers according to the project. The benchmark comes from the same Alibaba organization that built the tool, so read it as a vendor result. These are the two same-model pairs I read from the chart image on 2026-10-14 (OCR v1.3.1 vs. Claude Code v2.1.169):

What holds up: in those two pairs precision is 2.4x to 4.7x higher and token use 5.9x to 14.7x lower, and OCR reports far fewer comments (889 against 5,980 in the first pair), which is what “less noise” means in practice. What does not: recall is not better, and in absolute terms every row of the chart finds a minority of the annotated issues (OCR’s best is 20.00%). The chart also uses OCR v1.3.1 while the current release is v1.13.0, and the README text gives no repeat-run variance.

5. Quickstart: Your First Review

Prerequisites per the docs: Git 2.41+ and Node.js 18+. The commands below are copied from the README and the quickstart doc, verified against the README on 2026-10-14. I did not execute the installer or any of these commands for this post.

npm install -g @alibaba-group/open-code-review
ocr version

Other install paths exist (Homebrew, a curl install script, release binaries, building from source) and are listed in the installation doc. One detail worth knowing: the npm wrapper checks for new versions in the background and upgrades itself by default, and setting OCR_NO_UPDATE to any non-empty value turns that off.

ocr config provider          # Select a built-in provider or add a custom one
ocr config model             # Pick a model for the active provider
ocr llm test

The interactive UI walks you through provider, API key and model, then tests connectivity on its own. Delegation mode (section 9) skips this step entirely.

cd your-project

# Workspace mode — review all staged, unstaged, and untracked changes
ocr review

# Branch range — reviews feature-branch's changes since it diverged from main (merge-base mode)
ocr review --from main --to feature-branch

# Single commit
ocr review --commit abc123

Preview what would be reviewed before you spend tokens, and write machine-readable output for scripts:

# What would be reviewed, and why files were kept or dropped (no LLM cost)
ocr review --preview

# Save results to a file
ocr review --format json --output result.json

# Full-file scan of a directory instead of a diff
ocr scan --path internal/agent

6. Providers and Rules

OCR speaks the Anthropic, OpenAI-compatible and OpenAI Responses protocols, plus AWS Bedrock. The configuration doc lists built-in presets, so most of the time you pick one and paste a key:

ocr config set provider                    anthropic
ocr config set model                       claude-opus-4-6
ocr config set providers.anthropic.api_key_cmd "op read op://dev/anthropic/api-key"

Anything not in the preset list is a custom provider, which is how you point OCR at an internal gateway or a local model. This is the docs’ Ollama example:

ocr config set provider                          ollama
ocr config set custom_providers.ollama.url       http://127.0.0.1:11434/v1
ocr config set custom_providers.ollama.protocol  openai
ocr config set custom_providers.ollama.model     qwen3:32b
ocr config set custom_providers.ollama.api_key   ollama

Rules are where you encode your team’s standards. They are JSON, layered and matched per file by glob; the first matching entry wins. By default a matching entry replaces the built-in language rule for that file, so set merge_system_rule when you want to add to it:

{
  "rules": [
    {
      "path": "**/*",
      "rule": "Security review: flag hardcoded secrets, unvalidated redirects, and missing authz checks.",
      "merge_system_rule": true
    }
  ]
}

The embedded rules cover Java, Go, TypeScript and JavaScript, Rust, Python, Solidity, Vyper, Terraform, GitHub workflows and more. When a rule never fires, this command prints the layer and pattern that won:

ocr rules check src/main/java/com/example/UserService.java

7. Where It Fits in CI

The repo ships CI examples for GitHub Actions, GitLab CI, Bitbucket Pipelines, GitFlic CI and Gerrit. The GitHub path is a reusable action: it installs OCR, computes the merge base, runs ocr review --from <merge-base> --to <head> --format json, and posts inline comments plus a summary on the PR. Permissions are contents: read and pull-requests: write. The core step from the example README:

- uses: alibaba/open-code-review@main
  with:
    llm_url: ${{ secrets.OCR_LLM_URL }}
    llm_auth_token: ${{ secrets.OCR_LLM_AUTH_TOKEN }}
    llm_model: ${{ vars.OCR_LLM_MODEL }}
    llm_use_anthropic: ${{ vars.OCR_LLM_USE_ANTHROPIC }}

The example expects OCR_LLM_URL and OCR_LLM_AUTH_TOKEN as secrets, and OCR_LLM_MODEL and OCR_LLM_USE_ANTHROPIC as variables. Three things I would change before copying it:

- uses: alibaba/open-code-review@<full-commit-sha> # vX.Y.Z
  with:
    ocr_version: 'X.Y.Z'
    llm_url: ${{ secrets.OCR_LLM_URL }}
    llm_auth_token: ${{ secrets.OCR_LLM_AUTH_TOKEN }}
    llm_model: ${{ vars.OCR_LLM_MODEL }}
    llm_use_anthropic: ${{ vars.OCR_LLM_USE_ANTHROPIC }}

- uses: alibaba/open-code-review@<full-commit-sha> # vX.Y.Z
  with:
    ocr_version: 'X.Y.Z'
    effort: low
    max_tokens_budget: '2000000'

8. Privacy: What Actually Leaves Your Machine

Sending code to a third-party model is the real adoption question. This is what the FAQ and docs state:

For regulated code the config doc gives you levers: Bedrock through the AWS credential chain, a LiteLLM gateway inside your VPC via providers.<name>.url, or a local Ollama model. Some presets point at hosts in cn-beijing (volcengine, dashscope-tokenplan), so check data residency before choosing one.

9. Where It Shines: Use Cases

OCR earns its keep where you want consistent, low-noise review comments on every change:

The common thread: the pipeline makes the boring parts (what to review, which rules, where to comment) repeatable, and the model is spent only on judgment.

10. When NOT to Use It

I like the engineering, but this is a precision tool, not a safety net. Skip it when:

Conclusion

Open Code Review is a credible answer to a real problem: agent reviewers that skip files, drift off the right line and cost a fortune in tokens. Putting file selection, rule matching and comment positioning in code is the right instinct, and the docs are unusually candid about the cost: lower recall, no built-in redaction, failures that exit 0.

My advice is simple. Install it, run ocr review --preview on a real branch, replay a few merged PRs, and decide from your own numbers. If it finds the bugs you care about, pin the version and put it in CI as an advisory reviewer.