Welcome back!

Resume reading from where you left off?

Cover with a three-layer stack and a $0 badge beside the title Stop Paying for Tokens
AI/ML

Stop Paying for Tokens: Ship a Real AI Tool on a $0 Stack

Ship a local ticket-triage CLI on Ollama and SQLite. Schema-checked JSON, one stored decision per ticket, and no API token bill.

By Pankaj Sonani from aigrama.net • 13 min read
#ollama#llm#python#ai

OpenCode + a free “premium” LLM + automatic fallback + Cloudflare + Supabase — from empty folder to live URL, without entering a credit card.


A developer I know burned $63 in API credits last month building a tool that exactly zero people used.

He wasn’t training anything. He wasn’t serving traffic. He was paying frontier-model prices to ask an AI why his useEffect fired twice.

Here’s the part that stings: the model he was paying for had a free twin. Same weights. Same context window. Same output. Sitting on OpenRouter with a :free tag, behind a rate limit instead of a paywall.

If you’re building for yourself — learning, prototyping, shipping a weekend tool — you almost never need to pay for tokens. You need to know where the free door is, and what to do when it closes for 60 seconds.

That’s this entire post.

By the end you’ll have:

  • A local AI coding agent (OpenCode) running on a free premium model
  • A fallback ladder that silently swaps models when one hits a rate limit — so you never stare at a 429 again
  • A real, working tool: Plainly — paste anything confusing, get plain English
  • A live URL on Cloudflare, with saved history in Supabase
  • A total bill of $0.00

⏱️ ~30 minutes. ☕ One coffee. 💳 Zero cards. Step 5 is the one nobody writes about — it’s the difference between a demo and something you actually use daily.

📌 One honest note before we start: model slugs, free tiers, and rate limits change constantly. Treat every ID below as a template — copy the exact slug from OpenRouter’s model page. If baidu/ernie-5.5-flash:free has been renamed or un-freed by the time you read this, the architecture still works. That’s the whole point of the ladder.


The $0 stack, at a glance

Layer Tool What it costs Why it’s here
Coding agent OpenCode (open source) Free Runs in your terminal, edits real files
Model ERNIE 5.5 Flash (:free) Free Fast, cheap-to-serve, strong at code + reasoning
Gateway OpenRouter Free tier One API key, hundreds of models, free variants
Fallback Tiny Cloudflare Worker Free Turns “rate limited” into “slightly slower”
Database + auth Supabase Free tier Postgres, row-level security, login
Hosting Cloudflare Pages Free tier Global CDN, instant deploys

Now let’s build it.


Step 1 — One folder. One agent. Four minutes.

Most AI projects die in a swamp of test2_final_FINAL.js. Give the agent a clean room to work in.

mkdir plainly && cd plainly
mkdir -p app worker supabase
git init
printf "node_modules/\n.env\n.wrangler/\n" > .gitignore

Install OpenCode (pick one):

npm i -g opencode-ai
# or
curl -fsSL https://opencode.ai/install | bash

Then initialize it:

opencode

Inside the TUI, run /init. OpenCode reads your project and writes an AGENTS.md — the standing instructions it follows on every single task. This file is the highest-leverage 20 lines in your whole project.

# AGENTS.md — Plainly

## Product
Paste confusing text → get plain English. Two modes: Everyday and Developer.

## Stack
Static frontend (app/), Cloudflare Worker API (worker/), Supabase (supabase/).

## Rules
- No paid services. Free tiers only.
- Never put secrets in app/. Frontend gets the Supabase anon key only.
- Vanilla JS + no build step unless I explicitly ask.
- Every new endpoint must handle upstream 429 by falling back.
- Small files. Explain the diff before writing it.

💡 Why this matters: an agent without AGENTS.md is a talented intern with amnesia. With it, it’s a teammate who already knows the house rules.

✅ Checkpoint: opencode opens, AGENTS.md exists. No model configured yet — that’s next.


Step 2 — The key (and the mistake everyone makes)

  1. Sign in at openrouter.ai
  2. Go to Keys → Create Key
  3. Name it plainly-local, copy it once
echo 'export OPENROUTER_API_KEY="sk-or-v1-paste-yours-here"' >> ~/.zshrc
source ~/.zshrc

⚠️ Read this before you paste anything:

  • An API key in a frontend file is a public donation to strangers. It goes in your shell env or a Worker secret. Never in app/.
  • .env is already gitignored above. Good. Keep it that way.
  • OpenRouter’s privacy settings control whether providers may train on your prompts. Many free endpoints are free because of that trade. Free is fine for a public demo tool. Free is not fine for a client’s contract or your medical results. Toggle it consciously.

That last bullet is the real price of “free.” Know it, choose it, move on.


Step 3 — How to find actually free premium models (the 1-line trick)

Browsing openrouter.ai/models with the price filter set to $0 works. But it hides the thing that breaks coding agents.

A model can be free and still be useless to OpenCode — if it doesn’t support tool calling. Your agent will chat beautifully and then fail to edit a single file.

Ask the API instead of the eyeballs:

curl -s https://openrouter.ai/api/v1/models \
| jq -r '.data[]
  | select(.pricing.prompt == "0" and .pricing.completion == "0")
  | select(.supported_parameters | index("tools"))
  | "\(.id)\tctx=\(.context_length)"'

That’s the whole game: price = 0 AND tools supported. Sort by context length and pick your ladder.

A sane starting set:

Role Model slug Why
🥇 Primary baidu/ernie-5.5-flash:free Fast, strong instruction-following, great cost/perf profile
🥈 Backup qwen/qwen3-coder:free Code-specialist, tool-friendly
🥉 Safety net meta-llama/llama-3.3-70b-instruct:free Boring, reliable, almost always up

🔎 Verify the slugs. Copy them straight from each model page. If a model has no :free variant listed, it is not free — no suffix tricks will change that.

✅ Checkpoint: you have three real, tool-capable, $0 model IDs. Save them; Step 5 needs all three.


Step 4 — Wire OpenRouter into OpenCode, then prove it works

OpenCode already knows OpenRouter. Fastest path:

opencode auth login   # choose OpenRouter, paste your key

Then pin your choices in opencode.json at the project root:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "openrouter/baidu/ernie-5.5-flash:free",
  "small_model": "openrouter/meta-llama/llama-3.3-70b-instruct:free",
  "provider": {
    "openrouter": {
      "options": { "apiKey": "{env:OPENROUTER_API_KEY}" },
      "models": {
        "baidu/ernie-5.5-flash:free": { "name": "ERNIE 5.5 Flash (free)" },
        "qwen/qwen3-coder:free": { "name": "Qwen3 Coder (free)" },
        "meta-llama/llama-3.3-70b-instruct:free": { "name": "Llama 3.3 70B (free)" }
      }
    }
  }
}

Two details worth more than they look:

  • {env:OPENROUTER_API_KEY} — the key never touches a committed file.
  • small_model — OpenCode uses a cheap model for throwaway jobs like generating session titles. Point that at a free model too, or you’ll leak spend on work you never see.

Smoke test, headless:

opencode run --model openrouter/baidu/ernie-5.5-flash:free \
  "Reply with exactly: FREE TIER ONLINE"

Then the real test — does it act, not just talk?

opencode run "Create app/index.html: a single textarea, a button labeled Explain, and an empty results div. No frameworks."

Open the file. If it exists, your free model is doing agentic work. Inside the TUI, /models swaps models live — remember that command, you’ll use it.

🎉 You now have a free AI developer. And in about ten minutes of real usage, you’ll hit the wall.


Step 5 — The free-tier wall (and the ladder that makes it disappear)

Here’s what nobody tells you in the “free LLM!!” threads: free endpoints are rate limited, and they get congested. Roughly, OpenRouter’s free tier gives you a per-minute cap and a daily cap that rises once you’ve ever added a small amount of credit — check the current numbers in their docs, because they move.

So the naive setup dies like this: you’re mid-refactor, the model returns 429, and your flow is gone.

The fix isn’t a paid plan. It’s a ladder. Two layers:

Layer 1 — OpenRouter’s native fallback

OpenRouter accepts a models array instead of a single model. It tries them in order and only bills (or fails) after the list is exhausted:

{
  "models": [
    "baidu/ernie-5.5-flash:free",
    "qwen/qwen3-coder:free",
    "meta-llama/llama-3.3-70b-instruct:free"
  ],
  "messages": [{ "role": "user", "content": "Explain this stack trace..." }]
}

Beautiful. But your agent sends model, not models. So we put the ladder in front of it.

Layer 2 — A 40-line Worker that never says “rate limited”

cd worker && npm i -D wrangler

worker/wrangler.toml:

name = "plainly-router"
main = "src/index.js"
compatibility_date = "2026-11-01"

worker/src/index.js:

const LADDER = [
  "baidu/ernie-5.5-flash:free",
  "qwen/qwen3-coder:free",
  "meta-llama/llama-3.3-70b-instruct:free",
];
const RETRYABLE = new Set([408, 409, 429, 500, 502, 503, 504]);
const CORS = {
  "Access-Control-Allow-Origin": "*",
  "Access-Control-Allow-Headers": "content-type,authorization",
};

export default {
  async fetch(req, env) {
    if (req.method === "OPTIONS") return new Response(null, { headers: CORS });
    if (new URL(req.url).pathname !== "/v1/chat/completions")
      return new Response("not found", { status: 404 });

    // Don't run an open proxy to your key.
    if (req.headers.get("authorization") !== `Bearer ${env.APP_TOKEN}`)
      return new Response("unauthorized", { status: 401 });

    const body = await req.json();
    let lastError = "all free models unavailable";

    for (const model of LADDER) {
      const upstream = await fetch("https://openrouter.ai/api/v1/chat/completions", {
        method: "POST",
        headers: {
          Authorization: `Bearer ${env.OPENROUTER_API_KEY}`,
          "Content-Type": "application/json",
          "HTTP-Referer": "https://plainly.pages.dev",
          "X-Title": "Plainly",
        },
        body: JSON.stringify({ ...body, model, models: LADDER }),
      });

      if (upstream.ok) {
        const headers = new Headers(upstream.headers);
        headers.set("x-served-by", model);
        for (const [k, v] of Object.entries(CORS)) headers.set(k, v);
        return new Response(upstream.body, { status: 200, headers }); // streams through
      }

      lastError = await upstream.text();
      if (!RETRYABLE.has(upstream.status)) break; // 400 = your bug, not their outage
    }

    return new Response(lastError, { status: 503, headers: CORS });
  },
};

Deploy and load the secrets:

npx wrangler secret put OPENROUTER_API_KEY
npx wrangler secret put APP_TOKEN      # any long random string
npx wrangler deploy

Now teach OpenCode to use the ladder as if it were a normal provider:

"freeladder": {
  "npm": "@ai-sdk/openai-compatible",
  "name": "Free ladder (OpenRouter)",
  "options": {
    "baseURL": "https://plainly-router.YOUR-SUBDOMAIN.workers.dev/v1",
    "apiKey": "{env:APP_TOKEN}"
  },
  "models": {
    "auto-free": { "name": "Auto free ladder" }
  }
}
opencode run --model freeladder/auto-free "Refactor app/index.html to use a dark theme."

The auto-free name is cosmetic — the Worker overrides it and picks whoever is awake. Response header x-served-by tells you who actually answered.

🧠 The mindset shift: stop thinking “which free model is best?” Start thinking “what’s my ordered list, and who decides?” One model is a dependency. Three models behind a router is infrastructure.

✅ Checkpoint: temporarily break the first slug (...flash-free-BROKEN), redeploy, rerun. It still works — served by model #2. That’s the whole trick, proven.


Step 6 — Build the tool (this is where it gets fun)

Plainly: paste anything confusing → plain English. Everyday mode: a medical bill line item, a lease clause, a Terms of Service paragraph. Developer mode: a stack trace → likely cause, then the fix.

One product, two audiences, zero new infrastructure. Prompt it like a spec, not a wish:

Build app/index.html + app/app.js for Plainly.

UI: one textarea, a mode toggle (Everyday | Developer), an Explain button,
a results panel, and a "Save" button. Dark, single-column, system fonts, no build step.

Behavior: POST { text, mode } to /api/explain on the Worker.
Show a loading state. Render markdown-ish output as plain text with line breaks.
Handle 503 with: "All free models are busy — try again in a minute."

System prompt: explain at an 8th-grade reading level, lead with a one-sentence
summary, then 3 bullets. Developer mode adds "Likely cause" and "Fix".
Never invent facts; say when the input is too ambiguous.

Then iterate the way you’d talk to a junior dev:

Add a character counter capped at 4000, and disable Explain past the cap.
The results panel jumps on mobile. Fix the layout, don't add a library.

⚡ Free-tier pro move: batch your asks. One well-specified request beats five vague ones — and every round trip eats your daily quota. Prompt like tokens are scarce, because they are.


Step 7 — Supabase: give it memory, free

Create a free project at supabase.com, then run in the SQL editor:

create table public.explanations (
  id uuid primary key default gen_random_uuid(),
  user_id uuid not null default auth.uid() references auth.users on delete cascade,
  input text not null,
  output text not null,
  model text,
  created_at timestamptz not null default now()
);

alter table public.explanations enable row level security;

create policy "users manage their own rows"
  on public.explanations for all
  using (auth.uid() = user_id)
  with check (auth.uid() = user_id);

Wire the frontend with @supabase/supabase-js: magic-link auth, then insert on Save and select for History. RLS means even a stolen anon key can’t read someone else’s rows.

🚨 Two non-negotiables:

  1. Never ship the service_role key to the browser. It bypasses RLS entirely. Worker secrets only.
  2. Free Supabase projects pause after a period of inactivity. Fine for a portfolio piece; a cron ping or a paid plan is the answer if it’s real.

Step 8 — Cloudflare: put it on the internet

The Worker is already live. Now the frontend:

npx wrangler pages deploy ./app --project-name plainly

Seconds later: https://plainly.pages.dev. HTTPS, global CDN, custom domains supported, no card.

Last mile — in app/app.js, point the fetch at your Worker and send the APP_TOKEN… or better, route /api/* to the Worker via a Pages route so the browser never holds a token at all. Then add a rate-limit or Turnstile check before you share the link publicly, because “free” plus “unprotected” equals “someone else’s bot farm on your quota.”

✅ Done. Local agent on free models. Self-healing fallback. Live URL. Persistent history. $0.


The $0 fine print (read it, don’t discover it)

Reality What to do
Free models are rate limited per minute and per day The ladder + batched prompts
Free endpoints may allow provider training on your data Never send private/regulated text
Free variants get retired or repriced without notice Ladder + re-run the jq command monthly
Small free models can be weak at multi-file tool use Keep tasks small; /models up to a bigger free model
Cloudflare/Supabase free tiers have daily & storage caps Fine for prototypes; budget $5–10 when you get users
An exposed Worker is an open proxy to your key Auth token + rate limiting, always

When to stop being free: the day a real user complains about latency. Not before. Free tiers aren’t a lifestyle, they’re a runway — and runway is exactly what a side project needs.


Your 30-minute checklist

  • plainly/ folder + AGENTS.md written
  • OpenCode installed, OPENROUTER_API_KEY in shell env
  • Three free, tool-capable model slugs found via jq
  • opencode run returns text and creates a file
  • Fallback Worker deployed, x-served-by proves the swap
  • freeladder/auto-free working inside OpenCode
  • Supabase table + RLS + auth
  • wrangler pages deploy → live URL
  • git log clean, git grep sk-or-v1 returns nothing

FAQ

Is this actually legal / allowed? Yes — free tiers exist to win developers. Follow the rate limits and each provider’s terms. Don’t resell access or run a public unauthenticated proxy.

Do I need a credit card anywhere? Not for the stack as described. Some free-tier daily caps increase after a small one-time credit purchase on OpenRouter — optional, and check current policy.

Is a free model “worse”? Usually it’s the same model with a queue and a quota, not a dumber one. You pay in latency and limits, not IQ.

Why OpenCode instead of a paid IDE agent? Open source, terminal-native, model-agnostic, and it reads AGENTS.md. Swapping providers is a config edit — which is the entire reason this article is possible.

Why ERNIE 5.5 Flash as the primary? Flash-class models are tuned for speed and efficiency, which is exactly what an agent loop wants: many short calls, fast turnaround. But it’s a slot, not a religion. If tomorrow’s best free model is something else, change one line.


The real takeaway

The gap between “I have an idea” and “it’s live” was never money. It was friction — pricing pages, quota errors, five dashboards, and the quiet fear of a surprise bill.

You just removed all of it. One folder, one key, three free models, one router, two free hosts.

The next move is yours: what’s the tiny annoying thing you’d fix if it cost nothing to try?

Build it tonight. Ship the ugly version. Tell me what you made in the comments — and if a free model disappears next week, you already know what to do. 🪜


📎 Bonus: publishing kit

Title options (A/B these):

  1. Stop Paying for Tokens: Build and Ship a Real AI Tool on a $0 Stack
  2. My AI Coding Agent Costs $0/Month — Here’s the Exact Config
  3. The Free-Model Fallback Trick That Makes Rate Limits Disappear

Meta description (154 chars): Run OpenCode on free premium LLMs via OpenRouter, add automatic model fallback, and ship to Cloudflare + Supabase — full config, zero credit card.

Social hook:

He spent $63 asking AI why his CSS broke. The same model was free — behind a rate limit instead of a paywall. Here’s the $0 stack + the 40-line fallback router that makes limits invisible. 🧵

example of the contract

Each decision has four fields:

  • category: billing, access, bug, or other
  • priority: low, normal, or high
  • reply: a short draft a person can edit
  • needs_human: true when the ticket touches money, account access, or a defect

The CLI never sends email. It writes a draft and a flag. Billing, access, and bug tickets are supposed to set needs_human so a queue can hold them. A status-page question can come back with the flag false, and a person still reads the reply before it goes out.

Pass the same schema in the prompt and in format. Ollama’s docs recommend both: the grammar limits tokens, and the prompt tells the model what the fields mean. Set think to false so a thinking-capable tag does not spend the reply on a trace, and set temperature to 0 so the same model file repeats itself.

One file

Save this as triage.py. It uses only the standard library. Python 3.11 or newer is a safe choice.

#!/usr/bin/env python3
"""Triage one support ticket on a local Ollama model. Stdlib only."""

import argparse
import json
import os
import sqlite3
import sys
import urllib.error
import urllib.request
from datetime import datetime, timezone
from pathlib import Path

MODEL = os.environ.get("TRIAGE_MODEL", "qwen3:4b-instruct")
HOST = os.environ.get("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/")
DB_PATH = Path(os.environ.get("TRIAGE_DB", "triage.sqlite"))

CATEGORIES = ("billing", "access", "bug", "other")
PRIORITIES = ("low", "normal", "high")

SCHEMA = {
    "type": "object",
    "properties": {
        "category": {"type": "string", "enum": list(CATEGORIES)},
        "priority": {"type": "string", "enum": list(PRIORITIES)},
        "reply": {"type": "string"},
        "needs_human": {"type": "boolean"},
    },
    "required": ["category", "priority", "reply", "needs_human"],
}

SYSTEM = (
    "You triage one support ticket. "
    "Pick category and priority from the schema. "
    "Write a reply of one or two sentences that a human agent can send. "
    "Set needs_human to true when the ticket involves money, account access, or a product defect. "
    "Set needs_human to false only for general questions that do not need an account change."
)


def connect() -> sqlite3.Connection:
    connection = sqlite3.connect(DB_PATH)
    connection.execute(
        """
        CREATE TABLE IF NOT EXISTS decisions (
            ticket_key TEXT PRIMARY KEY,
            subject TEXT NOT NULL,
            body TEXT NOT NULL,
            category TEXT NOT NULL,
            priority TEXT NOT NULL,
            reply TEXT NOT NULL,
            needs_human INTEGER NOT NULL,
            model TEXT NOT NULL,
            created_at TEXT NOT NULL
        )
        """
    )
    return connection


def classify(subject: str, body: str) -> dict[str, object]:
    user = f"Subject: {subject}\n\n{body}\n\nJSON schema:\n{json.dumps(SCHEMA)}"
    payload = {
        "model": MODEL,
        "stream": False,
        "think": False,
        "format": SCHEMA,
        "options": {"temperature": 0},
        "messages": [
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": user},
        ],
    }
    request = urllib.request.Request(
        f"{HOST}/api/chat",
        data=json.dumps(payload).encode(),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urllib.request.urlopen(request, timeout=180) as response:
            raw = json.load(response)
    except urllib.error.HTTPError as exc:
        detail = exc.read().decode("utf-8", errors="replace")
        raise SystemExit(
            f"Ollama returned HTTP {exc.code} for model {MODEL}. "
            f"If the model is missing, run: ollama pull {MODEL}\n{detail}"
        ) from exc
    except urllib.error.URLError as exc:
        raise SystemExit(
            f"Ollama is not reachable at {HOST}. Start it, then run: ollama pull {MODEL}\n{exc.reason}"
        ) from exc
    except TimeoutError as exc:
        raise SystemExit(
            f"Ollama did not answer within 180s. The first call loads {MODEL} into memory.\n{exc}"
        ) from exc

    message = raw.get("message") if isinstance(raw, dict) else None
    content = message.get("content") if isinstance(message, dict) else None
    if not isinstance(content, str):
        raise SystemExit(f"Unexpected Ollama response: {raw}")
    return parse_decision(content)


def parse_decision(content: str) -> dict[str, object]:
    try:
        data = json.loads(content)
    except json.JSONDecodeError as exc:
        raise SystemExit(f"Model did not return JSON:\n{content}") from exc
    if not isinstance(data, dict):
        raise SystemExit("Model returned JSON that is not an object.")

    category = data.get("category")
    priority = data.get("priority")
    reply = data.get("reply")
    needs_human = data.get("needs_human")
    if category not in CATEGORIES or priority not in PRIORITIES:
        raise SystemExit(f"Model returned a value outside the schema: {data}")
    if not isinstance(reply, str) or not reply.strip():
        raise SystemExit("Model returned an empty reply.")
    if not isinstance(needs_human, bool):
        raise SystemExit("needs_human must be true or false.")
    return {
        "category": category,
        "priority": priority,
        "reply": reply.strip(),
        "needs_human": needs_human,
    }


def decide(
    connection: sqlite3.Connection,
    ticket_key: str,
    subject: str,
    body: str,
) -> dict[str, object]:
    row = connection.execute(
        """
        SELECT category, priority, reply, needs_human, model
        FROM decisions
        WHERE ticket_key = ?
        """,
        (ticket_key,),
    ).fetchone()
    if row is not None:
        return {
            "category": row[0],
            "priority": row[1],
            "reply": row[2],
            "needs_human": bool(row[3]),
            "model": row[4],
            "cached": True,
        }

    decision = classify(subject, body)
    connection.execute(
        """
        INSERT INTO decisions (
            ticket_key, subject, body, category, priority, reply,
            needs_human, model, created_at
        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
        """,
        (
            ticket_key,
            subject,
            body,
            decision["category"],
            decision["priority"],
            decision["reply"],
            int(decision["needs_human"] is True),
            MODEL,
            datetime.now(timezone.utc).isoformat(),
        ),
    )
    connection.commit()
    decision["model"] = MODEL
    decision["cached"] = False
    return decision


GOLDEN = (
    {
        "key": "t-100",
        "subject": "Charged twice for March",
        "body": "My card shows two charges of 49 dollars for the same month. Please refund one.",
        "category": "billing",
        "needs_human": True,
    },
    {
        "key": "t-101",
        "subject": "Cannot log in after password reset",
        "body": "The reset email arrived, the new password is accepted, then the site sends me back to login.",
        "category": "access",
        "needs_human": True,
    },
    {
        "key": "t-102",
        "subject": "Export button downloads an empty file",
        "body": "CSV export on the invoices page finishes and the file is 0 bytes. Started after yesterday's deploy.",
        "category": "bug",
        "needs_human": True,
    },
    {
        "key": "t-103",
        "subject": "Where is the status page?",
        "body": "Do you publish uptime somewhere? I could not find a link in the footer.",
        "category": "other",
        "needs_human": False,
    },
)


def cmd_triage(args: argparse.Namespace) -> None:
    connection = connect()
    try:
        decision = decide(connection, args.key, args.subject, args.body)
    finally:
        connection.close()
    json.dump(decision, sys.stdout, indent=2)
    sys.stdout.write("\n")


def cmd_eval(_args: argparse.Namespace) -> None:
    matched = 0
    for case in GOLDEN:
        decision = classify(case["subject"], case["body"])
        category_ok = decision["category"] == case["category"]
        human_ok = decision["needs_human"] == case["needs_human"]
        matched += int(category_ok and human_ok)
        mark = "ok" if category_ok and human_ok else "miss"
        print(
            f"{mark}  {case['key']}  "
            f"category {decision['category']} (want {case['category']})  "
            f"needs_human {decision['needs_human']} (want {case['needs_human']})"
        )
    print(f"{matched}/{len(GOLDEN)} matched")
    if matched != len(GOLDEN):
        raise SystemExit(1)


def main() -> None:
    parser = argparse.ArgumentParser(description="Triage one support ticket on a local model.")
    sub = parser.add_subparsers(dest="command", required=True)

    triage = sub.add_parser("triage", help="Classify one ticket and store the decision")
    triage.add_argument("--key", required=True)
    triage.add_argument("--subject", required=True)
    triage.add_argument("--body", required=True)
    triage.set_defaults(func=cmd_triage)

    evaluate = sub.add_parser("eval", help="Score the golden set against the live model")
    evaluate.set_defaults(func=cmd_eval)

    args = parser.parse_args()
    args.func(args)


if __name__ == "__main__":
    main()

triage stores the first answer for a ticket key and returns it on the next call. eval always calls the model, so a prompt edit shows up on the next run. It exits 1 when a label misses. That exit code is what you wire to a shell check before you point teammates at the file.

Temperature 0 makes one model file stable. It does not promise these four labels. A 4B model can miss other versus access. Read the miss, tighten the system prompt, or move to a larger tag such as qwen3:8b. Keep the golden set. A README claim is weaker than four lines of output.

Enjoyed this article?

Subscribe & Follow

Get notified of new technical articles on AI/ML, Java, Python, and system architecture.