# Run and evaluate a model locally
URL: /llms/run-locally

Load a downloaded .cact file with the open-source runtime, run it on CPU on any machine, and score it on your test file the way the platform does.

***

title: Run and evaluate a model locally
description: Load a downloaded .cact file with the open-source runtime, run it on CPU on any machine, and score it on your test file the way the platform does.
---------------------------------------------------------------------------------------------------------------------------------------------------------------

Every model the platform produces, and the base model, is a `.cact` file that the open-source `cactus-needle` package runs on CPU: macOS and Linux on Arm and x86, Windows on x64 and Arm64, Python 3.9 or newer, no GPU. The native engine is a few hundred kilobytes and is fetched on first use; a full-size model is 35 MB and answers in about 50 ms on a laptop.

## Install and run

```sh
pip install "cactus-needle>=3.0.2"
export NEEDLE_TELEMETRY=0
```

```python
import needle

tools = [  # flat definitions, no {"type":"function","function":…} wrapper
    {"name": "set_lights", "description": "Turn the lights in one room on or off.",
     "parameters": {"type": "object", "properties": {"room": {"type": "string"},
                    "state": {"type": "string", "enum": ["on", "off"]}}, "required": ["room", "state"]}},
]
agent = needle.Needle(tools=tools, weights="smart-home-8L.cact", auto_date=False)
reply = agent.complete("turn the kitchen lights off")
print(reply["function_calls"])   # [{"name": "set_lights", "arguments": {"room": "kitchen", "state": "off"}}]
```

Tools bind when you construct the agent; each user turn is one `complete()` call; a tool result goes back in as `complete(json.dumps(result))`. An empty `function_calls` list means the model chose not to call anything. Platform fine-tunes train the confidence head with the model, so `confidence` is calibrated on your tools; only locally trained adapters report it as `None`.

## Score a test file

The platform's own test file is chat-format `.jsonl` with the tools wrapped in OpenAI form, which is not what the runtime takes directly. [needle\_eval.py](https://cactuscompute.com/needle-fine-tuning/needle_eval.py) does the translation and scores exactly the way the platform does: an example is correct when every assistant turn makes the expected calls with the same arguments, no-call examples count, and lines with no `tools` are excluded.

```sh
curl -O https://cactuscompute.com/needle-fine-tuning/needle_eval.py
python needle_eval.py smart-home-8L.cact smart-home-test.jsonl
```

```
correct 91 / total 100, excluded 0  (91.0%)
line 17: expected [set_lights {"room":"hall","state":"off"}] got []
…
```

Run it against the base model too, `needle3.cact` or a smaller `needle3-8L.cact` from `/v1/models/needle-3-depth-8/content`, and you have the before and after on the same data. The failing lines tell you what to add to the training set before the next run.

## Things that bite

* Use `cactus-needle` 3.0.2 or newer for platform output. Older 2.x packages refuse the file with a format-tag error.
* Pass `auto_date=False`, or a fixed `system="date: 2026-09-19"`, when scoring. Otherwise today's date is injected and date-bearing examples drift from day to day.
* Keep tool and parameter names in snake\_case, in the schemas and the data.
* `needle finetune` reads platform files too, from `cactus-needle` 3.0.4 on; older packages read only their own `query`/`tools`/`answers` schema and skip chat-format lines.