# Cactus Compute

> Cactus builds foundation models for on-device intelligence on any hardware: phones, wearables, automotive, laptops and microcontrollers. Deploy our models (Needle 3 for tool calling, Whistle for speech) or bring your own. Our hybrid models report a confidence score, so a request falls back to the cloud only when needed. This file is the index for agents: the Needle fine-tuning platform comes first, then the models themselves, then a pointer to the on-device SDK docs.

## What Cactus does

- Small models on everyday hardware: our smallest models run CPU-only on budget phones, wearables and Raspberry Pis, offline or hybrid, with no GPU, no server and no data leaving the device. Needle 3 and Whistle are open and in production, including in the Pebble app. See https://cactuscompute.com/needle and https://cactuscompute.com/blog/whistle.
- Your models, made smaller: we compress third-party foundation models as well as our own so they fit the memory of the device you ship. See https://cactuscompute.com/research.
- Hybrid routing: our models know when they are likely wrong and hand those requests to the cloud; everything else stays on the device. See https://cactuscompute.com/hybrid.

## Instructions for agents

You are probably here because a human asked you to make Needle handle their product's tools. Read this section, then the platform pages below, before calling anything.

What you can do with no key: read these docs, GET https://cactuscompute.com/v1/plans for hosted fine-tuning prices, and download the base model from https://cactuscompute.com/v1/models/needle-3/content to run it locally (see "Run and evaluate a model locally"). Do that first: if the base model already scores well enough on your human's test cases, there is nothing to pay for.

What needs your human: purchasing Starter or subscribing to Pro, and creating an API key. Explain which plan fits their use case and its limits, then link them to https://cactuscompute.com/dashboard/billing to pay themselves. Afterwards they create a key at https://cactuscompute.com/dashboard/api-keys and put it in the NEEDLE_API_KEY environment variable of the shell you run in. Never print the key, never paste it into a chat, and never write it into a file that is committed.

# Needle hosted fine-tuning pricing

These plans cover hosted fine-tuning in the Cactus console (https://cactuscompute.com/dashboard): data generation, training on our GPUs and evaluation. Downloading the base model and running any model locally is not billed.

Fine-tune locally for free, or use the platform for hosted GPUs, full-model training, a trained confidence head and 2-bit exports. Run your finished models on your own devices.

Free: $0 for local LoRA fine-tuning with the open-source package. Bring your own hardware and data; no hosted compute, generation or platform storage is included.

Start locally: https://github.com/cactus-compute/needle#customisation

Starter: $19 one-time for 3 fine-tunes, 3,000 generated examples, and 10 GB storage. Use within 30 days. Once per account before subscribing; no automatic renewal.

Pro: $99 per month for 10 fine-tunes, 10,000 generated examples, and 10 GB storage. Run and generation allowances reset each billing period. Cancel anytime.

Custom: contact us for higher limits.

Starter uses API plan ID `starter`; Pro uses API plan ID `standard`. Existing purchases and checkout eligibility are unchanged.

## Local vs. platform fine-tuning

What trains — Free: LoRA adapters on attention projections. Base frozen; adapters merged at export. Platform (Starter, Pro, Custom): The full model, at every supported depth from 2 layers up.

Original capabilities — Free: Your training data only. Platform (Starter, Pro, Custom): Your data augmented with Needle’s original dataset to help retain base capabilities.

Confidence — Free: Confidence head untouched; confidence is not available on the fine-tuned model. Platform (Starter, Pro, Custom): Confidence head fine-tuned with the model and calibrated on your tools.

Export precision — Free: 4-bit. Platform (Starter, Pro, Custom): 2-bit, with the same post-training process as the shipped model.

Training data — Free: Bring your JSONL in query / answers or chat format. Platform (Starter, Pro, Custom): Upload your data or generate synthetic examples from your tool definitions, within your plan’s allowance.

Evaluation — Free: Validation loss. Platform (Starter, Pro, Custom): Validation and test accuracy for every depth, so you can compare model sizes.

Compute & storage — Free: Your machine and disk. JAX on CPU, CUDA or Metal. Platform (Starter, Pro, Custom): Cactus GPUs and platform storage for datasets and model exports.

Run from — Free: The CLI: needle finetune. Platform (Starter, Pro, Custom): The dashboard, CLI, Python, or a coding agent with your API key.

Choose Free for local LoRA training on your own hardware. Choose Starter to test the full hosted pipeline before committing; choose Pro for ongoing development. If the free base model already meets your requirements, you do not need to fine-tune.

The human purchases at https://cactuscompute.com/dashboard/billing and creates an API key at https://cactuscompute.com/dashboard/api-keys; agents do not purchase or manage subscriptions.

Starter is a one-time purchase for one account before subscribing. Access and unused allowances expire after 30 days; there is no automatic renewal.

Pro is a monthly subscription. Upgrading starts fresh Pro allowances and preserves files and models. Unused Starter allowances do not carry over, its payment is not credited, and it cannot be purchased again.

Starter and Pro include the same model sizes, full-model training on hosted GPUs, 2-bit exports, confidence-head training, original-dataset reinforcement, synthetic example generation, evaluation, and playground access. Training and generation spend separate allowances; generated datasets can be reused across runs.

Storage includes uploaded files, generated datasets, model exports, and reserved output space. Jobs reserve up to 512 MiB for output. Failed jobs return their allowance; cancelled jobs do not.

After access ends, files and models remain available to download for another 30 days before they may be deleted. An active paid plan preserves them. Downloaded models remain usable locally.

Generation creates synthetic examples from tool definitions and optional product context. Original-dataset augmentation happens during training; it does not rewrite or clean your uploaded files. Review generated data and evaluate on independent examples from your use case.

There is no hosted inference endpoint. Run downloaded models locally with the open-source runtime. Accuracy improvements depend on your data and task.

Hosted fine-tuning pricing is also available at https://cactuscompute.com/llms/pricing (markdown).

The flow, once you have a key (every endpoint is under https://cactuscompute.com/v1; full reference at https://cactuscompute.com/llms/api, local run and scoring at https://cactuscompute.com/llms/run-locally, spec at https://cactuscompute.com/v1/openapi.json; this whole file is also served as https://cactuscompute.com/llms-full.txt):
1. GET /billing: your plan, limits and what this period has already used.
2. POST /tool_schemas with a product description, or bring your own OpenAI-form tool definitions. snake_case names.
3. Either upload chat-format .jsonl files (POST /files, PUT the bytes, POST /files/{id}/complete) or POST /generations to have training, validation and test files written for you. Poll GET /generations/{id} until succeeded; the response lists the three files with their group.
4. POST /fine_tuning/jobs with the three file ids and a max_depth. Poll GET /fine_tuning/jobs/{id} until succeeded; evaluations holds validation and test accuracy for every size.
5. GET /models/{fine_tuned_model} for the sizes, GET /models/{variant}/content to download a .cact (follow the 307 within 60 seconds).
6. Evaluate locally with needle_eval.py on the test file, compare with the base model, report both numbers to your human, and put the failing cases into the next training set.

Rules:
- A fine-tune spends 1 of the active plan's runs and a generation spends its example count, at submission. A run that fails is refunded; a cancelled one is not. Check GET /billing, tell your human what a submission will spend, and get their yes before each POST to /generations or /fine_tuning/jobs.
- One job runs at a time per account. Poll every 5 to 10 seconds, no faster. Honour Retry-After on 429.
- Do not resubmit a run to "try again" without changing the data. Same data gives the same model.
- Error bodies carry a stable code and, for 401 and 402, a url: the console page your human opens to clear it.
- Run and evaluate models locally with the open-source runtime; there is no hosted inference endpoint.
- A skill with this flow is installable with: npx skills add https://cactuscompute.com

## The models you can run on-device

Two models, one C++ engine, one .cact file each, no dependencies and no GPU. The engine loads either or both at once.

- Needle 3: 121M parameters, tool calling and structured JSON extraction. Weights, prebuilt engine folders for seventeen targets and needle.h are at https://huggingface.co/Cactus-Compute/needle3, and the base model also downloads from https://cactuscompute.com/v1/models/needle-3/content with no key. Run and score it: https://cactuscompute.com/llms/run-locally. Write schemas it answers well: https://cactuscompute.com/blog/designing-tools-for-needle. Pull typed records out of text: https://cactuscompute.com/blog/structured-extraction-with-needle. Decide whether to execute a call: https://cactuscompute.com/blog/needle-confidence.
- Whistle: 16.9 MB speech to text in English, German, French, Spanish, Italian, Dutch and Polish, with word timestamps, live streaming transcription and speech embeddings, on the same engine as Needle. Weights are at https://huggingface.co/Cactus-Compute/whistle. What it is: https://cactuscompute.com/blog/whistle. How to get the most accuracy out of it: https://cactuscompute.com/blog/whistle-best-practices.
- Engine source for both: https://github.com/cactus-compute/needle. Python package: pip install cactus-needle.

For speech and tool calling together, first transcribe the clip with needle.transcribe(audio, keywords=[...]). Then pass the text to Needle(tools=...).complete(text). Use the values, names and descriptions from your tools as keywords. Write them the way people say them, so living_room becomes living room. More at https://cactuscompute.com/blog/whistle-best-practices.

## Needle fine-tuning platform


# API reference
URL: /llms/api

Every endpoint under /v1, in the order you use them, with curl for each. Machine-readable spec at /v1/openapi.json.

***

title: API reference
description: Every endpoint under /v1, in the order you use them, with curl for each. Machine-readable spec at /v1/openapi.json.
--------------------------------------------------------------------------------------------------------------------------------

Base URL `https://cactuscompute.com/v1`. Every response is JSON. The OpenAPI document is at [`/v1/openapi.json`](https://cactuscompute.com/v1/openapi.json).

## Authentication

Create a key in the console under [API Keys](https://cactuscompute.com/dashboard/api-keys) and send it as `Authorization: Bearer needle_ft_…`. A key reads and writes its owner's files, jobs and models and reads plan and usage. Two things need a website login instead of a key: creating keys, and changing billing. If you are an agent, ask your human to do those two and to put the key in the `NEEDLE_API_KEY` environment variable of the shell you run in. Never print the key.

```sh
export NEEDLE_API_KEY=needle_ft_...
API=https://cactuscompute.com/v1
auth="Authorization: Bearer $NEEDLE_API_KEY"
```

## Errors

```json
{"error":{"message":"…","type":"platform_error","code":"quota_exceeded","param":"max_depth","url":"https://cactuscompute.com/dashboard/usage"}}
```

`code` is stable and is what to branch on. `param` names the request field at fault. `url`, when present, is the console page that clears the error: hand it to your human.

| Status | Code                    | Meaning, and what to do                                                                                        |
| ------ | ----------------------- | -------------------------------------------------------------------------------------------------------------- |
| 401    | `unauthorized`          | Missing, revoked or expired key. Ask your human for a new one at the `url`.                                    |
| 403    | `session_required`      | This call needs a website login, not a key.                                                                    |
| 402    | `subscription_required` | An active paid plan is required. Your human purchases Starter or subscribes to Pro at the `url`.               |
| 402    | `quota_exceeded`        | The active plan’s allowance is used up. Starter does not reset: upgrade to Pro. Pro resets at its next period. |
| 409    | `insufficient_storage`  | Delete files or models you no longer need.                                                                     |
| 409    | `job_active`            | One job runs at a time. Poll the running one.                                                                  |
| 409    | `file_in_use`           | The file belongs to a running job. Wait for it.                                                                |
| 400    | `invalid_request`       | Read `param` and `message`, fix the field.                                                                     |
| 429    | `rate_limited`          | Wait the seconds in the `Retry-After` header.                                                                  |
| 429    | `queue_limit_reached`   | Ten jobs already waiting. Wait for one to start or cancel one.                                                 |

A job can also fail after it was accepted. Then `status` is `failed` and `error.code` is one of `invalid_input` (a line in your data is not in an accepted format), `execution_failed`, `deadline_exceeded`, `subscription_required`, `insufficient_storage` or `output_incomplete`.

## 1. Plans and your usage

`GET /plans` needs no key. It returns `price_usd`, `billing_type` (`free`, `one_time`, `monthly`, or `custom`), `duration_days`, `once_per_account`, and limits. Free is a local workflow, not a hosted entitlement; follow its `get_started_url`. Starter uses API plan ID `starter`; Pro uses API plan ID `standard`. Each entry includes `execution_mode` and `features`. The legacy `price_usd_per_month` is null for Starter. Read [pricing and included value](/llms/pricing) before recommending a purchase; the human pays at [Billing](https://cactuscompute.com/dashboard/billing). `GET /billing` returns your plan, its limits and what this period has used. Check it before you submit anything that spends allowance. `starter.eligible` reports purchase eligibility. For Starter, `period_end` is the expiry, not a renewal; `starter.paid_at`, `expires_at`, and `upgraded_at` describe its lifecycle.

```sh
curl $API/plans
curl -H "$auth" $API/billing
```

```json
{"plan":"standard","limits":{"jobs":10,"examples":10000,"storage_bytes":10000000000},"usage":{"jobs":2,"examples":1000,"storage_bytes":48213},"period_end":1761955200}
```

## 2. Draft tools (optional)

`POST /tool_schemas` with a product description of 10 to 1,000 characters returns three to five OpenAI-form tool definitions. Fifteen drafts per day. Read them over and edit before you use them.

```sh
curl -H "$auth" -H "Content-Type: application/json" $API/tool_schemas \
  -d '{"description":"Voice assistant for a smart home app. Users speak short commands to control lights, thermostat and door locks."}'
```

## 3. Upload files (if you have your own)

Three calls: reserve, PUT the bytes, complete. `bytes` must be the exact size.

```sh
size=$(wc -c < train.jsonl)
reservation=$(curl -s -H "$auth" -H "Content-Type: application/json" $API/files \
  -d "{\"name\":\"train.jsonl\",\"bytes\":$size}")
upload_id=$(echo "$reservation" | jq -r .id)
curl -X PUT -H "Content-Type: application/octet-stream" --data-binary @train.jsonl "$(echo "$reservation" | jq -r .url)"
curl -X POST -H "$auth" $API/files/$upload_id/complete
```

The last call returns the file record; its `id` starts with `file-`. Repeat for the validation and test files. Files stay until you delete them (30 days after a plan ends), and `GET /files` lists them.

## 4. Generate files (if you do not)

`POST /generations` with your tools, a count of 100 to 10,000 examples, and ideally a product description and a few example messages. The count is spent at submission.

```sh
curl -H "$auth" -H "Content-Type: application/json" $API/generations -d @- <<'EOF'
{"tools":[{"type":"function","function":{"name":"set_lights","parameters":{"type":"object","properties":{"room":{"type":"string"},"state":{"type":"string","enum":["on","off"]}},"required":["room","state"]}}}],
 "examples":1000,
 "description":"Voice assistant for a smart home app; users speak short commands to control devices.",
 "messages":["turn off the kitchen lights","is the front door locked?"],
 "suffix":"smart-home"}
EOF
```

Poll `GET /generations/{id}` every few seconds until `status` is `succeeded`. The response then lists the three files it published, each with a `group` of `train`, `validation` or `test`:

```json
{"id":"ftjob-…","object":"generation","status":"succeeded","files":[
  {"id":"file-…","filename":"smart-home-training.jsonl","group":"train","ordinal":0},
  {"id":"file-…","filename":"smart-home-validation.jsonl","group":"validation","ordinal":0},
  {"id":"file-…","filename":"smart-home-test.jsonl","group":"test","ordinal":0}]}
```

Download any of them with `GET /files/{id}/content`, which answers 307 to a link that lasts 60 seconds.

## 5. Fine-tune

`POST /fine_tuning/jobs` spends one run. `max_depth` is the largest size to train, from 2 up to the base model's depth; every smaller size is trained and scored too. Read the base depth from `GET /models/needle-3` rather than assuming it.

```sh
curl -H "$auth" -H "Content-Type: application/json" $API/fine_tuning/jobs -d @- <<'EOF'
{"model":"needle-3",
 "training_files":["file-TRAIN"],
 "validation_files":["file-VALIDATION"],
 "test_files":["file-TEST"],
 "max_depth":20,
 "suffix":"smart-home-v1"}
EOF
```

Poll `GET /fine_tuning/jobs/{id}`. `status` goes `queued`, `running`, then `succeeded`, `failed` or `cancelled`. While running, `completed_steps` counts to 5. On success, `evaluations` holds one entry per depth with validation and test scores, and `fine_tuned_model` is the id of the full-size model:

```json
{"status":"succeeded","max_depth":8,"fine_tuned_model":"model-…",
 "evaluations":[{"depth":8,"validation":{"correct":94,"total":100,"excluded":0},"test":{"correct":91,"total":100,"excluded":0}},
                {"depth":7,"validation":{"correct":93,"total":100,"excluded":0},"test":{"correct":90,"total":100,"excluded":0}}]}
```

Cancel with `POST /fine_tuning/jobs/{id}/cancel`. A run that fails returns its allowance; a cancelled one does not.

## 6. Download

`GET /models/{fine_tuned_model}` lists every size as `variants`. `GET /models/{variant_id}/content` answers 307 to a download link that lasts 60 seconds, so follow the redirect straight away:

```sh
curl -L -H "$auth" -o smart-home-8L.cact $API/models/model-…/content
```

The base model and its smaller sizes are there too: `GET /models/needle-3` and `GET /models/needle-3-depth-8/content`. Then run it locally: [https://cactuscompute.com/llms/run-locally](https://cactuscompute.com/llms/run-locally).

## Data format

Each line of a `.jsonl` file is one example: a JSON object with `messages` and `tools`, the same shape as an OpenAI chat request. Tool calls sit on the assistant turn as `tool_calls`, with `arguments` as a JSON string. Give every assistant turn a short `reasoning_content` line deriving each argument from the request, or saying why no tool fits: Needle writes this line before every answer, so a model trained without it scores lower. An example where the right answer is to make no call has an assistant turn with empty `content` and `"tool_calls": []`. Those count: staying silent where your data stays silent is a correct answer, and a model trained without them learns to call a tool for every message. Files written for local fine-tuning with `needle finetune`, one `query`, `tools` and `answers` object per line, upload as they are.

```json
{"messages":[{"role":"user","content":"turn the kitchen lights off"},{"role":"assistant","content":"","reasoning_content":"'kitchen' -> room; 'off' -> state off","tool_calls":[{"id":"call_1","type":"function","function":{"name":"set_lights","arguments":"{\"room\":\"kitchen\",\"state\":\"off\"}"}}]}],"tools":[{"type":"function","function":{"name":"set_lights","description":"Turn the lights in one room on or off.","parameters":{"type":"object","properties":{"room":{"type":"string"},"state":{"type":"string","enum":["on","off"]}},"required":["room","state"]}}}]}
{"messages":[{"role":"user","content":"what's the weather like?"},{"role":"assistant","content":"","reasoning_content":"set_lights only switches lights; nothing reports weather","tool_calls":[]}],"tools":[{"type":"function","function":{"name":"set_lights","description":"Turn the lights in one room on or off.","parameters":{"type":"object","properties":{"room":{"type":"string"},"state":{"type":"string","enum":["on","off"]}},"required":["room","state"]}}}]}
```

Name tools and parameters in snake\_case, in the schemas and in the examples. Needle rewrites names to snake\_case before the model sees them, so a model trained on camelCase learns names it is never shown at inference.

A fine-tune needs three files: training, validation and test. Aim for at least 1,000 training examples and 100 each for validation and test. Less still runs, but the scores get noisy. Nothing is cleaned, deduplicated or repaired: lines reach the trainer as sent, and a line in neither format fails the job.

## Scoring

An example is correct when every assistant turn makes exactly the calls the example expects, with the same arguments. Key order and spacing do not matter. Examples with no tools defined are excluded and counted separately. Scores use greedy inference with up to 512 response tokens per turn, which the local runtime reproduces.

## Limits

Use `GET /plans` for current prices and limits; [pricing](/llms/pricing) explains the one-time Starter package and monthly Pro subscription. Both use the same job pipeline and storage accounting. A fine-tune counts as one run at submission and a generation counts its requested examples at submission; a job that fails returns its allowance, a cancelled one does not. Request limits, per account across every key and browser session: 120 reads, 30 writes, 6 uploads, 6 job submissions and 10 cancellations per minute, 15 tool drafts per day, one running job at a time, up to 10 waiting per kind.

## Everything else

* `GET /files`, `GET /generations`, `GET /fine_tuning/jobs`, `GET /models` list with `limit` (1 to 100) and `after` (the last id of the previous page); files and models also take `search`.
* `DELETE` on a file, a model, or a finished job removes it and frees storage.
* `GET /api_keys` lists your keys without secrets; `DELETE /api_keys/{id}` revokes one, including the one you are using.
* `POST /billing/checkout`, `/billing/plan` and `/billing/portal` need a website login and answer 403 to a key.


# Run and evaluate a model locally
URL: /llms/run-locally

Load a downloaded .cact file with the open-source runtime, run it on CPU on any machine, and score it on your test file the way the platform does.

***

title: Run and evaluate a model locally
description: Load a downloaded .cact file with the open-source runtime, run it on CPU on any machine, and score it on your test file the way the platform does.
---------------------------------------------------------------------------------------------------------------------------------------------------------------

Every model the platform produces, and the base model, is a `.cact` file that the open-source `cactus-needle` package runs on CPU: macOS and Linux on Arm and x86, Windows on x64 and Arm64, Python 3.9 or newer, no GPU. The native engine is a few hundred kilobytes and is fetched on first use; a full-size model is 35 MB and answers in about 50 ms on a laptop.

## Install and run

```sh
pip install "cactus-needle>=3.0.2"
export NEEDLE_TELEMETRY=0
```

```python
import needle

tools = [  # flat definitions, no {"type":"function","function":…} wrapper
    {"name": "set_lights", "description": "Turn the lights in one room on or off.",
     "parameters": {"type": "object", "properties": {"room": {"type": "string"},
                    "state": {"type": "string", "enum": ["on", "off"]}}, "required": ["room", "state"]}},
]
agent = needle.Needle(tools=tools, weights="smart-home-8L.cact", auto_date=False)
reply = agent.complete("turn the kitchen lights off")
print(reply["function_calls"])   # [{"name": "set_lights", "arguments": {"room": "kitchen", "state": "off"}}]
```

Tools bind when you construct the agent; each user turn is one `complete()` call; a tool result goes back in as `complete(json.dumps(result))`. An empty `function_calls` list means the model chose not to call anything. Platform fine-tunes train the confidence head with the model, so `confidence` is calibrated on your tools; only locally trained adapters report it as `None`.

## Score a test file

The platform's own test file is chat-format `.jsonl` with the tools wrapped in OpenAI form, which is not what the runtime takes directly. [needle\_eval.py](https://cactuscompute.com/needle-fine-tuning/needle_eval.py) does the translation and scores exactly the way the platform does: an example is correct when every assistant turn makes the expected calls with the same arguments, no-call examples count, and lines with no `tools` are excluded.

```sh
curl -O https://cactuscompute.com/needle-fine-tuning/needle_eval.py
python needle_eval.py smart-home-8L.cact smart-home-test.jsonl
```

```
correct 91 / total 100, excluded 0  (91.0%)
line 17: expected [set_lights {"room":"hall","state":"off"}] got []
…
```

Run it against the base model too, `needle3.cact` or a smaller `needle3-8L.cact` from `/v1/models/needle-3-depth-8/content`, and you have the before and after on the same data. The failing lines tell you what to add to the training set before the next run.

## Things that bite

* Use `cactus-needle` 3.0.2 or newer for platform output. Older 2.x packages refuse the file with a format-tag error.
* Pass `auto_date=False`, or a fixed `system="date: 2026-09-19"`, when scoring. Otherwise today's date is injected and date-bearing examples drift from day to day.
* Keep tool and parameter names in snake\_case, in the schemas and the data.
* `needle finetune` reads platform files too, from `cactus-needle` 3.0.4 on; older packages read only their own `query`/`tools`/`answers` schema and skip chat-format lines.


## Cactus SDK, the on-device engine

The SDK runs Needle and other models inside apps. Its documentation lives in the README at https://github.com/cactus-compute/cactus#readme.