Needle Python Docs

cactus-needle is inference, LoRA fine-tuning and export for Needle 3 in one Python package. Text goes in, a JSON tool call comes back; a byte-level grammar compiled from your schemas constrains every token, so the call is always well-formed. A sub-1 MB engine and the needle3.cact weights are fetched once from Hugging Face and cached; nothing is compiled locally. This page is the reference. The guides on designing tools, confidence, extraction, fine-tuning and devices are the narrative.

Install

pip install cactus-needle                # runtime
pip install "cactus-needle[train]"       # adds JAX and the training deps
pip install "cactus-needle[train,gpu]"   # NVIDIA
pip install "cactus-needle[train,metal]" # Apple Silicon

import needle is lightweight and never imports JAX; only needle finetune and needle build do. The engine downloads on the first Needle(...). Anonymous usage counts (function name, package version, OS, a random install id; never prompts, outputs or data) can be switched off with NEEDLE_TELEMETRY=0 or DO_NOT_TRACK=1, and CI environments are excluded automatically.

API

Tools, three ways

The decorator reads the signature for types, the docstring for the description, a Google-style Args: block for per-argument descriptions; a default makes an argument optional and a Literal becomes a fixed set the model cannot leave:

import needle
from typing import Annotated, Literal

@needle.tool
def set_thermostat(temperature: int, mode: Literal["heat", "cool", "auto"] = "auto"):
    """Set the thermostat.

    Args:
        temperature: target temperature in Celsius
        mode: heating strategy to use
    """
    return {"temperature": temperature, "mode": mode}

@needle.tool(triggers=[r"\b(turn|switch|power|flip)\b.*\b(on|off)\b", r"\btoggle\b"])
def control_device(device: str, action: Literal["on", "off", "toggle"]):
    "Switch or toggle any named smart-home device."
    return {"device": device, "action": action}

needle.Field constrains values, and every constraint is compiled into the decode grammar:

@needle.tool
def send_money(
    amount: Annotated[float, needle.Field(gt=0, le=10000, description="USD, up to 10,000")],
    to:     Annotated[str,   needle.Field(pattern=r"^@[a-z0-9_]+$", description="recipient handle")],
    memo:   Annotated[str,   needle.Field(max_length=80)] = "",
):
    "Send money to a handle."
    return {"sent": amount, "to": to}

Field takes description, enum, const, ge/le/gt/lt, multiple_of, min_length/max_length, pattern, format, min_items/max_items and unique_items. A raw JSON schema is exactly what the engine consumes, and takes the same "triggers" list beside description:

tools = [{
    "name": "set_lights",
    "description": "Turn a room's lights on or off and set brightness",
    "parameters": {
        "type": "object",
        "properties": {
            "room": {"type": "string", "description": "which room to control"},
            "on": {"type": "boolean"},
            "brightness": {"type": "integer", "minimum": 0, "maximum": 100},
        },
        "required": ["room", "on"],
    },
}]
agent = needle.Needle(tools=tools)

OpenAI-style {"type": "function", "function": {...}} wrappers are accepted too. Java, JavaScript and camel-cased names are converted to collision-safe snake-case aliases for the model and restored exactly in the returned calls; descriptions, enum values and defaults are never rewritten.

The loop

import json

agent = needle.Needle(tools=[set_thermostat, control_device])
agent.run("make it 21 and cool the room")["results"]

r = agent.complete("toggle the garage door")
if r["type"] == "call":
    out = control_device(**r["function_calls"][0]["arguments"])
    r = agent.complete(json.dumps(out))

A turn after a call is a tool result only when it parses as JSON; any other text is a new user turn. Later arguments may depend on earlier results (search_for_contact first, then send_instant_message with the returned id). A final "type": "respond" with empty function_calls ends the loop; the answer is the tool results, which run() collects as results. No free text is generated.

The response

{
  "type": "call",
  "success": true,
  "error": null,
  "error_code": null,
  "function_calls": [ { "name": "set_lights", "arguments": { "room": "living room", "on": true, "brightness": 30 } } ],
  "suppressed_calls": [],
  "reasoning": "'living room' -> room; 'dim' -> on true, brightness 30",
  "confidence": 0.94,
  "prefill_tps": 4300.0,
  "decode_tps": 850.0,
  "peak_ram_mb": 88.5
}

• function_calls is a list of {"name", "arguments"}; arguments is grammar-guaranteed to match your schema. Empty is the refusal for off-topic input.
• suppressed_calls holds a call the engine withheld: confidence under 0.1, or a grounding gate. Show it to confirm, or treat the turn as a refusal.
• reasoning is the model's short, unconstrained derivation of each argument from its source span.
• confidence is calibrated for the base model; it is None for tuned weights.
• validation.ungrounded, when present, lists tool.field names whose value the package could not ground, including a date whose year appears nowhere in the conversation or the system facts. run() refuses those calls with {"error": "ungrounded field"} unless strict=False.

The contract

• A request no declared tool can serve is refused with the empty call []. There is no free-text fallback; always handle the empty case.
• Arguments contain only values evidenced by the input. An optional field with no evidence is omitted, not guessed; do not assume a key exists.
• Arguments arrive after the engine's deterministic repair, grounded in the request: split names, completed phone numbers, verbatim quotes, recovered place queries, weekday and March 4th dates against the date: fact, polarity from the request verb, enums moved to the option the request names, routes from from X to Y, by N into a lone number argument, invented required slots refilled from the request or the schema default, ungrounded optional values dropped, and calls the request excludes or negates dropped.
• One toolset per agent. Later turns are bare queries against the same tools; reset() rewinds the conversation. New tools mean a new Needle(...).
• Needle(system=...) prefixes the local date: fact automatically unless the text already carries one; auto_date=False opts out.

System facts

agent = needle.Needle(tools=tools, system="date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%")

Recognised keys: date, locale, device, battery, network, location, user, assistant. Facts, never instructions: the model resolves relative language against them and is not steered by prose placed there. Omitting the turn is safe.

Tool retrieval

Five or fewer tools render directly. Above that, every schema is embedded once at init by a built-in contrastive head, each turn embeds the query, and only the five highest-scoring tools enter the context with the grammar rebuilt over that subset. An unselected tool is unreachable. tool_index_path="tools.idx" persists the embeddings on disk, keyed by a fingerprint over the schemas and the model; a changed schema re-embeds only what changed.

Tuned weights and generations

weights="tuned.cact" loads an archive from needle build. Each tuned agent runs in its own worker process with an independent engine, KV cache and conversation. The engine cannot unload weights: once a tuned archive is bound in a process, constructing a base-model agent there raises rather than silently answering with those weights, so construct base agents first or use separate processes. extract(..., weights=None) inherits a single active tuned archive; with both a Needle 2 and a Needle 3 tune loaded, pass weights= explicitly. Needle 2 and Needle 3 archives carry different format tags and the package never feeds one generation's weights to the other engine.

Offline devices

Each generation's engine is cached under ~/.cache/cactus-needle/v2/ or v3/, and Needle 3 caches needle3.cact beside it. Inference never touches the network, so an air-gapped device only needs the files in place:

1. needle fetch downloads the engine for the current machine into the cache and prints the path; --platform-tag manylinux2014_aarch64 fetches another wheel platform (macosx_11_0_arm64, manylinux2014_x86_64, musllinux_1_2_aarch64, win_amd64, win_arm64); needle download needle3 fetches the archive; --out <dir> places either elsewhere.
2. Copy the files to the same cache path on the device, or drop them inside the installed needle/ package directory, which wins over the cache.
3. NEEDLE3_LIB_PATH=/path/to/libneedle.so overrides the Needle 3 engine (NEEDLE2_LIB_PATH for Needle 2; the legacy NEEDLE_LIB_PATH is a Needle 2 alias and is ignored for Needle 3 on purpose).

The package itself installs offline the standard way: pip download cactus-needle on a connected machine, then pip install --no-index --find-links <dir> cactus-needle on the device. Set HF_HUB_OFFLINE=1 on a device that must never attempt the network, so a missing engine fails fast.

Environments

needle.environments ships six ready-made tool surfaces: smart_home, media_player, productivity, wearable, kitchen_appliance and data_capture. Each module exposes TOOLS, SYSTEM, a lazily constructed agent, a frozen TEST_CASES suite of 32 cases and run_tests(min_confidence=0.0); python -m needle.environments.smart_home runs one suite and exits 0 on pass.

from needle.environments import smart_home

smart_home.agent.complete("dim the study lights to 30 percent")
smart_home.run_tests(min_confidence=0.4)

Adapt one by swapping the Literal values for your own and keeping the shapes: closed sets as enums, bounded numbers, verbatim copy for free text, five tools or fewer.

CLI

Mistakes to avoid

• Expecting free-text answers. Unsupported input returns an empty call; handle it.
• Reading arguments keys that were not evidenced in the input. Optional fields may be absent.
• Creating one Needle per turn. Reuse the instance so context carries; a new toolset is a new instance.
• Passing a .safetensors or .pkl to weights=. It expects a .cact from needle build.
• Loading an archive exported by an older package. The .cact format is tied to the engine version; rebuild it.
• Routing tuned weights on confidence. It is None there; use your own validation.