Jev Tool Selection Tutorial

An agent loop spends most of its cycles answering one question: what happens next? Most of the time that means picking a tool. This tutorial builds a tool selector with Jev, gates it on confidence, and handles the part most demos skip — deciding when not to call a tool at all.

The scenario

An agent is debugging a production issue. It has a fixed set of tools available:

const TOOLS = {
	kubectl: "Inspect Kubernetes cluster state: pods, logs, events, deployments",
	prometheus: "Query metrics and time-series data for graphs and thresholds",
	github: "Read repositories, pull requests, issues, and commit history",
	"web-search": "Find external documentation, changelogs, and known issues",
	calculator: "Perform arithmetic or unit conversions",
} as const;

The user asks: "Why are my pods restarting every 10 minutes?"

The tools are known and the question is known, which makes this a Choice question rather than a generation task.

Step 1: Ask the routing question

type ToolChoice = {
	type: "choice";
	choice: keyof typeof TOOLS;
	confidence: number;
	probabilities: Record<keyof typeof TOOLS, number>;
};

async function selectTool(task: string): Promise<ToolChoice> {
	const response = await fetch("https://api.typesafe.ai/v1/systemone", {
		method: "POST",
		headers: {
			Authorization: `Bearer ${process.env.TYPESAFE_API_KEY}`,
			"Content-Type": "application/json",
		},
		body: JSON.stringify({
			state: task,
			model: "jev-latest",
			questions: {
				tool: {
					type: "choice",
					instructions: "Which tool should be used next",
					criteria: TOOLS,
				},
			},
		}),
	});

	if (!response.ok) {
		throw new Error(`TypeSafe API error: ${response.status}`);
	}

	return (await response.json()).answers.tool;
}

Two details from the official docs are worth internalizing:

criteria descriptions define the boundaries between tools. "Inspect Kubernetes cluster state" versus "Query metrics" is what lets the model separate kubectl from prometheus on a pod-restart question.
The probabilities object matters as much as the winner. A close race between kubectl and prometheus is your signal that the request is ambiguous, not that the model is bad.

Step 2: Gate execution on confidence

Tool calls have side effects, so the threshold should be higher than for read-only classification. TypeSafe's confidence guidance frames it as risk-scaled thresholds: read-only actions can proceed at moderate confidence, destructive ones need high confidence plus confirmation.

const READ_ONLY = new Set(["kubectl", "prometheus", "github", "web-search"]);

async function chooseTool(task: string) {
	const tool = await selectTool(task);

	if (tool.confidence < 0.5) {
		// The model is telling you the task doesn't map cleanly to any tool.
		return { action: "ask-user" as const, probabilities: tool.probabilities };
	}

	const threshold = READ_ONLY.has(tool.choice) ? 0.7 : 0.9;

	if (tool.confidence >= threshold) {
		return { action: "execute" as const, tool: tool.choice };
	}

	return { action: "confirm" as const, tool: tool.choice };
}

Notice what happens at low confidence. Instead of executing the least-bad tool, the loop asks the user. An agent that confidently calls the wrong tool is worse than one that admits ambiguity.

Step 3: Pre-check side effects with Noul

Before executing a tool that can change state, ask a Noul question: is this operation destructive? Noul answers are 0–1 values, and per the docs they do not carry a confidence value — the number itself is the signal.

async function isDestructive(command: string): Promise<number> {
	const response = await fetch("https://api.typesafe.ai/v1/systemone", {
		method: "POST",
		headers: {
			Authorization: `Bearer ${process.env.TYPESAFE_API_KEY}`,
			"Content-Type": "application/json",
		},
		body: JSON.stringify({
			state: command,
			model: "jev-latest",
			questions: {
				destructive: {
					type: "noul",
					instructions: "This command deletes, restarts, or otherwise mutates production state",
				},
			},
		}),
	});

	return (await response.json()).answers.destructive.noul;
}

Because questions evaluate in parallel, you can fold this into the same request as the tool choice. One round trip, two typed answers:

{
	"questions": {
		"tool": {
			"type": "choice",
			"instructions": "Which tool should be used next",
			"criteria": { "...": "..." }
		},
		"destructive": { "type": "noul", "instructions": "This step mutates production state" }
	}
}

Step 4: Compose in code, not in the prompt

The docs are direct about this: ask one well-scoped thing per question and combine the results with your own logic. The loop becomes explicit policy:

const { tool, destructive } = await classify(task);

if (tool.confidence < 0.5) return askUser();
if (destructive.noul > 0.7 && tool.confidence < 0.9) return confirmWithUser(tool.choice);
if (tool.confidence >= 0.7) return execute(tool.choice, task);

return confirmWithUser(tool.choice);

Every branch is readable, testable, and tunable without rewriting a prompt. When priorities change, you change a coefficient rather than the model's instructions.

Step 5: Watch the distribution, not just the pick

Log probabilities with every decision and review the ambiguous ones. A few patterns to expect:

Flat across two tools — the task genuinely spans both. Consider whether a multi-step plan (call kubectl, then prometheus) beats forcing a single choice.
calculator or web-search winning unexpectedly — your criteria descriptions are too broad; tighten them.
High confidence on the wrong tool — usually a state problem. The model judged what you sent, and what you sent was missing context.

Keep the loop fast

Tool selection runs on every step of the loop, so it has to stay cheap. Jev 1.13 bills $0.042 per million input tokens with output free, so a selector that sends a compact state and a criteria list costs a fraction of a cent per decision. See Jev Pricing Explained for the math. And because questions are evaluated in parallel against one state, adding the destructive pre-check to the same request barely changes response time.

Try it

The Jev Agent playground includes a tool-selection scenario. Define your own tool set, paste a task, and watch the probabilities. It is the fastest way to find overlapping criteria before they become production misroutes.