Skip to main content

Documentation Index

Fetch the complete documentation index at: https://internal.september.wtf/llms.txt

Use this file to discover all available pages before exploring further.

Sometimes you don’t want a paragraph; you want JSON that fits a specific shape so your downstream code can use it directly. This page covers two approaches the Engine supports and when each is the right call.

​
Two approaches

​
1. Provider-native structured outputs

OpenAI and Anthropic both support a “give me JSON that matches this schema” mode at the API level. The Engine surfaces this through a request flag (when running on a provider that supports it) and the provider enforces the schema during decoding. The output is guaranteed to match the schema — no malformed JSON, no missing required fields. This is the right answer when:
  • You have a fixed schema you control.
  • The schema is meaningful to the model (good field names, descriptions).
  • You’re on a provider that supports it for the model you’re using.

​
2. Tool-as-extractor

Define a tool whose only purpose is to return the structured data. The model “calls” the tool with the structured JSON as the input; you read the JSON from the tool_call.input rather than executing anything: 
{
  "name": "submit_summary",
  "description": "Submit the final structured summary.",
  "input_schema": {
    "type": "object",
    "properties": {
      "title": { "type": "string" },
      "key_points": { "type": "array", "items": { "type": "string" } },
      "sentiment": { "enum": ["positive", "neutral", "negative"] }
    },
    "required": ["title", "key_points", "sentiment"]
  }
}
The agent’s system prompt instructs it: “When you’ve finished analyzing, call submit_summary with the result. Do not write the summary as text.” When you see tool_call.input for submit_summary, that’s your structured output. This works on every provider. It also gives you context — the model can write some preamble (which you can ignore or surface as “thinking”) and then commit to the structured output as a “submit.”

​
Pick the right one

Provider-nativeTool-as-extractor
Provider supportOpenAI, Anthropic (varies by model)All
Schema enforcedYes, at decodeAt validation, not decode
Mid-task usageWhole turn is structuredCan interleave with text
Streaming behaviorField-by-field as decodedComes through tool_call
Best forPure extractor tasksMulti-step tasks ending in structure
If you want one structured object as the answer, native is cleaner. If you want the agent to think, search, and finally commit a structure, tool-as-extractor is better.

​
Schema design

Whichever approach you pick, schema quality affects output quality.

​
Use descriptive field names

"name": "user_name"           // ok
"name": "u"                   // bad
"name": "preferred_full_name" // better — tells the model what to fill

​
Add descriptions

The model uses field descriptions like documentation. They guide the fill: 
{
  "type": "string",
  "description": "ISO 4217 currency code, uppercase. Example: USD, EUR."
}

​
Use enums for closed sets

If a field has 3–5 valid values, use enum: 
{
  "type": "string",
  "enum": ["high", "medium", "low"],
  "description": "Severity level."
}
The model picks one of the listed values. Without enum, you’ll see “Medium” vs. “medium” vs. “MEDIUM” inconsistently.

​
Mark required fields

Optional fields that the model thinks are unnecessary will be omitted. Required fields force the model to fill them. Use required to express your contract.

​
Avoid deeply nested structures

Models do well with one or two levels of nesting. Three levels deep hurts accuracy. If your data is naturally hierarchical, consider:
  • A flat structure with parent IDs.
  • Separate calls for nested levels.
  • Two passes: first the outline, then the details.

​
Validation

The Engine does NOT silently accept invalid JSON. If you use provider-native structured outputs, the provider guarantees validity. If you use tool-as-extractor, the Engine validates the input against the schema before emitting the tool_call event. Invalid input becomes a tool_error and feeds back into context — the model can correct and retry. For your application code, treat the JSON as already validated against the schema, but still validate against your business rules. The schema can’t enforce “the timestamp must be in the future” or “the user_id must exist.”

​
Pattern: extractor-with-confidence

Add a confidence field to your schema: 
{
  "title": "...",
  "key_points": [...],
  "sentiment": "positive",
  "confidence": 0.85
}
The model self-reports how confident it is. Use this in your code to route low-confidence outputs to a human review queue instead of acting on them automatically. It’s not perfectly calibrated, but it’s a useful signal in aggregate.

​
Pattern: parsed-then-acted

For tasks where the structure is the input to a subsequent action, run the agent in two turns:
  1. Turn 1: agent calls submit_summary with structured JSON.
  2. Your code reads the JSON, performs the action.
  3. Turn 2: agent gets a tool_result describing what you did, replies with confirmation text to the user.
This separates “thinking and structuring” from “acting” cleanly, and keeps your application in the loop for the destructive part.

​
Pitfalls

  • Schema drift. Updating the schema without updating the agent’s system prompt or tool description leads to garbage output. Treat schemas like APIs — version and migrate.
  • Free-form fields in structured outputs. A string field with no description ends up holding everything from a code snippet to a paragraph of explanation. Constrain with description and length hints.
  • Enums that grow. If you keep adding to an enum, the model gets confused as the list lengthens. At some point, you want a classifier-as-tool instead.
  • Streaming partial JSON. Provider-native structured outputs stream the JSON token-by-token. You can’t parse it until it’s complete unless you use a streaming JSON parser. Tool-as-extractor sends the full input in one event.

​
See also