Here's the trap I see most teams fall into: they treat the LLM node like a creative writing partner that happens to return data. They ask GPT-4o or Claude to "analyze this and give me the result," then they wire a Code node downstream that hunts for patterns, splits strings, or prays that the model formatted the response the same way it did five minutes ago. In the demo, it works. In production, at request 847, the model decides to wrap the JSON in a markdown code block, or it translates a field name to Spanish because the input was bilingual, or it emits "true" inside a sentence that should have been a boolean.
The workflow doesn't throw an error; it silently routes a customer complaint to the auto-responder path because a regex matched the word "true" in the wrong place.
I call this post-hoc parsing, and it is the single biggest source of AI workflow breakage I get called in to fix. The alternative is to stop treating structure as an afterthought and start treating it as the contract between the model and the rest of your pipeline.
The schema is the API between the AI and the rest of your workflow. Not a guideline. Not a suggestion. Treat it with the same rigour as a third-party REST contract.
When I redesign a brittle AI workflow, the first thing I do is replace the free-text prompt with a rigid JSON schema that defines every field, type, and enum value the model is allowed to return. The system prompt doesn't ask; it instructs. The model receives the schema inline, and the parser — either the platform's native structured output mode or a dedicated parser node — enforces compliance before any downstream logic sees the data.
This fundamentally changes how you build. Instead of:
[LLM generates text] → [Code node regex extracts] → [IF node guesses intent]
You get:
[LLM returns JSON] → [Schema validator confirms shape] → [IF node reads exact field]
Downstream nodes should not know that an LLM was involved. They should consume the output the same way they consume a Stripe webhook or a REST API response: predictable keys, typed values, and enumerated options that map directly to routing logic. If a downstream IF node is evaluating $json.urgency === "critical", then "urgency" must be defined in the schema as an enum with exactly those four values — "low", "medium", "high", "critical" — and nothing else.
The moment you adopt this pattern, your error handling simplifies. You stop debugging string offsets and start debugging schema violations, which are explicit and loggable. You also stop over-paying for cognition that isn't happening.
Reasoning and formatting are different tasks. We don't hire senior architects to do data entry, yet we routinely fire the same large language model at both jobs. The big model should do the thinking, and a small, cheap model should do the parsing.
In my experience, the majority of production AI workflows that bleed money are sending everything to Claude Sonnet or GPT-4o because those models "understand context better." They do. But once the big model has done the hard work — drafted the support reply, analyzed the sentiment, synthesized the research brief — forcing it to also worry about whether a field is camelCase or whether a boolean is unquoted is a waste of tokens and latency.
Instead, I use a two-stage architecture. Stage one is the reasoning model. It receives the full context, the conversation history, the RAG-retrieved documents, whatever it needs to form a decision or draft a response. It returns its conclusion in plain text or lightly structured form. Stage two is a small model — gpt-4o-mini, GPT-4.1 Mini, whatever is cheapest and fastest — that receives the reasoning output along with the strict schema, and its only job is to map that reasoning into the exact JSON shape the workflow requires.
This pattern appears in the most reliable multi-model stacks I run. The big model handles nuance, empathy, and open-ended research. The parser model handles enums, nesting, and type compliance. The cost difference is dramatic: a parser model runs at roughly $0.40 per million input tokens, while a reasoning model can run $3 to $15 per million tokens. If your parser catches even a few formatting errors per thousand requests, it pays for itself. If it lets you downgrade the main model because the main model no longer needs to be paranoid about output format, it can cut total AI spend by 60 to 80 percent.
The parse-vs-think split also makes failure modes obvious. When the reasoning model hallucinates, the parser doesn't fix the content — but it does contain the damage. The parser can only emit values from the allowed set. The bad idea gets boxed into a predictable shape.
A schema is only as good as its field definitions. I follow two rules that keep workflows maintainable: use enums for any value that drives a branch, and use nested objects for any context that travels more than one step downstream.
Enums turn ambiguous model intent into deterministic routing. If you have an IF node that checks whether a support ticket needs a human, do not let the model return a boolean as free text. Define the field as:
{
"escalation": {
"required": true,
"reason": ["sales_opportunity", "complaint", "technical", "payment_issue", null]
}
}
Now your IF node checks $json.escalation.required === true, and your routing logic checks $json.escalation.reason. There is no string-matching, no "contains" logic, no surprise nulls. The model is physically incapable of returning "escalate_to_human" when the enum only contains the four allowed values plus null.
Nested objects prevent namespace pollution and make schemas self-documenting. A flat schema with twelve top-level keys is a mess to maintain. A nested schema groups intent:
{
"routing": {
"auto_respond": true,
"escalate": false,
"queue": "billing"
},
"content": {
"reply": "...",
"tone": "empathetic",
"citations": ["https://..."]
},
"meta": {
"confidence": "high",
"model": "claude-sonnet-4-20250514",
"parser_version": "gpt-4o-mini-v2"
}
}
Downstream nodes pick the branch they care about. The Slack notifier reads content.reply. The CRM updater reads routing.queue. The audit logger reads meta. If you need to add a new field six months later, you add it inside the relevant nest without risking collision with every existing reference.
When models struggle with consistency — dates, time normalisation, or nested conditional fields — I inject two or three few-shot examples directly into the system prompt. Not examples of reasoning; examples of exact output format. Show the model what "15:00" looks like versus "3pm", or how a multi-day event populates start_date and end_date versus a single-day event. This typically drops format errors from roughly 15 percent to under 3 percent, which means fewer failed executions and fewer 3 AM pages.
Small, cheap, and instruction-tuned is usually the right profile for a parser. You do not need a frontier model to reformat text into JSON. You need a model that follows directions exactly and does not get creative.
My default is gpt-4o-mini or GPT-4.1 Mini for the parser role. These models are fast, their token costs are negligible at parsing scale, and they are surprisingly stubborn about schema compliance when you tell them their entire purpose is to map reasoning into a fixed shape. If the platform I'm using supports native structured output or JSON mode — where the API itself constrains generation to valid schema — I enable it, but I still keep the parser node as a second layer. Native JSON mode catches syntax errors. A parser node catches semantic drift.
If the task is pure classification — routing an email to one of four departments — a small model with native JSON mode is often all you need. The model both thinks and parses in one call because the reasoning is trivial. But if the task involves reading ten messages of thread history, researching documentation, and drafting a nuanced reply, the parse-vs-think split is non-negotiable.
Adding a second model call adds network time. In my experience, the extra 300–800 ms is worth the elimination of parse-related failures. If you are latency-sensitive, run the parser in parallel with other non-dependent steps. Do not optimise for milliseconds by optimising for broken JSON at scale.
Even with schema-driven parsing, things go wrong. The API throws a 429. The model returns a truncated response because the max_tokens limit was too tight. The reasoning output is so garbled that even the parser cannot map it cleanly. You need fallback layers, not hope.
I use three layers of defense:
const delay = Math.pow(2, attempt) * 1000 + Math.random() * 1000;
await new Promise(r => setTimeout(r, delay));
I also enforce a lightweight validation layer after parsing. A Code node that checks required fields, validates enum values against the allowed set, and confirms booleans are actual booleans — not the strings "true" and "false". This takes ten lines of code and runs in milliseconds. It has caught schema-adjacent errors that made it past the parser because the parser was configured too permissively during a schema migration.
If you have AI workflows running in production today, you can harden them this week without rewriting your entire stack.
Audit every LLM node that feeds downstream logic. If you see JSON.parse wrapped in a
try/catch, or worse, regex extraction, replace it with a schema-driven parser. Define one
strict JSON schema per LLM output.
Include enums for every field that drives an IF node or a switch. Use nested objects to group routing signals, content, and metadata.
If your reasoning model is burning tokens on formatting anxiety, split the work. Let the big model think. Let a mini model parse.
Log malformed outputs. Retry with backoff. Default to human escalation when the machine is uncertain.
Structured output does not eliminate model unpredictability; it cages it. The cage is only as strong as the error handling you wrap around it.
Build the schema first. Write the prompt to serve the schema. Treat the model's output like an API response, not a conversation. That's the difference between an AI demo and an AI system that stays sane at 2 AM on a Sunday.