Building Context-Aware Applications

The trap is treating every API call as a fresh conversation with a stranger. I see it constantly: teams build a support bot that asks the user for information the system already has. Name, account tier, timezone, the product they are currently looking at -- all of it sitting in the session store, none of it reaching Claude. The model gives a generic answer. The user gives a generic review. Everyone blames the AI.

The problem is never Claude's reasoning. The problem is that nobody told Claude what it was reasoning about.

Context in the Claude API is the complete set of information the model uses to understand intent, constraints, and user expectations. Think of it as the briefing folder you hand a consultant before they walk into a meeting. System instructions, user messages, conversation history, structured metadata, external data -- all of it lands in that folder. Stronger context leads directly to better accuracy, higher relevance, and safer responses. Weaker context leads to the kind of vague, hedge-everything output that makes stakeholders question whether AI was the right bet.

The Five Layers of Context

Claude does not rely on a single type of context. It integrates several components, and understanding the hierarchy matters because each layer has a different cost-to-value ratio.

System instructions define the model's role, rules, and constraints. This is the cheapest, most stable context you can provide. Write it once, reuse it across every request in the session.

User instructions represent the immediate request -- the question, the task, the thing the user typed. This is what most developers focus on, and it is the least interesting layer to optimize because the user controls it.

Long-term context captures the ongoing conversation or session data. This is where continuity lives. Without it, every message is a cold start.

Structured inputs include tool definitions, JSON schemas, function signatures -- anything with a defined shape. These give Claude precision that prose instructions cannot.

External context allows the model to incorporate information from APIs, databases, or files. This is the highest-value, highest-cost layer. It makes the difference between a model that knows what a user asked and one that knows the full situation.

I have seen teams pour effort into retrieval-augmented generation pipelines to inject documentation into every request while leaving the system prompt as a single line: "You are a helpful assistant." That is building the penthouse before pouring the foundation.

Building Runtime Context

The real power of context-aware applications shows up when you inject runtime information -- data the system knows at request time but the user never explicitly provides. User name, account metadata, current timestamp, the page they are viewing, the feature flags enabled for their account. This is the context that makes a response feel personal without the user lifting a finger.

Here is the pattern I use in production. A function builds a context dictionary from whatever the system knows about the current request, and that dictionary gets injected into the prompt alongside the user's question:

import anthropic
import json
from datetime import datetime, timezone

client = anthropic.Anthropic()

def build_runtime_context(user: dict, session: dict) -> dict:
    return {
        "user_profile": {
            "name": user.get("name", "Unknown"),
            "account_tier": user.get("tier", "free"),
            "timezone": user.get("timezone", "UTC"),
            "signup_date": user.get("signup_date"),
        },
        "session": {
            "channel": session.get("channel", "web"),
            "current_page": session.get("page", "/"),
            "timestamp": datetime.now(timezone.utc).isoformat(),
            "locale": session.get("locale", "en-US"),
        },
    }


def ask_with_context(question: str, user: dict, session: dict) -> str:
    context = build_runtime_context(user, session)

    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        system=(
            "You are a support assistant for Acme SaaS. "
            "Use the provided runtime context to personalize your response. "
            "Address the user by name. Adjust technical depth based on their account tier. "
            "Always respond in the user's locale language."
        ),
        messages=[
            {
                "role": "user",
                "content": (
                    f"Runtime context:\n{json.dumps(context, indent=2)}\n\n"
                    f"User question: {question}"
                ),
            }
        ],
    )

    return message.content[0].text

The context dictionary is dead simple -- a flat structure with user profile and session metadata. No clever abstractions. No ORM. The function builds it from whatever data sources your system already has: your auth middleware, your session store, your feature-flag service. The important move is that this context is assembled on every request, not stored in the conversation history. It is always fresh, always accurate, and it costs a fixed number of tokens regardless of how long the conversation runs.

Managing Conversation History

Conversation history is the most expensive context you will manage. Every previous message becomes part of the next prompt. A 20-turn conversation means Claude processes 20 messages of input before generating a single token of output. The token meter is running on every word the user said three minutes ago, ten minutes ago, at the start of the session.

The naive approach -- appending every message to a list and sending the full list on every call -- works fine for a demo and collapses in production. I have seen chat applications hit the context window ceiling in under fifteen minutes of active use.

The pattern that works is a sliding window with a summary checkpoint:

import anthropic

client = anthropic.Anthropic()

MAX_HISTORY = 10  # Keep the last 10 messages


def trim_history(history: list[dict]) -> list[dict]:
    """Keep only the most recent messages to control token usage."""
    if len(history) <= MAX_HISTORY:
        return history
    return history[-MAX_HISTORY:]


def summarize_old_context(history: list[dict]) -> str:
    """Ask Claude to compress older messages into a summary."""
    old_messages = history[:-MAX_HISTORY]
    if not old_messages:
        return ""

    transcript = "\n".join(
        f"{msg['role']}: {msg['content']}" for msg in old_messages
    )

    response = client.messages.create(
        model="claude-haiku-4-20250514",
        max_tokens=300,
        messages=[
            {
                "role": "user",
                "content": (
                    "Summarize this conversation in 2-3 sentences. "
                    "Focus on decisions made and questions still open.\n\n"
                    f"{transcript}"
                ),
            }
        ],
    )
    return response.content[0].text


def chat(user_message: str, history: list[dict], summary: str = "") -> str:
    history.append({"role": "user", "content": user_message})

    system_prompt = "You are a helpful technical assistant."
    if summary:
        system_prompt += f"\n\nPrevious conversation summary: {summary}"

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        system=system_prompt,
        messages=trim_history(history),
    )

    reply = response.content[0].text
    history.append({"role": "assistant", "content": reply})
    return reply

The trick is using a cheap, fast model like Claude Haiku for the summarization step. The summary compresses twenty messages into two sentences and gets injected into the system prompt, where it costs a fixed number of tokens. The recent messages stay intact for immediate context. Old messages get replaced by the summary. Total token cost stays bounded regardless of conversation length.

Structured Context Injection

The most underused technique I see is structured context injection -- passing Claude a well-defined JSON or schema alongside the user's question. Prose instructions are ambiguous. Structured context is not.

Compare these two approaches to telling Claude about the current user:

Here is what structured context injection looks like in practice:

import anthropic
import json

client = anthropic.Anthropic()


def ask_with_structured_context(
    question: str,
    user_profile: dict,
    mood: str = "neutral",
) -> str:
    context = {
        "user_profile": user_profile,
        "session_context": {
            "mood": mood,
            "response_language": "en",
            "assistant_role": "supportive technical mentor",
        },
    }

    prompt = (
        "You will receive a JSON context object and a user question. "
        "Always use the context to tailor your response:\n"
        "- Address the user by name\n"
        "- Match technical depth to their skill level\n"
        "- Adjust tone to their current mood\n\n"
        f"Context:\n```json\n{json.dumps(context, indent=2)}\n```\n\n"
        f"Question: {question}"
    )

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )

    return response.content[0].text

The JSON context is explicit. Every field has a name. Nothing is ambiguous. When a teammate reads this code six months from now, they know exactly what context the model is receiving because the schema is the documentation.

How Claude Processes Context Internally

Understanding what happens inside the model clarifies why context quality matters more than context quantity. When you send a prompt, Claude transforms every token of your input into a mathematical representation in latent space. This is where relationships, meaning, and intent get encoded. The model then uses attention mechanisms to determine which parts of the context are most relevant to the current token being generated.

Here is the practical implication: if your context is a 3,000-token wall of text with the critical piece of information buried in paragraph seven, the attention mechanism will still find it -- but the signal-to-noise ratio is lower. A tight, focused context with high information density produces better results than a sprawling context that "covers everything just in case."

Longer context allows Claude to make more informed decisions, but it also increases computation cost. Every additional token adds to the pre-fill time -- the delay before the model starts generating its first output token. On a 200-token prompt, pre-fill is negligible. On a 50,000-token prompt with full conversation history, injected documents, and a verbose system prompt, pre-fill can add seconds of latency that users feel on every single request.

The engineering discipline is managing this balance. More context is not always better context. The right question is not "what else can I include?" but "what can I remove without degrading the response?"

The Context Budget

Here is the reality nobody talks about in tutorials. Context is not free. Every token of context is a token you pay for on every single request. A 500-token system prompt costs 500 input tokens per API call. If that system prompt includes a 200-token biography of the author that has nothing to do with the user's question, you are burning money on every call.

I think about context the same way I think about database indexes: every one you add speeds up a specific query and slows down writes. Every piece of context you inject improves a specific class of responses and increases the cost and latency of every response.

The discipline is knowing what to include and what to leave out. System prompts should be short, stable, and role-focused. Runtime context should contain only information that changes the response. Conversation history should be trimmed aggressively. External data should be retrieved only when the question requires it, not preloaded "just in case."

Here is a set of rules I keep pinned above my desk:

• Keep system prompts under 300 tokens. If yours is longer, you are probably encoding behavior that belongs in the user message or in structured context.
• Carry forward only essential context. "Essential" means: removing it changes the response for the worse. If removing it changes nothing, it was dead weight.
• Use structured prompting -- rules, examples, schemas -- to guide the model instead of verbose natural-language instructions. Three JSON fields communicate more precision than three paragraphs of prose.
• Add metadata (user preferences, session state, feature flags) only when you can demonstrate that it changes the output. Do not inject context prophylactically.
• Deploy conversation windows. Trim old messages. Summarize what came before. Every token of history you carry is a token you pay for on the next turn.
• Validate outputs using schemas. If you asked for JSON, parse the JSON. If a field is missing, handle it before the rest of your application sees the response.

What to Do Monday Morning