AI UI Patterns

An LLM feature is an unusually hostile UI problem: responses take seconds instead of milliseconds, arrive incrementally, fail partway through, cost money per attempt, and sometimes contain content that tries to manipulate the surrounding application. A chat box wired to a completion endpoint handles none of that.

The latency contract

Users tolerate slow generation and do not tolerate silence. Show state within 100ms of submit, first token as soon as it exists, and never a spinner with no other signal.

Four states, all of which need a design: pending (request sent, nothing back yet), streaming (tokens arriving), complete, failed or cancelled. The failure state is the one most implementations skip, and it is the one users hit on flaky connections.

Streaming

Stream by default. A 40-token response feels instant when streamed and slow when buffered, even at identical total latency.

// app/api/chat/route.ts
export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({ model: openai('gpt-4o'), messages });
  return result.toDataStreamResponse();
}

Three details that separate a demo from production:

Cancellation. Pass the request's AbortSignal through to the provider so a stopped generation stops being billed. Without it, "Stop" hides the output while the tokens keep costing money.

Backpressure. Rendering on every token thrashes React at high token rates. Batch with requestAnimationFrame or a short interval so you re-render at frame rate, not token rate.

Reconnection. A dropped SSE connection mid-stream leaves a truncated message. Either resume from a persisted stream id or mark the message as incomplete and offer retry — never present a truncated answer as if it finished.

Autoscroll deserves its own rule: follow the stream only while the user is already at the bottom. Yanking the viewport down while someone reads earlier output is the single most common complaint about chat UIs.

Rendering model output safely

Model output is untrusted input. It reaches your DOM, so treat it as you would a comment field on a public forum.

• Never dangerouslySetInnerHTML on raw output. Render markdown through a sanitizing pipeline (react-markdown with rehype-sanitize), not a raw HTML converter.
• Strip or neutralize links and images pointing at untrusted origins. A markdown image whose URL encodes conversation content into a query string is a real exfiltration path in RAG applications.
• Never let model output trigger actions directly. If a response says to call a tool or navigate somewhere, that must route through your own validated tool-call path, not through text interpretation.
• Escape inside code blocks and render them as text; a fenced block is content, not markup.

This matters most when the model has read anything a third party wrote — a retrieved document, a scraped page, a support ticket. That content can contain instructions aimed at your application, and the UI layer is one of the places that either contains the blast radius or spreads it.

Tool calls and agent steps

When the model calls tools, show the work. A ten-second silence during which an agent searches, reads, and computes reads as a hang unless the UI narrates it.

Render each step as it resolves — the tool name in human terms ("Searching orders"), its status, and a collapsed result the user can expand. Keep raw arguments available but hidden; they are essential for debugging and noise for everyone else.

Anything with side effects — sending, purchasing, deleting, writing — gets an explicit confirmation step showing the exact arguments before execution. Model confidence is not authorization.

Cost, limits, and failure

Every request costs money and can be rate limited, so build for it rather than discovering it in production:

• Surface remaining quota before the user hits the wall, not after
• Distinguish the failure types in the UI — rate limited (retry with backoff, show when), context too long (offer to summarize or trim), content filtered (explain, do not retry), provider down (fall back to a smaller model or queue)
• Retry with exponential jitter on 429 and 5xx only; retrying a filtered response just burns quota
• Debounce or gate expensive calls behind an explicit action; an autocomplete firing a completion per keystroke is a billing incident
• Cache aggressively for identical prompts, and keep a stable request id so a retry after a network blip does not double-charge

Editable and regenerable history

Chat is a document the user should be able to edit, not an append-only log. Support editing an earlier message and regenerating from that point, which means storing messages as an ordered structure you can truncate and replay — not a string you append to.

Offer regeneration with a variation (different model, longer, shorter) rather than an identical retry, since an identical retry usually produces an equally unsatisfying answer.

Accessibility

Streaming text in an aria-live region announces every token, which is unusable. Instead, keep the streaming region aria-live="off" and announce once on completion with a polite status ("Response complete, 3 paragraphs"). Ensure the stop control is keyboard reachable at all times, and that focus returns to the composer after a response finishes.

Implementation reference

references/streaming.md has working code for the pieces described above: a custom SSE route with abort propagation, frame-batched client rendering, scroll-position-aware autoscroll, a sanitized markdown renderer, and the error classifier. Read it when implementing rather than reviewing — several of the details there (the buffering header, abort on unmount) are the difference between a demo that works locally and one that works behind a proxy.

Review checklist

1. Cancellation wired through to the provider, not just the UI
2. Untrusted-content rendering path sanitized; no raw HTML injection
3. Tool side effects gated behind confirmation
4. All four request states designed, including partial failure
5. Autoscroll respects the user's scroll position
6. Rate limit and quota errors distinguished and actionable
7. Streaming region does not spam screen readers
8. Token rendering batched rather than per-token re-render
9. Costs bounded — no per-keystroke completions, no unbounded context growth
10. API keys server-side only; never a provider key in a client bundle