Stream AI Responses in Production

The model is already generating. Your loading spinner is hiding it.

Stream the response and users can read while the model writes. The total generation time may not change, but the interface stops feeling stuck.

Here is the current pattern with Hono and the Vercel AI SDK.

The route

bun add ai @ai-sdk/anthropic @ai-sdk/react hono

import { anthropic } from "@ai-sdk/anthropic";
import {
  convertToModelMessages,
  streamText,
  type UIMessage,
} from "ai";
import { Hono } from "hono";

const app = new Hono();

app.post("/api/chat", async (c) => {
  const { messages }: { messages: UIMessage[] } = await c.req.json();

  const result = streamText({
    model: anthropic(process.env.ANTHROPIC_MODEL!),
    messages: await convertToModelMessages(messages),
    abortSignal: c.req.raw.signal,
  });

  return result.toUIMessageStreamResponse();
});

export default app;

That is the server.

streamText starts generation. toUIMessageStreamResponse() returns the protocol expected by AI SDK UI. abortSignal stops generation when the browser disconnects.

On the client, useChat handles the stream and exposes each message as parts:

import { useChat } from "@ai-sdk/react";
import { DefaultChatTransport } from "ai";

export function useStreamingChat() {
  return useChat({
    transport: new DefaultChatTransport({
      api: "/api/chat",
    }),
  });
}

Three production rules

1. Cancel abandoned requests. Pass the request signal into streamText. Tokens generated after a disconnect still cost money.
2. Budget for long responses. Configure your platform timeout for the longest generation you allow. Streaming keeps a connection open; it does not remove runtime limits.
3. Design for partial failure. A stream can fail after useful text has arrived. Render the partial response, expose the error, and give the user a clear retry.

The AI SDK keeps the model provider behind one interface. Hono keeps the HTTP layer on Web-standard Request and Response objects. That is enough abstraction for most chat routes.

Streaming is not a finishing touch. If the product generates text, make progress visible from the first version.

Resources

• Vercel AI SDK
• streamText reference
• Hono request API