Skip to content
Arcjet Docs

AI budget control for Next.js

AI providers bill per token. Without per-user limits, a single user can exhaust your entire monthly budget – through prompt attacks, runaway loops, or just heavy legitimate use.

A token bucket rate limit maps directly onto how AI billing works. You estimate the cost of each request in tokens, deduct it from the user’s bucket, and deny requests when the bucket is empty. The bucket refills over time, giving each user a sustained allowance without sharp rate-limit cliffs.

Alternatively, you can use a fixed window or sliding window limit to enforce a hard cap on spend per user per day, week, or month. For details on different approaches, see the rate limiting algorithms reference.

Arcjet handles bucket state across all instances of your application – no Redis or external state store required.

Get started

Section titled “Get started”

In this example we use the Vercel AI SDK to create a simple AI chat endpoint with Next.js, and Arcjet to enforce per-user token budgets to prevent cost overruns. The same principles can be applied to any AI application, including those built with other frameworks.

We assume you already have a Next.js app set up.

Install the dependencies:

Terminal window
# Export your Arcjet API key from https://console.arcjet.com
export ARCJET_KEY="ajkey_..."


npm install @arcjet/next ai @ai-sdk/openai

Create an AI chat endpoint:

/app/api/chat/route.ts
import { openai } from "@ai-sdk/openai";
import arcjet, { tokenBucket } from "@arcjet/next";
import type { UIMessage } from "ai";
import { convertToModelMessages, streamText } from "ai";


const aj = arcjet({
  key: process.env.ARCJET_KEY!, // Get your site key from https://console.arcjet.com
  // Track budgets per user – replace "userId" with any stable identifier
  characteristics: ["userId"],
  rules: [
    tokenBucket({
      mode: "LIVE", // Blocks requests. Use "DRY_RUN" to log only
      refillRate: 2_000, // Refill 2,000 tokens per hour
      interval: "1h",
      capacity: 5_000, // Maximum 5,000 tokens in the bucket
    }),
  ],
});


export async function POST(req: Request) {
  // Replace with your session/auth lookup to get a stable user ID
  const userId = "user-123";
  const { messages }: { messages: UIMessage[] } = await req.json();
  const modelMessages = await convertToModelMessages(messages);


  // Estimate token cost: ~1 token per 4 characters of text (rough heuristic).
  // For accurate counts use https://www.npmjs.com/package/tiktoken
  const totalChars = modelMessages.reduce((sum, m) => {
    const content =
      typeof m.content === "string" ? m.content : JSON.stringify(m.content);
    return sum + content.length;
  }, 0);
  const estimate = Math.ceil(totalChars / 4);


  // Deduct the estimated tokens from the user's budget
  const decision = await aj.protect(req, { userId, requested: estimate });


  if (decision.isDenied()) {
    return new Response("AI usage limit exceeded", { status: 429 });
  }


  const result = await streamText({
    model: openai("gpt-4o"),
    messages: modelMessages,
  });


  return result.toUIMessageStreamResponse();
}

And hook it up to a chat UI:

/app/page.tsx
"use client";


import { useChat } from "@ai-sdk/react";
import { useState } from "react";


export default function Chat() {
  const [input, setInput] = useState("");
  const [errorMessage, setErrorMessage] = useState<string | null>(null);
  const { messages, sendMessage } = useChat({
    onError: async (e) => setErrorMessage(e.message),
  });
  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      {messages.map((message) => (
        <div key={message.id} className="whitespace-pre-wrap">
          {message.role === "user" ? "User: " : "AI: "}
          {message.parts.map((part, i) => {
            switch (part.type) {
              case "text":
                return <div key={`${message.id}-${i}`}>{part.text}</div>;
            }
          })}
        </div>
      ))}


      {errorMessage && (
        <div className="text-red-500 text-sm mb-4">{errorMessage}</div>
      )}


      <form
        onSubmit={(e) => {
          e.preventDefault();
          sendMessage({ text: input });
          setInput("");
          setErrorMessage(null);
        }}
      >
        <input
          className="fixed dark:bg-zinc-900 bottom-0 w-full max-w-md p-2 mb-8 border border-zinc-300 dark:border-zinc-800 rounded shadow-xl"
          value={input}
          placeholder="Say something..."
          onChange={(e) => setInput(e.currentTarget.value)}
        />
      </form>
    </div>
  );
}

Then run the server:

Terminal window
npm run dev

Requests appear in your Arcjet dashboard in real time.

Configure the rate limit

Section titled “Configure the rate limit”

characteristics: ["userId"] - Tracks the bucket per user. Replace "userId" with the characteristic that identifies a unique user in your application, such as a session token, API key, or authenticated user ID. Pass the value to aj.protect() as a named argument.

refillRate and interval - Set the sustained allowance. refillRate: 2_000, interval: "1h" gives each user 2,000 tokens per hour. Adjust to match your AI provider’s pricing and your cost targets. These are hard coded in this example, but you can also calculate them dynamically based on user subscription level or other factors. Pass the calculated values to the rule.

capacity - The maximum tokens a user can accumulate. Setting capacity: 5_000 with refillRate: 2_000 lets users burst up to 5,000 tokens if they haven’t used their allowance recently.

Token estimation

Section titled “Token estimation”

The example uses a characters / 4 heuristic (~1 token per 4 characters for common English text). This is a reasonable starting point – it avoids introducing extra dependencies and works well enough for budget enforcement where a small margin of error is acceptable.

For accurate counts, use a tokenizer:

Learn more

Section titled “Learn more”