Arcjet rate limiting algorithms

Arcjet rate limiting lets you define rules which limit the number of requests a client can make over a period of time. It supports 3 different rate limit algorithms: fixed window, sliding window and token bucket.

What is Arcjet?

Arcjet is the runtime security platform that ships with your code. Enforce budgets, stop prompt injection, detect bots, and protect personal information with Arcjet's AI security building blocks.

Fixed window

This algorithm tracks the number of requests made by a client over a fixed time window, such as 60 seconds. If the client exceeds the limit, they are blocked until the window expires.

When to use fixed window

This algorithm is useful when you want to apply a fixed limit in a fixed time window. For example, a flat limit on the total number of requests a client can make.

Downsides of fixed window

• A burst of requests at the start of the window exhausts the limit for newer requests.
• A client can make a burst of requests at the end of the window and continue to burst as the window resets. This can cause request stampedes where clients suddenly become unblocked at the start of a new window.

Fixed window example

In this example, we define a fixed window rate limit rule which allows a maximum of 100 requests in a 60 second window. The rule is applied to all requests to /api/hello and is tracked by the client’s IP address.

import arcjet, { fixedWindow } from "@arcjet/next";

const aj = arcjet({
  key: process.env.ARCJET_KEY!,
  characteristics: ["ip.src"], // track requests by IP address
  rules: [
    fixedWindow({
      mode: "LIVE", // will block requests. Use "DRY_RUN" to log only
      window: "60s", // 60 second fixed window
      max: 100, // allow a maximum of 100 requests
    }),
  ],
});

Suppose the time is 12:00:00 and a client makes a burst of 50 requests, then another 50 requests over the following 30 seconds. Arcjet blocks any further requests until 12:01:00. At 12:01:00 the window resets and they could make another 100 requests immediately.

Sliding window

This algorithm tracks the number of requests made by a client over a sliding window so that the window moves with time.

When to use sliding window

This algorithm is useful to avoid the stampede problem of the fixed window. It provides smoother rate limiting over time and can prevent a client from making a burst of requests at the start of a window and then being blocked for the rest of the window.

Downsides of sliding window

• The limit is not as strict as fixed window because the time window is continuously moving.
• It can be more difficult to reason about the rate limit because the window is always moving.

Sliding window example

In this example, we define a sliding window rate limit which allows a maximum of 100 requests in a 60 second window. The rule is applied to all requests to /api/hello and is tracked by the client’s IP address.

import arcjet, { slidingWindow } from "@arcjet/next";

const aj = arcjet({
  key: process.env.ARCJET_KEY!,
  rules: [
    slidingWindow({
      mode: "LIVE", // will block requests. Use "DRY_RUN" to log only
      // Tracked by IP address by default, but this can be customized
      // See https://docs.arcjet.com/fingerprints
      //characteristics: ["ip.src"],
      interval: 60, // 60 second sliding window
      max: 100, // allow a maximum of 100 requests
    }),
  ],
});

Suppose the client made 50 requests between 12:00:00 and 12:01:00, then made another 50 requests in the current time period as of 12:01:30. Arcjet calculates the remaining rate limit as follows:

// 50 requests in the previous interval
// 60 seconds per interval
// 30 seconds elapsed in the current interval
// 50 requests made in the current interval
rate = 50 * ((60 - 30) / 60) + 50 = 75
limit = 100
remaining = limit - rate = 25

In this case, the client has 25 requests remaining in the current interval.

Token bucket

This algorithm is based on a bucket filled with a specific number of tokens. Each request withdraws a token from the bucket and the bucket is refilled at a fixed rate. Once the bucket is empty, the client is blocked until the bucket refills.

When to use token bucket

This algorithm is useful when you want to allow clients to make a burst of requests and then still be able to make requests at a slower rate.

Downsides of token bucket

A burst of requests can exhaust the bucket, which then takes a long time to refill, depending on your configured refill rate. This can block a client for a long time.

Token bucket example

In this example, we define a token bucket rate limit with a bucket capacity of 100 tokens and a refill rate of 10 tokens per minute. The rule is applied to all requests to /api/hello and is tracked by the client’s IP address.

import arcjet, { tokenBucket } from "@arcjet/next";

const aj = arcjet({
  key: process.env.ARCJET_KEY!,
  rules: [
    tokenBucket({
      mode: "LIVE", // will block requests. Use "DRY_RUN" to log only
      // Tracked by IP address by default, but this can be customized
      // See https://docs.arcjet.com/fingerprints
      //characteristics: ["ip.src"],
      refillRate: 10, // refill 10 tokens per interval
      interval: 60, // 60 second interval
      capacity: 100, // bucket maximum capacity of 100 tokens
    }),
  ],
});

The amount of tokens to deduct from the bucket is specified in the requested option when calling the Arcjet protect function. You can request any number of tokens up to the capacity of the bucket.

Note

The requested amount must be a positive integer.

JavaScript / TypeScript

// Deduct 1 token from the bucket
const decision = await aj.protect(req, { requested: 1 });

Python

# Deduct 1 token from the bucket
decision = await aj.protect(request, requested=1)  # use aj.protect(request, ...) for arcjet_sync

For example, if it is 12:00:00 and a client makes 100 requests each with 1 token deducted, further requests are blocked until 12:01:00, when the bucket refills by 10 tokens. The client could then make 10 requests over the following 60 seconds. If they make no requests, the bucket refills to 20 tokens at 12:02:00. If they make no requests between 12:01:00 and 12:10:00 then the bucket refills to 100 tokens at 12:10:00. No further tokens are added after it reaches 100 tokens.

Discussion