AI prompt injection detection for Python + Flask

Prompt injection attacks trick AI models into ignoring their instructions. Users paste in jailbreaks such as “DAN” prompts, role-play escapes, or instruction overrides designed to bypass your system prompt, extract restricted information, or make your AI behave in unintended ways.

Arcjet prompt injection detection evaluates each incoming message for injection patterns inside your application before it reaches the AI provider. Detected attacks are blocked before the AI call is made, protecting both your application behavior and your AI budget.

Get started

In this example we use LangChain to create a simple AI chat server with Flask, and Arcjet to block prompt injection attacks before they reach the AI model. The same principles can be applied to any AI application, including those built with other frameworks.

Set up the environment and install dependencies (uses uv, but you can also use pip to install the Arcjet Python SDK):

# Export your Arcjet API key from https://console.arcjet.com
export ARCJET_KEY="ajkey_..."
export ARCJET_ENV=development

# Export your OpenAI API key (used by LangChain)
export OPENAI_API_KEY="sk-..."

# Install dependencies
uv add arcjet flask langchain langchain-openai

Create the chat server:

app.py

import logging
import os

from arcjet import Mode, arcjet_sync, detect_prompt_injection, shield
from flask import Flask, jsonify, request
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

app = Flask(__name__)

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

arcjet_key = os.getenv("ARCJET_KEY")
if not arcjet_key:
    raise RuntimeError("ARCJET_KEY is required. Get one at https://console.arcjet.com")

openai_api_key = os.getenv("OPENAI_API_KEY")
if not openai_api_key:
    raise RuntimeError(
        "OPENAI_API_KEY is required. Get one at https://platform.openai.com"
    )

llm = ChatOpenAI(model="gpt-4o-mini", api_key=openai_api_key)

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful assistant."),
        ("human", "{message}"),
    ]
)

chain = prompt | llm | StrOutputParser()

# Create a single Arcjet client at startup and reuse it across requests
aj = arcjet_sync(
    key=arcjet_key,  # Get your key from https://console.arcjet.com
    rules=[
        # Shield protects against common web attacks e.g. SQL injection
        shield(mode=Mode.LIVE),
        # Detect prompt injection attacks before they reach your AI model
        detect_prompt_injection(
            mode=Mode.LIVE,  # Blocks requests. Use Mode.DRY_RUN to log only
        ),
    ],
)


@app.post("/chat")
def chat():
    body = request.get_json()
    message = body.get("message", "") if body else ""

    # Pass the user message so detect_prompt_injection can evaluate it
    decision = aj.protect(request, detect_prompt_injection_message=message)

    if decision.is_denied():
        if decision.reason_v2.type == "PROMPT_INJECTION":
            logger.warning("Request blocked due to prompt injection")
            return jsonify(
                error="Prompt injection detected — please rephrase your message"
            ), 400
        # SHIELD or any other denial
        return jsonify(error="Forbidden"), 403

    # Arcjet approved — call the AI model
    reply = chain.invoke({"message": message})

    return jsonify(reply=reply)


if __name__ == "__main__":
    app.run(debug=True)

Then run the server:

uv run python app.py

And send a message to the API endpoint:

curl -X POST http://localhost:5000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the capital of France?"}'

Requests appear in your Arcjet dashboard in real time.

Protect a production chat endpoint

A production chat endpoint needs more than one guardrail. Some requests contain hostile instructions designed to override your system prompt. Others may be legitimate user requests that still contain sensitive data you do not want entering model context. And like any other public route, AI endpoints still need protection from common web attacks.

Combining Arcjet rules gives you layered enforcement before the model runs:

• Shield blocks common web attacks against the endpoint
• Prompt injection detection catches hostile instructions before inference
• Sensitive information detection prevents PII from entering model context

The following example uses the Vercel AI SDK (JS) / LangChain (Python):

JavaScript / TypeScript

import { openai } from "@ai-sdk/openai";
import arcjet, {
  detectPromptInjection,
  sensitiveInfo,
  shield,
} from "@arcjet/next";
import type { UIMessage } from "ai";
import { convertToModelMessages, isTextUIPart, streamText } from "ai";

const aj = arcjet({
  key: process.env.ARCJET_KEY!, // Get your site key from https://console.arcjet.com
  rules: [
    // Shield protects against common web attacks e.g. SQL injection
    shield({ mode: "LIVE" }),

    // Detect prompt injection attacks before they reach your AI model
    detectPromptInjection({
      mode: "LIVE",
    }),

    // Block sensitive data from entering model context
    sensitiveInfo({
      mode: "LIVE",
      // Block PII types that should never appear in AI prompts.
      // Remove types your app legitimately handles (e.g. EMAIL for a support bot).
      deny: ["CREDIT_CARD_NUMBER", "EMAIL"],
    }),
  ],
});

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  // Check the most recent user message.
  // Pass the full conversation if you want to scan all messages.
  const lastMessage: string = (messages.at(-1)?.parts ?? [])
    .filter(isTextUIPart)
    .map((p) => p.text)
    .join(" ");

  const decision = await aj.protect(req, {
    detectPromptInjectionMessage: lastMessage,
    sensitiveInfoValue: lastMessage,
  });

  if (decision.isDenied()) {
    if (decision.reason.isPromptInjection()) {
      console.warn("Request blocked due to prompt injection");
      return new Response(
        "Prompt injection detected — please rephrase your message",
        { status: 403 },
      );
    }

    if (decision.reason.isSensitiveInfo()) {
      console.warn("Request blocked due to sensitive information");
      return new Response(
        "Sensitive information detected — please remove it from your prompt",
        { status: 400 },
      );
    }

    return new Response("Forbidden", { status: 403 });
  }

  // Arcjet approved — call your AI provider
  const result = await streamText({
    model: openai("gpt-4o"),
    messages: await convertToModelMessages(messages),
  });

  return result.toUIMessageStreamResponse();
}

Python

import logging
import os

from arcjet import (
    Mode,
    SensitiveInfoEntityType,
    arcjet,
    detect_prompt_injection,
    detect_sensitive_info,
    shield,
)
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from pydantic import BaseModel

app = FastAPI()
logger = logging.getLogger(__name__)

llm = ChatOpenAI(model="gpt-4o", api_key=os.environ["OPENAI_API_KEY"])
chain = ChatPromptTemplate.from_messages(
    [("system", "You are a helpful assistant."), ("human", "{message}")]
) | llm | StrOutputParser()


class ChatRequest(BaseModel):
    message: str


aj = arcjet(
    key=os.environ["ARCJET_KEY"],  # Get your site key from https://console.arcjet.com
    rules=[
        # Shield protects against common web attacks e.g. SQL injection
        shield(mode=Mode.LIVE),
        # Detect prompt injection attacks before they reach your AI model
        detect_prompt_injection(mode=Mode.LIVE),
        # Block sensitive data from entering model context
        detect_sensitive_info(
            mode=Mode.LIVE,
            # Block PII types that should never appear in AI prompts.
            # Remove types your app legitimately handles (e.g. EMAIL for a
            # support bot).
            deny=[
                SensitiveInfoEntityType.CREDIT_CARD_NUMBER,
                SensitiveInfoEntityType.EMAIL,
            ],
        ),
    ],
)


@app.post("/chat")
async def chat(request: Request, body: ChatRequest):
    # Pass the user message to both scanners. Pass the full conversation if
    # you want to scan all messages.
    decision = await aj.protect(
        request,
        detect_prompt_injection_message=body.message,
        sensitive_info_value=body.message,
    )

    if decision.is_denied():
        if decision.reason_v2.type == "PROMPT_INJECTION":
            logger.warning("Request blocked due to prompt injection")
            return JSONResponse(
                {"error": "Prompt injection detected — please rephrase your message"},
                status_code=400,
            )
        if decision.reason_v2.type == "SENSITIVE_INFO":
            logger.warning("Request blocked due to sensitive information")
            return JSONResponse(
                {"error": "Sensitive information detected — please remove it from your prompt"},
                status_code=400,
            )
        return JSONResponse({"error": "Forbidden"}, status_code=403)

    # Arcjet approved — call your AI provider
    reply = await chain.ainvoke({"message": body.message})
    return {"reply": reply}

Keep denied responses generic – do not leak detector details or explain exactly what was flagged. A generic message asking the user to rephrase is the right default.

Configure prompt injection detection

detectPromptInjectionMessage - the text to evaluate. Pass the user’s most recent message, or the full conversation history if you want to scan all messages.

mode - "LIVE" blocks detections. "DRY_RUN" logs detections without blocking. Use "DRY_RUN" to measure the false-positive rate in production before switching to "LIVE". JavaScript detectPromptInjection accepts only mode.

The verdict is binary. In JavaScript, use decision.reason.isPromptInjection() or decision.reason.injectionDetected. In Python, check decision.reason_v2.type == "PROMPT_INJECTION".

Python detect_prompt_injection accepts only mode, which is required. Omitting mode or passing threshold= raises TypeError.

Combine with abuse protection

Prompt injection detection controls what your AI model receives. To also block automated clients and enforce per-user budgets, combine it with AI abuse protection and AI budget control.

Prompt injection is one class of AI abuse. Automated traffic is another – if you expose a public AI endpoint, attackers can drive up costs with automated traffic without needing to bypass your system prompt. Bot detection composes cleanly with prompt injection protection:

JavaScript / TypeScript

import arcjet, {
  detectBot,
  detectPromptInjection,
  sensitiveInfo,
  shield,
} from "@arcjet/next";

const aj = arcjet({
  key: process.env.ARCJET_KEY!,
  rules: [
    shield({ mode: "LIVE" }),
    detectBot({ mode: "LIVE", allow: [] }),
    detectPromptInjection({ mode: "LIVE" }),
    sensitiveInfo({
      mode: "LIVE",
      deny: ["CREDIT_CARD_NUMBER", "EMAIL"],
    }),
  ],
});

Python

import os
from arcjet import (
    Mode,
    SensitiveInfoEntityType,
    arcjet,
    detect_bot,
    detect_prompt_injection,
    detect_sensitive_info,
    shield,
)

aj = arcjet(
    key=os.environ["ARCJET_KEY"],
    rules=[
        shield(mode=Mode.LIVE),
        detect_bot(mode=Mode.LIVE, allow=[]),
        detect_prompt_injection(mode=Mode.LIVE),
        detect_sensitive_info(
            mode=Mode.LIVE,
            deny=[
                SensitiveInfoEntityType.CREDIT_CARD_NUMBER,
                SensitiveInfoEntityType.EMAIL,
            ],
        ),
    ],
)

Protect tool calls

The preceding examples protect an HTTP endpoint – the boundary where user input enters your application. But prompt injection can also arrive through tool results: a fetch tool retrieves a page that contains injected instructions, which then re-enter the model context when the tool result is passed back.

Agent guards run the same prompt injection detection inside tool handlers, without a Request object. You can also use a remote policy to apply server-side prompt injection detection to an explicitly mapped input. Install the skill to add an SDK guard automatically:

npx skills add arcjet/skills

Or wire it up manually – create a guard client once at module scope, then call .guard() inline in each tool handler:

JavaScript / TypeScript

import { launchArcjet, detectPromptInjection } from "@arcjet/guard";

const arcjet = launchArcjet({ key: process.env.ARCJET_KEY! });

const piRule = detectPromptInjection();

// Inside your fetch tool handler
export async function fetchTool({ url }: { url: string }, userId: string) {
  const content = await fetch(url).then((r) => r.text());

  const decision = await arcjet.guard({
    label: "tools.fetch",
    metadata: { userId },
    rules: [piRule(content)],
  });

  if (decision.conclusion === "DENY") {
    // Return a safe placeholder rather than the injected content
    return { content: "[Content blocked: prompt injection detected]" };
  }

  return { content };
}

Python

import os
import httpx
from arcjet.guard import DetectPromptInjection, launch_arcjet

arcjet = launch_arcjet(key=os.environ["ARCJET_KEY"])

pi_rule = DetectPromptInjection()


# Inside your fetch tool handler
async def fetch_tool(url: str, user_id: str) -> dict:
    async with httpx.AsyncClient() as client:
        content = (await client.get(url)).text

    decision = await arcjet.guard(
        label="tools.fetch",
        metadata={"user_id": user_id},
        rules=[pi_rule(content)],
    )

    if decision.conclusion == "DENY":
        # Return a safe placeholder rather than the injected content
        return {"content": "[Content blocked: prompt injection detected]"}

    return {"content": content}

Learn more

• Prompt injection detection introduction
• Prompt injection detection quick start
• Sensitive info detection introduction
• Full sensitive info reference
• Agent guards – guard one tool call
• Framework integrations
• Vercel AI SDK agent guard
• LangChain agent guard
• LangChain JS agent guard