Design a Scalable API Rate Limiting Architecture

Get a complete technical blueprint for implementing API rate limiting using Redis, including data structures, tiered limit logic, and standard response headers.

Prompt

Prompt
Act as a Senior Backend Architect. Design a robust API rate limiting and throttling system. Before providing the design, ask me these three questions: 1. What is your primary programming language and web framework? 2. What is the expected request volume in requests per second during peak hours? 3. Do you have existing infrastructure like Redis or Nginx already in place? After I answer, provide the architecture following these rules: 1. Use the Token Bucket algorithm for throttling. 2. Define specific Redis data structures for tracking tokens and timestamps. 3. Include standardized HTTP response headers for rate limit status (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After). 4. Create distinct tiers for authenticated users and anonymous visitors. Output format: 1. Architecture Overview (a short paragraph). 2. Redis Data Structure Schema (key naming conventions and types). 3. Logic Flow (step-by-step processing for an incoming request). 4. Response Header Specification (table with name, description, and purpose). 5. Tiered Limit Strategy (table comparing anonymous vs authenticated users).

Example output

Example for: A Node.js Express API handling 500 requests per second with Redis available.

Architecture Overview

This design utilizes a centralized Redis store to maintain atomic state for rate limiting across multiple API instances. Each request triggers a Lua script in Redis to ensure thread-safe token consumption, preventing race conditions during high concurrency.

Redis Data Structure Schema

Keys follow the pattern: rate:limit:<user_id_or_ip>:<window>. We use the Hash data type to store two fields: tokens (Integer) and last_refill_timestamp (Integer). This allows constant-time access and updates per request.

Logic Flow
  1. Receive request and identify user via JWT or IP address.
  2. Execute Lua script to fetch the user key.
  3. Calculate elapsed time since last refill and add new tokens based on fill rate.
  4. If tokens >= 1, decrement and return status 200.
  5. If tokens < 1, return status 429 Too Many Requests.
Response Header Specification
HeaderDescriptionPurpose
X-RateLimit-LimitMax requests allowedInforms client of total capacity
X-RateLimit-RemainingRemaining tokensHelps client manage concurrency
X-RateLimit-ResetSeconds until resetTells client when to retry
Retry-AfterSeconds to waitStandard HTTP signal for 429 status
Tiered Limit Strategy
TierRequest LimitWindow Size
Anonymous60 requests1 minute
Authenticated2000 requests1 minute
Written by AI from this prompt. Your results will vary.

Tips and common edits

Why this works

  • Enforces a specific, industry-standard algorithm (Token Bucket) for consistency.
  • Forces the use of Lua scripts to ensure atomicity in high-concurrency Redis environments.
  • Structures output into clear, actionable sections: schema, logic, and headers.

Use this prompt when you need to architect a production-grade rate limiting system. It works best in ChatGPT, Claude, or Gemini. Answer the three questions clearly after pasting the prompt.

Common edits

  • Adjust the tiered limits table to match your specific business requirements.
  • Add a section on handling distributed denial-of-service (DDoS) scenarios if needed.

Make it repeatable

Keep using this prompt without pasting it in every time.

Keep exploring

More in Coding