Rate Limiting and Anti-Automation Defenses

Rate Limiting and Anti-Automation Defenses - article cover image Web Exploitation
Time it takes to read this article 6 minutes.

Disclaimer: This article is provided strictly for educational purposes and authorized security testing. Only run these techniques against systems you own or have explicit written permission to assess. Unauthorized access to computer systems is illegal in virtually every jurisdiction and can carry severe penalties.

Introduction

Rate limiting and broader anti-automation controls exist to make high-volume, scripted abuse of an application uneconomical: credential stuffing against a login endpoint, content scraping, account enumeration, coupon/inventory abuse, and brute-force OTP guessing all depend on an attacker being able to send far more requests than a human ever would. OWASP’s Automated Threat Handbook catalogs these as distinct threat patterns (e.g. OAT-008 Credential Stuffing, OAT-007 Credential Cracking) precisely because each has its own detection signature and needs its own control, not a single generic “requests per minute” cap.

Rate limiting done well is layered and keyed on more than just source IP, because IP is the easiest signal for an attacker to rotate away from. Done poorly — a single global limit, keyed on a client-supplied or spoofable value, checked with a non-atomic counter — it either fails to stop the abuse it was built for or becomes a denial-of-service vector against legitimate users sharing the same key (e.g. NAT’d office IPs).

Attack Prerequisites

Bypassing a rate limit, or exploiting a gap in one, generally requires:

  • A limiting key that is attacker-controlled or trivially rotated — IP-only limiting defeated by a proxy/botnet pool, or a limit keyed on a client-supplied header rather than a value the server assigns.
  • No per-account or per-credential limiting, so distributing requests across many source IPs (low-and-slow credential stuffing) evades any single IP’s threshold entirely.
  • A non-atomic counter implementation — a check-then-increment pattern with a race window that lets concurrent requests all read the same “count” before any of them increments it, exceeding the intended limit.
  • No behavioral/device signal, so a request that presents valid syntax and stays under the numeric threshold is treated identically to a real user regardless of timing pattern or fingerprint.

How It Works

Rate limiting algorithms trade off burst tolerance against implementation simplicity. Fixed window counters (e.g. “100 requests per IP per 60-second window”) are simple but allow a burst of up to 2x the limit across a window boundary. Sliding window and leaky bucket algorithms smooth this by tracking requests continuously rather than resetting at fixed boundaries. Token bucket allows controlled bursts (a bucket refills at a steady rate and each request consumes a token) which suits APIs that see legitimate bursty traffic. All of these require the counter increment to be atomic — typically implemented with Redis INCR plus EXPIRE, or a Lua script that combines both operations so concurrent requests cannot race past the limit.

Effective anti-automation layers the limit at multiple points: an edge/CDN or WAF layer catches high-volume, obviously-automated traffic before it reaches the origin; an API gateway enforces per-API-key or per-client quotas; and the application layer enforces business-logic-aware limits (e.g. failed login attempts per account, regardless of source IP). Risk-based/adaptive throttling adds behavioral signals — request timing regularity, missing browser fingerprint entropy, known bad ASN/proxy ranges — to tighten limits for suspicious traffic without penalizing normal users.

The key an attacker cannot rotate is the one worth limiting on. IP address is trivial to rotate via residential proxy networks or botnets; an account identifier, a device fingerprint, or (for unauthenticated endpoints) a proof-of-work challenge is far more expensive to churn, which is why mature anti-automation stacks combine several keys rather than relying on IP alone.

Vulnerable Code / Configuration

A common real-world gap is an Nginx rate limit keyed on the client IP as seen by Nginx, deployed behind a load balancer or CDN without restricting which upstream is trusted to set X-Forwarded-For — an attacker who can reach Nginx directly, or whose proxy forwards a client-supplied header unmodified, can set an arbitrary value per request and get a fresh limit bucket every time:

# VULNERABLE: limits on $binary_remote_addr, but real_ip is derived
# from a header the client can set if any upstream trusts it blindly.
limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;

server {
    location /login {
        limit_req zone=login burst=2 nodelay;
        proxy_pass http://app_backend;
    }
}
# No `set_real_ip_from` / `real_ip_header` restricted to a known,
# trusted proxy CIDR -- if the attacker can hit this server directly,
# X-Forwarded-For is fully attacker-controlled input.
NGINX

Corrected: trust X-Forwarded-For only from a known proxy range, and add a second limit keyed on the submitted account/username so distributing across IPs does not bypass per-account protection:

set_real_ip_from 10.0.0.0/8;      # only the internal LB range
real_ip_header X-Forwarded-For;
real_ip_recursive on;

limit_req_zone $binary_remote_addr zone=login_ip:10m rate=5r/m;
# Application layer additionally enforces, e.g. via Redis:
#   INCR failed_login:{account_id}; EXPIRE ... ; lock after N in window
NGINX

Walkthrough / Exploitation

Assessing a rate limit as a pentester starts with identifying the actual limiting key:

# Does spoofing a forwarded-for header reset the limit?
for i in $(seq 1 20); do
  curl -s -o /dev/null -w "%{http_code}\n" \
    -H "X-Forwarded-For: 10.0.0.$i" \
    -d 'user=victim&pass=guess123' https://app.example/login
done
Bash

Next, test for a race condition in the counter by firing concurrent requests instead of sequential ones — many limits that hold under sequential testing fail under true concurrency because the check-then-increment is not atomic:

seq 1 50 | xargs -P50 -I{} curl -s -o /dev/null -w "%{http_code} " \
  -d 'user=victim&pass=guess{}' https://app.example/login
# If significantly more than the configured limit returns 200/401
# (rather than 429), the increment is not atomic.
Bash

Finally, test whether username enumeration is possible via a timing or response-content side channel that the rate limit itself does not address — a different lockout message or response latency for valid versus invalid usernames leaks account existence independent of the request volume control.

Note: Aggressive account lockout after N failed attempts is itself abusable as a denial-of-service against legitimate users if the lockout key is just the username — an attacker can lock any known account by deliberately failing logins for it. Prefer progressive delays or CAPTCHA challenges over hard lockout for public-facing authentication endpoints.

Opsec: When load-testing rate limits during an authorized assessment, coordinate volume and timing with the target — concurrent-request tests deliberately designed to find race conditions can degrade service if run against a production system without warning.

Detection and Defense

  • Use atomic counter operations (Redis INCR+EXPIRE or a Lua script, not separate GET/SET calls) to close race-condition bypasses.
  • Layer limits by multiple keys — IP, account/credential, API key, and device fingerprint — so rotating any single one does not fully evade protection.
  • Only trust forwarded-IP headers from known proxy infrastructure; never derive the rate-limit key from a header the client can set directly.
  • Prefer progressive friction over hard lockout for authentication endpoints — increasing delay, CAPTCHA, or step-up verification instead of an account-wide lock an attacker can trigger at will.
  • Monitor for low-and-slow distributed patterns at the SIEM/WAF level — many source IPs each staying under the per-IP threshold but collectively hammering one account or endpoint.

Real-World Impact

Credential stuffing, enabled by the enormous volume of credentials circulating from unrelated prior breaches, is one of the highest-volume automated threats web applications face and is explicitly cataloged by OWASP’s Automated Threat Handbook precisely because naive IP-based rate limiting alone does not stop it — botnets and residential proxy services exist specifically to defeat that single control. Applications that layer per-account throttling, breached-credential checks at login, and behavioral signals see materially lower automated account-takeover rates than those relying on a single IP-based limit.

Conclusion

A rate limit is only as strong as the key it is enforced on and the atomicity of the counter behind it — IP-only, non-atomic implementations are routinely bypassed by rotation or concurrency. Layer limits across IP, account, and device signals, verify the counter is race-free under real concurrent load, and prefer progressive friction to hard lockouts that an attacker can weaponize against your own users.

You Might Also Like

If you found this useful, these related deep-dives cover adjacent techniques and their defenses:

Comments

Copied title and URL