Rate limit
In short
A rate limit is a cap on how much you can use an online service in a set period, such as requests or tokens per minute. AI companies use rate limits to stop abuse and share computing capacity fairly between customers.
- Also called
- Rate limiting, API rate limit, 429 error, Too Many Requests
- Topic
- Using AI tools
- Level
- Beginner
- First described
- The standard web signal for hitting a rate limit, HTTP status 429 Too Many Requests, was defined by Mark Nottingham and Roy Fielding in RFC 6585, published in April 20123.
OpenAI describes rate limits as restrictions on how many times a user or client can access its services within a specified period1. For AI models the period is usually a minute or a day, and the unit can be requests or tokens, the chunks of text models read and write. OpenAI counts requests per minute and per day, tokens per minute and per day, and images per minute, and you are stopped by whichever cap you hit first1. Anthropic limits requests per minute plus input and output tokens per minute, separately for each model2.
The stated reasons are to prevent abuse, keep access fair, and cope with spikes in demand12.
Go over a limit and the API refuses the request with HTTP status 429, "Too Many Requests"3. The reply often carries a Retry-After header saying how long to wait32. OpenAI recommends retrying with exponential backoff, waiting longer after each failure and adding some randomness so that many clients do not all retry at once1.
Limits grow with use. Both companies sort accounts into usage tiers and move them up automatically as they spend more or build up history12. Anthropic uses a "token bucket" method, so capacity refills continuously rather than resetting at the top of each minute2. A rate limit is not a spend limit: Anthropic sets those separately, as a cap on monthly cost2.
Example
Anthropic notes that a limit of 60 requests per minute might be enforced as one request per second2. So a UK start-up that fires off 20 requests in the same instant can get 429 errors, even though it is well under 60 for the minute.
Why it matters
It is why an AI app sometimes says "too many requests, try again later". For businesses building on someone else's model, rate limits decide how many customers they can serve at once.
Questions people ask
What does error 429 mean?
It means "Too Many Requests": you have sent more requests than the service allows in a given amount of time3. Both OpenAI and Anthropic return it when a rate limit is exceeded12.
How do I fix a rate limit error?
Wait and retry. Follow the Retry-After header if there is one, otherwise back off for longer after each failure. OpenAI notes that failed requests still count towards per-minute limits, so instantly resending the same request does not help1. For lasting problems, move to a higher usage tier.
Do cached prompts count towards Claude's rate limits?
For most Claude models, no. Anthropic says input tokens read from its prompt cache do not count towards the input-tokens-per-minute limit, so caching long, repeated instructions raises effective throughput2.