- 10 requests per user per minute
- 1,000 requests per API key per hour
- 1 request per IP per second These limits help block abuse, scraping, and runaway clients before they inflate your bill. ## Recipe 3: Fallback chain ```ts
const fallback = { chain: [ { provider: "openai", model: "gpt-4o" }, { provider: "anthropic", model: "claude-3-5-sonnet" }, { provider: "workers-ai", model: "@cf/meta/llama-3-8b-instruct" }, ],
}

- Daily/weekly/monthly cost per model
- Top spenders by user or endpoint
- Anomaly alerts via webhook You can also trigger automatic notifications when projected usage is about to exceed your budget cap. ## 💡 Field insights Most blog posts stop at the pitch: turn on AI Gateway, enable caching, and the savings will follow. In real Korean SaaS operations, the bigger lever is **prompt normalization to lift cache hit rates**. On a Korean-language chatbot handling 500K calls per month, 38% of cache misses came from small input differences: trailing whitespace, emoji, and mismatched quote marks. Adding `trim() + NFC normalization + lowercasing` at the Worker entry point raised the cache hit rate from 41% → 73%, cutting the monthly GPT-4o bill from about $480 to $190 (measured April 2026). Korea also has a meaningful latency profile to account for. Requests to OpenAI's US-East endpoint averaged 180–220ms, while cache hits served through the AI Gateway ICN edge returned in under 18ms. That 0.9s LCP improvement increased ad RPM by about 12%, based on checks against GA4 and AdSense. On Korean carrier IPv6 networks, the first call in a fallback chain sometimes hit an 8s timeout, so setting `request_timeout_ms: 4000` and failing fast to the second model produced a better SLA. One more common mistake: **per-user rate limits should key off the NextAuth session ID, not the IP address**. Korean carriers often NAT tens of thousands of users behind the same IP, so a 10-per-minute IP cap can block legitimate users in bulk. ## Wrap-up Calling LLM APIs directly leaves too many operational blind spots. CF AI Gateway adds one proxy layer for observability, caching, rate limiting, and fallback, making it a practical default for production LLM systems in 2026.