Rate Limits

Quotas, throttling, and best practices

2 min read

KLOAKD meters usage based on residential bandwidth consumption, not request counts. Each plan includes a monthly residential bandwidth allocation that covers all discovery, fetching, and extraction operations.

How metering works

Every operation that triggers a residential proxy fetch consumes bandwidth. The platform tracks this internally and enforces it at the gateway level. You never need to count requests or pages — just monitor your bandwidth usage in the dashboard.

Bandwidth allocation by plan

| Plan | Monthly residential bandwidth | Overage | |------|-------------------------------|---------| | Playground | Limited (free tier) | Not available — upgrade | | Pro | 10 GB | Pause until renewal or upgrade | | Developer | 30 GB | Pause until renewal or upgrade | | Enterprise | Custom | Custom |

See Pricing for full tier comparison.

Response headers

| Header | Description | |--------|-------------| | X-RateLimit-Limit | Bandwidth allocation in bytes | | X-RateLimit-Remaining | Remaining bandwidth in bytes | | X-RateLimit-Reset | Unix timestamp of monthly reset | | Retry-After | Seconds to wait (on 429 only) |

Handling 429

from kloakd.errors import RateLimitError

try:
    result = client.crawl("https://example.com", max_depth=2)
except RateLimitError as e:
    print(f"Bandwidth limit reached. Retry after {e.retry_after}s")

import { RateLimitError } from 'kloakd-sdk';

try {
  const result = await client.crawl('https://example.com', { maxDepth: 2 });
} catch (e) {
  if (e instanceof RateLimitError) {
    console.log(`Bandwidth limit reached. Retry after ${e.retryAfter}s`);
  }
}

Tier enforcement

When you exceed your monthly bandwidth allocation:

1. Gateway returns 429 Too Many Requests
2. Response includes Retry-After header (seconds until monthly reset)
3. SDKs automatically retry with exponential backoff (up to max_retries)
4. Operations resume automatically after the reset window

Best practices

• Use client.crawl() — the orchestrator reuses artifacts across discovery, fetching, and extraction, minimizing redundant residential traffic
• Use artifact chaining — one fetch() reused by extract() avoids re-fetching the same page
• Use extract_schema with crawl() — extraction runs on already-fetched HTML, zero additional bandwidth
• Use SSE streaming — crawlStream() and fetchStream() avoid polling overhead
• Limit crawl depth — max_depth=2 is usually enough for most sites