Tim's Wiki

API Gateway

Properties2
descriptionEntry-point service that routes, secures, and governs API traffic between clients and backend systems.
tagstech, cloud, mature

3 min read

API Gateway

Definition

An API gateway is a managed entry point that accepts client requests and routes them to backend systems. It centralizes cross-cutting concerns such as authentication, throttling, request transformation, and protocol handling.

Core Ideas

Common Responsibilities

  • route requests to services or functions
  • enforce authentication and authorization
  • apply throttling and usage controls
  • handle CORS and header transformations
  • support monitoring and request tracing

Why It Matters

In distributed and serverless systems, the gateway becomes the public edge of the platform. It separates clients from backend topology and lets teams evolve internal systems without exposing every change externally.

In Serverless Systems

The source notes repeatedly connect API Gateway with:

  • Lambda invocations
  • direct integrations to AWS services
  • asynchronous handoff to SQS for long-running work
  • WebSocket support for status updates

Tiered Rate Limiting at the Gateway

A worked design for consolidating plan-based limits in the gateway rather than in each service:

  • One base limiter for everyone, including paid tiers (e.g. 1000/min, 20000/day) as a hard ceiling.
  • Per-tier multipliers below the base: free 100/min · 2000/day, beta 800/min · 16000/day, paid inherits the base.
  • Internal callers bypass entirely via a server key + internal tier — first-party server-side traffic is not rate limited.
  • Per-endpoint extras stack on top of the tier limit, not instead of it:
    • resource-heavy data endpoints get a ticker cap (e.g. 10 / 100 / ∞ per month by tier)
    • historical endpoints get a date-range cap (free 1 year, beta 5 years, paid full history)
    • upstream-proxied endpoints (realtime quotes, news, sentiment) get a flat low limit across all tiers because cost is external
    • LLM endpoints skip request limiting and count tokens instead
  • Edge burst limiting is separate (e.g. 20 req/sec per IP) and applies to every public request.
  • Keying: user_<sub> for authenticated sessions, key_<hash> for API keys.

Endpoints served outside the gateway (edge workers, admin UI behind a session) sit outside this scheme — which is itself an argument for keeping the tier/limit table in one place.

The design tension worth noting: the simpler the tier vocabulary (free / beta / paid), the easier the limiter is to reason about. Plan proliferation from product history is what makes gateway limit logic unmaintainable.

Trade-Offs

  • too much logic in the gateway can create a new bottleneck
  • shared gateways can become governance choke points
  • synchronous integrations need careful attention to rate limits and failure handling

Relationships

  • RESTful API — API gateways often expose RESTful interfaces
  • Serverless — gateways are a common HTTP entry layer for serverless systems
  • Event-Driven Architecture — a gateway can accept requests and then hand them off asynchronously through queues or topics
  • Backend for Frontend — a BFF may sit behind or alongside an API gateway to tailor client-facing behavior
  • Serverless on AWS — AWS API Gateway appears throughout the serverless architecture notes

References

  • Architecting Serverless Solutions on AWS
  • AWS SAP C02

Graph View

Backlinks