gRPC Service Mesh for Internal APIs

Overview

When services call each other thousands of times per second, JSON-over-HTTP overhead and loose contracts become costly. gRPC offers binary Protocol Buffers serialization, HTTP/2 multiplexing, and strongly typed contracts generated from a schema. Running gRPC over a service mesh adds mTLS, retries, and observability without burdening application code. This architecture targets internal east-west traffic where performance and contract safety matter most.

Use it for latency-sensitive microservices, streaming RPCs, or polyglot fleets that need a shared, versioned contract language.

Components

• gRPC services: domain services exposing Protobuf-defined RPCs, generated into multiple languages.
• Linkerd: a lightweight sidecar mesh providing mTLS, transparent retries, load balancing, and golden metrics.
• Protobuf registry: stores and lints .proto schemas, enforcing backward compatibility in CI.
• Ingress gateway: translates external REST/JSON or gRPC-Web into internal gRPC.
• PostgreSQL: per-service relational storage.
• NATS: carries asynchronous events between services for non-request/response flows.
• Grafana: dashboards over mesh-emitted RED metrics.

Data Flow

External clients reach the ingress gateway, which authenticates and forwards to the appropriate gRPC service. Service-to-service calls travel through Linkerd sidecars that handle mTLS, load balancing, and retries. Long-lived streaming RPCs push incremental updates over HTTP/2. Side effects that do not need a synchronous reply are published to NATS for decoupled consumers.

Scaling and Resilience

HTTP/2 connection pooling and the mesh's per-request load balancing distribute traffic evenly across pods. Services scale horizontally on CPU and request concurrency. Linkerd applies retry budgets and timeouts so transient failures recover without amplifying load. Circuit breaking sheds traffic from unhealthy endpoints. Deadlines propagate across the call chain to prevent runaway latency.

Security

The mesh enforces automatic mTLS between all pods, so internal traffic is encrypted and identity-verified by default. The ingress gateway validates external tokens before any internal call. The Protobuf registry blocks incompatible schema changes, reducing the chance of contract drift between teams. Network policies restrict which services may call which RPCs.

Trade-offs and Alternatives

gRPC is harder to debug than JSON over HTTP, browser support requires gRPC-Web or a translation layer, and tooling is less ubiquitous. For public, browser-facing, or third-party APIs, REST or GraphQL remain better choices. The mesh adds per-pod resource overhead; very small systems may not justify it. gRPC shines specifically for internal, high-throughput, typed communication.