Introduction
Kubernetes networking can feel like navigating a maze — pods communicate across nodes, services abstract away dynamic IP addresses, and external traffic needs a way into the cluster. Two critical components solve these challenges: Ingress controllers manage how external HTTP/HTTPS traffic reaches your services, while service meshes handle the complex web of internal service-to-service communication.
Understanding the difference between Ingress and service mesh is crucial for building production-ready Kubernetes deployments. Ingress operates at Layer 7 (HTTP/HTTPS), routing external requests based on hostnames and paths. Service meshes operate at Layer 4-7, managing internal traffic with features like mutual TLS, circuit breaking, retries, and observability — all without changing application code.
This guide provides a deep dive into both technologies, covering architecture, configuration, real-world deployment patterns, and how they complement each other in modern microservice architectures. Whether you're running a simple web application or a complex microservices platform with hundreds of services, understanding these networking primitives is essential for building reliable, secure, and observable systems.
Understanding Kubernetes Networking: Core Concepts
Before diving into Ingress and service meshes, it's essential to understand Kubernetes' networking model. Every pod gets its own IP address, and pods can communicate with each other directly without NAT. Services provide stable virtual IPs that route to a set of pods, and DNS-based service discovery lets applications find each other by name.
The Service Abstraction
Kubernetes Services are the foundation of internal networking. They provide a stable endpoint for a set of pods using label selectors.
# ClusterIP Service — internal only
apiVersion: v1
kind: Service
metadata:
name: user-service
namespace: production
spec:
selector:
app: user-service
ports:
- port: 80
targetPort: 8080
protocol: TCP
type: ClusterIP
---
# NodePort Service — exposes on each node's IP
apiVersion: v1
kind: Service
metadata:
name: user-service-nodeport
spec:
selector:
app: user-service
ports:
- port: 80
targetPort: 8080
nodePort: 30080
type: NodePort
---
# LoadBalancer Service — cloud provider load balancer
apiVersion: v1
kind: Service
metadata:
name: user-service-lb
spec:
selector:
app: user-service
ports:
- port: 80
targetPort: 8080
type: LoadBalancerThe Problem Ingress Solves
Without Ingress, exposing multiple services externally requires either a LoadBalancer service per application (expensive on cloud providers) or NodePort assignments (hard to manage, non-standard ports). Ingress consolidates HTTP routing into a single entry point.
Ingress Controllers: The Gateway to Your Cluster
An Ingress resource defines routing rules, but it needs an Ingress Controller to actually implement them. The controller is a reverse proxy running inside the cluster that watches for Ingress resources and configures itself accordingly.
NGINX Ingress Controller
The NGINX Ingress Controller is the most widely deployed option, used by over 60% of Kubernetes clusters. It's battle-tested, well-documented, and supports a rich set of annotations for configuration.
# Install NGINX Ingress Controller with Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--set controller.replicaCount=2 \
--set controller.metrics.enabled=true \
--set controller.podAnnotations."prometheus\.io/scrape"="true" \
--set controller.resources.requests.cpu=100m \
--set controller.resources.requests.memory=128Mi \
--set controller.config.use-gzip="true" \
--set controller.config.gzip-level="5" \
--set controller.config.proxy-body-size="50m" \
--set controller.config.proxy-connect-timeout="60" \
--set controller.config.proxy-read-timeout="60"
# Verify installation
kubectl get pods -n ingress-nginx
kubectl get svc -n ingress-nginxNGINX Ingress strengths include mature annotation-based configuration, extensive rate limiting, custom error pages, and robust TLS handling. Weaknesses include annotation sprawl (configuration scattered across YAML annotations rather than structured resources), static configuration requiring reloads, and limited dynamic reconfiguration.
Traefik Ingress Controller
Traefik differentiates itself with dynamic configuration via CRDs (IngressRoute) and built-in Let's Encrypt integration. Unlike NGINX, Traefik can update its configuration without restarting or reloading.
# Traefik IngressRoute — cleaner than NGINX annotations
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: api-route
namespace: production
spec:
entryPoints:
- websecure
routes:
- match: Host(`api.example.com`) && PathPrefix(`/v1`)
kind: Rule
services:
- name: api-v1
port: 80
middlewares:
- name: rate-limit
- name: compress
- match: Host(`api.example.com`) && PathPrefix(`/v2`)
kind: Rule
services:
- name: api-v2
port: 80
tls:
certResolver: letsencrypt
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: rate-limit
spec:
rateLimit:
average: 100
burst: 200
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: compress
spec:
compress: {}Traefik's key strengths: automatic service discovery, built-in Let's Encrypt ACME, and a real-time dashboard. It handles TCP/UDP alongside HTTP, making it versatile for non-HTTP workloads like databases or gRPC.
Envoy Gateway
Envoy Gateway is the newest contender, built on the Envoy proxy that powers most service meshes. It implements the Kubernetes Gateway API natively, making it the most future-proof option.
# Install Envoy Gateway
helm install eg oci://docker.io/envoyproxy/gateway-helm \
--version v1.0.0 \
-n envoy-gateway-system \
--create-namespaceEnvoy Gateway advantages include native Gateway API support, high performance (Envoy is written in C++), extensibility via WASM filters, and seamless integration with service meshes. It's the best choice for teams planning to adopt a service mesh later.
Controller Comparison
| Feature | NGINX Ingress | Traefik | Envoy Gateway |
|---|---|---|---|
| Configuration | Annotations | CRDs | Gateway API |
| Dynamic Updates | Reload required | Automatic | Automatic |
| TLS Management | cert-manager | Built-in ACME | cert-manager |
| Rate Limiting | Built-in | Middleware | Built-in |
| TCP/UDP Support | Yes | Yes | Yes |
| Service Mesh Integration | Manual | Manual | Native (Istio) |
| Dashboard | None | Built-in | None |
| Complexity | Low | Medium | Medium |
| Best For | Simple HTTP routing | Dynamic config | Future-proofing |
Ingress Resource Deep Dive
Path and Host-Based Routing
The Ingress resource defines how external traffic reaches internal services. The most common patterns are path-based and host-based routing.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: multi-app-ingress
namespace: production
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "50m"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"
nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
nginx.ingress.kubernetes.io/rate-limit: "100"
nginx.ingress.kubernetes.io/rate-limit-window: "1m"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-origin: "https://example.com"
spec:
ingressClassName: nginx
tls:
- hosts:
- app.example.com
- api.example.com
secretName: tls-secret
rules:
# Host-based routing
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: frontend-service
port:
number: 80
# Path-based routing
- host: api.example.com
http:
paths:
- path: /users
pathType: Prefix
backend:
service:
name: user-service
port:
number: 80
- path: /orders
pathType: Prefix
backend:
service:
name: order-service
port:
number: 80
- path: /api/v2
pathType: Prefix
backend:
service:
name: api-v2-service
port:
number: 80Path types matter more than most people realize:
- Prefix: Matches based on URL path prefix (e.g.,
/apimatches/api/users,/api/orders) - Exact: Matches the URL path exactly (e.g.,
/api/usersdoes NOT match/api/users/123) - ImplementationSpecific: Delegates to the Ingress controller's default behavior
TLS Termination and Certificate Management
TLS termination at the Ingress level means all internal traffic can use plain HTTP, simplifying service configuration. cert-manager automates certificate lifecycle.
# Install cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.0/cert-manager.yaml# ClusterIssuer for Let's Encrypt
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
---
# Certificate resource
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: app-tls
namespace: production
spec:
secretName: app-tls-secret
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
dnsNames:
- app.example.com
- api.example.comcert-manager handles automatic renewal 30 days before expiry. For wildcard certificates, use DNS-01 challenge instead of HTTP-01, which requires DNS provider integration (Route53, Cloudflare, etc.).
Service Mesh Architecture
A service mesh adds a sidecar proxy to every pod. These sidecars intercept all network traffic, enabling features like mutual TLS, traffic shaping, and observability without application changes.
The Sidecar Pattern
Pod A Pod B
┌──────────────────┐ ┌──────────────────┐
│ App Container │ │ App Container │
│ (port 8080) │ │ (port 8080) │
│ ↕ │ │ ↕ │
│ Envoy Sidecar │ ←→ │ Envoy Sidecar │
│ (port 15001) │ │ (port 15001) │
└──────────────────┘ └──────────────────┘
↕ ↕
┌────────────────────────────────┐
│ Istio Control Plane │
│ (istiod: Pilot, Citadel) │
└────────────────────────────────┘
The sidecar proxy intercepts all inbound and outbound traffic using iptables rules. When your application makes an HTTP request to user-service:80, the sidecar intercepts it, applies routing rules, adds tracing headers, enforces mTLS, and forwards the request to the destination sidecar. The application code remains completely unaware.
Istio: The Industry Standard
Istio consists of two planes:
- Data Plane: Envoy sidecar proxies that handle all traffic between services
- Control Plane: istiod (Pilot + Citadel + Galley) that configures the proxies, manages certificates, and validates configurations
# Download and install Istio
curl -L https://istio.io/downloadIstio | sh -
cd istio-*
export PATH=$PWD/bin:$PATH
# Install with default profile
istioctl install --set profile=default -y
# Verify installation
istioctl verify-install
kubectl get pods -n istio-system
# Enable sidecar injection for a namespace
kubectl label namespace production istio-injection=enabledLinkerd: The Lightweight Alternative
Linkerd is a lightweight, Rust-based service mesh that prioritizes simplicity and performance. Where Istio offers maximum configurability, Linkerd offers minimal operational overhead.
# Install Linkerd CLI
curl -fsL https://run.linkerd.io/install | sh
export PATH=$HOME/.linkerd2/bin:$PATH
# Install Linkerd control plane
linkerd install --crds | kubectl apply -f -
linkerd install | kubectl apply -f -
# Verify installation
linkerd check
# Inject sidecars into a namespace
kubectl get deploy -n production -o yaml | linkerd inject - | kubectl apply -f -Linkerd's key advantages:
- Smaller footprint: The Linkerd sidecar uses ~10MB memory vs. Envoy's ~50MB
- Faster startup: Linkerd proxy starts in milliseconds vs. seconds for Envoy
- Simpler operations: Fewer CRDs, simpler configuration, easier debugging
- Rust-based proxy: Memory-safe, no CVEs from buffer overflows
Linkerd limitations:
- Fewer advanced traffic management features than Istio
- Smaller ecosystem and community
- Less flexible for complex multi-cluster scenarios
Cilium: eBPF-Powered Service Mesh
Cilium takes a fundamentally different approach using eBPF (extended Berkeley Packet Filter) to implement networking, security, and observability directly in the Linux kernel, bypassing sidecar proxies entirely.
# Install Cilium
cilium install
cilium status
# Enable service mesh features
cilium upgrade --set kubeProxyReplacement=true \
--set hubble.enabled=true \
--set hubble.relay.enabled=true \
--set hubble.ui.enabled=trueCilium advantages:
- No sidecars: Networking and security happen in the kernel, not in per-pod proxies
- Lower latency: Kernel-level processing is faster than userspace proxies
- Better resource efficiency: No per-pod memory overhead for sidecars
- Hubble: Built-in observability platform with network flow visualization
Cilium is the foundation of Google's GKE Dataplane V2 and is gaining rapid adoption for teams that want service mesh capabilities without the operational overhead of sidecar management.
Mesh Comparison
| Feature | Istio | Linkerd | Cilium |
|---|---|---|---|
| Proxy | Envoy (C++) | linkerd2-proxy (Rust) | eBPF (kernel) |
| Sidecar Memory | ~50MB per pod | ~10MB per pod | 0 (kernel-level) |
| mTLS | Yes | Yes | Yes |
| Traffic Management | Advanced | Basic | Moderate |
| Multi-cluster | Yes | Yes | Yes |
| Observability | Kiali, Jaeger | Grafana dashboards | Hubble |
| Complexity | High | Low | Medium |
| Best For | Enterprise, complex routing | Simplicity, performance | High-performance, kernel-level |
Mutual TLS and Security
Enabling mTLS
Mutual TLS ensures both the client and server verify each other's identity. In a service mesh, this happens automatically — the sidecar proxies handle certificate exchange without any application code changes.
# PeerAuthentication — enforce mTLS across the mesh
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: production
spec:
mtls:
mode: STRICT
---
# AuthorizationPolicy — fine-grained access control
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: user-service-policy
namespace: production
spec:
selector:
matchLabels:
app: user-service
rules:
- from:
- source:
principals: ["cluster.local/ns/production/sa/api-gateway"]
- to:
- operation:
methods: ["GET", "POST"]
paths: ["/api/users/*"]mTLS modes:
- STRICT: All traffic must be mTLS encrypted. Plain HTTP is rejected.
- PERMISSIVE: Accepts both mTLS and plain HTTP. Use during migration.
- DISABLE: No mTLS. Only for legacy services that can't support it.
- UNSET: Inherits from parent scope (namespace or mesh level).
Certificate Rotation
Istio's Citadel component automatically rotates certificates every 24 hours. The sidecar proxies hot-reload new certificates without dropping connections. This means you get automatic certificate rotation with zero downtime — a significant operational advantage over manual certificate management.
Traffic Management
Canary Deployments
Canary deployments route a small percentage of traffic to a new version, allowing you to validate changes before full rollout.
# VirtualService — fine-grained traffic routing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: user-service-vs
namespace: production
spec:
hosts:
- user-service
http:
# Header-based routing for internal testing
- match:
- headers:
x-canary:
exact: "true"
route:
- destination:
host: user-service
subset: canary
weight: 100
# Weight-based canary
- route:
- destination:
host: user-service
subset: stable
weight: 90
- destination:
host: user-service
subset: canary
weight: 10
retries:
attempts: 3
perTryTimeout: 2s
retryOn: 5xx,reset,connect-failure
timeout: 10s
---
# DestinationRule — defines subsets and traffic policies
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: user-service-dr
namespace: production
spec:
host: user-service
trafficPolicy:
connectionPool:
tcp:
maxConnections: 100
http:
h2UpgradePolicy: DEFAULT
http1MaxPendingRequests: 100
http2MaxRequests: 1000
outlierDetection:
consecutive5xxErrors: 5
interval: 30s
baseEjectionTime: 30s
maxEjectionPercent: 50
loadBalancer:
simple: LEAST_REQUEST
subsets:
- name: stable
labels:
version: stable
- name: canary
labels:
version: canaryBlue-Green Deployments
Blue-green deployments maintain two identical environments and switch traffic instantly.
# Blue-green with Istio
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: order-service-vs
spec:
hosts:
- order-service
http:
- route:
- destination:
host: order-service
subset: green
weight: 100
---
# Switch to blue by changing weight:
# subset: blue weight: 100
# subset: green weight: 0A/B Testing
A/B testing routes specific user segments to different versions based on headers, cookies, or other request attributes.
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: frontend-vs
spec:
hosts:
- frontend
http:
# Route beta users to new UI
- match:
- headers:
x-user-group:
exact: "beta"
route:
- destination:
host: frontend
subset: v2
# Everyone else gets stable
- route:
- destination:
host: frontend
subset: v1Circuit Breaking and Resilience
Circuit Breaker Pattern
Circuit breakers prevent cascading failures by stopping requests to unhealthy services.
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: payment-service-dr
spec:
host: payment-service
trafficPolicy:
outlierDetection:
# Eject hosts returning 5xx errors
consecutive5xxErrors: 3
interval: 10s
baseEjectionTime: 30s
maxEjectionPercent: 50
connectionPool:
tcp:
maxConnections: 100
connectTimeout: 5s
http:
http1MaxPendingRequests: 50
http2MaxRequests: 100
maxRequestsPerConnection: 10
maxRetries: 3When a service starts returning errors, the circuit breaker ejects it from the load balancing pool. After the ejection period, it allows a few requests through to test recovery. This prevents one failing service from consuming all resources and bringing down dependent services.
Retry Policies
Retries handle transient failures automatically, but must be configured carefully to avoid amplifying problems.
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: api-retry-policy
spec:
hosts:
- api-service
http:
- route:
- destination:
host: api-service
retries:
attempts: 3
perTryTimeout: 2s
retryOn: 5xx,reset,connect-failure,retriable-4xxRetry budgets prevent retry storms. If 20% of requests are already retries, stop retrying to avoid overwhelming the failing service. Istio's outlier detection combined with retries creates a robust resilience layer.
Observability: Tracing, Metrics, and Kiali
Service mesh observability requires three pillars: metrics, traces, and topology visualization.
# Install Kiali + Jaeger + Prometheus addon
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.21/samples/addons/kiali.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.21/samples/addons/jaeger.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.21/samples/addons/prometheus.yaml
# Access Kiali dashboard
istioctl dashboard kialiKiali provides a real-time service topology graph showing request flow, latency, error rates, and mTLS status between all services. Jaeger traces individual requests across service boundaries, revealing bottlenecks and latency sources invisible in per-service metrics.
For Ingress-level observability, enable Prometheus metrics on the NGINX Ingress controller:
controller:
metrics:
enabled: true
serviceMonitor:
enabled: true
namespace: monitoring
customHeaders: "X-Request-ID:$req_id"Key metrics to monitor:
| Metric | What It Tells You |
|---|---|
nginx_ingress_controller_requests | Request rate, status codes per ingress |
nginx_ingress_controller_request_duration_seconds | Latency distribution per route |
nginx_ingress_controller_connections | Active connections, connection rate |
istio_requests_total | Mesh-level request rate with source/destination labels |
istio_request_duration_milliseconds | Mesh-level latency with full attribution |
Gateway API: The Future of Kubernetes Ingress
The Kubernetes Gateway API is the successor to Ingress, providing a more expressive, role-oriented, and extensible approach to traffic management.
# GatewayClass — infrastructure provider defines this
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: istio
spec:
controllerName: istio.io/gateway-controller
---
# Gateway — cluster operator configures this
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: production-gateway
namespace: istio-system
spec:
gatewayClassName: istio
listeners:
- name: https
protocol: HTTPS
port: 443
tls:
mode: Terminate
certificateRefs:
- name: wildcard-tls
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
gateway-access: "true"
---
# HTTPRoute — application developer configures this
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: api-routes
namespace: production
spec:
parentRefs:
- name: production-gateway
namespace: istio-system
hostnames:
- "api.example.com"
rules:
- matches:
- path:
type: PathPrefix
value: /v1
backendRefs:
- name: api-v1
port: 80
weight: 90
- name: api-v2
port: 80
weight: 10
- matches:
- path:
type: PathPrefix
value: /v2
backendRefs:
- name: api-v2
port: 80The key advantage is role separation: infrastructure teams manage GatewayClass and Gateway, while application teams manage HTTPRoute. This eliminates the annotation sprawl problem that plagues NGINX Ingress in large clusters.
Real-World Architecture Patterns
Pattern 1: Ingress + Service Mesh
The most common production pattern uses Ingress for external traffic and service mesh for internal traffic.
Internet → Cloud LB → Ingress Controller → Service → Sidecar → App
(mesh boundary)
The Ingress controller handles TLS termination, path routing, and rate limiting for external traffic. Once traffic enters the mesh, sidecar proxies handle mTLS, circuit breaking, retries, and observability for all internal communication.
Pattern 2: Ingress at the Edge, Mesh for Internal
External traffic: Internet → NGINX Ingress → Frontend Service
Internal traffic: Frontend → (mesh) → API → (mesh) → Database
This pattern keeps the Ingress layer simple (just routing) while the mesh handles all the complex internal traffic management.
Pattern 3: Multi-Cluster with Federation
Cluster A (US) ←→ Istio East-West Gateway ←→ Cluster B (EU)
↕ ↕
Services Services
For global deployments, service meshes enable cross-cluster communication with mTLS and consistent policy enforcement. Istio's multi-cluster support allows services in different clusters to communicate as if they were in the same cluster.
Production Best Practices
-
Always enable TLS: Use cert-manager with Let's Encrypt for automatic certificate management. Redirect HTTP to HTTPS at the Ingress level.
-
Set resource limits on Ingress controllers: Ingress controllers are critical infrastructure. Set CPU and memory limits, run multiple replicas, and use pod disruption budgets.
-
Use NetworkPolicies alongside Ingress: Ingress handles Layer 7 routing, but NetworkPolicies provide Layer 3/4 isolation. Use both for defense in depth.
-
Monitor Ingress metrics: Enable Prometheus metrics on your Ingress controller. Track request rates, latency percentiles, error rates, and active connections.
-
Start with mTLS in permissive mode: When introducing a service mesh, start with
PERMISSIVEmTLS mode to allow both encrypted and unencrypted traffic, then gradually move toSTRICTmode. -
Use VirtualServices for retries and timeouts: Define retry policies and timeouts at the mesh level rather than in application code. This provides consistent behavior across all services.
-
Implement distributed tracing: Istio integrates with Jaeger and Zipkin for distributed tracing. This is invaluable for debugging latency issues in microservice architectures.
-
Use Gateway resources for edge traffic: Istio's Gateway resource controls load balancing at the mesh edge, similar to Ingress but with Istio-native features.
Common Pitfalls and Solutions
| Pitfall | Impact | Solution |
|---|---|---|
| No Ingress controller installed | Ingress resources have no effect | Always install an Ingress Controller before creating Ingress resources |
| Missing TLS configuration | Traffic sent in plaintext | Configure TLS secrets and cert-manager for automatic renewal |
| Service mesh sidecar injection disabled | Services bypass mesh, no mTLS | Label namespaces with istio-injection=enabled and verify sidecars |
| Incorrect path matching | Routes return 404 or hit wrong service | Use pathType: Prefix carefully; test with exact paths first |
| Overly permissive AuthorizationPolicies | Security vulnerabilities | Follow least-privilege principle; define explicit policies per service |
| Not setting circuit breakers | Cascading failures under load | Configure outlier detection in DestinationRules for all services |
| Retry storms | Amplified failures during outages | Set retry budgets and limit retry attempts |
| Sidecar resource consumption | Memory overhead per pod | Tune proxy resources; consider ambient mesh for large clusters |
Performance Optimization
Service mesh adds latency due to sidecar proxies. Here's how to minimize the impact:
# Tune Istio proxy resources
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
defaultConfig:
concurrency: 2
holdApplicationUntilProxyStarts: true
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 256MiFor latency-sensitive services, consider disabling sidecar injection for specific pods and using Istio's ambient mesh mode, which moves proxy functionality to the node level instead of per-pod sidecars.
Testing Your Ingress and Mesh
Testing Ingress and service mesh configurations requires validating routing, TLS, canary distribution, and mTLS status.
# Test Ingress routing
curl -H "Host: app.example.com" http://<INGRESS_IP>/
curl -H "Host: api.example.com" http://<INGRESS_IP>/users
# Test TLS
curl -v https://app.example.com/
# Test canary routing — verify weight distribution
for i in $(seq 1 100); do
curl -s https://api.example.com/version | jq .version
done | sort | uniq -c
# Verify mTLS is active between services
istioctl authn tls-check user-service.production.svc.cluster.local
# Check sidecar injection
kubectl get pods -n production -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.containers[*]}{.name}{" "}{end}{"\n"}{end}'
# View mesh topology
istioctl dashboard kiali
# Analyze Istio configuration for errors
istioctl analyze -n production
# Debug Envoy configuration for a specific pod
istioctl proxy-config routes <pod-name> -n production
istioctl proxy-config clusters <pod-name> -n production
istioctl proxy-config listeners <pod-name> -n productionFor load testing your Ingress and mesh, use tools like hey, wrk, or k6 to simulate traffic patterns and validate that rate limiting, circuit breaking, and retry policies behave as expected under load. Always test canary deployments with synthetic traffic before routing real users to a new version.
Ambient Mesh: The Next Evolution
Istio's ambient mesh mode eliminates sidecar proxies, reducing resource overhead and operational complexity. Instead of injecting an Envoy sidecar into every pod, ambient mesh uses two distinct components that separate L4 and L7 concerns:
- ztunnel: A per-node daemon that handles L4 networking, mTLS, and basic traffic routing
- Waypoint proxies: Per-namespace or per-service proxies that handle L7 features like routing, retries, and observability
This architecture reduces per-pod memory overhead from ~50MB (Envoy sidecar) to nearly zero for L4-only workloads. It also eliminates the init container that hijacks iptables rules, simplifying pod startup and reducing the blast radius of proxy failures. Ambient mesh is currently in beta and represents the future direction of service mesh architecture for teams that want mesh capabilities without the operational complexity of sidecar management. As ambient mesh matures, it will likely become the default deployment model for most service mesh adopters.
Conclusion
Ingress controllers and service meshes are complementary technologies that solve different networking challenges in Kubernetes. Ingress handles external HTTP/HTTPS traffic routing with path and host-based rules. Service meshes manage internal service-to-service communication with mTLS, traffic shaping, and observability.
Key takeaways:
- Use Ingress for external HTTP routing — it's simpler and sufficient for most use cases
- Add a service mesh when you need mTLS, advanced traffic management, or deep observability
- Always enable TLS with automatic certificate management via cert-manager
- Start with permissive mTLS mode and gradually enforce strict encryption
- Monitor everything — Ingress metrics, mesh metrics, and distributed traces
- Use NetworkPolicies for network-level isolation alongside Ingress and mesh
- The Gateway API is the future — plan your migration from Ingress to Gateway API
- Choose the right mesh: Istio for enterprise complexity, Linkerd for simplicity, Cilium for performance