Fix: Multiple Nginx Pods Port Conflict on Same K8s Node

In high-performance Kubernetes environments, such as those running on AWS EKS or bare-metal Linux clusters, architects often attempt to bypass the latency of the kube-proxy by using hostNetwork: true or specific hostPort mappings.

This architectural choice creates a bottleneck: because the Nginx process attempts to bind directly to the Node’s network interface (the OS kernel’s IP stack), only the first Pod to initialize can successfully claim the port (usually 80 or 443). Subsequent Pods on the same node will enter a CrashLoopBackOff state or remain in Running but with a failed internal Nginx process.

The Exact Error Log

When you inspect the logs of the failing Nginx Pods using kubectl logs <pod-name>, you will see the following output from the Nginx master process:

2023/10/27 14:30:15 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2023/10/27 14:30:15 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
2023/10/27 14:30:15 [emerg] 1#1: still could not bind()

If you describe the pod, the events will indicate a back-off:

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  1m                 default-scheduler  Successfully assigned default/nginx-67k2 to ip-10-0-1-50.ec2.internal
  Normal   Pulled     50s                kubelet            Container image "nginx:latest" already present on machine
  Normal   Created    45s                kubelet            Created container nginx
  Normal   Started    45s                kubelet            Started container nginx
  Warning  BackOff    10s (x3 over 40s)  kubelet            Back-off restarting failed container

Diagnostic Checklist

Verify Host Network Usage: Check if the deployment manifest explicitly requests the host’s IP namespace.
Terminal window
```
kubectl get pod <pod-name> -o jsonpath='{.spec.hostNetwork}'
```
Check Port Bindings on Node: SSH into the specific Kubernetes worker node and identify which process owns the port.
Terminal window
```
# Run on the worker node OS
sudo ss -tulpn | grep :80
```
Inspect hostPort Configurations: Even without hostNetwork, a hostPort definition will prevent multiple pods from using the same port on a single IP.
Terminal window
```
kubectl get pod <pod-name> -o jsonpath='{.spec.containers[*].ports[*].hostPort}'
```
Resource Contention: Ensure multiple instances aren’t trying to mount the same hostPath volume for PID files or logs, which can cause locking issues on the underlying Linux filesystem (EXT4/XFS).

The Fix

Option 1: Remove Host Networking (Recommended)

The most stable fix is to use standard Kubernetes networking. Remove hostNetwork: true and hostPort from your YAML. This allows the CNI (like Amazon VPC CNI or Calico) to assign unique IP addresses to every Pod.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
        ports:
        - containerPort: 80  # Remove hostPort here
      # hostNetwork: true    # Delete or set to false

Option 2: Implement Pod Anti-Affinity

If your architecture strictly requires hostNetwork or hostPort (e.g., for certain Ingress Controllers or specialized telco workloads), you must ensure that Kubernetes never schedules more than one Nginx Pod on the same node.

spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - nginx
        topologyKey: "kubernetes.io/hostname"

Option 3: Dynamic Port Mapping via NodePort

If you need external access without a Load Balancer, use a NodePort service. This allows the kernel to manage port distribution across a high-range (30000-32767) while keeping the Nginx instances isolated on containerPort: 80.

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080

By applying Option 1, you allow the Linux kernel within the container to manage its own network stack independently of the host, enabling infinite scaling on a single node until CPU/RAM exhaustion.