Kubernetes Ingress, Nginx Controller, Blue-Green Deployment

This post describes how we use Kubernetes Ingress to create multiple deployments that allow blue-green deployments.We run Kubernetes on AWS.

Initially we took the approach of creating our services with the type LoadBalancer that create AWS ELBs for each service. With the multiplication of services, the number and cost of ELB grew quickly. We also had to make sure that our Route53 records match their respective ELBs, it was error prone and costly.

Enter Kubernetes Ingress, here is what we have now:
- Services of type ClusterIP for each of our deployments
- Nginx Ingress Controller
- Service of type LoadBalancer for Nginx Ingress Controller
- Ingress resources to define the routing of the requests by the nginx ingress controller to the right service

In our case the routing is based on host names. With that solution, our Route53 records all refer to the same ELB which delegates the requests to the nginx ingress controller service.

Blue Green Deployment

To do a blue/green deployment:

• First update the deployment (green) that is inactive/not used with the new version
• Wait until the green deployment is ready 
• Update the service selector to make it use the green deployment
• Scale down the previous deployment "blue" to zero replicas

Here you go, the service is now sending client requests to the new version of the application.

References

https://kubernetes.io/docs/concepts/services-networking/ingress/
https://github.com/kubernetes/ingress/blob/master/controllers/nginx/README.md
https://github.com/kubernetes/ingress/tree/master/examples/aws/nginx