Seamless Migration from Ingress NGINX to Gateway API on DOKS

Introduction

The digital landscape is constantly evolving, and so should your cloud architecture. With the deprecation of the Ingress NGINX controller, it’s time to embrace the Gateway API on DigitalOcean Kubernetes (DOKS). This migration can seem daunting, but fear not! This guide will walk you through a step-by-step process that ensures a smooth transition with zero downtime for your workloads.

By leveraging the capabilities of Cilium, you can run both Ingress and Gateway side-by-side, validating the production readiness of your new setup before making the final switch. Understanding the nuances between these two systems is crucial, and this guide will cover everything from handling TLS certificates to configuring DigitalOcean LoadBalancer for your new gateway.

Key Considerations for Your Migration

Migrating to the Gateway API is not just about switching controllers; it requires careful planning and understanding of the differences in configurations. Here are some key takeaways to keep in mind:

• Zero Downtime: Run both controllers concurrently and perform a controlled DNS cutover.
• LoadBalancer Endpoint Changes: Expect a new LoadBalancer with a different IP address.
• Annotation Migration: Different configurations between Ingress NGINX and Gateway API; pay attention to the changes.
• Explicit Certificate Management: Transition from annotations to separate Certificate resources.

Prerequisites for Migration

Before starting the migration, ensure that you meet the following prerequisites:

1. A VPC-integrated DOKS cluster version 1.33+.
2. Configured kubectl for your cluster.
3. An existing Ingress NGINX deployment with cert-manager.
4. Domain name with DNS access.
5. Budget for temporary dual LoadBalancers during migration.

Step-by-Step Migration Process

The migration process follows a blue-green deployment strategy, allowing you to validate your new Gateway setup without affecting your existing traffic. Here’s how to go about it:

Phase 1: Prepare Your Gateway API Stack

Start by enabling Gateway API in cert-manager. This will allow cert-manager to manage certificates for Gateway-managed routes. Execute the following Helm command:

helm upgrade cert-manager jetstack/cert-manager --namespace cert-manager --reuse-values --set extraArgs="{--enable-gateway-api=true}"

Next, create a Gateway resource using a manifest file (gateway.yaml) that defines your Gateway settings, including the annotations for your DigitalOcean LoadBalancer.

Once created, ensure the Gateway is ready by applying the configuration:

kubectl apply -f gateway.yaml

Then verify that the Gateway is programmed correctly and has an assigned address:

kubectl get gateway my-gateway

Phase 2: Validate Your Gateway Setup

Before cutting over DNS records, you need to validate that your Gateway is functioning correctly. Test the setup using the Gateway LoadBalancer IP:

GATEWAY_IP=$(kubectl get gateway my-gateway -o jsonpath='{.status.addresses[0].value}')

Use curl to check both HTTP and HTTPS traffic:

curl -I --resolve www.example.com:80:$GATEWAY_IP http://www.example.com

If all tests return the expected results, you are ready for the next phase.

Phase 3: Execute the DNS Cutover

The final step is to update your DNS records to point to the new Gateway LoadBalancer IP. If you're using manual DNS updates, ensure to lower the TTL beforehand:

doctl compute domain records update example.com --record-id  --record-ttl 60

After updating the A record, monitor the propagation of DNS changes to ensure a smooth transition.

Post-Migration Best Practices

After successfully migrating to the Gateway API, it’s essential to establish proper certificate management to ensure your certificates do not expire. Create a Certificate resource that references your ClusterIssuer:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: www-tls-gateway
spec:
  secretName: www-tls
  issuerRef:
    name: letsencrypt-prod-gateway
    kind: ClusterIssuer
  dnsNames:
  - www.example.com

Monitor the stability of your new setup for at least 24-48 hours before removing the old Ingress. Keep an eye on traffic volume, error rates, and response times to ensure everything is functioning as expected.

Conclusion

Successfully migrating from Ingress NGINX to Gateway API on DOKS with zero downtime is a significant achievement that positions your workloads for enhanced performance and security. By following the outlined steps, you can harness the modern features of the Gateway API, leading to better traffic management and simplified configurations. As you settle into your new Gateway setup, consider exploring additional features like advanced routing and observability to further enhance your cloud architecture.