Skip to main content
Version: Next

Migrating to Gateway API

warning

Gateway API support in metal-stack is still in development. Changes are expected, especially around certificate management.

From version x.y.z metal-stack supports Gateway API as a replacement for Kubernetes Ingress resources.

This guide explains why we are moving to Gateway API, what changes for you as an operator, and how to migrate an existing metal-stack installation.

Motivation​

ingress-nginx has been deprecated and we currently depend on it. metal-stack has control-plane components that are not served via HTTP/GRPC. Those are exposed as TCP Services via ingress-nginx. As a result we are going to move all metal-stack components to Gateway API.

What will change​

Certificate provisioning and TLS termination​

Certificate provisioning and TLS termination will move into the Gateway resource. metal-stack deployed via metal-roles will expect TLS termination to happen on the Gateway.

Exceptions for TLS termination:

  • NSQ will for now still terminate its own certificates
  • The GRPC endpoint of metal-api will stay a TCPRoute and TLS termination will stay in the application. As metal-api will be superseded by metal-apiserver in the near future, we do not see the value in migrating it fully

Continued need for an Ingress Controller​

For some vendor dependencies you will still require an ingress controller. Examples of services still relying on Ingress are Gardener and Thanos. As ingress-nginx is EoL we recommend switching to a different Ingress Controller implementation.

Before you begin​

Familiarize yourself with Gateway API​

Gateway API is more complex than Ingress and consists of multiple different resources. An introduction can be found here

Gateway API implementation requirements​

metal-stack does not require one specific Gateway API implementation. You can choose any Gateway API implementation that provides the following features:

  • HTTPRoute
  • TCPRoute

In our mini-lab demo environment we are running Envoy Gateway.

Migration Path decisions​

  • Select a Gateway API implementation, e.g. Envoy Gateway
  • Certificate management: Gateway API now requires the Gateway to know

Migration steps​

Create a Gateway resource​

Gateway deployment​

As we require TCPRoutes you will have to make changes to your existing Gateways, if applicable. Still, we suggest you create a dedicated metal-stack gateway, like we do for mini-lab.

Certificate management​

Use automated certificate management. cert-manager can handle your certificate renewal and supports Gateway API. You can use Let's Encrypt for valid public certificates or use a private CA.

As an aside: you can issue valid public TLS certs using Let's Encrypt, including for IP addresses.

Migrating HTTP/GRPC services​

Migrating TCP services​

Remaining ingress controller dependencies​

For some vendor dependencies you will still require an ingress controller:

  • Gardener
  • Thanos

Verification​

Rollback​

Open questions​

  • Are we going to do TLS termination for nsq (for example) on the Gateway? This determines whether TLSRoute is a hard requirement.

References​