On-Premises (kube-vip-cloud-controller)

kube-vip On-Prem

We've designed kube-vip to be as decoupled or agnostic from other components that may exist within a Kubernetes cluster as possible. This has lead to kube-vip having a very simplistic but robust approach to advertising Kubernetes Services to the outside world and marking these Services as ready to use.

Cloud Controller Manager

kube-vip isn't coupled to anything other than the Kubernetes API and will only act upon an existing Kubernetes primitive (in this case the object of type Service). This makes it easy for existing cloud controller managers (CCMs) to simply apply their logic to services of type LoadBalancer and leave kube-vip to take the next steps to advertise these load balancers to the outside world.

Using the kube-vip Cloud Provider

The kube-vip cloud provider can be used to populate an IP address for Services of type LoadBalancer similar to what public cloud providers allow through a Kubernetes CCM. The below instructions should just work on Kubernetes regardless of the architecture (a Linux OS being the only requirement) and will install the latest components.

Install the kube-vip Cloud Provider

The kube-vip cloud provider can be installed from the latest release in the main branch by using the following command:

1kubectl apply -f https://raw.githubusercontent.com/kube-vip/kube-vip-cloud-provider/main/manifest/kube-vip-cloud-controller.yaml

Create a global CIDR or IP Range

In order for kube-vip to set an IP address for a Service of type LoadBalancer, it needs to have an availability of IP address to assign. This information is stored in a Kubernetes ConfigMap to which kube-vip has access. You control the scope of the IP allocations with the key within the ConfigMap. Either CIDR blocks or IP ranges may be specified and scoped either globally (cluster-side) or per-Namespace.

To allow a global (cluster-wide) CIDR block which kube-vip can use to allocate an IP to Services of type LoadBalancer in any Namespace, create a ConfigMap named kubevip with the key cidr-global and value equal to a CIDR block available in your environment. For example, the below command creates a global CIDR with value 192.168.0.220/29 from which kube-vip will allocate IP addresses.

1kubectl create configmap -n kube-system kubevip --from-literal cidr-global=192.168.0.220/29

To use a global range instead, create the key range-global with the value set to a valid range of IP addresses. For example, the below command creates a global range using the pool 192.168.1.220-192.168.1.230.

1kubectl create configmap -n kube-system kubevip --from-literal range-global=192.168.1.220-192.168.1.230

Creating services of type LoadBalancer in any Namespace will now take addresses from one of the global pools defined in the ConfigMap unless a Namespace-specific pool is created.

The kube-vip Cloud Provider ConfigMap

To manage the IP address ranges for Services of type LoadBalancer, the kube-vip-cloud-provider uses a ConfigMap held in the kube-system Namespace. IP addresses can be configured using one or multiple formats:

CIDR blocks
IP ranges [start address - end address]
Multiple pools by CIDR per Namespace
Multiple IP ranges per Namespace (handles overlapping ranges)
Setting of static addresses through --load-balancer-ip=x.x.x.x (kubectl expose command)

To control which IP address range is used for which Service, the following rules are applied:

Global address pools (cidr-global or range-global) are available for use by any Service in any Namespace
Namespace specific address pools (cidr-<namespace> or range-<namespace>) are only available for use by a Service in the specific Namespace
Static IP addresses can be applied to a Service of type LoadBalancer using the spec.loadBalancerIP field, even outside of the assigned ranges

Example Configmap:

 1apiVersion: v1
 2kind: ConfigMap
 3metadata:
 4  name: kubevip
 5  namespace: kube-system
 6data:
 7  cidr-default: 192.168.0.200/29                      # CIDR-based IP range for use in the default Namespace
 8  range-development: 192.168.0.210-192.168.0.219      # Range-based IP range for use in the development Namespace
 9  cidr-finance: 192.168.0.220/29,192.168.0.230/29     # Multiple CIDR-based ranges for use in the finance Namespace
10  cidr-global: 192.168.0.240/29                       # CIDR-based range which can be used in any Namespace

Expose a Service

We can now expose a Service and once the cloud provider has provided an address, kube-vip will start to advertise that address to the outside world as shown below:

1kubectl expose deployment nginx-deployment --port=80 --type=LoadBalancer --name=nginx

or via a Service YAML definition:

 1apiVersion: v1
 2kind: Service
 3metadata:
 4  name: nginx
 5spec:
 6  ports:
 7  - name: http
 8    port: 80
 9    protocol: TCP
10  selector:
11    app: nginx
12  type: LoadBalancer

We can also expose a specific address by specifying it imperatively on the command line:

1kubectl expose deployment nginx-deployment --port=80 --type=LoadBalancer --name=nginx --load-balancer-ip=1.1.1.1

or including it in the Service definition:

 1apiVersion: v1
 2kind: Service
 3metadata:
 4  name: nginx
 5spec:
 6  ports:
 7  - name: http
 8    port: 80
 9    protocol: TCP
10  selector:
11    app: nginx
12  type: LoadBalancer
13  loadBalancerIP: "1.1.1.1"