写在最前
在 Kubernetes 高可用部署中,通常涉及到 3 个 master 节点和多个 worker 节点。对于私有化部署,尤其在没有额外云服务资源(如阿里云 SLB)的情况下,我们不能依赖额外的机器,这样会增加不必要的开销。
为了实现 master 节点的高可用,通常使用 keepalived 和 HAProxy,但这种方式不适合将域名访问流量直接引入 master 节点,否则会影响其性能,甚至可能导致宕机。
虽然可以在 worker 节点上继续使用 keepalived 来创建第二个虚拟 IP(VIP)以供 Ingress 使用,但考虑到我们已经实现了容器化部署,继续依赖 keepalived 这样的传统工具显得不够优雅。此时,kube-vip 提供了一种更加简洁高效的解决方案。通过在指定节点中打上标签并部署 DaemonSet,使用容器化方式部署提供VIP是不是非常优雅呢。
1. kube-vip
https://github.com/kube-vip/kube-vip
1.1 rabc
kube-vip-rabc.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-vip
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
name: system:kube-vip-role
rules:
- apiGroups: [""]
resources: ["services/status"]
verbs: ["update"]
- apiGroups: [""]
resources: ["services", "endpoints"]
verbs: ["list","get","watch", "update"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["list","get","watch", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["list", "get", "watch", "update", "create"]
- apiGroups: ["discovery.k8s.io"]
resources: ["endpointslices"]
verbs: ["list","get","watch", "update"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:kube-vip-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-vip-role
subjects:
- kind: ServiceAccount
name: kube-vip
namespace: kube-system1.2 daemonset
在 kube-vip.yaml 中,虚拟 IP(VIP)需要自行配置,例如将 172.31.0.99 更改为所需的 VIP 地址,按你喜好自行决定。最重要的是节点亲和性配置,需要在目标机器上打上 kube-vip 标签,以确保 kube-vip 仅在这些节点上运行。可以使用以下命令为节点打标签:kubectl label nodes worker1 kube-vip=""
apiVersion: apps/v1
kind: DaemonSet
metadata:
creationTimestamp: null
name: kube-vip
namespace: kube-system
spec:
selector:
matchLabels:
name: kube-vip
template:
metadata:
creationTimestamp: null
labels:
name: kube-vip
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kube-vip
operator: Exists
containers:
- args:
- manager
env:
- name: vip_arp
value: "true"
- name: vip_interface
value: eth0
- name: vip_cidr
value: "32"
- name: vipSubnet
value: ""
- name: cp_enable
value: "true"
- name: cp_namespace
value: kube-system
- name: vip_ddns
value: "false"
- name: svc_enable
value: "true"
- name: vip_leaderelection
value: "true"
- name: vip_leaseduration
value: "5"
- name: vip_renewdeadline
value: "3"
- name: vip_retryperiod
value: "1"
- name: address
value: 172.31.0.99
image: plndr/kube-vip:v1.0.2
imagePullPolicy: Always
name: kube-vip
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
- SYS_TIME
hostNetwork: true
serviceAccountName: kube-vip
tolerations:
- effect: NoSchedule
operator: Exists
- effect: NoExecute
operator: Exists
updateStrategy: {}
status:
currentNumberScheduled: 0
desiredNumberScheduled: 0
numberMisscheduled: 0
numberReady: 02. kube-vip-cloud-provider
https://github.com/kube-vip/kube-vip-cloud-provider
kube-vip-cloud-provider 是一个 Kubernetes cloud provider 插件,它为 Kubernetes 集群提供了云环境中的 虚拟 IP (VIP) 功能。这使得 Kubernetes 集群可以使用 高可用性负载均衡 和 动态 IP 管理,尤其是在没有云负载均衡器(如 AWS ELB 或 Google Cloud Load Balancer)的环境中,提供类似云负载均衡的功能。
2.1 configmap
配置 kube-vip 时,如果只有一个虚拟 IP 可用,可以将其范围设置为相同的 IP 地址,即将 range-global 配置为 172.31.0.99-172.31.0.99,这样确保使用的唯一 VIP 地址在配置中左右一致。
kind: ConfigMap
apiVersion: v1
metadata:
name: kubevip
namespace: kube-system
annotations:
kubesphere.io/creator: admin
data:
range-global: 172.31.0.99-172.31.0.99
2.2 deployment
kube-vip-cloud-controller.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-vip-cloud-controller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
name: system:kube-vip-cloud-controller-role
rules:
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "update", "list", "put"]
- apiGroups: [""]
resources: ["configmaps", "endpoints","events","services/status", "leases"]
verbs: ["*"]
- apiGroups: [""]
resources: ["nodes", "services"]
verbs: ["list","get","watch","update"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:kube-vip-cloud-controller-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-vip-cloud-controller-role
subjects:
- kind: ServiceAccount
name: kube-vip-cloud-controller
namespace: kube-system
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kube-vip-cloud-provider
namespace: kube-system
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: kube-vip
component: kube-vip-cloud-provider
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: kube-vip
component: kube-vip-cloud-provider
spec:
containers:
- command:
- /kube-vip-cloud-provider
- --leader-elect-resource-name=kube-vip-cloud-controller
image: kubevip/kube-vip-cloud-provider:v0.0.12
name: kube-vip-cloud-provider
imagePullPolicy: Always
dnsPolicy: ClusterFirst
restartPolicy: Always
terminationGracePeriodSeconds: 30
serviceAccountName: kube-vip-cloud-controller
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 10
preference:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
- weight: 10
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
评论