集群高可用配置

• apiserver组件：提供 HTTP REST 服务，可以水平扩展，多个kube-apiserver同时提供服务
• scheduler和kube-controller-manager组件：这两个组件都采用了一主多从的高可用方案。使用--leader-elect参数启动，kube-scheduler和kube-controller-manager将会进行leader选举过程，由leader提供服务

集群高可用配置如下，每个worker节点以静态pod的形式跑一个nginx，用来代理kubelet和kube-proxy对apiserver的访问 nginx会对apiserver进行健康检测，流量不转发到挂掉的apiserver节点

kubelet维护本节点nginx静态pod的生命周期

配置nginx静态 pod yml文件

将下面生成的nginx-proxy.yml 静态pod配置文件分发到worker节点（master节点不需要）/etc/kubernetes/manifests目录
master节点下的组件直接连接本机的apiserver
这里使用8089作为nginx的自身健康检测端口

cat > nginx-proxy.yml <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx-proxy
  namespace: kube-system
  labels:
    k8s-app: kube-nginx
spec:
  hostNetwork: true
  nodeSelector:
    beta.kubernetes.io/os: linux
  priorityClassName: system-node-critical
  containers:
  - name: nginx-proxy
    image: nginx:1.15
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        cpu: 25m
        memory: 32M
    securityContext:
      privileged: true
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8089
    readinessProbe:
      httpGet:
        path: /healthz
        port: 8089
    volumeMounts:
    - mountPath: /etc/nginx/nginx.conf
      name: nginx-conf
      readOnly: true
  volumes:
  - name: nginx-conf
    hostPath:
      path: /etc/kubernetes/nginx/nginx.conf
      type: FileOrCreate
EOF

nginx配置文件

将下面生成的nginx.conf配置文件分发到worker节点（master节点不需要）/etc/kubernetes/nginx目录

master1_ip=172.16.16.112
master2_ip=172.16.16.113
master3_ip=172.16.16.114
cat > nginx.conf <<EOF
error_log stderr notice;
worker_processes 2;
worker_rlimit_nofile 130048;
worker_shutdown_timeout 10s;
events {
  multi_accept on;
  use epoll;
  worker_connections 16384;
}
stream {
  upstream kube_apiserver {
    least_conn;
    server ${master1_ip}:6443;
    server ${master2_ip}:6443;
    server ${master3_ip}:6443;
    }
  server {
    listen        127.0.0.1:6443;
    proxy_pass    kube_apiserver;
    proxy_timeout 10m;
    proxy_connect_timeout 1s;
  }
}
http {
  aio threads;
  aio_write on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 75s;
  keepalive_requests 100;
  reset_timedout_connection on;
  server_tokens off;
  autoindex off;
  server {
    listen 8089;
    location /healthz {
      access_log off;
      return 200;
    }
  }
}
EOF