签发证书

kubernetes集群相关证书依然是使用openssl签发。

证书说明

kubernetes master节点相关的证书如下：

• admin证书，kubectl命令行工具使用
  admin.crt
admin.key
• etcd客户端证书，由etcd ca签发，apiserver作为客户端访问etcd集群使用
  apiserver-etcd-client.crt
apiserver-etcd-client.key
• kubelet客户端证书，kubelet开启https安全访问使用
  apiserver-kubelet-client.crt
apiserver-kubelet-client.key
• apiserver服务器证书，apiserver对外提供服务器使用
  apiserver.crt
apiserver.key
• ca
  ca.crt
ca.key
  其中ca.key在controller-manager中使用，自动签发kubelet证书时使用，如果你不需要自动签发kubelet证书，ca.key可以不出现在集群中
• front-proxy ca
  front-proxy-ca.crt
front-proxy-ca.key
• front-proxy证书
  front-proxy-client.crt
front-proxy-client.key
• controller-manager客户端证书，访问apiserver使用
  kube-controller-manager.crt
kube-controller-manager.key
• kube-scheduler客户端证书，访问apiserver使用
  kube-scheduler.crt
kube-scheduler.crt
• 公私钥串，用来签名 ServiceAccount Token
  sa.key
sa.pub

kubernetes worker节点相关的证书如下：

• kube-proxy客户端证书，访问apiserver使用
  kube-proxy.crt
kube-proxy.crt
• kubelet证书，使用tls-bootstrap自动签发，后面章节会讲到

openssl签发证书

• 配置IP变量并进入pki文件夹

  cd pki
  master1=172.16.16.112
  master2=172.16.16.113
  master3=172.16.16.114
  

• 准备证书签发配置kubernetes-csr.conf
  DNS的值中包含kubernetes、kubernetes.default、kubernetes.default.svc、kubernetes.default.svc.cluster.local
  IP包含各个master节点的IP（如果为每个节点单独签发apiserver证书，那么包含本节点的IP即可）、集群中用来访问apiserver的scv IP，通常使用service-cluster-ip-range中的第一个IP，因为我们这里svc cidr是10.96.0.0/12，所以用的是10.96.0.1

  cat > kubernetes-csr.conf <<EOF
  [ req ]
  default_bits = 2048
  default_md = sha256
  distinguished_name = dn
  
  [ dn ]
  
  [ v3_ca ]
  keyUsage = critical, digitalSignature, keyEncipherment, keyCertSign
  basicConstraints = critical, CA:true
  
  [ v3_ext_client ]
  keyUsage= critical, digitalSignature, keyEncipherment
  extendedKeyUsage = clientAuth
  
  [ v3_ext_peer ]
  keyUsage = critical, digitalSignature, keyEncipherment
  extendedKeyUsage = serverAuth, clientAuth
  subjectAltName = @alt_names
  
  [ v3_ext_server ]
  keyUsage = critical, digitalSignature, keyEncipherment
  extendedKeyUsage = serverAuth
  subjectAltName = @alt_names
  
  [ alt_names ]
  DNS.1 = localhost
  DNS.2 = kubernetes
  DNS.3 = kubernetes.default
  DNS.4 = kubernetes.default.svc
  DNS.5 = kubernetes.default.svc.cluster.local
  IP.1 = 127.0.0.1
  IP.2 = 10.96.0.1
  IP.3 = ${master1}
  IP.4 = ${master1}
  IP.5 = ${master1}
  EOF
  

• 生成 kubernetes ca

  openssl genrsa -out ca.key 2048
  
  openssl req -x509 -new -nodes -key ca.key -config kubernetes-csr.conf \
  -subj "/CN=kubernetes" -extensions v3_ca -days 3650 -out ca.crt
  

• 生成 apiserver服务端证书

  openssl genrsa -out apiserver.key 2048
  
  openssl req -new  -key apiserver.key -subj "/CN=kubernetes-apiserver" \
  -reqexts v3_ext_server -config kubernetes-csr.conf -out apiserver.csr
  
  openssl x509 -in apiserver.csr -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -extensions v3_ext_server -extfile kubernetes-csr.conf -days 3650 -out apiserver.crt
  

• 生成 apiserver-kubelet-client证书

  openssl genrsa -out apiserver-kubelet-client.key 2048
  
  openssl req -new  -key apiserver-kubelet-client.key \
  -subj "/O=system:masters/CN=kube-apiserver-kubelet-client" \
  -reqexts v3_ext_client -config kubernetes-csr.conf -out apiserver-kubelet-client.csr
  
  openssl x509 -in apiserver-kubelet-client.csr \
  -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf \
  -days 3650 -out apiserver-kubelet-client.crt
  

• 生成 admin证书

  openssl genrsa -out admin.key 2048
  
  openssl req -new  -key admin.key -subj "/O=system:masters/CN=kubernetes-admin" \
  -reqexts v3_ext_client -config kubernetes-csr.conf -out admin.csr
  
  openssl x509 -in admin.csr -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf -days 3650 -out admin.crt
  

• 生成 kube-proxy证书 这个证书是部署worker节点的使用使用的，我们现在一并签发这个证书

  openssl genrsa -out kube-proxy.key 2048
  
  openssl req -new  -key kube-proxy.key -subj "/CN=system:kube-proxy" \
  -reqexts v3_ext_client -config kubernetes-csr.conf -out kube-proxy.csr
  
  openssl x509 -in kube-proxy.csr -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf -days 3650 -out kube-proxy.crt
  

• 生成 kube-controller-manager证书

  openssl genrsa -out kube-controller-manager.key 2048
  
  openssl req -new  -key kube-controller-manager.key \
  -subj "/CN=system:kube-controller-manager" \
  -reqexts v3_ext_client -config kubernetes-csr.conf \
  -out kube-controller-manager.csr
  
  openssl x509 -in kube-controller-manager.csr \
  -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf \
  -days 3650 -out kube-controller-manager.crt
  

• 生成 kube-scheduler证书

  openssl genrsa -out kube-scheduler.key 2048
  
  openssl req -new  -key kube-scheduler.key -subj "/CN=system:kube-scheduler" \
  -reqexts v3_ext_client -config kubernetes-csr.conf -out kube-scheduler.csr
  
  openssl x509 -in kube-scheduler.csr -req -CA ca.crt -CAkey ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf -days 3650 -out kube-scheduler.crt
  

• 生成 front-proxy-ca

  openssl genrsa -out front-proxy-ca.key 2048
  
  openssl req -x509 -new -nodes -key front-proxy-ca.key \
  -config kubernetes-csr.conf -subj "/CN=front-proxy-ca" \
  -extensions v3_ca -days 3650 -out front-proxy-ca.crt
  

• 生成 front-proxy-client证书

  openssl genrsa -out front-proxy-client.key 2048
  
  openssl req -new  -key front-proxy-client.key -subj "/CN=kube-front-proxy-client" \
  -reqexts v3_ext_client -config kubernetes-csr.conf -out front-proxy-client.csr
  
  openssl x509 -in front-proxy-client.csr \
  -req -CA front-proxy-ca.crt -CAkey front-proxy-ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf \
  -days 3650 -out front-proxy-client.crt
  

• 生成 apiserver-etcd-client证书

  openssl genrsa -out apiserver-etcd-client.key 2048
  
  openssl req -new  -key apiserver-etcd-client.key \
  -subj "/O=system:masters/CN=kube-apiserver-etcd-client" \
  -reqexts v3_ext_client -config kubernetes-csr.conf \
  -out apiserver-etcd-client.csr
  
  openssl x509 -in apiserver-etcd-client.csr \
  -req -CA etcd/ca.crt -CAkey etcd/ca.key -CAcreateserial \
  -extensions v3_ext_client -extfile kubernetes-csr.conf -days 3650 \
  -out apiserver-etcd-client.crt
  

• 生成 sa.key、sa.pub

  openssl genrsa -out sa.key 2048
  openssl rsa -in sa.key -pubout -out sa.pub
  

生产kubeconfig文件

组件访问apiserver时，使用kubeconf进行认证，文件里包含了apiserver地址和证书文件，其中证书文件可以直接指定路径，或者转换正base64编码直接放到文件里面
admin.kubeconf是给kubectl命令行工具用的

将controller-manager.kubeconf、scheduler.kubeconf分发到各个master节点的/etc/kubernetes路径下
将admin.kubeconf分发到master节点的~/.kube路径下，并命名为config（在你需要所使用kubectl命令的账号的用户目录下的.kube）

• controller-manager.kubeconf

  ca_base64=$(base64 -w 0 /etc/kubernetes/pki/ca.crt)
  controller_manager_cert_base64=$(base64 -w 0 /etc/kubernetes/pki/kube-controller-manager.crt)
  controller_manager_key_base64=$(base64 -w 0 /etc/kubernetes/pki/kube-controller-manager.key)
  
  cat > controller-manager.kubeconf <<EOF
  apiVersion: v1
  kind: Config
  preferences: {}
  clusters:
  - cluster:
      certificate-authority-data: ${ca_base64}
      server: https://127.0.0.1:6443
    name: kubernetes
  users:
  - name: system:kube-controller-manager
    user:
      client-certificate-data: ${controller_manager_cert_base64}
      client-key-data: ${controller_manager_key_base64}
  contexts:
  - context:
      cluster: kubernetes
      user: system:kube-controller-manager
    name: system:kube-controller-manager@kubernetes
  current-context: system:kube-controller-manager@kubernetes
  EOF
  

• scheduler.kubeconf

  ca_base64=$(base64 -w 0 /etc/kubernetes/pki/ca.crt)
  scheduler_cert_base64=$(base64 -w 0 /etc/kubernetes/pki/kube-scheduler.crt)
  scheduler_key_base64=$(base64 -w 0 /etc/kubernetes/pki/kube-scheduler.key)
  
  cat > scheduler.kubeconf <<EOF
  apiVersion: v1
  kind: Config
  preferences: {}
  clusters:
  - cluster:
      certificate-authority-data: ${ca_base64}
      server: https://127.0.0.1:6443
    name: kubernetes
  users:
  - name: system:kube-scheduler
    user:
      client-certificate-data: ${scheduler_cert_base64}
      client-key-data: ${scheduler_key_base64}
  contexts:
  - context:
      cluster: kubernetes
      user: system:kube-scheduler
    name: system:kube-scheduler@kubernetes
  current-context: system:kube-scheduler@kubernetes
  EOF
  

• admin kubeconf

  ca_base64=$(base64 -w 0 /etc/kubernetes/pki/ca.crt)
  admin_cert_base64=$(base64 -w 0 /etc/kubernetes/pki/admin.crt)
  admin_key_base64=$(base64 -w 0 /etc/kubernetes/pki/admin.key)
  
  cat > admin.kubeconf <<EOF
  apiVersion: v1
  kind: Config
  preferences: {}
  clusters:
  - cluster:
      certificate-authority-data: ${ca_base64}
      server: https://127.0.0.1:6443
    name: kubernetes
  users:
  - name: kubernetes-admin
    user:
      client-certificate-data: ${admin_cert_base64}
      client-key-data: ${admin_key_base64}
  contexts:
  - context:
      cluster: kubernetes
      user: kubernetes-admin
    name: kubernetes-admin@kubernetes
  current-context: kubernetes-admin@kubernetes
  EOF
  

• kubeconf文件

  （该文件在后面部署worker节点将会用到）

  kube-proxy连接apiserver时使用kube-proxy.kubeconf做身份认证
  将下面生成的kube-proxy.kubeconf分发到各个worker节点的/etc/kubernetes目录

  cat > kube-proxy.kubeconf <<EOF
  apiVersion: v1
  kind: Config
  preferences: {}
  clusters:
  - cluster:
      certificate-authority-data: /etc/kubernetes/pki/ca.crt
      server: https://127.0.0.1:6443
    name: kubernetes
  users:
  - name: kube-proxy
    user:
      client-certificate-data: /etc/kubernetes/pki/kube-proxy.crt
      client-key-data: /etc/kubernetes/pki/kube-proxy.key
  contexts:
  - context:
      cluster: kubernetes
      user: kube-proxy
    name: kube-proxy@kubernetes
  current-context: kube-proxy@kubernetes
  EOF
  

以下证书不再需要（已经制作成kubeconf文件了），可以删除

admin.crt
admin.key
kube-controller-manager.crt
kube-controller-manager.key
kube-scheduler.crt
kube-scheduler.key
kube-proxy.crt
kube-proxy.key

分发证书

将以下证书分发到各个master节点的/etc/kubernetes/pki/目录

apiserver-etcd-client.crt
apiserver-etcd-client.key
apiserver-kubelet-client.crt
apiserver-kubelet-client.key
apiserver.crt
apiserver.key
ca.crt
ca.key
front-proxy-ca.crt
front-proxy-ca.key
front-proxy-client.crt
front-proxy-client.key
sa.key
sa.pub

将以kubeconf文件分发到各个master节点的/etc/kubernetes目录

controller-manager.kubeconf
scheduler.kubeconf

将admin.kubeconfig文件分发到各个master节点的~/.kube目录，并命名为config（kubectl默认会取读取~/.kube/config的配置）