etcd TLS 部署
- 自签名证书部署
- etcd 自动证书

1. etcd TLS 部署

etcd 支持通过 TLS 协议的加密通讯。TLS 通道可以用于加密伙伴间的内部集群通讯，也可以用于加密客户端请求。这个章节提供例子来搭建使用伙伴和客户端 TLS 的集群。详细描述 etcd 的 TLS 支持的额外信息可以在加密指南

1.1. 自签名证书部署

使用自签名证书证书(self-signed certificates)的集群同时加密请求并认证它的连接。要启动使用自签名证书的集群，每个集群成员应该有一个唯一的通过共享的集群CA证书来签名的键对(key pair) (member.crt, member.key)，用于伙伴连接和客户端连接。证书可以通过仿照 etcd 搭建TLS 的例子来生成。

etcd 支持 TLS 安全方案。etcd 的 TLS 有两对，一对是 etcd 和 client 端的 TLS 配置。一对是 etcd 之间的 peer 的 TLS 配置。

1.1.1. 核心配置

如果需要开启 TLS 认证，需要配置调整参数，默认值如下

--client-cert-auth=false \
--trusted-ca-file=none \
--cert-file=none \
--key-file=none \
--peer-client-cert-auth=false \
--peer-cert-file=none \
--peer-key-file=none \
--peer-trusted-ca-file=none \

同时，配置中所有通讯的协议都需要由 http:// 调整为 https://

1.1.2. 配置示例

cat /etc/etcd/etcd.conf

修改后配置内容

ETCD_NAME="etcd01"
ETCD_DATA_DIR="/data/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://10.100.0.100:2380"
ETCD_LISTEN_CLIENT_URLS="https://10.100.0.100:2379,https://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.100.0.100:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://10.100.0.100:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://10.100.0.100:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

ETCD_CLIENT_CERT_AUTH=true
ETCD_TRUSTED_CA_FILE=/etc/etcd/pki/ca.crt
ETCD_CERT_FILE=/etc/etcd/pki/client.crt
ETCD_KEY_FILE=/etc/etcd/pki/client.key
ETCD_PEER_CLIENT_CERT_AUTH=true
ETCD_PEER_CERT_FILE=/etc/etcd/pki/peer.crt
ETCD_PEER_KEY_FILE=/etc/etcd/pki/peer.key
ETCD_PEER_TRUSTED_CA_FILE=/etc/etcd/pki/ca.crt

证书文件内容

该证书内容由下文中脚本生成

$ ls -lh /etc/etcd/pki/
total 40K
-rw-r--r-- 1 root root 1.1K Jan  2 15:25 ca.crt
-rw-r--r-- 1 root root 1.7K Jan  2 15:25 ca.key
-rw-r--r-- 1 root root   17 Jan  2 15:27 ca.srl
-rw-r--r-- 1 root root 1.2K Jan  2 15:27 client.crt
-rw-r--r-- 1 root root 1.1K Jan  2 15:27 client.csr
-rw-r--r-- 1 root root 1.7K Jan  2 15:25 client.key
-rw-r--r-- 1 root root  369 Jan  2 15:27 openssl.conf
-rw-r--r-- 1 root root 1.2K Jan  2 15:27 peer.crt
-rw-r--r-- 1 root root 1.1K Jan  2 15:27 peer.csr
-rw-r--r-- 1 root root 1.7K Jan  2 15:25 peer.key

1.1.3. 启动服务并验证

启动服务

systemctl daemon-reload
systemctl enable etcd
systemctl start etcd

查看集群状态

 etcdctl --endpoints https://10.100.0.100:2379 --cacert=/etc/etcd/pki/ca.crt --cert=/etc/etcd/pki/client.crt --key=/etc/etcd/pki/client.key  endpoint status --cluster  -w table
+---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|         ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://10.100.0.100:2379 | 65231cb33388f16d |  3.4.14 |   20 kB |      true |      false |         7 |         14 |                 14 |        |
+---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

写入数据

$ etcdctl --endpoints https://10.100.0.100:2379 --cacert=/etc/etcd/pki/ca.crt --cert=/etc/etcd/pki/client.crt --key=/etc/etcd/pki/client.key  put tls-key tls-value
OK

读取数据

[root@cust56-gitlab default.etcd]# etcdctl --endpoints https://10.100.0.100:2379 --cacert=/etc/etcd/pki/ca.crt --cert=/etc/etcd/pki/client.crt --key=/etc/etcd/pki/client.key  get  tls-key
tls-key
tls-value

1.1.4. 证书制作脚本

#!/bin/bash -x
export NAME1=etcd01
export ADDRESS1=10.100.0.100

export NAME2=etcd02
export ADDRESS2=10.100.0.101

export NAME3=etcd03
export ADDRESS3=10.100.0.102

days=3650

cat > openssl.conf << EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = $NAME1
DNS.2 = $NAME2
DNS.3 = $NAME3
IP.1 = 127.0.0.1
IP.2 = $ADDRESS1
IP.3 = $ADDRESS2
IP.4 = $ADDRESS3
EOF


# 准备 CA 证书
[ -f ca.key ] || openssl genrsa -out ca.key 2048
[ -f ca.crt ] || openssl req -x509 -new -nodes -key ca.key -subj "/CN=etcd-ca" -days ${days} -out ca.crt

# 创建 etcd client 证书
[ -f client.key ] || openssl genrsa -out client.key 2048
[ -f client.csr ] || openssl req -new -key client.key -subj "/CN=kube-etcd" -out client.csr -config openssl.conf
[ -f client.crt ] || openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days ${days} -extensions v3_req  -extfile openssl.conf

# 创建 etcd 集群 peer 间证书
[ -f peer.key ] || openssl genrsa -out peer.key 2048
[ -f peer.csr ] || openssl req -new -key peer.key -subj "/CN=kube-etcd-peer" -out peer.csr -config openssl.conf
[ -f peer.crt ] || openssl x509 -req -in peer.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out peer.crt -days ${days} -extensions v3_req  -extfile openssl.conf

1.2. etcd 自动证书

如果集群需要加密通讯，但是不需要认证连接，etcd 可以配置为自动生成 key。在初始化时，每个集群基于它的通告IP(advertised IP) 地址和主机名创建它自己的 key 集合。

在每台机器上，etcd 使用这些标记启动：

infra0 节点

$ etcd --name infra0 --initial-advertise-peer-urls https://10.0.1.10:2380 \
  --listen-peer-urls https://10.0.1.10:2380 \
  --listen-client-urls https://10.0.1.10:2379,https://127.0.0.1:2379 \
  --advertise-client-urls https://10.0.1.10:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=https://10.0.1.10:2380,infra1=https://10.0.1.11:2380,infra2=https://10.0.1.12:2380 \
  --initial-cluster-state new \
  --auto-tls \
  --peer-auto-tls

infra1 节点

$ etcd --name infra1 --initial-advertise-peer-urls https://10.0.1.11:2380 \
  --listen-peer-urls https://10.0.1.11:2380 \
  --listen-client-urls https://10.0.1.11:2379,https://127.0.0.1:2379 \
  --advertise-client-urls https://10.0.1.11:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=https://10.0.1.10:2380,infra1=https://10.0.1.11:2380,infra2=https://10.0.1.12:2380 \
  --initial-cluster-state new \
  --auto-tls \
  --peer-auto-tls

infra2 节点

$ etcd --name infra2 --initial-advertise-peer-urls https://10.0.1.12:2380 \
  --listen-peer-urls https://10.0.1.12:2380 \
  --listen-client-urls https://10.0.1.12:2379,https://127.0.0.1:2379 \
  --advertise-client-urls https://10.0.1.12:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=https://10.0.1.10:2380,infra1=https://10.0.1.11:2380,infra2=https://10.0.1.12:2380 \
  --initial-cluster-state new \
  --auto-tls \
  --peer-auto-tls

etcd TLS 部署