系统:ubuntu 24.04.4
集群规划(3 节点)
master节点,主机名:k8s-master-01,ip:172.29.128.1
工作节点1:主机名:k8s-worker-01,ip:172.29.132.68
工作节点2:主机名:k8s-worker-02,ip:172.29.132.69
1, 前置统一操作(所有 3 台机器全部执行)
1. 配置主机名与 hosts 解析
1.1 设置对应主机名
# master节点执行 hostnamectl set-hostname k8s-master-01 # worker01执行 hostnamectl set-hostname k8s-worker-01 # worker02执行 hostnamectl set-hostname k8s-worker-02
1.2 统一写入 /etc/hosts(三台都执行)
cat >> /etc/hosts << EOF 172.29.128.1 k8s-master-01 172.29.132.68 k8s-worker-01 172.29.132.69 k8s-worker-02 EOF
2. 关闭交换分区(必须永久关闭)
# 临时关闭 swapoff -a # 永久注释fstab交换分区 sed -i '/swap/s/^/#/' /etc/fstab # 验证无swap输出 cat /etc/fstab | grep swap
3,关闭防火墙
systemctl stop ufw
systemctl disable ufw
如果不关闭防火墙,要放行端口
1) Master 节点放行端口:
ufw allow 6443/tcp ufw allow 2379:2380/tcp ufw allow 10250/tcp ufw allow 10259/tcp ufw allow 10257/tcp ufw allow 30000:32767/tcp ufw reload
2) Worker 节点放行端口:
ufw allow 10250/tcp ufw allow 30000:32767/tcp ufw reload
4. 内核网络参数转发(容器网络必备)
# 加载内核模块modprobe overlay modprobe ip_tables modprobe br_netfilter modprobe ip_vs modprobe ip_vs_rr modprobe ip_vs_wrr modprobe ip_vs_sh# 持久化sysctl配置 cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 EOFsysctl --system # 校验输出全部为1 sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward
4,安装依赖
apt update && apt install -y ca-certificates curl gnupg lsb-release
二. 安装 containerd
# 阿里云镜像站docker公钥
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/trusted.gpg.d/docker.gpg
# 添加阿里云docker-ce源,可以安装containerd
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/trusted.gpg.d/docker.gpg] https://mirrors.aliyun.com/docker-ce/linux/ubuntu noble stable" | tee /etc/apt/sources.list.d/docker.listapt update
查看 containerd.io 可用版本
apt-cache madison containerd.io
2,安装containerd
apt install -y containerd.io
3,初始化 containerd 配置 + 开启 systemd cgroup(K8s 强制要求)
# 生成默认配置文件
containerd config default > /etc/containerd/config.toml# 修改配置:systemd_cgroup=true、pause镜像适配1.34版本
sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml
sed -i 's#sandbox_image = "registry.k8s.io/pause:3.8"#sandbox_image = "registry.k8s.io/pause:3.10"#g' /etc/containerd/config.toml# 重启并设置开机自启
systemctl restart containerd
systemctl enable containerd
systemctl status containerd
三,安装 Kubernetes v1.34.10 工具链(kubeadm/kubelet/kubectl)
# 导入k8s密钥 curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | gpg --dearmor -o /usr/share/keyrings/kubernetes-apt-keyring.gpg # 写入apt源 echo 'deb [signed-by=/usr/share/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /' | tee /etc/apt/sources.list.d/kubernetes.listapt update
重要避坑:
K8s 源是分支锁定
1) /v1.34/deb/ = 只能装 1.34.x;想要 1.35 必须改源地址为 /v1.35/deb/,不能一个源同时看到多个大版本。
2) containerd.io 版本 和 K8s 版本没有强制绑定,但有兼容矩阵,kubeadm 1.34 推荐 containerd ≥1.7;生产建议 2.0+,推荐 2.3.x。
2,查看源里有哪些包版本
# 查询kubeadm所有可用版本 apt-cache madison kubeadm # kubectl apt-cache madison kubectl # kubelet apt-cache madison kubelet
输出版本:
root@k8s-master-01:/data/k8s-app-deploy# apt-cache madison kubeadmkubeadm | 1.34.10-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.9-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.8-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.7-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.6-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.5-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.4-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.3-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packageskubeadm | 1.34.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb Packages
3,锁定版本安装 v1.34.10
K8S_VERSION="1.34.10-1.1" apt install -y kubelet=${K8S_VERSION} kubeadm=${K8S_VERSION} kubectl=${K8S_VERSION}# 锁定版本防止自动升级 apt-mark hold kubelet kubeadm kubectl # 校验版本 kubeadm version kubectl version --client
四,master节点初始化集群
1,kubeadm init 命令(适配 v1.34.10)
kubeadm init \ --apiserver-advertise-address=172.29.128.1 \ --kubernetes-version=v1.34.10 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=192.168.0.0/16 \ --cri-socket unix:///run/containerd/containerd.sock \ --image-repository registry.aliyuncs.com/google_containers #关键,走国内镜像源
注意:k8s-v1.22.x及以前的版本,镜像地址用:registry.aliyuncs.com/k8sxio
参数说明
--apiserver-advertise-address:master 本机内网 IPpod-network-cidr=192.168.0.0/16:适配 Calico 标准网段- cri-socket 指定 containerd 套接字,避免默认 docker 适配报错
如果初始化master节点失败,看kubelet日志
journalctl -u kubelet -f
2, 配置 kubectl 普通用户权限(master 执行)
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 验证集群状态(此时节点NotReady,未装CNI)
kubectl get nodes
3,如果初化集群失改,先导出镜像,再初始化
1)执行kubeadm init之前,可以先把kubeadm镜像拉下来:
kubeadm config images pull \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.34.10
或者配置 containerd 全局镜像加速(一劳永逸,无需每次加参数)
编辑 /etc/containerd/config.toml,添加 registry.k8s.io 镜像转发:
[plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.k8s.io"]endpoint = ["https://registry.aliyuncs.com/google_containers"]
配置全局镜像转发后,重启containerd
systemctl restart containerd
2)备选国内镜像
中科大:mirrors.ustc.edu.cn/google_containers
kubeadm config images pull --image-repository mirrors.ustc.edu.cn/google_containers --kubernetes-version v1.34.10
华为云:swr.cn-north-4.myhuaweicloud.com/ddn-k8s
ubeadm config images pull --image-repository swr.cn-north-4.myhuaweicloud.com/ddn-k8s --kubernetes-version v1.34.10
3)如果安装的时候镜像拉取失败,导出镜像,上传到所有节点并导入
#导出镜像 kubeadm config images pull \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.34.10#镜像改名 nerdctl tag registry.aliyuncs.com/google_containers/etcd:3.6.5-0 registry.k8s.io/etcd:3.6.5-0 nerdctl tag registry.aliyuncs.com/google_containers/pause:3.10.1 registry.k8s.io/pause:3.10.1 nerdctl tag registry.aliyuncs.com/google_containers/coredns:v1.12.1 registry.k8s.io/coredns/coredns:v1.12.1 nerdctl tag registry.aliyuncs.com/google_containers/kube-proxy:v1.34.10 registry.k8s.io/kube-proxy:v1.34.10 nerdctl tag registry.aliyuncs.com/google_containers/kube-scheduler:v1.34.10 registry.k8s.io/kube-scheduler:v1.34.10 nerdctl tag registry.aliyuncs.com/google_containers/kube-controller-manager:v1.34.10 registry.k8s.io/kube-controller-manager:v1.34.10 nerdctl tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.34.10 registry.k8s.io/kube-apiserver:v1.34.10#导出镜像 nerdctl save -o k8s-1.34.10-images.tar \ registry.k8s.io/kube-apiserver:v1.34.10 \ registry.k8s.io/kube-controller-manager:v1.34.10 \ registry.k8s.io/kube-scheduler:v1.34.10 \ registry.k8s.io/kube-proxy:v1.34.10 \ registry.k8s.io/etcd:3.6.5-0 \ registry.k8s.io/coredns/coredns:v1.12.1 \ registry.k8s.io/pause:3.10.1 \#上传到所有节点,并导入 nerdctl -n k8s.io load -i k8s-1.34.10-images.tar
再执行kubeadm init,去掉 --image-repository 参数
kubeadm init \ --apiserver-advertise-address=172.29.128.1 \ --kubernetes-version=v1.34.10 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=192.168.0.0/16 \ --cri-socket unix:///run/containerd/containerd.sock
5,如果安装失败,先清除安装的残留缓存,再重新初始化
kubeadm reset -f rm -rf /etc/kubernetes /var/lib/kubelet /var/lib/etcd rm -rf /var/run/kubernetes iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X ipvsadm -C
6,安装成功输出:
Your Kubernetes control-plane has initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:export KUBECONFIG=/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root:kubeadm join 172.29.128.1:6443 --token oy1nul.n1xz32q53qds022t \--discovery-token-ca-cert-hash sha256:23f8cc787eb0dd830c05003974eaba3460ac05fe72b000fe5fbec755d572b48e
五, worker 节点加入集群
1,保存上面输出的 join 命令
kubeadm join 172.29.128.1:6443 --token xxx --discovery-token-ca-cert-hash sha256:xxx
2,如果忘记了join命令,重新生成token(在master 执行)
# 重新生成join命令 kubeadm token create --print-join-command
输出:
kubeadm join 172.29.128.1:6443 --token xxxxx --discovery-token-ca-cert-hash sha256:xxxx
3, 工作节点加入集群
所有工作节点执行上面的 kubeadm join命令加入集群
4,Master 节点验证集群完整状态
# 查看所有节点 kubectl get nodes # 查看集群组件健康状态 kubectl get cs # 查看所有系统pod kubectl get pods -n kube-system
六,安装calico网络
1 部署 Calico CNI 网络插件(master 执行)
Calico v3.32.1 完美兼容 K8s v1.34
curl -O calico-v3.32.1.yaml https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/calico.yaml
2,替换为国内镜像
原镜像:quay.io/calico/xxx
替换为:m.daocloud.io/quay.io/calico/xxx
sed -i 's|quay.io/calico|m.daocloud.io/quay.io/calico|g' calico-v3.32.1.yaml
查看镜像配置
grep "image:" calico-v3.32.1.yaml
#只显示镜像名
awk '/image:\s*/{print $2}' calico-v3.32.1.yaml | sort -u
输出:
m.daocloud.io/quay.io/calico/cni:v3.32.1 m.daocloud.io/quay.io/calico/kube-controllers:v3.32.1 m.daocloud.io/quay.io/calico/node:v3.32.1
3,部署calico
kubectl apply -f calico-v3.32.1.yaml
# 等待calico全部pod就绪
kubectl get pods -n kube-system -w
# 全部ready后节点状态变为Ready
kubectl get nodes
其他配套清单(按需下载)
# 带Typha(50节点以上集群)
curl -O calico-typha-v3.32.1.yaml https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/calico-typha.yaml
# canal(calico网络策略+flannel底层)
curl -O canal-v3.32.1.yaml https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/canal.yaml
七,运维命令
1. Master 节点允许调度 Pod(单 master 无多余机器时)
kubectl taint nodes k8s-master-01 node-role.kubernetes.io/control-plane:NoSchedule-
2. 配置 kubectl 命令自动补全
apt install bash-completion -y echo 'source <(kubectl completion bash)' >> ~/.bashrc echo 'alias k=kubectl' >> ~/.bashrc echo 'complete -F __start_kubectl k' >> ~/.bashrc source ~/.bashrc
3. 常见故障排查命令
# kubelet日志 journalctl -u kubelet -f # containerd日志 journalctl -u containerd # 查看集群事件 kubectl get events -w # 重置集群(重装场景) kubeadm reset -f rm -rf /var/lib/kubelet /etc/kubernetes