Kubernetes自动扩缩容策略深度解析
Kubernetes自动扩缩容策略深度解析
引言
自动扩缩容是 Kubernetes 实现弹性伸缩的核心功能，能够根据实际负载自动调整 Pod 副本数。本文将深入探讨 Kubernetes 自动扩缩容的核心概念、配置方法和最佳实践。
一、自动扩缩容架构
1.1 扩缩容层次结构
┌─────────────────────────────────────────────────────────────┐ │ 自动扩缩容架构 │ ├─────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ 集群层扩缩容 │ │ │ │ - Cluster Autoscaler │ │ │ │ - 节点自动增删 │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ Pod 层扩缩容 │ │ │ │ - Horizontal Pod Autoscaler │ │ │ │ - 副本数自动调整 │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ 容器层扩缩容 │ │ │ │ - Vertical Pod Autoscaler │ │ │ │ - 资源请求/限制自动调整 │ │ │ └─────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────┘1.2 扩缩容类型对比
| 类型 | 描述 | 作用范围 | 触发条件 |
|---|---|---|---|
| HPA | 水平扩缩容 | Pod 副本数 | CPU/内存/自定义指标 |
| VPA | 垂直扩缩容 | 资源请求/限制 | 资源使用情况 |
| CA | 集群扩缩容 | 节点数量 | Pod 调度失败 |
二、Horizontal Pod Autoscaler 配置
2.1 基础 HPA 配置
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 802.2 自定义指标 HPA
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: api-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: api-server minReplicas: 3 maxReplicas: 15 metrics: - type: Pods pods: metric: name: requests-per-second target: type: AverageValue averageValue: 100 - type: Object object: describedObject: apiVersion: v1 kind: Service name: api-service metric: name: queue-length target: type: Value value: 502.3 外部指标 HPA
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: external-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: worker minReplicas: 1 maxReplicas: 20 metrics: - type: External external: metric: name: custom_queue_length selector: matchLabels: queue: processing target: type: AverageValue averageValue: 1000三、扩缩容行为配置
3.1 行为策略配置
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: controlled-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 2 maxReplicas: 10 behavior: scaleUp: stabilizationWindowSeconds: 60 policies: - type: Percent value: 50 periodSeconds: 60 - type: Pods value: 4 periodSeconds: 60 selectPolicy: Max scaleDown: stabilizationWindowSeconds: 300 policies: - type: Percent value: 30 periodSeconds: 60 - type: Pods value: 2 periodSeconds: 60 selectPolicy: Min metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 703.2 行为策略参数说明
| 参数 | 说明 | 默认值 |
|---|---|---|
stabilizationWindowSeconds | 稳定窗口时间 | 300s (scaleDown) / 0s (scaleUp) |
periodSeconds | 策略生效周期 | 60s |
value | 扩缩容数量/百分比 | - |
selectPolicy | 策略选择方式 | Max (scaleUp) / Min (scaleDown) |
四、Vertical Pod Autoscaler 配置
4.1 VPA 配置
apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: my-app-vpa spec: targetRef: apiVersion: apps/v1 kind: Deployment name: my-app updatePolicy: updateMode: Auto resourcePolicy: containerPolicies: - containerName: '*' minAllowed: cpu: 100m memory: 128Mi maxAllowed: cpu: 2 memory: 4Gi controlledResources: - cpu - memory4.2 VPA 更新模式
| 模式 | 描述 | 适用场景 |
|---|---|---|
| Off | 仅建议，不自动更新 | 测试阶段 |
| Initial | 仅在 Pod 创建时应用 | 生产环境初期 |
| Recreate | 自动重建 Pod 应用 | 需要立即生效 |
| Auto | 自动更新，优雅处理 | 成熟生产环境 |
五、Cluster Autoscaler 配置
5.1 CA 配置
apiVersion: v1 kind: ConfigMap metadata: name: cluster-autoscaler-config data: config.yaml: | scan-interval: 10s scale-down-delay-after-add: 10m scale-down-delay-after-delete: 5m scale-down-delay-after-failure: 3m scale-down-unneeded-time: 10m scale-down-unready-time: 20m scale-down-critical-add-pod-timeout: 10m max-graceful-termination-sec: 6005.2 CA 部署
apiVersion: apps/v1 kind: Deployment metadata: name: cluster-autoscaler spec: replicas: 1 selector: matchLabels: app: cluster-autoscaler template: metadata: labels: app: cluster-autoscaler spec: serviceAccountName: cluster-autoscaler containers: - name: cluster-autoscaler image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.25.0 command: - ./cluster-autoscaler - --v=4 - --cloud-provider=aws - --skip-nodes-with-local-storage=false - --expander=least-waste - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster六、自定义指标适配器
6.1 部署指标适配器
apiVersion: apps/v1 kind: Deployment metadata: name: custom-metrics-adapter spec: replicas: 1 selector: matchLabels: app: custom-metrics-adapter template: metadata: labels: app: custom-metrics-adapter spec: containers: - name: adapter image: registry.k8s.io/custom-metrics-apiserver:v0.11.0 args: - --secure-port=6443 - --cert-dir=/tmp - --metrics-relist-interval=30s6.2 指标 API 服务
apiVersion: apiregistration.k8s.io/v1 kind: APIService metadata: name: v1beta1.custom.metrics.k8s.io spec: service: name: custom-metrics-adapter namespace: kube-system group: custom.metrics.k8s.io version: v1beta1 insecureSkipTLSVerify: true七、扩缩容最佳实践
7.1 资源请求配置
resources: requests: cpu: "200m" memory: "512Mi" limits: cpu: "1" memory: "2Gi"7.2 监控告警配置
apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: autoscaler-alerts spec: groups: - name: autoscaler_rules rules: - alert: HPAScaleUpFailed expr: hpa_scaling_failed_total{reason!="FailedGetScale"} > 0 for: 5m labels: severity: critical annotations: summary: "HPA 扩缩容失败" description: "{{ $labels.namespace }}/{{ $labels.name }} 扩缩容失败" - alert: HPAAtMaxReplicas expr: hpa_replicas_current / hpa_replicas_max >= 0.9 for: 10m labels: severity: warning annotations: summary: "HPA 达到最大副本数" description: "{{ $labels.namespace }}/{{ $labels.name }} 已达到最大副本数"7.3 扩缩容测试
apiVersion: batch/v1 kind: Job metadata: name: hpa-load-test spec: template: spec: containers: - name: load-test image: busybox command: ["sh", "-c", "while true; do wget -q -O /dev/null http://my-app.default.svc.cluster.local; sleep 0.1; done"] restartPolicy: Never parallelism: 10八、常见问题与解决方案
8.1 HPA 不扩缩容
问题分析：
- 指标数据未正确采集
- 目标指标未达到阈值
- 扩缩容策略配置不当
解决方案：
# 检查 HPA 状态 kubectl get hpa my-app-hpa # 检查指标数据 kubectl top pods # 查看 HPA 事件 kubectl describe hpa my-app-hpa8.2 扩缩容抖动
问题分析：
- 稳定窗口时间过短
- 指标波动较大
- 策略配置过于激进
解决方案：
behavior: scaleUp: stabilizationWindowSeconds: 120 scaleDown: stabilizationWindowSeconds: 6008.3 VPA 与 HPA 冲突
问题分析：
- VPA 修改资源请求影响 HPA 计算
- 两个控制器同时调整
解决方案：
# VPA 只控制内存，HPA 控制 CPU resourcePolicy: containerPolicies: - containerName: '*' controlledResources: - memory结论
自动扩缩容是 Kubernetes 弹性能力的核心体现。通过合理配置 HPA、VPA 和 Cluster Autoscaler，可以实现集群资源的高效利用和自动调优。在实际应用中，需要根据业务特点配置合适的扩缩容策略和阈值，避免抖动和资源浪费。