云原生Kubernetes: K8S 1.29版本部署GitLab

一、实验

1.环境

2.搭建NFS

3.K8S 1.29版本部署Redis

4.K8S 1.29版本部署Postgresql

5.K8S 1.29版本部署GitLab

6.K8S 部署istio微服务

7.K8S 部署ingress应用路由

二、问题

1.K8S部署gitlab报错

2.gitlab创建失败

3.生成网关资源报错

4.安装istio 报错

5.istio-ingressgateway 一直处于pending状态

6.istio如何实现自动注入 sidecar

7.K8S容器从公钥接收失败

一、实验

1.环境

（1）主机

表1 主机

主机	架构	版本	IP	备注
master	K8S master节点	1.29.0	192.168.204.8
node1	K8S node节点	1.29.0	192.168.204.9
node2	K8S node节点	1.29.0	192.168.204.10	已部署Kuboard

（2）master节点查看集群

1）查看node
kubectl get node
 
2)查看node详细信息
kubectl get node -o wide

（3）查看pod

[root@master ~]# kubectl get pod -A

(4) 访问Kuboard

http://192.168.204.10:30080/kuboard/cluster

查看节点

2.搭建NFS

(1)检查并安装rpcbind和nfs-utils软件包

[root@master ~]# rpm -q rpcbind nfs-utils

（2）创建目录并授权

[root@master ~]# mkdir -p /opt/k8s

[root@master ~]# chmod 777 k8s/

(3)打开nfs的配置文件

[root@master opt]# vim /etc/exports

（4）配置文件

给所有网段用户赋予读写权限、同步内容、不压缩共享对象root用户权限

/opt/k8s *(rw,sync,no_root_squash)

（5）先后开启rpcbind、nfs服务并热加载配置文件内容，查看本机发布的nfs共享目录

[root@master opt]# systemctl start rpcbind
[root@master opt]# systemctl start nfs

（6）监听端口

[root@master opt]# ss -antp | grep rpcbind

（7）查看共享

[root@master opt]# showmount -e

其他节点查看

[root@node1 ~]# showmount -e master

3.K8S 1.29版本部署Redis

（1）查阅

第三方镜像仓库

https://hub.docker.com/u/sameersbn

镜像(Gitlab主要涉及到3个应用：Redis、Postgresql、Gitlab 核心程序，实际上只要将这3个应用分别启动起来，然后加上对应的配置就可以方便快速的安装 Gitlab )

1）redis
sameersbn/redis

2）postgresql
sameersbn/postgresql

3）gitlab
sameersbn/gitlab

（2）创建redis的pv

[root@master ~]# vim pv-redis.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-redis
spec:
  capacity:
    storage: 2Gi 
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "pv-redis"
  nfs:
    path: /opt/k8s
    server: 192.168.204.8

（3）生成资源

[root@master ~]# kubectl apply -f pv-redis.yaml

（4）查看pv

[root@master ~]# kubectl get pv

(5)拉取镜像

node1

[root@node1 ~]# docker pull sameersbn/redis:latest

(6) 导出镜像

[root@node1 ~]# docker save -o redis.tar sameersbn/redis:latest

（7）复制Docker镜像到node2节点

[root@node1 ~]# scp redis.tar  root@node2:~

（8）node2节点导入Docker镜像

[root@node2 ~]# docker load -i redis.tar

(9)创建名称空间

[root@master ~]# kubectl create ns devops

(10)部署redis

[root@master ~]# vim redis.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc
  namespace: devops
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: "pv-redis"
  resources:
    requests:
      storage: 2Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: devops
  labels:
    name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      name: redis
  template:
    metadata:
      name: redis
      labels:
        name: redis
    spec:
      containers:
        - name: redis
          image: sameersbn/redis:latest
          imagePullPolicy: IfNotPresent
          ports:
            - name: redis
              containerPort: 6379
          volumeMounts:
            - mountPath: /var/lib/redis
              name: data
              subPath: redis
          livenessProbe:
            exec:
              command:
                - redis-cli
                - ping
            initialDelaySeconds: 30
            timeoutSeconds: 5
          readinessProbe:
            exec:
              command:
                - redis-cli
                - ping
            initialDelaySeconds: 5
            timeoutSeconds: 1
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: redis-pvc

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: devops
  labels:
    name: redis
spec:
  ports:
    - name: redis
      port: 6379
      targetPort: redis
  selector:
    name: redis

（11）生成资源

[root@master ~]# kubectl apply -f redis.yaml

（12）查看pv,pvc

[root@master ~]# kubectl get pv

[root@master ~]# kubectl get pvc -n devops

4.K8S 1.29版本部署Postgresql

(1)创建postgresql的pv

[root@master ~]# vim pv-postgresql.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-postgresql
spec:
  capacity:
    storage: 2Gi 
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "pv-postgresql"
  nfs:
    path: /opt/k8s
    server: 192.168.204.8

（2）生成资源

[root@master ~]# kubectl apply -f pv-postgresql.yaml

(3)拉取镜像

node1

[root@node1 ~]# docker pull sameersbn/postgresql:12-20200524

(4) 导出镜像

[root@node1 ~]# docker save -o postgresql.tar sameersbn/postgresql:12-20200524

（7）复制Docker镜像到node2节点

[root@node1 ~]# scp postgresql.tar  root@node2:~

（8）node2节点导入Docker镜像

[root@node2 ~]# docker load -i postgresql.tar

(9)部署postgresql

[root@master ~]# vim postgresql.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgresql-pvc
  namespace: devops
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: "pv-postgresql"
  resources:
    requests:
      storage: 2Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  namespace: devops
  labels:
    name: postgresql
spec:
  replicas: 1
  selector:
    matchLabels:
      name: postgresql
  template:
    metadata:
      name: postgresql
      labels:
        name: postgresql
    spec:
      containers:
        - name: postgresql
          image: sameersbn/postgresql:12-20200524
          imagePullPolicy: IfNotPresent
          env:
            - name: DB_USER
              value: gitlab
            - name: DB_PASS
              value: passw0rd
            - name: DB_NAME
              value: gitlab_production
            - name: DB_EXTENSION
              value: pg_trgm,btree_gist
          ports:
            - name: postgres
              containerPort: 5432
          volumeMounts:
            - mountPath: /var/lib/postgresql
              name: data
              subPath: postgresql
          livenessProbe:
            exec:
              command:
                - pg_isready
                - -h
                - localhost
                - -U
                - postgres
            initialDelaySeconds: 5
            timeoutSeconds: 1
          readinessProbe:
            exec:
              command:
                - pg_isready
                - -h
                - localhost
                - -U
                - postgres
            initialDelaySeconds: 5
            timeoutSeconds: 1
          startupProbe:
            exec:
              command:
                - pg_isready
                - -h
                - localhost
                - -U
                - postgres
            initialDelaySeconds: 90
            periodSeconds: 5
            failureThreshold: 100
            timeoutSeconds: 1
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: postgresql-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: devops
  labels:
    name: postgresql
spec:
  ports:
    - name: postgres
      port: 5432
      targetPort: 5432
  selector:
    name: postgresql

(10) 生成资源

[root@master ~]# kubectl apply -f postgresql.yaml

（11）查看pv,pvc

[root@master ~]# kubectl get pv -n devops

[root@master ~]# kubectl get pvc -n devops

5.K8S 1.29版本部署GitLab

(1)创建gitlab的pv

[root@master ~]# vim pv-gitlab.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-gitlab
spec:
  capacity:
    storage: 2Gi 
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: "pv-gitlab"
  nfs:
    path: /opt/k8s
    server: 192.168.204.8

（2）生成资源

[root@master ~]# kubectl apply -f pv-gitlab.yaml

(3)拉取镜像

node2

[root@node1 ~]# docker pull sameersbn/gitlab:15.6.0

(4) 导出镜像

[root@node2 ~]# docker save -o gitlab.tar sameersbn/gitlab:15.6.0

（7）复制Docker镜像到node1节点

[root@node2 ~]# scp gitlab.tar  root@node1:~

（8）node1节点导入Docker镜像

[root@node1 ~]# docker load -i gitlab.tar

(9) 部署gitlab

[root@master ~]# vim gitlab.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gitlab-pvc
  namespace: devops
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: "pv-gitlab"
  resources:
    requests:
      storage: 2Gi
---

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: devops
  name: gitlab-sa
  labels:
    account: gitlab

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab
  namespace: devops
  labels:
    app: gitlab
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitlab
      version: v1
  template:
    metadata:
      labels:
        app: gitlab
        version: v1
    spec:
      serviceAccountName: gitlab-sa
      containers:
        - name: gitlab
          image: sameersbn/gitlab:15.6.0
          imagePullPolicy: IfNotPresent
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: GITLAB_TIMEZONE
              value: Beijing
            - name: GITLAB_SECRETS_DB_KEY_BASE
              value: long-and-random-alpha-numeric-string
            - name: GITLAB_SECRETS_SECRET_KEY_BASE
              value: long-and-random-alpha-numeric-string
            - name: GITLAB_SECRETS_OTP_KEY_BASE
              value: long-and-random-alpha-numeric-string
            - name: GITLAB_ROOT_PASSWORD
              value: admin123
            - name: GITLAB_ROOT_EMAIL
              value: 7jjw@163.com
            - name: GITLAB_HOST
              value: gitlab.site
            - name: GITLAB_PORT
              value: "80"
            - name: GITLAB_SSH_PORT
              value: "31022"
            - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
              value: "true"
            - name: GITLAB_NOTIFY_PUSHER
              value: "false"
            - name: GITLAB_BACKUP_SCHEDULE
              value: daily
            - name: GITLAB_BACKUP_TIME
              value: 01:00
            - name: DB_TYPE
              value: postgres
            - name: DB_HOST
              value: postgresql
            - name: DB_PORT
              value: "5432"
            - name: DB_USER
              value: gitlab
            - name: DB_PASS
              value: passw0rd
            - name: DB_NAME
              value: gitlab_production
            - name: REDIS_HOST
              value: redis
            - name: REDIS_PORT
              value: "6379"
          ports:
            - name: http
              containerPort: 80
            - name: ssh
              containerPort: 22
          volumeMounts:
            - mountPath: /home/git/data
              name: data
              subPath: gitlab
          livenessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 180
            timeoutSeconds: 5
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 5
            timeoutSeconds: 1
          startupProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 90
            periodSeconds: 5
            failureThreshold: 100
            timeoutSeconds: 1
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: gitlab-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  namespace: devops
  labels:
    app: gitlab
    service: gitlab
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: ssh
      port: 22
      targetPort: ssh
  selector:
    app: gitlab

(10) 生成资源

[root@master ~]# kubectl apply -f gitlab.yaml

（11）查看pv,pvc

[root@master ~]# kubectl get pv -n devops

[root@master ~]# kubectl get pvc -n devops

(12) 查看pod,svc

[root@master ~]# kubectl get pod,svc -n devops

（13）Kuboard查看

工作负载

容器组

服务

存储

6.K8S 部署istio微服务

（1）查阅

https://github.com/istio/istio/releases

（2）选择版本

https://github.com/istio/istio/releases/tag/1.18.2

（3）master节点解压

[root@master ~]# tar zxvf istio-1.18.2-linux-amd64.tar.gz

（4）切换到istio包所在目录

[root@master ~]# cd istio-1.18.2/
[root@master istio-1.18.2]# ls

samples/目录下，有示例应用程序；

bin/目录下，有istioctl客户端文件。istioctl工具用于手动注入Envoy sidecar代理。

（5）把istioctl这个可执行文件拷贝到/bin目录

[root@master istio-1.18.2]# cp /root/istio-1.18.2/bin/istioctl /bin/

（6）node节点导入镜像

node1

[root@node1 ~]# docker load -i istio1.18.tar.gz

node2

[root@node2 ~]# docker load -i istio1.18.tar.gz

(7) 安装istio

[root@master istio-1.18.2]# istioctl install --set profile=demo -y
? Istio core installed                                                                                                    
? Istiod installed                                                                                                        
? Ingress gateways installed                                                                                              
? Egress gateways installed                                                                                               
? Installation complete                                                                                                   Making this installation the default for injection and validation.

（8）验证

[root@master istio-1.18.2]# kubectl get pods -n istio-system

（9）Kuboard查看

（10）创建网关

[root@master ~]# vim gitlab-gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gitlab-gateway
  namespace: devops
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "gitlab.site"

[root@master ~]#  kubectl apply -f gitlab-gateway.yaml

（11）创建虚拟服务

[root@master ~]# vim gitlab-vs.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: gitlab-vs
  namespace: devops
spec:
  hosts:
  - "gitlab.site"
  gateways:
  - gitlab-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: gitlab
        port:
          number: 80

[root@master ~]# kubectl apply -f gitlab-vs.yaml

（12）查看网关

[root@master ~]# kubectl get gateway -n devops

（13）查看虚拟服务

[root@master ~]# kubectl get virtualservice -n devops

（14）通过istio提供的入口网关访问pod

[root@master ~]# kubectl get svc -n istio-system

（15）查看关联

[root@master ~]# kubectl get pods -n istio-system -owide

istio-ingressgateway是service资源，关联的pod是istio-system名称空间叫做iistio-ingressgateway-6d9f6c64cb-nldhf的pod

（16）查看istio-ingressgateway这个service的详细信息

[root@master ~]# kubectl describe svc istio-ingressgateway -n istio-system
Name:                     istio-ingressgateway
Namespace:                istio-system
Labels:                   app=istio-ingressgateway
                          install.operator.istio.io/owning-resource=unknown
                          install.operator.istio.io/owning-resource-namespace=istio-system
                          istio=ingressgateway
                          istio.io/rev=default
                          operator.istio.io/component=IngressGateways
                          operator.istio.io/managed=Reconcile
                          operator.istio.io/version=1.18.2
                          release=istio
Annotations:              <none>
Selector:                 app=istio-ingressgateway,istio=ingressgateway
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.97.137.224
IPs:                      10.97.137.224
Port:                     status-port  15021/TCP
TargetPort:               15021/TCP
NodePort:                 status-port  30820/TCP
Endpoints:                10.244.166.162:15021
Port:                     http2  80/TCP
TargetPort:               8080/TCP
NodePort:                 http2  31447/TCP
Endpoints:                10.244.166.162:8080
Port:                     https  443/TCP
TargetPort:               8443/TCP
NodePort:                 https  31205/TCP
Endpoints:                10.244.166.162:8443
Port:                     tcp  31400/TCP
TargetPort:               31400/TCP
NodePort:                 tcp  30086/TCP
Endpoints:                10.244.166.162:31400
Port:                     tls  15443/TCP
TargetPort:               15443/TCP
NodePort:                 tls  32071/TCP
Endpoints:                10.244.166.162:15443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

（17）Kuboard查看

工作负载

容器组

服务

7.K8S 部署ingress应用路由

(1)K8S进入容器查看

[root@master ~]# kubectl exec -it gitlab-84d7ff8cc6-k2kh9 -n devops /bin/bash

安装net-tools

root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# apt-get install net-tools

安装lsof

root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# apt-get install lsof

（2）监听端口

root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# netstat -antlp

curl测试

curl 127.0.0.1

lsof

lsof -i

 lsof -i:80

（3）master节点查看svc

ingress-nginx-controller 默认是LoadBalancer，一直为pending状态

[root@master ~]# kubectl get svc -n ingress-nginx

（4）修改svc

[root@master ~]# kubectl edit svc ingress-nginx-controller -n ingress-nginx

修改前：

修改后：

（5）Kuboard查看

（6）部署ingress

[root@master ~]# vim ingress-gitlab.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-gitlab
  namespace: devops
spec:
  ingressClassName: "nginx"
  rules:
  - host: gitlab.site
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: gitlab
            port:
              number: 80

（7）生成资源

[root@master ~]# kubectl apply -f ingress-gitlab.yaml

（8）查看ingress

[root@master ~]# kubectl get ingress -n devops

（9）详细查看

[root@master ~]# kubectl describe ingress ingress-gitlab -n devops
Name:             ingress-gitlab
Labels:           <none>
Namespace:        devops
Address:          10.101.23.182
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host         Path  Backends
  ----         ----  --------
  gitlab.site  
               /   gitlab:80 (10.244.166.159:80)
Annotations:   <none>
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    17m (x2 over 17m)  nginx-ingress-controller  Scheduled for sync
  Normal  Sync    17m (x2 over 17m)  nginx-ingress-controller  Scheduled for sync

（10）Kuboard查看

应用路由

详细信息

（11）master节点修改hosts

[root@master ~]# vim /etc/hosts

（11）curl测试

[root@master ~]# curl gitlab.site:31820

（12）物理机修改hosts

（13）访问系统

http://gitlab.site:31820

（14）输入用户名和密码

账号：root
密码：admin123

（15）成功进入系统

二、问题

1.K8S部署gitlab报错

（1）报错

Warning  Unhealthy  2m43s (x15 over 3m53s)  kubelet            Startup probe failed: Get "http://10.244.166.144:80/": dial tcp 10.244.166.144:80: connect: connection refused
  Warning  Unhealthy  23s (x28 over 2m38s)    kubelet            Startup probe failed: HTTP probe failed with statuscode: 502

（2）原因分析

gitlab镜像版本的问题，使用的版本有问题导致启动失败。

1）修改sameersbn仓库镜像：
sameersbn/gitlab:15.6.0

2）其他支持的gitlab仓库镜像：
gitlab/gitlab-ce:14.0.0-ce.0或者gitlab/gitlab-ce:15.6.0-ce.0

（3）解决方法

删除资源

修改部署文件的gitlab镜像版本：

换了镜像后，启动pod成功，但用describe命令查看描述日志，仍然出现了开始的警告内容
此时可尝试修改readinessProbe参数中的initialDelaySeconds和timeoutSeconds
分别修改为180和5。

修改前：

修改后：（此举用意在于增加初始化延迟时间和超时时间来避免时间过短导致步骤未成功走完就报错。）

2.gitlab创建失败

（1）报错

gitlab的pod启动失败

（2）原因分析

查看日志

[root@master ~]# kubectl logs -f gitlab-84d7ff8cc6-k2kh9 -n devops
Loading /etc/docker-gitlab/runtime/env-defaults
Initializing logdir...
Initializing datadir...
Generating OpenSSH host keys... RSA DSA ECDSA ED25519 
Container TimeZone -> Asia/Shanghai
Installing configuration templates...
Configuring gitlab...
Configuring gitlab::database....
Configuring gitlab::redis..
Configuring gitlab::actioncable
Configuring gitlab::secrets...
Configuring gitlab::sidekiq...
Configuring gitlab::gitaly...
Configuring gitlab::monitoring...
Configuring gitlab::gitlab-workhorse...
Configuring gitlab::puma...
Configuring gitlab::timezone...
Configuring gitlab::rack_attack...
Configuring gitlab::ci...
Configuring gitlab::artifacts...
Configuring gitlab::packages...
Configuring gitlab::terraform_state...
Configuring gitlab::lfs...
Configuring gitlab::uploads...
Configuring gitlab::mattermost...
Configuring gitlab::project_features...
Configuring gitlab::oauth...
Configuring gitlab::ldap...
Configuring gitlab::cron_jobs...
Configuring gitlab::backups...
Configuring gitlab::backups::schedule...
Configuring gitlab::registry...
Configuring gitlab::pages...
Configuring gitlab::sentry...
Configuring gitlab::content_security_policy...
Configuring gitlab-shell...
Configuring nginx...
Configuring nginx::gitlab...
2024-04-23 21:25:23,390 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in tu intend to run as root, you can set user=root in the config file to avoid this message.
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
2024-04-23 21:25:23,390 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2024-04-23 21:25:23,397 INFO RPC interface 'supervisor' initialized
2024-04-23 21:25:23,398 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-04-23 21:25:23,398 INFO supervisord started with pid 753
2024-04-23 21:25:24,402 INFO spawned: 'gitaly' with pid 763
2024-04-23 21:25:24,405 INFO spawned: 'puma' with pid 764
2024-04-23 21:25:24,409 INFO spawned: 'gitlab-workhorse' with pid 765
2024-04-23 21:25:24,412 INFO spawned: 'sidekiq' with pid 766
2024-04-23 21:25:24,415 INFO spawned: 'sshd' with pid 772
2024-04-23 21:25:24,418 INFO spawned: 'nginx' with pid 773
2024-04-23 21:25:24,421 INFO spawned: 'cron' with pid 778
2024-04-23 21:25:25,911 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:25:25,911 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:25:25,911 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (
2024-04-23 21:25:25,911 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
2024-04-23 21:25:25,911 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:25:25,911 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:25:25,912 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
psql: error: could not translate host name "postgresql" to address: Temporary failure in name resolution

重点是最后一行：

psql: error: could not translate host name "postgresql" to address: Temporary failure in name resolution

（3）解决方法

查看容器地址

[root@master ~]# kubectl get pod -o wide -n devops
NAME                          READY   STATUS    RESTARTS      AGE   IP               NODE    NOMINATED NODE   READINESS GATES
gitlab-84d7ff8cc6-k2kh9       0/1     Running   4 (66s ago)   14m   10.244.166.159   node1   <none>           <none>
postgresql-6d7dfcf685-nhmw5   1/1     Running   0             26m   10.244.166.157   node1   <none>           <none>
redis-6948bd4c7f-gp2ml        1/1     Running   0             49m   10.244.166.151   node1   <none>           <none>

K8S 进入容器添加hosts

[root@master ~]# kubectl exec -it gitlab-84d7ff8cc6-k2kh9 -n devops /bin/bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# cat /etc/hosts
# Kubernetes-managed hosts file.
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
fe00::0	ip6-mcastprefix
fe00::1	ip6-allnodes
fe00::2	ip6-allrouters
10.244.166.159	gitlab-84d7ff8cc6-k2kh9
root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# echo "10.244.166.157 postgresql" >> /etc/hosts
root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# echo "10.244.166.151 redis" >> /etc/hosts

查看

root@gitlab-84d7ff8cc6-k2kh9:/home/git/gitlab# cat /etc/hosts
# Kubernetes-managed hosts file.
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
fe00::0	ip6-mcastprefix
fe00::1	ip6-allnodes
fe00::2	ip6-allrouters
10.244.166.159	gitlab-84d7ff8cc6-k2kh9
10.244.166.157 postgresql
10.244.166.151 redis

再次查看日志

[root@master ~]# kubectl logs -f gitlab-84d7ff8cc6-k2kh9 -n devops
Loading /etc/docker-gitlab/runtime/env-defaults
Initializing logdir...
Initializing datadir...
Container TimeZone -> Asia/Shanghai
Installing configuration templates...
Configuring gitlab...
Configuring gitlab::database..
Configuring gitlab::redis...
Configuring gitlab::actioncable
Configuring gitlab::secrets...
Configuring gitlab::sidekiq...
Configuring gitlab::gitaly...
Configuring gitlab::monitoring...
Configuring gitlab::gitlab-workhorse...
Configuring gitlab::puma...
Configuring gitlab::timezone...
Configuring gitlab::rack_attack...
Configuring gitlab::ci...
Configuring gitlab::artifacts...
Configuring gitlab::packages...
Configuring gitlab::terraform_state...
Configuring gitlab::lfs...
Configuring gitlab::uploads...
Configuring gitlab::mattermost...
Configuring gitlab::project_features...
Configuring gitlab::oauth...
Configuring gitlab::ldap...
Configuring gitlab::cron_jobs...
Configuring gitlab::backups...
Configuring gitlab::backups::schedule...
Configuring gitlab::registry...
Configuring gitlab::pages...
Configuring gitlab::sentry...
Configuring gitlab::content_security_policy...
Configuring gitlab-shell...
Configuring nginx...
Configuring nginx::gitlab...
Setting up GitLab for firstrun. Please be patient, this could take a while...
2024-04-23 21:39:06,958 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in tu intend to run as root, you can set user=root in the config file to avoid this message.
2024-04-23 21:39:06,958 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2024-04-23 21:39:06,958 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
2024-04-23 21:39:06,958 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
2024-04-23 21:39:06,958 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
2024-04-23 21:39:06,958 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
2024-04-23 21:39:06,959 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2024-04-23 21:39:06,959 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
2024-04-23 21:39:06,959 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
2024-04-23 21:39:06,959 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2024-04-23 21:39:06,966 INFO RPC interface 'supervisor' initialized
2024-04-23 21:39:06,966 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-04-23 21:39:06,966 INFO supervisord started with pid 755
2024-04-23 21:39:07,970 INFO spawned: 'gitaly' with pid 768
2024-04-23 21:39:07,974 INFO spawned: 'puma' with pid 769
2024-04-23 21:39:07,977 INFO spawned: 'gitlab-workhorse' with pid 770
2024-04-23 21:39:07,980 INFO spawned: 'sidekiq' with pid 771
2024-04-23 21:39:07,983 INFO spawned: 'sshd' with pid 777
2024-04-23 21:39:07,986 INFO spawned: 'nginx' with pid 778
2024-04-23 21:39:07,989 INFO spawned: 'cron' with pid 782
2024-04-23 21:39:09,462 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:39:09,463 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:39:09,463 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (
2024-04-23 21:39:09,463 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
2024-04-23 21:39:09,463 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:39:09,463 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:39:09,463 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:39:40,078 INFO exited: puma (exit status 1; not expected)
2024-04-23 21:39:40,081 INFO spawned: 'puma' with pid 886
/home/git/gitlab/lib/gitlab/instrumentation/redis.rb:9: warning: already initialized constant Gitlab::Instrumentation::Red
/home/git/gitlab/lib/gitlab/instrumentation/redis.rb:9: warning: previous definition of ActionCable was here
2024-04-23 21:39:41,387 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:39:41,388 INFO exited: sidekiq (exit status 1; not expected)
2024-04-23 21:39:41,620 INFO spawned: 'sidekiq' with pid 887
2024-04-23 21:39:43,017 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
Database 'gitlab_production' already exists
psql:/home/git/gitlab/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping
psql:/home/git/gitlab/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping
2024-04-23 21:40:10,686 INFO exited: puma (exit status 1; not expected)
2024-04-23 21:40:10,689 INFO spawned: 'puma' with pid 919
2024-04-23 21:40:11,692 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:40:12,042 INFO exited: sidekiq (exit status 1; not expected)
2024-04-23 21:40:12,213 INFO spawned: 'sidekiq' with pid 920
2024-04-23 21:40:13,217 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
2024-04-23 21:40:40,234 INFO exited: puma (exit status 1; not expected)
2024-04-23 21:40:41,236 INFO spawned: 'puma' with pid 929
2024-04-23 21:40:42,140 INFO exited: sidekiq (exit status 1; not expected)
2024-04-23 21:40:42,832 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:40:42,835 INFO spawned: 'sidekiq' with pid 930
2024-04-23 21:40:43,837 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
2024-04-23 21:41:33,889 INFO exited: puma (exit status 1; not expected)
2024-04-23 21:41:34,767 INFO spawned: 'puma' with pid 942
2024-04-23 21:41:34,854 INFO exited: sidekiq (exit status 1; not expected)
2024-04-23 21:41:34,857 INFO spawned: 'sidekiq' with pid 943
2024-04-23 21:41:35,859 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:41:35,859 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
Migrating database...
/home/git/gitlab/lib/gitlab/instrumentation/redis.rb:9: warning: already initialized constant Gitlab::Instrumentation::Red
/home/git/gitlab/lib/gitlab/instrumentation/redis.rb:9: warning: previous definition of ActionCable was here
Clearing cache...
2024-04-23 21:43:41,207 WARN received SIGTERM indicating exit request
2024-04-23 21:43:41,208 INFO waiting for gitaly, puma, gitlab-workhorse, sidekiq, sshd, nginx, cron to die
2024-04-23 21:43:41,209 INFO stopped: cron (terminated by SIGTERM)
2024-04-23 21:43:41,209 INFO stopped: sshd (exit status 0)
2024-04-23 21:43:41,214 INFO stopped: nginx (exit status 0)
2024-04-23 21:43:44,231 INFO stopped: sidekiq (exit status 0)
2024-04-23 21:43:44,232 INFO waiting for gitaly, puma, gitlab-workhorse to die
2024-04-23 21:43:44,234 INFO stopped: gitlab-workhorse (exit status 1)
2024-04-23 21:43:47,238 INFO stopped: puma (terminated by SIGQUIT (core dumped))
2024-04-23 21:43:47,238 INFO waiting for gitaly to die
2024-04-23 21:43:47,274 INFO stopped: gitaly (exit status 1)
2024-04-23 21:43:47,533 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in tu intend to run as root, you can set user=root in the config file to avoid this message.
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
2024-04-23 21:43:47,534 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2024-04-23 21:43:47,541 INFO RPC interface 'supervisor' initialized
2024-04-23 21:43:47,541 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-04-23 21:43:47,542 INFO supervisord started with pid 1
2024-04-23 21:43:48,545 INFO spawned: 'gitaly' with pid 1093
2024-04-23 21:43:48,548 INFO spawned: 'puma' with pid 1094
2024-04-23 21:43:48,551 INFO spawned: 'gitlab-workhorse' with pid 1095
2024-04-23 21:43:48,555 INFO spawned: 'sidekiq' with pid 1096
2024-04-23 21:43:48,557 INFO spawned: 'sshd' with pid 1099
2024-04-23 21:43:48,560 INFO spawned: 'nginx' with pid 1103
2024-04-23 21:43:48,563 INFO spawned: 'cron' with pid 1108
2024-04-23 21:43:50,020 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:43:50,020 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:43:50,020 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (
2024-04-23 21:43:50,020 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
2024-04-23 21:43:50,020 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:43:50,021 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:43:50,021 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:44:08,020 WARN received SIGTERM indicating exit request
2024-04-23 21:44:08,020 INFO waiting for gitaly, puma, gitlab-workhorse, sidekiq, sshd, nginx, cron to die
2024-04-23 21:44:08,021 INFO stopped: cron (terminated by SIGTERM)
2024-04-23 21:44:08,022 INFO stopped: sshd (exit status 0)
2024-04-23 21:44:08,024 INFO stopped: nginx (exit status 0)
2024-04-23 21:44:08,066 INFO stopped: sidekiq (terminated by SIGTERM)
2024-04-23 21:44:08,068 INFO stopped: gitlab-workhorse (exit status 1)
2024-04-23 21:44:08,718 INFO stopped: puma (terminated by SIGQUIT (core dumped))
2024-04-23 21:44:08,752 INFO stopped: gitaly (exit status 1)
[root@master ~]# kubectl logs -f gitlab-84d7ff8cc6-k2kh9 -n devops
Loading /etc/docker-gitlab/runtime/env-defaults
Initializing logdir...
Initializing datadir...
Container TimeZone -> Asia/Shanghai
Installing configuration templates...
Configuring gitlab...
Configuring gitlab::database...
Configuring gitlab::redis...
Configuring gitlab::actioncable...
Configuring gitlab::secrets...
Configuring gitlab::sidekiq...
Configuring gitlab::gitaly...
Configuring gitlab::monitoring...
Configuring gitlab::gitlab-workhorse...
Configuring gitlab::puma...
Configuring gitlab::timezone...
Configuring gitlab::rack_attack...
Configuring gitlab::ci...
Configuring gitlab::artifacts...
Configuring gitlab::packages...
Configuring gitlab::terraform_state...
Configuring gitlab::lfs...
Configuring gitlab::uploads...
Configuring gitlab::mattermost...
Configuring gitlab::project_features...
Configuring gitlab::oauth...
Configuring gitlab::ldap...
Configuring gitlab::cron_jobs...
Configuring gitlab::backups...
Configuring gitlab::backups::schedule...
Configuring gitlab::registry...
Configuring gitlab::pages...
Configuring gitlab::sentry...
Configuring gitlab::content_security_policy...
Configuring gitlab-shell...
Configuring nginx...
Configuring nginx::gitlab...
2024-04-23 21:48:22,675 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in tu intend to run as root, you can set user=root in the config file to avoid this message.
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
2024-04-23 21:48:22,675 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2024-04-23 21:48:22,683 INFO RPC interface 'supervisor' initialized
2024-04-23 21:48:22,683 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-04-23 21:48:22,683 INFO supervisord started with pid 1
2024-04-23 21:48:23,688 INFO spawned: 'gitaly' with pid 772
2024-04-23 21:48:23,691 INFO spawned: 'puma' with pid 773
2024-04-23 21:48:23,695 INFO spawned: 'gitlab-workhorse' with pid 774
2024-04-23 21:48:23,698 INFO spawned: 'sidekiq' with pid 775
2024-04-23 21:48:23,701 INFO spawned: 'sshd' with pid 781
2024-04-23 21:48:23,704 INFO spawned: 'nginx' with pid 782
2024-04-23 21:48:23,707 INFO spawned: 'cron' with pid 785
2024-04-23 21:48:25,192 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:48:25,192 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:48:25,192 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (
2024-04-23 21:48:25,192 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
2024-04-23 21:48:25,192 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:48:25,192 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:48:25,192 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-04-23 21:51:20,541 INFO exited: sidekiq (exit status 1; not expected)
2024-04-23 21:51:21,546 INFO spawned: 'sidekiq' with pid 911
2024-04-23 21:51:23,016 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs
^C

成功：

3.生成网关资源报错

（1）报错

error: resource mapping not found for name: "gitlab-gateway" namespace: "devops" from "gitlab-gateway.yaml": no matches for kind "Gateway" in version "networking.istio.io/v1alpha3"
ensure CRDs are installed first

（2）原因分析

未安装istio。

（3）解决方法

安装istio:

成功：

4.安装istio 报错

（1）报错

? Egress gateways encountered an error: failed to wait for resource: resources not ready after 5m0s: context deadline exce
  Deployment/istio-system/istio-egressgateway (containers with unready status: [istio-proxy])
- Pruning removed resources

（2）原因分析

Egress的pod还未完全启动。

（3）解决方法

重新安装，等待egress加载完成。

5.istio-ingressgateway 一直处于pending状态

（1）报错

（2）原因分析

因为istio-ingressgateway的默认类型为LoadBalancer,没有公有云的话，可以修改为NodePort.

（3）解决方法

istio-ingressgateway的类型修改为NodePort：

[root@master ~]# kubectl edit svc istio-ingressgateway -n istio-system

修改前：

修改后：

成功：

[root@master ~]# kubectl get pods -n istio-system -owide

查看：

[root@master ~]# kubectl describe svc istio-ingressgateway -n istio-system
Name:                     istio-ingressgateway
Namespace:                istio-system
Labels:                   app=istio-ingressgateway
                          install.operator.istio.io/owning-resource=unknown
                          install.operator.istio.io/owning-resource-namespace=istio-system
                          istio=ingressgateway
                          istio.io/rev=default
                          operator.istio.io/component=IngressGateways
                          operator.istio.io/managed=Reconcile
                          operator.istio.io/version=1.18.2
                          release=istio
Annotations:              <none>
Selector:                 app=istio-ingressgateway,istio=ingressgateway
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.97.137.224
IPs:                      10.97.137.224
Port:                     status-port  15021/TCP
TargetPort:               15021/TCP
NodePort:                 status-port  30820/TCP
Endpoints:                10.244.166.162:15021
Port:                     http2  80/TCP
TargetPort:               8080/TCP
NodePort:                 http2  31447/TCP
Endpoints:                10.244.166.162:8080
Port:                     https  443/TCP
TargetPort:               8443/TCP
NodePort:                 https  31205/TCP
Endpoints:                10.244.166.162:8443
Port:                     tcp  31400/TCP
TargetPort:               31400/TCP
NodePort:                 tcp  30086/TCP
Endpoints:                10.244.166.162:31400
Port:                     tls  15443/TCP
TargetPort:               15443/TCP
NodePort:                 tls  32071/TCP
Endpoints:                10.244.166.162:15443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

6.istio如何实现自动注入 sidecar

（1）命令

需要为default命名空间打上标签istio-injection=enabled

[root@master ~]# kubectl label namespace default istio-injection=enabled

7.K8S容器从公钥接收失败

（1）报错

进入容器

[root@master ~]# kubectl exec -it gitlab-84d7ff8cc6-k2kh9 -n devops /bin/bash

更新源报错

W: GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>

（2）原因分析

无法检查签名：找不到公钥

（3）解决方法

备份更换源

cp sources.list source.list.bak

sudo sed -i 's/cn.archive.ubuntu.com/mirrors.ustc.edu.cn/g' /etc/apt/sources.list

sudo sed -i 's/security.ubuntu.com/mirrors.ustc.edu.cn/g' /etc/apt/sources.list

更新还是报错

清空源

echo > /etc/apt/source.list

更新源

echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic main restricted" >> /etc/apt/sources.list
echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates main restricted" >> /etc/apt/sources.list
echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic universe" >> /etc/apt/sources.list
echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates universe" >> /etc/apt/sources.list
echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic multiverse" >> /etc/apt/sources.list
echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates multiverse" >> /etc/apt/sources.list
echo "deb http://us.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb http://security.ubuntu.com/ubuntu bionic-security main restricted" >> /etc/apt/sources.list
echo "deb http://security.ubuntu.com/ubuntu bionic-security universe" >> /etc/apt/sources.list
echo "deb http://security.ubuntu.com/ubuntu bionic-security multiverse" >> /etc/apt/sources.list
echo "deb http://mirrors.ustc.edu.cn/ubuntu/ xenial main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-security main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-security main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse" >> /etc/apt/sources.list
echo "deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse" >> /etc/apt/sources.list

修改DNS服务器

echo "nameserver 8.8.8.8" >> /etc/resolv.conf

echo "nameserver 8.8.4.4" >> /etc/resolv.conf

导入

gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 23E7166788B63E1E

加入

sudo gpg --armor --export 23E7166788B63E1E | sudo apt-key add -

软件源更新成功：

apt-get update

软件更新（输入Y）

apt-get upgrade