分类： KUBERNETES学习

发布于

K8S容器时区问题解决

方法一

在制作容器镜像时候，将时区问题解决

RUN rm -f /etc/localtime && ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone

方法二

采用hostpath方式将主机时区挂载至容器内部
          volumeMounts:
            - mountPath: /etc/localtime
              name: time_name
      volumes:
        - hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai
            type: ''
          name: time_name
发布于

hpa自动扩容缩

kubectl autoscale deployment 名称 --cpu-percent=80 --min=3 --max=6 -n 命名空间
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: gsvauth-v1
  namespace: istio-system
spec:
  maxReplicas: 6
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: gsvauth-v1
  targetCPUUtilizationPercentage: 80
发布于

secret模板

生成系统变量的模板

apiVersion: v1
kind: Secret
metadata:
  name: gitlogin
  namespace: template
type: Opaque
data:
  username: xxxxx ## 直接base64直接转码后挂进来 使用"echo 字符串 | base64加密，使用echo 字符串 | base64 -d解密"
  password: xxxxx ## 此处密码被挂载到git服务中，其中含有@字符，必须将“@”转义成“%40”，然后进行base64转换成密码
---
apiVersion: batch/v1
kind: Job
metadata:
  name: git
  namespace: template
spec:
  template:
    metadata:
      name: git
      labels:
        app: git
        version: v1
      spec:
        containers:
          - name: git
            image: bitnami/git:latest
            workingDir: /tmp
            command:
              - "sh"
              - "-c"
              - git clone -b dev https://${username}:${password}@xxx.git && echo "job done!"
              #- env
            env:
              - name: username
                valueFrom:
                  secretKeyRef:
                    name: gitlogin
                    key: username
              - name: password
                valueFrom:
                  secretKeyRef:
                    name: gitlogin
                    key: password
        restartPolicy: OnFailure
发布于

k8s更新镜像的简单脚本

用途：

当镜像实际发生改变，但是tag不变的情况下进行操作

updateimage.sh

#!/bin/env bash
kubectl set image -n demo deployments.apps/demo-deployment  demo=demo.host.com/demo/demo:latest
if [ $? -ne 0 ]; then
        echo "set image faild"
        exit 1
else
        echo "set image successed"
        kubectl rollout restart -n demo deployments.apps/demo-deployment
fi
发布于

一组基础的k8s模板

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: template-nginx
    release: template-nginx
  name: template-deployment
  namespace: template
spec:
  replicas: 1
  selector:
    matchLabels:
      app: template-nginx
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      name: template-nginx
      namespace: template
      labels:
        app: template-nginx
        release: template-nginx
        releaseRevision: "v1"
    spec:
      containers:
        - name: template-nginx
          image: nginx:1.20-alpine
          imagePullPolicy: IfNotPresent
          ports:
          - name: template
            containerPort: 80
          volumeMounts:
          - name: template
            mountPath: /etc/nginx/conf.d/
      volumes:
          - name: template
            configMap:
              name: template

service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: template-nginx
    release: template-nginx
  name: template-nginx
  namespace: template
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: template-nginx
  type: ClusterIP

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: template-nginx
  namespace: template
  labels:
    ingress: template-nginx
    release: template-nginx
spec:
    rules:
      - host: temp.host.com
        http:
          paths:
            - path: /admin/
              backend: 
                service:
                  name: template-nginx
                  port:
                    number: 80
              pathType: Prefix
            - path: /
              backend:
                service:
                  name: template-nginx
                  port:
                    number: 80
              pathType: ImplementationSpecific

namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  labels:
    kubernetes.io/metadata.name: template
  name: template
spec:
  finalizers:
  - kubernetes

configmap.yaml

apiVersion: v1
data:
  default.conf: |+
    server {
        listen       80;
        listen  [::]:80;
        server_name  temp.host.com;
        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }

kind: ConfigMap
metadata:
  name: template
  namespace: template
发布于

ubuntu安装kubernetes

基础环境

四台主机，一主三从，ubuntu20.04

重点：主机名必须不同！必须不同！必须不同！

修改系统参数

  • 禁用swap
vim /etc/fstab
注释掉swap行
  • 确保时区和时间正确
timedatectl set-timezone Asia/Shanghai
  • master安装ntp服务
apt install ntp -y
  • node节点变更ntp服务器
vim /etc/ntp.conf
清空，只写如下行
server master节点ip或者内部域名
  • net.bridge.bridge-nf-call-iptables
vim /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
  • rp_filter
vim /etc/sysctl.d/10-network-security.conf

#将下面两个参数的值从2修改为1
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.rp_filter=1

部署docker

  • installdocker.sh
#!/bin/env bash
apt update
apt upgrade -y
apt install apt-transport-https ca-certificates curl software-properties-common -y
curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add - 
add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
apt update
apt install docker-ce docker-ce-cli containerd.io -y
  • daemon.json

最后一行是重点

 {
	    "registry-mirrors": ["https://0xj5rnq5.mirror.aliyuncs.com"],
	    "log-driver":"json-file",
	    "log-opts": {"max-size":"500m", "max-file":"3"},
	    "exec-opts": ["native.cgroupdriver=systemd"]
 }

部署kubernetes master

  • installk8s.sh
#!/bin/env bash
k8sversion=1.22.7-00
apt update && apt install -y ca-certificates curl software-properties-common apt-transport-https curl
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
tee /etc/apt/sources.list.d/kubernetes.list <<EOF 
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt update
apt install -y kubelet=${k8sversion} kubeadm=${k8sversion} kubectl=${k8sversion}
apt-mark hold kubelet kubeadm kubectl
  • 初始化master

kubeadm init --kubernetes-version=1.22.7 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/16 --apiserver-advertise-address=0.0.0.0 --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

  • 非root用户要做的
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
  • root用户要做的
/etc/profile
export KUBECONFIG=/etc/kubernetes/admin.conf
  • 安装calico网络插件（可选）
第一种：
wget -c https://docs.projectcalico.org/v3.21/manifests/calico.yaml
修改CALICO_IPV4POOL_CIDR，不能与宿主机冲突
kubectl apply -f calico.yaml

第二种：(安装正确，但是安装istio时候，ca过不去，不知道具体原因，暂时不适用这个方式)
wget https://docs.projectcalico.org/manifests/tigera-operator.yaml
wget https://docs.projectcalico.org/manifests/custom-resources.yaml
修改custom-resources.yaml,ippool修改的于pod-network一致
kubectl create -f tigera-operator.yaml
kubectl create -f custom-resources.yaml

给master打上污点
kubectl taint nodes --all node-role.kubernetes.io/master-
  • 安装flannel网络插件（可选）
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
  • 初始化后的输出
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.31.0.99:6443 --token 08gygc.dgz7t1gf1plglllc \
	--discovery-token-ca-cert-hash sha256:f4fbe6d79db14d9a038a01ba34e5fd460a3807b967c664e8658f0e4948e3beba

安装K8s worker节点

  • 执行 installk8s.sh
  • 加入节点

kubeadm join 172.31.0.99:6443 --token 08gygc.dgz7t1gf1plglllc \
	--discovery-token-ca-cert-hash sha256:f4fbe6d79db14d9a038a01ba34e5fd460a3807b967c664e8658f0e4948e3beba

删除节点

  • 在master执行
kubectl drain tqu4jnp1sot5qxtx-0003  --delete-emptydir-data --force --ignore-daemonsets
kuberctl delete nods tqu4jnp1sot5qxtx-0003
  • 在client执行
kuberctl reset
  • token失效后的处理
kubeadm token create --print-join-command

安装helm

installhelm.sh

#!/bin/env bash
curl https://baltocdn.com/helm/signing.asc | apt-key add -
apt install apt-transport-https --yes
echo "deb https://baltocdn.com/helm/stable/debian/ all main" | tee /etc/apt/sources.list.d/helm-stable-debian.list
apt update
apt install helm

istio

  • 安装istio
wget -c https://github.com/istio/istio/releases/download/1.13.1/istio-1.13.1-linux-amd64.tar.gz
tar -zxvf istio-1.13.1-linux-amd64.tar.gz
mv istio-1.13.1 istio
export $PWD/bin:$PATH
cd istio
istioctl install --set profile=demo -y
kubectl label namespace default istio-injection=enabled
  • 删除istio
istioctl experimental uninstall --purge 
kubectl delete namespace istio-system

更新证书到100年

下载kubernetes源码

wget -c https://github.com/kubernetes/kubernetes/archive/refs/tags/v1.22.7.tar.gz
tar -zxvf v1.22.7.tar.gz
mv ./kubernetes-1.22.7 /tmp/kubernetes
修改./staging/src/k8s.io/client-go/util/cert/cert.go
    // NotAfter: now.Add(duration365d * 10).UTC(),
    NotAfter:              now.Add(duration365d * 100).UTC(),
修改：./cmd/kubeadm/app/constants/constants.go
    // CertificateValidity = time.Hour * 24 * 365
    CertificateValidity = time.Hour * 24 * 365 * 100

docker run --rm -it -v /tmp/kubernetes:/go/src/k8s.io/kubernetes larryq/kube-cross:v-go1.17.7
#cd /go/src/k8s.io/kubernetes
make all WHAT=cmd/kubeadm GOFLAGS=-v
以下两个可以不编译
make all WHAT=cmd/kubectl GOFLAGS=-v
make all WHAT=cmd/kubelet GOFLAGS=-v

输出文件路径：/tmp/kubernetes/_output/local/bin/linux/amd64
将原kubeadm备份
mv /usr/bin/kubeadm /usr/bin/kubeadm-bak
cp /tmp/kubernetes/_output/local/bin/linux/amd64/kubeadm /usr/bin/
chmod a+x /usr/bin/kubeadm

如果在kubeadm init之前就处理了kubeadm，直接init即可，如果是之后，则需进行更新操作
检查有效期kubeadm certs check-expiration
更新证书：kubeadm certs renew all

发布于

kubernetes初始化错误

kunernetes初始化命令：

kubeadm init --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/16 --apiserver-advertise-address=0.0.0.0 --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

报错：

[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.

解决：
修改配置文件/etc/docker/daemon.json，添加以下行：

{"exec-opts": ["native.cgroupdriver=systemd"]}

原因：

这是cgroup驱动问题。