部署jenkins-master
创建相应的role、rolebinding、sa 创建jenkins master的image
环境说明:
- k8s 1.11
- docker: ce-18.09.0
- storageclass: nfs-dynamic-class
- 私有仓库:harbor 192.168.3.27:8888
创建相应的role、rolebinding、sa
因为k8s 1.6引入了RBAC,所以在部署jenkins的时候也应该加入相应的元素
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jenkins
rules:
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["services"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins
创建jenkins master的image
Dockerfile如下:
FROM jenkins/jenkins:lts-alpine
USER root
打包生成镜像
docker build -t 192.168.3.27:8888/ops/jenkins:lts-alpine .
docker push 192.168.3.27:8888/ops/jenkins:lts-alpine
创建拉取镜像的sercet
因为是在k8s中使用docker拉取镜像,所以需要生成secret,详见 私有镜像仓库介绍
[root@master k8s-jenkins]# cat registry-pull-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: registry-pull-secret
namespace: default
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjMuMjc6ODg4OCI6IHsKCQkJImF1dGgiOiAiZVdaNmVEb3hjV0Y2UUZkVFdBPT0iCgkJfQoJfSwKCSJIdHRwSGVhZGVycyI6IHsKCQkiVXNlci1BZ2VudCI6ICJEb2NrZXItQ2xp
ZW50LzE4LjA5LjAgKGxpbnV4KSIKCX0KfQ==
kubectl create -f registry-pull-secret.yaml
在k8s集群中部署jenkins master
部署pvc
因为我已经配置好了storageclass,所以在这个地方直接使用的是PVC
# kubectl get storageclass
NAME PROVISIONER AGE
nfs-dynamic-class fuseim.pri/ifs 46d
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-jenkins-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: nfs-dynamic-class
resources:
requests:
storage: 10Gi
部署jenkins
我在这个地方选择的是StatefulSet控制器, 当然也可以选择deployment的控制器
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: jenkins
labels:
name: jenkins
spec:
serviceName: jenkins
replicas: 1
updateStrategy:
type: RollingUpdate
template:
metadata:
name: jenkins
labels:
name: jenkins
spec:
terminationGracePeriodSeconds: 10
serviceAccountName: jenkins
imagePullSecrets:
- name: registry-pull-secret
containers:
- name: jenkins
image: 192.168.3.27:8888/ops/jenkins:lts-alpine
imagePullPolicy: Always
ports:
- containerPort: 8080
- containerPort: 50000
resources:
limits:
cpu: 1
memory: 1Gi
requests:
cpu: 0.5
memory: 500Mi
env:
- name: LIMITS_MEMORY
valueFrom:
resourceFieldRef:
resource: limits.memory
divisor: 1Mi
- name: JAVA_OPTS
value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
livenessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
volumes:
- name: jenkins-home
persistentVolumeClaim:
claimName: nfs-jenkins-pvc
注意几个地方：
1.
imagePullSecrets:
- name: registry-pull-secret
2.
image: 192.168.3.27:8888/ops/jenkins:lts-alpine
3.
volumes:
- name: jenkins-home
persistentVolumeClaim:
claimName: nfs-jenkins-pvc
部署service
apiVersion: v1
kind: Service
metadata:
name: jenkins
annotations:
spec:
type: NodePort
selector:
name: jenkins
ports:
-
name: http
port: 80
targetPort: 8080
protocol: TCP
nodePort: 30001
-
name: agent
port: 50000
protocol: TCP
最终要达到的效果
[root@master k8s-jenkins]# kubectl get pvc,statefulset,pod,svc | grep jenkins
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/nfs-jenkins-pvc Bound pvc-219bc06f-f3a7-11e8-bc01-525400088c01 10Gi RWO nfs-dynamic-class 14d
NAME DESIRED CURRENT AGE
statefulset.apps/jenkins 1 1 14d
NAME READY STATUS RESTARTS AGE
pod/jenkins-0 1/1 Running 1 4d
pod/nfs-client-provisioner-6db86bc775-5zbn7 1/1 Unknown 21 23d
pod/nfs-client-provisioner-6db86bc775-lb945 1/1 Running 7 3d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/jenkins NodePort 10.1.210.74 <none> 80:30001/TCP,50000:20990/TCP 14d
service/kubernetes ClusterIP 10.1.0.1 <none> 443/TCP 55d