部署云上Jenkins
- 创建 namespace
$ kubectl create namespace devops
- 创建 PV 和 PVC
yaml文件准备(jenkins-pv-pvc.yaml)
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-pv
spec:
capacity:
storage: 20Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Delete
nfs:
path: /backup/data/jenkins
server: 10.16.16.41
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: jenkins-work-claim
namespace: devops
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
创建PVC:
$ kubectl create -f jenkins-pv-pvc.yaml
- 创建RBAC
yaml文件准备,jenkins-rbac.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: devops
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: jenkins
rules:
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["services"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get","list","watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: jenkins
namespace: devops
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins
namespace: devops
创建 RBAC
$ kubectl create -f jenkins-rbac.yaml
- 创建 Service 和 Deployment
jenkins-deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
namespace: devops
spec:
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
terminationGracePeriodSeconds: 10
serviceAccount: jenkins
containers:
- name: jenkins
image: 10.15.15.247/devops/jenkins:2.368
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: web
protocol: TCP
- containerPort: 50000
name: agent
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 1000m
memory: 1Gi
livenessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
readinessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
env:
- name: JAVA_OPTS
value: -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true
volumeMounts:
- name: jenkinshome
subPath: jenkins
mountPath: /var/jenkins_home
- name: jenkinshome
subPath: workspace
mountPath: /var/jenkins_home/workspace
securityContext:
fsGroup: 1000
volumes:
- name: jenkinshome
persistentVolumeClaim:
claimName: jenkins-work-claim
jenkins-svc.yaml
---
apiVersion: v1
kind: Service
metadata:
name: jenkins
namespace: devops
labels:
app: jenkins
spec:
selector:
app: jenkins
type: NodePort
ports:
- name: web
port: 8080
targetPort: web
nodePort: 30081
- name: agent
port: 50000
targetPort: agent
nodePort: 30082
创建 Service 和 Deployment,将 Jenkins 部署到 Kubernetes 集群中:
$ kubectl create -f jenkins-deployment.yaml
$ kubectl create -f jenkins-svc.yaml
Jenkins 初始化配置
等到服务启动成功后,可以根据 Node 节点的 IP:30081 端口就可以访问 jenkins 服务了,可以根据提示信息进行安装配置即可
获取初始密码:
$ kubectl exec -it $(kubectl get pods -n devops | awk '{print $1}' | grep jenkins) -n devops -- cat /var/jenkins_home/secrets/initialAdminPassword
配置动态Slave节点
登入 Jenkins 平台,安装 Kubernetes 插件, 依次点击系统管理(Manage Jenkins) -> 插件管理(Manage Plugins) -> Available -> 输入 Kubernetes 勾选安装,等待安装完成,然后重启jenkins,使之生效。
依次点击系统管理(Manage Jenkins)→节点管理(Manage Nodes and Clouds)→Configure Clouds→Add a new cloud→选择Kubernetes→Kubernetes Cloud details,然后填写Kubernetes和Jenkins配置信息。
注意namespace,这里填 devops(即当前Jenkins应用所属namespace),然后点击连接测试,如果出现 Connected to Kubernetes [K8S版本号] 的提示信息,就说明 Jenkins 已经可以和 Kubernetes 系统正常通信了。
Jenkins地址:http://jenkins.devops.svc.cluster.local:8080 其格式为:http://{服务名}.{namespace}.svc.cluster.local:8080
- 新增一个Pod Template,其名称和标签均取为ci
- 在Pod Template中添加容器模板列表
1) Git 容器
2) Maven 容器
3) Docker 容器
4) JNLP 容器
在以上容器列表中,jnlp 这个 Container Template 的运行命令和命令参数这两个参数内容都需要置空,其余 Container Template 里的一般都保持默认值即可。 在每个 Pod Template 中都需要添加上一个 jnlp 容器(名称必须取为 jnlp),其作用是让 Jenkins 的 Master 和 Slave 节点之间保持交互通信。
- 主机目录挂载
挂载以下主机目录,用于Pod中的容器能够共享宿主机的Docker,实现Docker in Docker的方式
- 工作空间卷
在容器云环境新增一个PVC,取名为jenkins-slave-yc-work-claim(参考jenkins-pv-pvc.yaml),其目的在于将动态Slave节点运行时生成的工作空间持久化。
补充:因Pod Slave每次执行完就自己销毁了,如果一个流水线周期中会有启动两个或多个Pod的情况,一般可将工作目录挂载到PVC上,实现Pod之间的数据共享。
- Pod Template在被流水线调度创建时的内部情况
流水线执行调度生成动态Slave节点时,如果指定的节点(即Pod Template)的容器列表配置了多少容器信息,那么在节点创建时,就会启动多少容器。
此示例中的 ci 这个 Pod Template,里边配置了 jnlp、git 、maven 以及 docker 这四个容器。当流水线执行时指定了此 Pod Template,运行期间节点内部将存在有四个容器,分别为 jnlp、git、maven 以及 docker。