Jenkins Checkout插件逐步优化指南:每步优化效果对比分析
在持续集成和持续部署(CI/CD)流程中,代码检出阶段往往是构建任务中最耗时的环节之一。通过逐步优化Jenkins的Checkout插件配置,我们可以显著提升代码下载速度并减少磁盘空间占用。本文将详细分析每一步优化相对于前一步的效果,帮助您更好地理解优化原理和实际效果。
基准配置:基础Checkout配置
首先,我们从一个基础配置开始,该配置获取所有分支和标签:
pipeline {
agent {
label '192.168.4.5'
}
stages {
stage('下载代码') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: '**']],
extensions: [],
userRemoteConfigs: [
[
credentialsId: 'gitlab_id',
url: 'http://192.168.3.3/howlaisi/argocd-gitops.git'
]
]
])
sh "du -sh ./"
sh "git count-objects -v"
sh "du -sh .git"
sh "git branch -a"
sh "git tag -l"
}
}
}
}
执行后输出结果:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on 192.168.4.5 in /home/jenkins/workspace/下载代码
[Pipeline] {
[Pipeline] stage
[Pipeline] { (下载代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential gitlab_id
Cloning the remote Git repository
Cloning repository http://192.168.3.3/howlaisi/argocd-gitops.git
> git init /home/jenkins/workspace/下载代码 # timeout=10
Fetching upstream changes from http://192.168.3.3/howlaisi/argocd-gitops.git
> git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials gitlab_id
> git fetch --tags --progress http://192.168.3.3/howlaisi/argocd-gitops.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Multiple candidate revisions
Checking out Revision 750b6cf7015b7393c6de428e6434669b902a2e2b (origin/test1)
Commit message: "上传新文件"
First time build. Skipping changelog.
> git config remote.origin.url http://192.168.3.3/howlaisi/argocd-gitops.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Seen branch in repository origin/master
Seen branch in repository origin/test1
Seen branch in repository origin/test2
Seen 3 remote branches
> git show-ref --tags -d # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f 750b6cf7015b7393c6de428e6434669b902a2e2b # timeout=10
[Pipeline] sh
+ du -sh ./
19M ./
[Pipeline] sh
+ git count-objects -v
count: 0
size: 0
in-pack: 27426
packs: 1
size-pack: 15253
prune-packable: 0
garbage: 0
size-garbage: 0
[Pipeline] sh
+ du -sh .git
16M .git
[Pipeline] sh
+ git branch -a
* (detached from 750b6cf)
remotes/origin/master
remotes/origin/test1
remotes/origin/test2
[Pipeline] sh
+ git tag -l
tag_test1
tag_test2
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
基准配置分析
-
磁盘使用情况:
- 工作区大小:19M
- .git目录大小:16M
-
Git对象统计:
- Git对象数量:27426个
- 打包文件大小:15253KB(约15MB)
-
分支和标签信息:
- 获取了所有3个远程分支
- 获取了2个标签
这种配置会获取所有分支和标签,对于大型仓库可能会导致下载时间较长、占用更多磁盘空间和网络传输数据量大的问题。
优化步骤一:指定具体分支
第一步优化是明确指定需要检出的分支,而不是使用通配符**
:
pipeline {
agent {
label '192.168.4.5'
}
stages {
stage('下载代码') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: 'master']],
extensions: [],
userRemoteConfigs: [
[
credentialsId: 'gitlab_id',
url: 'http://192.168.3.3/howlaisi/argocd-gitops.git'
]
]
])
sh "du -sh ./"
sh "git count-objects -v"
sh "du -sh .git"
sh "git branch -a"
sh "git tag -l"
}
}
}
}
执行后输出结果:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on 192.168.4.5 in /home/jenkins/workspace/下载代码
[Pipeline] {
[Pipeline] stage
[Pipeline] { (下载代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential gitlab_id
Fetching changes from the remote Git repository
Checking out Revision a2851ebafe0eec1b56c0c863d9c2f792375f0d48 (origin/master)
Commit message: "build: automatic update of ruoyi-cloud-prod"
> git rev-parse --is-inside-work-tree # timeout=10
> git config remote.origin.url http://192.168.3.3/howlaisi/argocd-gitops.git # timeout=10
Fetching upstream changes from http://192.168.3.3/howlaisi/argocd-gitops.git
> git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials gitlab_id
> git fetch --tags --progress http://192.168.3.3/howlaisi/argocd-gitops.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
First time build. Skipping changelog.
[Pipeline] sh
+ du -sh ./
16M ./
[Pipeline] sh
+ git count-objects -v
count: 0
size: 0
in-pack: 27426
packs: 1
size-pack: 15253
prune-packable: 0
garbage: 0
size-garbage: 0
[Pipeline] sh
+ du -sh .git
16M .git
[Pipeline] sh
+ git branch -a
* (detached from a2851eb)
remotes/origin/master
remotes/origin/test1
remotes/origin/test2
[Pipeline] sh
+ git tag -l
tag_test1
tag_test2
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
优化步骤一对比分析
相对于基准配置的变化:
-
磁盘使用情况:
- 工作区大小:从19M减少到16M(减少15.8%)
- .git目录大小:保持16M不变
-
Git对象统计:
- Git对象数量:保持27426个不变
- 打包文件大小:保持15253KB不变
-
操作行为变化:
- 从"Cloning the remote Git repository"变为"Fetching changes from the remote Git repository"
- 仍然获取所有分支和标签
分析总结:
指定具体分支后,工作区大小有所减少,但Git对象数量和.git目录大小保持不变。这是因为虽然我们只检出了master分支,但仍获取了所有分支和标签的信息。下一步需要禁用标签获取以进一步优化。
优化步骤二:禁用标签获取
第二步优化是禁用标签获取以减少下载数据量:
pipeline {
agent {
label '192.168.4.5'
}
stages {
stage('下载代码') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: 'master']],
extensions: [
[
$class: 'CloneOption',
noTags: true
]
],
userRemoteConfigs: [
[
credentialsId: 'gitlab_id',
url: 'http://192.168.3.3/howlaisi/argocd-gitops.git'
]
]
])
sh "du -sh ./"
sh "git count-objects -v"
sh "du -sh .git"
sh "git branch -a"
sh "git tag -l"
}
}
}
}
执行后输出结果:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on 192.168.4.5 in /home/jenkins/workspace/下载代码
[Pipeline] {
[Pipeline] stage
[Pipeline] { (下载代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential gitlab_id
Cloning the remote Git repository
Avoid fetching tags
Cloning repository http://192.168.3.3/howlaisi/argocd-gitops.git
> git init /home/jenkins/workspace/下载代码 # timeout=10
Fetching upstream changes from http://192.168.3.3/howlaisi/argocd-gitops.git
> git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials gitlab_id
> git fetch --no-tags --progress http://192.168.3.3/howlaisi/argocd-gitops.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision a2851ebafe0eec1b56c0c863d9c2f792375f0d48 (origin/master)
Commit message: "build: automatic update of ruoyi-cloud-prod"
> git config remote.origin.url http://192.168.3.3/howlaisi/argocd-gitops.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
> git rev-list --no-walk a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
[Pipeline] sh
+ du -sh ./
16M ./
[Pipeline] sh
+ git count-objects -v
count: 0
size: 0
in-pack: 27424
packs: 1
size-pack: 15253
prune-packable: 0
garbage: 0
size-garbage: 0
[Pipeline] sh
+ du -sh .git
16M .git
[Pipeline] sh
+ git branch -a
* (detached from a2851eb)
remotes/origin/master
remotes/origin/test1
remotes/origin/test2
[Pipeline] sh
+ git tag -l
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
优化步骤二对比分析
相对于优化步骤一的变化:
-
磁盘使用情况:
- 工作区大小:保持16M不变
- .git目录大小:保持16M不变
-
Git对象统计:
- Git对象数量:从27426个减少到27424个(减少2个)
- 打包文件大小:保持15253KB不变
-
操作行为变化:
- 添加了"Avoid fetching tags"提示
git fetch
命令从--tags
变为--no-tags
分析总结:
禁用标签获取后,Git对象数量略微减少(减少了2个标签对象),但工作区和.git目录大小基本保持不变。这表明标签在该仓库中占用的空间很小。下一步需要启用浅层克隆以大幅减少Git对象数量。
优化步骤三:启用浅层克隆
第三步优化是启用浅层克隆,只获取最新的提交:
pipeline {
agent {
label '192.168.4.5'
}
stages {
stage('下载代码') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: 'master']],
extensions: [
[
$class: 'CloneOption',
noTags: true,
depth: 1,
shallow: true
]
],
userRemoteConfigs: [
[
credentialsId: 'gitlab_id',
url: 'http://192.168.3.3/howlaisi/argocd-gitops.git'
]
]
])
sh "du -sh ./"
sh "git count-objects -v"
sh "du -sh .git"
sh "git branch -a"
sh "git tag -l"
}
}
}
}
执行后输出结果:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on 192.168.4.5 in /home/jenkins/workspace/下载代码
[Pipeline] {
[Pipeline] stage
[Pipeline] { (下载代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential gitlab_id
Cloning the remote Git repository
Using shallow clone with depth 1
Avoid fetching tags
Cloning repository http://192.168.3.3/howlaisi/argocd-gitops.git
> git init /home/jenkins/workspace/下载代码 # timeout=10
Fetching upstream changes from http://192.168.3.3/howlaisi/argocd-gitops.git
> git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials gitlab_id
> git fetch --no-tags --progress --depth=1 http://192.168.3.3/howlaisi/argocd-gitops.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision a2851ebafe0eec1b56c0c863d9c2f792375f0d48 (origin/master)
Commit message: "build: automatic update of ruoyi-cloud-prod"
> git config remote.origin.url http://192.168.3.3/howlaisi/argocd-gitops.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
> git rev-list --no-walk a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
[Pipeline] sh
+ du -sh ./
14M ./
[Pipeline] sh
+ git count-objects -v
count: 0
size: 0
in-pack: 289
packs: 1
size-pack: 12612
prune-packable: 0
garbage: 0
size-garbage: 0
[Pipeline] sh
+ du -sh .git
13M .git
[Pipeline] sh
+ git branch -a
* (detached from a2851eb)
remotes/origin/master
remotes/origin/test1
remotes/origin/test2
[Pipeline] sh
+ git tag -l
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
优化步骤三对比分析
相对于优化步骤二的变化:
-
磁盘使用情况:
- 工作区大小:从16M减少到14M(减少12.5%)
- .git目录大小:从16M减少到13M(减少18.75%)
-
Git对象统计:
- Git对象数量:从27424个锐减到289个(减少98.9%)
- 打包文件大小:从15253KB减少到12612KB(减少17.3%)
-
操作行为变化:
- 添加了"Using shallow clone with depth 1"提示
git fetch
命令增加了--depth=1
参数
分析总结:
启用浅层克隆后,效果非常显著。Git对象数量从27424个锐减到289个,减少了98.9%,这是所有优化步骤中效果最明显的一步。同时,工作区和.git目录大小也有了明显减少。下一步可以通过添加refspec限制进一步优化。
优化步骤四:添加refspec限制
第四步优化是添加refspec参数,进一步限制获取的分支范围:
pipeline {
agent {
label '192.168.4.5'
}
stages {
stage('下载代码') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: 'master']],
extensions: [
[
$class: 'CloneOption',
noTags: true,
depth: 1,
shallow: true
]
],
userRemoteConfigs: [
[
credentialsId: 'gitlab_id',
url: 'http://192.168.3.3/howlaisi/argocd-gitops.git',
refspec: "+refs/heads/master:refs/remotes/origin/master"
]
]
])
sh "du -sh ./"
sh "git count-objects -v"
sh "du -sh .git"
sh "git branch -a"
sh "git tag -l"
}
}
}
}
执行后输出结果:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on 192.168.4.5 in /home/jenkins/workspace/下载代码
[Pipeline] {
[Pipeline] stage
[Pipeline] { (下载代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential gitlab_id
Fetching changes from the remote Git repository
Using shallow fetch with depth 1
Checking out Revision a2851ebafe0eec1b56c0c863d9c2f792375f0d48 (origin/master)
Commit message: "build: automatic update of ruoyi-cloud-prod"
> git rev-parse --is-inside-work-tree # timeout=10
> git config remote.origin.url http://192.168.3.3/howlaisi/argocd-gitops.git # timeout=10
Fetching upstream changes from http://192.168.3.3/howlaisi/argocd-gitops.git
> git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials gitlab_id
> git fetch --no-tags --progress --depth=1 http://192.168.3.3/howlaisi/argocd-gitops.git +refs/heads/master:refs/remotes/origin/master # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
> git rev-list --no-walk a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
[Pipeline] sh
+ du -sh ./
14M ./
[Pipeline] sh
+ git count-objects -v
count: 0
size: 0
in-pack: 289
packs: 1
size-pack: 12612
prune-packable: 0
garbage: 0
size-garbage: 0
[Pipeline] sh
+ du -sh .git
13M .git
[Pipeline] sh
+ git branch -a
* (detached from a2851eb)
remotes/origin/master
remotes/origin/test1
remotes/origin/test2
[Pipeline] sh
+ git tag -l
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
优化步骤四对比分析
相对于优化步骤三的变化:
-
磁盘使用情况:
- 工作区大小:保持14M不变
- .git目录大小:保持13M不变
-
Git对象统计:
- Git对象数量:保持289个不变
- 打包文件大小:保持12612KB不变
-
操作行为变化:
- 从"Cloning the remote Git repository"变为"Fetching changes from the remote Git repository"
git fetch
命令的refspec参数从+refs/heads/*:refs/remotes/origin/*
变为+refs/heads/master:refs/remotes/origin/master
分析总结:
添加refspec限制后,磁盘使用情况和Git对象统计没有发生变化,但操作行为有所优化。从日志可以看出,这次操作没有重新克隆仓库,而是直接从远程仓库获取更新并检出指定的master分支,这在重复构建时可以节省时间。
优化步骤五:添加超时和清理选项
第五步优化是添加超时设置和构建前清理选项以确保构建过程的稳定性:
pipeline {
agent {
label '192.168.4.5'
}
stages {
stage('下载代码') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: 'master']],
extensions: [
[
$class: 'CloneOption',
noTags: true,
depth: 1,
shallow: true,
timeout: 30
],
[
$class: 'CleanBeforeCheckout'
],
[
$class: 'CheckoutOption',
timeout: 30
]
],
userRemoteConfigs: [
[
credentialsId: 'gitlab_id',
url: 'http://192.168.3.3/howlaisi/argocd-gitops.git',
refspec: "+refs/heads/master:refs/remotes/origin/master"
]
]
])
sh "du -sh ./"
sh "git count-objects -v"
sh "du -sh .git"
sh "git branch -a"
sh "git tag -l"
}
}
}
}
执行后输出结果:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on 192.168.4.5 in /home/jenkins/workspace/下载代码
[Pipeline] {
[Pipeline] stage
[Pipeline] { (下载代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential gitlab_id
Cloning the remote Git repository
Using shallow clone with depth 1
Avoid fetching tags
Cloning repository http://192.168.3.3/howlaisi/argocd-gitops.git
> git init /home/jenkins/workspace/下载代码 # timeout=10
Fetching upstream changes from http://192.168.3.3/howlaisi/argocd-gitops.git
> git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials gitlab_id
> git fetch --no-tags --progress --depth=1 http://192.168.3.3/howlaisi/argocd-gitops.git +refs/heads/*:refs/remotes/origin/* # timeout=30
Avoid second fetch
Checking out Revision a2851ebafe0eec1b56c0c863d9c2f792375f0d48 (origin/master)
Commit message: "build: automatic update of ruoyi-cloud-prod"
> git config remote.origin.url http://192.168.3.3/howlaisi/argocd-gitops.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=30
> git rev-list --no-walk a2851ebafe0eec1b56c0c863d9c2f792375f0d48 # timeout=10
[Pipeline] sh
+ du -sh ./
14M ./
[Pipeline] sh
+ git count-objects -v
count: 0
size: 0
in-pack: 289
packs: 1
size-pack: 12612
prune-packable: 0
garbage: 0
size-garbage: 0
[Pipeline] sh
+ du -sh .git
13M .git
[Pipeline] sh
+ git branch -a
* (detached from a2851eb)
remotes/origin/master
remotes/origin/test1
remotes/origin/test2
[Pipeline] sh
+ git tag -l
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
优化步骤五对比分析
相对于优化步骤四的变化:
-
磁盘使用情况:
- 工作区大小:保持14M不变
- .git目录大小:保持13M不变
-
Git对象统计:
- Git对象数量:保持289个不变
- 打包文件大小:保持12612KB不变
-
操作行为变化:
git fetch
和git checkout
命令的timeout参数从默认值变为30- 由于CleanBeforeCheckout的存在,每次构建都会重新克隆仓库
分析总结:
添加超时和清理选项后,磁盘使用情况和Git对象统计保持不变,但增加了构建过程的稳定性和可预测性。超时设置可以避免网络问题导致的构建挂起,而清理选项确保每次构建都在干净的环境中进行。
总体优化效果对比
让我们总结一下从基准配置到完全优化配置的总体效果:
优化步骤 | 工作区大小 | .git目录大小 | Git对象数 | 相对于基准的改善 |
---|---|---|---|---|
基准配置 | 19M | 16M | 27426 | 基准 |
步骤一:指定分支 | 16M | 16M | 27426 | 工作区减小15.8% |
步骤二:禁用标签 | 16M | 16M | 27424 | Git对象减少2个 |
步骤三:浅层克隆 | 14M | 13M | 289 | 工作区减小26.3%,Git对象减少98.9% |
步骤四:refspec限制 | 14M | 13M | 289 | 无明显变化 |
步骤五:超时和清理 | 14M | 13M | 289 | 无明显变化,但增加稳定性 |
最佳实践建议
-
逐步优化:按照本文的步骤逐步优化,每一步都可以带来一定的改善。
-
关注关键优化点:浅层克隆是最有效的优化手段,可以显著减少Git对象数量和磁盘占用。
-
根据项目需求调整:并非所有项目都需要启用所有优化项,应根据具体需求选择合适的优化策略。
-
监控优化效果:建议在实施优化后持续监控效果,确保优化策略符合预期。
总结
通过逐步优化Jenkins Checkout插件的配置,我们可以显著提升CI/CD流程的效率和稳定性。最关键的优化是浅层克隆,它能将Git对象数量减少98%以上。其他优化措施如指定分支、禁用标签获取和添加refspec限制也能带来一定的改善。
合理使用这些配置项,不仅能提升构建速度,还能减少资源消耗,提高系统的整体稳定性。
评论区