Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:流水线并发运行时,支持限制并发个数和排队 #10718

Closed
fayewong-v5 opened this issue Jul 25, 2024 · 2 comments
Closed
Assignees
Labels
done Production environment in tencent has been deploy kind/enhancement 功能改进特性 service/process 流水线 todo todo

Comments

@fayewong-v5
Copy link
Contributor

fayewong-v5 commented Jul 25, 2024

【背景】
某业务,云端构建资源有限、测试环境部署资源也有限,需要控制并发的个数,超限时排队:

  • 超限时不希望取消运行,每一次触发都需要测试
  • 流水线里使用公共构建机来进行构建,但构建过程中会使用业务自己的云端资源
  • 测试环境是 IDC,需从无编译环境下发部署

现有的能力满足不了此场景

【需求】
当流水线并发设置为“可并发运行”时,可以:

  1. 配置同时可并发运行的最大任务个数

    • 最大不超过 200 个
  2. 并发达到上限时,设置排队的超时时间

    • 最大不超过 1440 分钟
    • 缺省时,超时时间默认为 10 分种
    • 当任务任务排队时长达到阈值、还未获得运行资格,结束排队,任务终止
  3. code

concurrency:
  max-parallel: 5
  queue-timeout-minutes: 10
企业微信截图_e55578ac-28db-4e4c-a948-f813403ccd7b
@fayewong-v5 fayewong-v5 added the kind/enhancement 功能改进特性 label Jul 25, 2024
@fayewong-v5
Copy link
Contributor Author

需求变更:并不是每一次触发都需要运行测试,相同分支可以只运行最新的构建(同一分支上短时间多次提交,以最新一次提交后的代码为准部署测试环境即可)。但同时部署的测试环境不希望超过指定的个数(部署资源有限)。

最快的方式可以使用现有功能来实现:

  1. 流水线并发策略,配置为按照分组,组内同一时间只运行1个最新的构建:
    企业微信截图_f0524ff9-b18b-4a26-8da7-ad8f164405c5
  2. 部署测试环境的 Job ,使用第三方构建机集群的方式,限制 Job 的并发个数
    image

@fayewong-v5
Copy link
Contributor Author

需求变更:并不是每一次触发都需要运行测试,相同分支可以只运行最新的构建(同一分支上短时间多次提交,以最新一次提交后的代码为准部署测试环境即可)。但同时部署的测试环境不希望超过指定的个数(部署资源有限)。

最快的方式可以使用现有功能来实现:

  1. 流水线并发策略,配置为按照分组,组内同一时间只运行1个最新的构建:
    企业微信截图_f0524ff9-b18b-4a26-8da7-ad8f164405c5
  2. 部署测试环境的 Job ,使用第三方构建机集群的方式,限制 Job 的并发个数
    image

第 2 点,优化功能,公共构建机/无编译环境 Job 也支持排队:#10721

@bkci-bot bkci-bot added the todo todo label Sep 2, 2024
vhwweng added a commit to vhwweng/bk-ci that referenced this issue Sep 12, 2024
@bkci-bot bkci-bot added for test Test environment in tencent has been deploy and removed todo todo labels Sep 13, 2024
@yongyiduan yongyiduan added the tested 测试环境通过/test patest passed for test stage label Sep 13, 2024
vhwweng added a commit to vhwweng/bk-ci that referenced this issue Sep 13, 2024
@bkci-bot bkci-bot added todo todo and removed todo todo labels Sep 13, 2024
@bkci-bot bkci-bot added the todo todo label Sep 19, 2024
@bkci-bot bkci-bot removed the todo todo label Sep 19, 2024
@bkci-bot bkci-bot added todo todo and removed todo todo labels Sep 25, 2024
vhwweng added a commit to vhwweng/bk-ci that referenced this issue Sep 29, 2024
vhwweng added a commit to vhwweng/bk-ci that referenced this issue Oct 10, 2024
@bkci-bot bkci-bot added the todo todo label Oct 10, 2024
@bkci-bot bkci-bot removed the todo todo label Oct 10, 2024
bkci-bot added a commit that referenced this issue Oct 15, 2024
feat:流水线并发运行时,支持限制并发个数和排队 #10718
bkci-bot added a commit that referenced this issue Oct 15, 2024
feat:流水线并发运行时,支持限制并发个数和排队 #10718
@bkci-bot bkci-bot added for gray UAT environment in tencent has been deploy todo todo and removed for test Test environment in tencent has been deploy tested 测试环境通过/test patest passed for test stage labels Oct 15, 2024
@bkci-bot bkci-bot added done Production environment in tencent has been deploy and removed for gray UAT environment in tencent has been deploy labels Oct 27, 2024
@yongyiduan yongyiduan added the tested 测试环境通过/test patest passed for test stage label Nov 1, 2024
@bkci-bot bkci-bot added for test Test environment in tencent has been deploy todo todo for gray UAT environment in tencent has been deploy and removed todo todo for test Test environment in tencent has been deploy tested 测试环境通过/test patest passed for test stage labels Nov 1, 2024
@mingshewhe mingshewhe added the service/process 流水线 label Nov 21, 2024
@bkci-bot bkci-bot removed the for gray UAT environment in tencent has been deploy label Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
done Production environment in tencent has been deploy kind/enhancement 功能改进特性 service/process 流水线 todo todo
Projects
None yet
Development

No branches or pull requests

8 participants