Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: feat: merge changes #359

Closed
wants to merge 1 commit into from
Closed

WIP: feat: merge changes #359

wants to merge 1 commit into from

Conversation

elrrrrrrr
Copy link
Member

目前 syncPackage 任务执行时,会分别执行:

  1. versions 表更新
  2. versions 写入 pkg.manifest
  3. tag 写入 pkg.manifest
    在并发场景下,versions 表可能被其他 task 更新完
    但可能 versions 还未 写入pkg.manifest,tag 在这种情况下也会写入 pkg.manifest 中
    导致期间依赖查询失败(manifest 有 tag 没 version)致使下游的 regsitry 产生脏数据

此外,同名 task 可能还会导致 version change 和 tag change 互相冲突覆盖的问题 ref

为了尽可能减少任务并发带来的影响:

  1. 在任务分发时,合并相同 id 的 change,version 和 tag change 进行合并,忽略前者
  2. 在任务消费时,syncPackgeWoker 添加锁,减少请求并发带来的影响

部分极端场景,可能会致使 tag 更新延迟 60s 更新,牺牲时效性确保数据一致性 😈

@elrrrrrrr elrrrrrrr added the bug Something isn't working label Nov 24, 2022
// 创建同步任务
for await (const change of stream) {
let change: ChangesStreamChange = (await stream.next()).value;
let nextChange: ChangesStreamChange = (await stream.next()).value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要这么做。change 可能没有,nextChange 可能也没有。

@elrrrrrrr elrrrrrrr changed the title feat: merge changes WIP: feat: merge changes Nov 24, 2022
@codecov
Copy link

codecov bot commented Nov 25, 2022

Codecov Report

Merging #359 (e46c439) into main (9dd2d4b) will decrease coverage by 0.27%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #359      +/-   ##
==========================================
- Coverage   95.96%   95.68%   -0.28%     
==========================================
  Files         148      148              
  Lines        4512     4521       +9     
  Branches      390      389       -1     
==========================================
- Hits         4330     4326       -4     
- Misses        182      195      +13     
Impacted Files Coverage Δ
app/common/adapter/CacheAdapter.ts 97.56% <100.00%> (+0.33%) ⬆️
app/port/schedule/SyncPackageWorker.ts 92.30% <100.00%> (+1.39%) ⬆️
config/config.default.ts 75.75% <0.00%> (-15.16%) ⬇️
app/infra/NFSClientAdapter.ts 74.07% <0.00%> (-14.82%) ⬇️
app/port/controller/BinarySyncController.ts 92.68% <0.00%> (-4.88%) ⬇️
app/common/adapter/NFSAdapter.ts 96.66% <0.00%> (-3.34%) ⬇️
app/core/service/TaskService.ts 97.22% <0.00%> (-1.39%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@elrrrrrrr
Copy link
Member Author

通过 ref 实现

@elrrrrrrr elrrrrrrr closed this Nov 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants