Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve data sync performance #13216

Closed
88250 opened this issue Nov 20, 2024 · 0 comments
Closed

Improve data sync performance #13216

88250 opened this issue Nov 20, 2024 · 0 comments
Assignees
@88250
Labels
Enhancement
Milestone
3.1.14

Comments

@88250
Copy link
Member

88250 commented Nov 20, 2024 •
edited
Loading

  • 并行 数仓索引 和 下载云端快照索引 这两个过程来降低启动时间，在固态盘上测试 1W 个文件且网络良好的情况下大致可以比之前节省 1/3 的启动时间，3K 个文件的情况下大致节省 1/2 的启动时间
  • 数仓索引 时使用更高效的遍历函数 filepath.WalkDir 替换 filepath.Walk，大约可以节省 1/5 的遍历时间
  • 移动端不再校验分块完整性，因为没有必要，移动端上私有数据空间不会出现分块损坏（被外部删除），大约可以节省 2/5 的数仓索引时间
@88250 88250 added this to the 3.1.14 milestone Nov 20, 2024
@88250 88250 self-assigned this Nov 20, 2024
@88250 88250 changed the title Improve data sync performance for booting Improve data sync performance Nov 21, 2024
88250 added a commit to siyuan-note/filelock that referenced this issue Nov 21, 2024
@88250
⚡ Improve walk dir performance siyuan-note/siyuan#13216
1cf0626
88250 added a commit to siyuan-note/dejavu that referenced this issue Nov 21, 2024
@88250
⚡ Improve data sync performance siyuan-note/siyuan#13216
e418aab
88250 added a commit to siyuan-note/dejavu that referenced this issue Nov 21, 2024
@88250
⚡ Improve data sync performance siyuan-note/siyuan#13216
95e24cd
88250 added a commit that referenced this issue Nov 21, 2024
@88250
⚡ Improve data sync performance for booting #13216
eba4dfa
88250 added a commit that referenced this issue Nov 21, 2024
@88250
⚡ Improve data sync performance for booting #13216
89f1887
@88250 88250 closed this as completed Nov 21, 2024
88250 added a commit that referenced this issue Nov 21, 2024
@88250
⚡ Improve data sync performance for booting #13216
3ebc1a4
@88250 88250 reopened this Nov 24, 2024
88250 added a commit to siyuan-note/dejavu that referenced this issue Nov 24, 2024
@88250
⚡ Improve data sync performance siyuan-note/siyuan#13216
df73bba
88250 added a commit that referenced this issue Nov 24, 2024
@88250
⚡ Improve data sync performance #13216
4443f3e
@88250 88250 closed this as completed Nov 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@88250 88250

Labels
Enhancement
Projects
None yet
Milestone
3.1.14
Development

No branches or pull requests
1 participant
@88250