Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: 大批量上传很多文件会很慢 #141

Open
hunjixin opened this issue Mar 17, 2024 · 6 comments
Open

fix: 大批量上传很多文件会很慢 #141

hunjixin opened this issue Mar 17, 2024 · 6 comments
Labels
bug Something isn't working P1 High - we should be working on this now or in the immediate future
Milestone
m2: Completion

Comments

@hunjixin
Copy link
Contributor

数据小的时候批量传刷的hen'k很快， 大量的传数据会变得非常慢。

val.zip
@hunjixin hunjixin added bug Something isn't working P1 High - we should be working on this now or in the immediate future labels Mar 17, 2024
@hunjixin hunjixin added this to the m2: Completion milestone Mar 17, 2024
@TsumikiQAQ
Copy link
Contributor

TsumikiQAQ commented Mar 20, 2024 •
edited
Loading

image
应该是网速问题 本地很快就传了 clound那边不开vpn加载会很慢

@hunjixin hunjixin reopened this Mar 20, 2024
@hunjixin
Copy link
Contributor Author

我说的是数量，把我那个压缩吧，打开，用那个，那里面有里几千个小文件。

@hunjixin
Copy link
Contributor Author

怀疑是不是元素太多，每次上传一个，全部都会状态检查。

@TsumikiQAQ
Copy link
Contributor

现在是把文件做成一个文件数组，然后遍历发送请求上传，因为之前是限制一次最多上传5个，防止上传大文件一次性内存占用太大崩溃。
ui上传文件应该不需要上传太多小文件吧，以后不是会增加命令行上传吗？

@TsumikiQAQ
Copy link
Contributor

现在几千个文件遍历上传，每个都单独发送请求，每次都会有几十到几百ms延迟，打包一起上传就会快很多 但是传大文件崩溃的风险就会加大

@hunjixin
Copy link
Contributor Author

晚上我在试试，问题是上传速度和数量不是线性的。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working P1 High - we should be working on this now or in the immediate future
Projects
None yet
Milestone
m2: Completion
Development

No branches or pull requests
2 participants
@hunjixin @TsumikiQAQ