Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[leo_storage] Setting a large num_of_containers causes leo_storage not to work properly #1086

Closed
mocchira opened this issue Jul 27, 2018 · 2 comments
Closed

[leo_storage] Setting a large num_of_containers causes leo_storage not to work properly #1086

mocchira opened this issue Jul 27, 2018 · 2 comments
Assignees
@mocchira
Labels
Improve _leo_backend_db _leo_storage Priority-MIDDLE survey
Milestone
1.4.3

Comments

@mocchira
Copy link
Member

mocchira commented Jul 27, 2018 •
edited
Loading

Going through #1076 and #1078, it turns out that Setting a large num_of_containers causes leo_storage not to work properly (many errors will start to appear on the node with such setting)
As I mentioned at #1078 (comment), some resource contention may happen on underlying leveldb instances.
@mocchira mocchira added this to the 1.4.3 milestone Jul 27, 2018
@mocchira mocchira self-assigned this Jul 27, 2018
@mocchira mocchira mentioned this issue Jul 27, 2018
Open
@mocchira
Copy link
Member Author

mocchira commented Aug 2, 2018

WIP. the number of native OS threads created by eleveldb could be problem. I'll test with num_of_containers set to a large value to check if the assumption is correct.

mocchira added a commit to mocchira/leofs that referenced this issue Aug 22, 2018
@mocchira
docs: Add a FAQ for leo-project#1086
8f187bb
@mocchira mocchira mentioned this issue Aug 22, 2018
Merged
@mocchira
Copy link
Member Author

No resource conflicts have been confirmed. The problem seems that a large num_of_containers with many cpu cores causes the high disk utilization resulting in errors on leo_storage. The workaround is already described on our official doc https://leo-project.net/leofs/docs/faq/administration/#what-should-i-do-when-many-errors-have-started-to-appear-in-log-files-on-leostorage so I'll close the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mocchira mocchira

Labels
Improve _leo_backend_db _leo_storage Priority-MIDDLE survey
Projects
None yet
Milestone
1.4.3
Development

No branches or pull requests
1 participant
@mocchira