Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

colossus: fully qualified path for temp folder #4977

Closed
mnaamani opened this issue Nov 28, 2023 · 0 comments · Fixed by #5000
Closed

colossus: fully qualified path for temp folder #4977

mnaamani opened this issue Nov 28, 2023 · 0 comments · Fixed by #5000
Assignees
Labels

Comments

@mnaamani
Copy link
Member

mnaamani commented Nov 28, 2023

Background

The temp folder path where active uploads to the node and objects being fetched by sync worker from other nodes is not configurable, but is always created as a sub folder under the uploads directory (long term storage of data objects).

Some ideas for scaling storage capacity of the node will require separation of temp files from the long term storage path. If we decide to use object storage (s3 compatible) then its not possible to store incomplete download files there.

Solution

Add new parameter to server command to specify a fully qualified path for the temp folder to use.

Challenges ?

Typically the path to the uploads folder will be on a volume with considerable capacity, and this volume might be mounted from a network storage cluster. It might be more suitable for the temp folder to be a path on the local host file system with not as much disk capacity. Should the node actively try to manage the number of downloads and uploads at a rate that takes the temp folder size into consideration?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant