You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem:
The zboxcli uses commands like list, list-all and the other CLIs in progress like s3 and nfs will also have list operations. There is currently no support for pagination, so the entire list is given as a respondse. This is fine if there are smaller number of files but we need to consider that an allocation can contain thousands of such files. For about 5 directories and 5 files the response size was about 60KB so for large number of files it will be large sized response making blobber busy to serve request for certain amount of time as metadata can be of for example; 20MB which obviously stalls the Blobber.
And this is just for single allocation. Blobbers however are not confined to single allocation and there can be multitude of clients requests.
Techincal description:
These repos currently use the GoSDK function NewListRequest in gosdk/zboxcore/zboxutil/http.go. This makes an GET request to the endpoint /v1/file/list/ which only takes in two params, path_hash and auth_token . It would be good if this endpoint allows a param for pagination (offset or limit or something similar) which the GoSDK can levarage.
Problem:
The zboxcli uses commands like
list
,list-all
and the other CLIs in progress likes3
andnfs
will also have list operations. There is currently no support for pagination, so the entire list is given as a respondse. This is fine if there are smaller number of files but we need to consider that an allocation can contain thousands of such files. For about 5 directories and 5 files the response size was about 60KB so for large number of files it will be large sized response making blobber busy to serve request for certain amount of time as metadata can be of for example; 20MB which obviously stalls the Blobber.And this is just for single allocation. Blobbers however are not confined to single allocation and there can be multitude of clients requests.
Techincal description:
These repos currently use the GoSDK function
NewListRequest
ingosdk/zboxcore/zboxutil/http.go
. This makes an GET request to the endpoint/v1/file/list/
which only takes in two params,path_hash
andauth_token
. It would be good if this endpoint allows a param for pagination (offset or limit or something similar) which the GoSDK can levarage.Follows issue 117
The text was updated successfully, but these errors were encountered: