-
Notifications
You must be signed in to change notification settings - Fork 361
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add export action and action manager #807
Conversation
and action manager draft
add tests for delete and touch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Very nice stuff.
Some operational comments on the manager. Also I think the block adapter copy needs some operation: for export it is not writing to a destination namespace and hash, just to some path.
Co-authored-by: arielshaqed <ariels@treeverse.io>
Co-authored-by: arielshaqed <ariels@treeverse.io>
document action_manager add data to logs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
I'm approving, but would really like to see improved numbers on the benchmark from using this! That can happen in a future PR, too.
Workers int // number of goroutines handling tasks | ||
ChannelSize int // size of the channel containing tasks for workers | ||
MaxTasks int // max tasks requested in every ownTasks request | ||
WaitTime *time.Duration // time to wait if OwnTasks returned no tasks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure why these are pointers and the int
s are values, identified by being zero.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arielshaqed Because 0 is a valid argument for the pointers.
And I didn't wan't to override it with a default value.
parade/action_manager.go
Outdated
ownedTasks, err := a.parade.OwnTasks(a.handler.Actor(), a.properties.MaxTasks, a.handler.Actions(), a.properties.MaxDuration) | ||
if err != nil { | ||
logging.Default().WithField("actor", a.handler.Actor()).Errorf("manager failed to receive tasks: %s", err) | ||
time.Sleep(*a.properties.WaitTime) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
time.Sleep(*a.properties.WaitTime) | |
time.Sleep(*a.properties.ErrWaitTime) | |
continue |
?
The first line is in order to wait the time I think you meant. The second line is in order not to wait WaitTime
after waiting ErrWaitTime
.
res := a.handler.Handle(task.Action, task.Body) | ||
err := a.parade.ReturnTask(task.ID, task.Token, res.Status, res.StatusCode) | ||
if err != nil { | ||
logging.Default().WithFields(logging.Fields{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we maybe add these fields to a logger and use that on all the handlers?
(Something to consider, certainly not in any way required for this PR!)
Co-authored-by: arielshaqed <ariels@treeverse.io>
document actor
draft