Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create generic worker pool wrapper #271

Closed
plastikfan opened this issue May 27, 2024 · 3 comments · Fixed by #272
Closed

create generic worker pool wrapper #271

plastikfan opened this issue May 27, 2024 · 3 comments · Fixed by #272
Assignees
Labels
feature New feature or request

Comments

@plastikfan
Copy link
Contributor

plastikfan commented May 27, 2024

The boost wrapper will wrap the underlying ants worker pool and expose functionality that matches the legacy worker pool.

@plastikfan plastikfan added the feature New feature or request label May 27, 2024
@plastikfan plastikfan self-assigned this May 27, 2024
@plastikfan
Copy link
Contributor Author

plastikfan commented May 28, 2024

Just some notes about the design of the ants worker pool that should be acknowledged:

  • the go routine associated with each worker, has a lifetime that exists only for the current job
  • workers are stored in a sync.Pool
  • each work has its own job queue
  • workers are created on demand
  • there are 2 types of pool, withFunc and without Func
  • with func: CTOR: NewPoolWithFunc, UNIT-OF-WORK: func(i ants.InputParam)
  • without func: CTOR: NewPool, UNIT-OF-WORK: func()
  • when working with waitgroups and posting new jobs, there has to be a waitgroup.Add for each posted job, because the go routine's lifetime spans only the timespan of the job (the wrapper will take care of the Add)

@plastikfan
Copy link
Contributor Author

plastikfan commented May 28, 2024

In the wrapper, to make the pool easier to work with, we could various different UNIT-OF-WORK interfaces:

  • simple/ants-native: func()
  • simpleE: func() error
  • withInput: func[I](input I)
  • withInputE: func[I](input I) error
  • withOutput: func[O]() O
  • withOutputE: func[O]() O, error
  • withManifold: func[I, O](input I) O
  • withManifoldE: func[O](input I) O, error

@plastikfan
Copy link
Contributor Author

plastikfan commented May 29, 2024

ants contains a proprietry Logger interface. This needs to be replaced by the standard structured logger.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant