Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support aborting the mapping #28

Closed
niftylettuce opened this issue Jul 14, 2020 · 11 comments
Closed

Support aborting the mapping #28

niftylettuce opened this issue Jul 14, 2020 · 11 comments

Comments

@niftylettuce
Copy link

e.g. a few seconds into waiting for the promise to finish, we could increase concurrency due to CPU limit being lowered below 20%, to a concurrency of 5x or something. Basically dynamic concurrency

@niftylettuce niftylettuce changed the title Best way to manipulate concurrency while process is running Best way to manipulate concurrency while process is running? Jul 14, 2020
@niftylettuce
Copy link
Author

Perhaps expose or pass a Symbol?

@sindresorhus
Copy link
Owner

Where would you change the concurrency? Inside the map function?

@sindresorhus
Copy link
Owner

Perhaps expose or pass a Symbol?

Can you show an example of how that would be used?

@niftylettuce
Copy link
Author

Here's an example:

https://github.com/breejs/bree#nodejs-email-queue-job-scheduling-example

Regarding the Symbol, I'm not sure to be honest because there would need to be support for multiple instances somehow.

@niftylettuce
Copy link
Author

In the prior link shared, I am referencing this snippet:

// handle cancellation (this is a very simple example)
if (parentPort)
  parentPort.once('message', message => {
    //
    // TODO: once we can manipulate concurrency option to p-map
    // we could make it `Number.MAX_VALUE` here to speed cancellation up
    // <https://github.com/sindresorhus/p-map/issues/28>
    //
    if (message === 'cancel') isCancelled = true;
  });

@sindresorhus
Copy link
Owner

While I agree that the ability to dynamically change the concurrency could be useful, in the example you reference, I think what you actually want is the ability to abort the p-map call. Could maybe be something like pMap.abort(mapper).

@niftylettuce
Copy link
Author

Yes that would be great!

@niftylettuce
Copy link
Author

Both would, but the latter you mention would solve the immediate issue.

@papb
Copy link
Contributor

papb commented Jul 14, 2020

@sindresorhus Interesting idea of using the mapper reference itself to abort, but wouldn't it be better to add an .abort method to the returned promise? Otherwise if someone calls pMap twice with the same mapper they might get an undesired behavior.

- await pMap(foo, bar);
+ const mapping = pMap(foo, bar);
+ setTimeout(mapping.abort, 5000);
+ await mapping;

@sindresorhus
Copy link
Owner

Otherwise if someone calls pMap twice with the same mapper they might get an undesired behavior.

Yeah, good point.

Maybe supporting AbortController would be an even better solution.

@sindresorhus sindresorhus changed the title Best way to manipulate concurrency while process is running? Support aborting the mapping Feb 21, 2021
@sindresorhus
Copy link
Owner

Duplicate of #31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants