Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provides for multiple executors operating under a single runner #123

Merged
merged 1 commit into from
Dec 20, 2023

Conversation

robacarp
Copy link
Collaborator

@robacarp robacarp commented Nov 21, 2023

This will fix #122 #18 and #80

My goal here is to minimally make mosquito network bound workloads significantly faster. Testing with @mamantoha shards.info codebase it’s trivial to cut the scrape time for the whole run by 60-70% because the whole workload is network bound. CPU abound workloads will have almost no benefit here, but the framework is laid for multi process work in the future.

As always, comments welcome and appreciated.

@robacarp robacarp force-pushed the runner-interface branch 3 times, most recently from b9d939b to 8d612cc Compare November 23, 2023 05:48
@robacarp robacarp force-pushed the runner-interface branch 6 times, most recently from b539365 to 478221a Compare November 29, 2023 03:28
@robacarp robacarp force-pushed the runner-interface branch 4 times, most recently from b6423ec to 6245c0a Compare December 12, 2023 03:52
@robacarp robacarp marked this pull request as ready for review December 17, 2023 01:46
@robacarp robacarp force-pushed the runner-interface branch 3 times, most recently from d5580eb to 559588d Compare December 18, 2023 15:24
Copy link

@crimson-knight crimson-knight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, everything looks really nice :)

I'm not as familiar with the inner workings as I should be to leave any meaningful comments/feedback, but one thing that sticks out is the lack of documentation comments and using return types.

Even if you only add comments to generate public docs, I'd highly recommend doing that while everything is fresh in your mind and :nodoc: everything you want to be in the private API

src/mosquito/queue.cr Show resolved Hide resolved
@robacarp robacarp merged commit 51904a0 into master Dec 20, 2023
3 checks passed
@robacarp robacarp deleted the runner-interface branch December 20, 2023 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Graceful shutdown when receiving the TERM signal
2 participants