Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multithreaded startrail processing. #649

Merged
merged 4 commits into from
Oct 14, 2021
Merged

Multithreaded startrail processing. #649

merged 4 commits into from
Oct 14, 2021

Commits on Oct 14, 2021

  1. Multithreaded startrail processing. 

    Inspired by @sebmueller. This splits up the input images into batches of work
more or less evenly divided across the number of processors.

Two new flags are added:
* -n / --nice to make the workers be nice. default = 10
* -m / --max-threads to control the number of worker threads. default = ncpus

Allows me to process a night of startrails on my workstation in under 3.5s.
On my pi3b+, it'll use between 350% - 395% CPU without making it feel slow

```
root@skycam:/home/allsky/src# time ./startrails -b 0.255 -d ../images/20210912/ -e jpg -o /tmp/junk.jpg -s 1280x960
Minimum: 0.0265446 maximum: 0.804403 mean: 0.143959 median: 0.137787

real	5m5.951s
user	11m22.243s
sys	1m5.347s

root@skycam:/home/allsky/src# time ./startrails -b 0.255 -d ../images/20210912/ -e jpg -o /tmp/junk2.jpg -s 1280x960 --max-threads 1
Minimum: 0.0265446 maximum: 0.804403 mean: 0.143959 median: 0.137787

real	12m8.758s
user	11m3.073s
sys	0m54.200s

```

Fixes #628
    @ckuethe
    ckuethe committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    22c4c15 View commit details
    Browse the repository at this point in the history

  2. fix concurrency option

    @ckuethe
    ckuethe committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    1e7dc32 View commit details
    Browse the repository at this point in the history

  3. better concurrency limit check

    @ckuethe
    ckuethe committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    426fa79 View commit details
    Browse the repository at this point in the history

  4. clarify behavior of concurrency limit

    @ckuethe
    ckuethe committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    f428f11 View commit details
    Browse the repository at this point in the history