Skip to content

Commit

Permalink
Parallel Stream File Parsing with ray.io (#260)
Browse files Browse the repository at this point in the history
* add ray dep for parallelization

* add ray as testing dep

* fix up docstrings

* add proper capitalization, fix dtypes in A / eV

* use Union instead of |

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* maybe fix build issue

* remove unused imports, attr

* add average cell method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix wavelength for empty peak lists

* fix numpy v1 support

* add numpy v1 to matrix

* fix numpy versioning

* pass through kwargs to ray.init

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* enable read only stream parsing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused imports

* fix column spec, add option to disable ray, make num_cpus explicit

* test with and without ray

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add error message

* clean up dataset creation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused variable

* removed unused timing stuff

* removed unused DataSeries

---------

Co-authored-by: Kevin Dalton <kmdalton@slac.stanford.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kevin Dalton <kmdalton@pop-os.localdomain>
  • Loading branch information
4 people authored Aug 5, 2024
1 parent f8765bf commit 870a019
Show file tree
Hide file tree
Showing 6 changed files with 595 additions and 242 deletions.
1 change: 1 addition & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
python -m pip install --upgrade pip
pip install "${{ matrix.numpy-version }}"
pip install -e .
pip install ray
- name: Test with pytest
run: |
Expand Down
Loading

0 comments on commit 870a019

Please sign in to comment.