Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: array.dtype.type will never be in ak.types.numpytype._dtype_to_primitive_dict #1841

Merged
merged 11 commits into from
Oct 27, 2022

Conversation

lgray
Copy link
Contributor

@lgray lgray commented Oct 26, 2022

Fixes #1840

Original input to square brackets would never work?

@lgray lgray changed the title array.dtype.type will never be in ak.types.numpytype._dtype_to_primitive_dict fix: array.dtype.type will never be in ak.types.numpytype._dtype_to_primitive_dict Oct 26, 2022
@codecov
Copy link

codecov bot commented Oct 26, 2022

Codecov Report

Merging #1841 (678aed9) into main (c033697) will increase coverage by 0.02%.
The diff coverage is 81.25%.

Additional details and impacted files
Impacted Files Coverage Δ
src/awkward/operations/ak_type.py 73.91% <81.25%> (+18.64%) ⬆️
src/awkward/contents/indexedarray.py 76.77% <0.00%> (-0.19%) ⬇️
src/awkward/contents/recordarray.py 83.94% <0.00%> (-0.19%) ⬇️
src/awkward/_util.py 82.15% <0.00%> (-0.18%) ⬇️
src/awkward/contents/indexedoptionarray.py 88.96% <0.00%> (-0.16%) ⬇️
src/awkward/_connect/avro.py 87.17% <0.00%> (-0.14%) ⬇️
src/awkward/types/_awkward_datashape_parser.py 47.72% <0.00%> (-0.01%) ⬇️
src/awkward/contents/content.py 77.31% <0.00%> (+0.10%) ⬆️

lgray added a commit to scikit-hep/coffea that referenced this pull request Oct 26, 2022
@agoose77
Copy link
Collaborator

Would you be able to add a test_1840-type-numpy-array.py to tests/ containing the following:

# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE

import numpy as np  # noqa: F401
import pytest  # noqa: F401

import awkward as ak  # noqa: F401


def test():
    array = np.random.random(size=512).astype(dtype=np.float64)
    assert ak.type(array) == ak.types.ArrayType(
        ak.types.NumpyType("float64"), 512
    )

@lgray
Copy link
Contributor Author

lgray commented Oct 27, 2022

Sure! Just a bit.

@agoose77 agoose77 self-requested a review October 27, 2022 14:16
@agoose77 agoose77 enabled auto-merge (squash) October 27, 2022 14:16
@jpivarski
Copy link
Member

I'm working on fixes to this, but I'm in a meeting now, so that's paused.

@jpivarski
Copy link
Member

(Do not merge!)

@jpivarski jpivarski disabled auto-merge October 27, 2022 14:18
@agoose77
Copy link
Collaborator

agoose77 commented Oct 27, 2022

Weirdly enough, I was actually looking at this code for another issue when I noticed that we have a dtype_to_primitive function that we can use here.

@jpivarski
Copy link
Member

I got the 10 minutes to do this, and now realize that I had misunderstood the original issue #1840. I thought it was about making ak.type(np.float64) work, but it had been more broken than that: ak.type(np.array([1, 2, 3])) didn't work.

Well, all of these cases work now. Since it's doubly approved, I'll reenable auto-merge.

@jpivarski jpivarski enabled auto-merge (squash) October 27, 2022 14:51
@agoose77 agoose77 disabled auto-merge October 27, 2022 14:55
@agoose77
Copy link
Collaborator

I've held this from merging, as we probably want to use numpytype.dtype_to_primitive - but we might need to do more with respect to the loop that checks for membership in the type dictionary. We could use a try/except, but perhaps there's a more elegant way of writing this code.

@agoose77 agoose77 requested a review from jpivarski October 27, 2022 16:49
Copy link
Member

@jpivarski jpivarski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dtype_to_primitive function is much better. I think it includes some logic for getting the time units of datetime/timedelta types right. Even if it doesn't, it's the right function to use here.

Just one problem below. I'll self-accept my suggestion and enable auto-merge.

src/awkward/operations/ak_type.py Outdated Show resolved Hide resolved
@jpivarski
Copy link
Member

I think this is done now; I'm reenabling auto-merge.

@jpivarski jpivarski enabled auto-merge (squash) October 27, 2022 20:21
@jpivarski jpivarski merged commit 178b4e9 into scikit-hep:main Oct 27, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Nov 3, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Dec 2, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Dec 12, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Dec 12, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Dec 12, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Dec 13, 2022
lgray added a commit to scikit-hep/coffea that referenced this pull request Feb 4, 2023
lgray added a commit to scikit-hep/coffea that referenced this pull request Feb 22, 2023
lgray added a commit to scikit-hep/coffea that referenced this pull request Mar 1, 2023
lgray added a commit to scikit-hep/coffea that referenced this pull request Mar 15, 2023
lgray added a commit to scikit-hep/coffea that referenced this pull request Apr 6, 2023
* start on updating to awkward v2 - this will require a few phases and need a transition to dask-awkward as well

uproot5 RCs as well

* remove deprecated coffea.hist

* fix Weight, etc. pending scikit-hep/awkward#1841

* lumi tools and btagging tests are back

get fixes in awkward/uproot

min python is 3.7 now

3.11 everywhere.

fix up deps (sorry spark)

bring back extras later

3.11 has problems

* fixes for clib wrapper, analysis tools (weights)

* remove old btag csv parser, slow and unmaintained

* jec lookup tools fixes

* restore all lookup tools tests that to not require nanoevents

stash interim changes

flake8

* ressurect more tests, awkward is dropping behaviors in some cases

more tests running but needs fixes in awkward as well

preloaded works now as well

enforce awkward2rc3

* stash wip for dask_awkward work

update pins

black

turn off noisy debugging info

* use/update version pinnings in ci

ci updates

update rc pins to consistent versions

remove overspecification of gh-actions

ditto

* _nplikes is now a private interface, need to move to backend instead

* auto schema passes tests

* rename auto schema file to something more obvious

* pdune schema passes tests

* treemaker schema passes tests

* try extended install extras again (except spark)

hmm quoting?

just dask

add parsl

* physlite schema passes tests

* delphes schema passes tests

* repin to spark3.3 (mapInArrow)

* remove python3.8 restrictions on parsl and spark

* bump to java17 for spark

black

* de-lazify Correct*Factory in anticiptation for dask_awkward, skip spark tests while being worked on

* skip tests if no spark

use importerskip to import pyspark

remove old arrow fix

* awkward2 is out of rc

* uproot 5 is out of rc

* do any of parsl/dask/spark work on windows?

interestingly - dask functions on windows

remove spark and parsl which assume linux

add java to windows, remove parsl from macos

figure out spark on windows later - seems doable

comparison ops are hard

* expose awkward forth flag in NanoEventsFactory.from_root

more allowed words

* passes pre-commit

add pre-commit hook instead of old lint

correct naming

remove more old ci

maybe this works

update manifest rules to pass tests

more manifest

more manifest

ignore symlink

* python 3.11 finally?

use miniforge instead of miniconda

perhaps quotes?

* will need to figure out python 3.11 elsewhere

* allow unsecured commands for miniforge

* add isort with black compatibility

* remove errant print from physlite

* pin numpy past security vulnerability

* get local spark tests to work

* Skip a step in normalizing buffers for NanoEvents

It seems now all internal layout and index types have a __array__
implementation for np.array() to work. And, the intermediate ak.Array()
  call was triggering from_iter, slowing things down.

* Unused import

* dask-awkward is out of alpha

* Workaround uproot5 issue with @* leaves, fix TLorentzVector

* Very sneaky comma

* use dask in factorized jet corrector

* jet resolution is daskified

* all individual correction components daskified

* CorrectedXFactories are daskified but produces really nasty graph

* remove workaround / use ak forth by default

* don't use awkward forth in physlite for now

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix not yet released?

* all default ak forth to True

* better implementation of handling dask in lookup_tools, rewrite of dask-aware jet tools

* update dask(-awkward) pins, add dask-histogram, test dak in lookup_tools a bit

* provide more readable names for components of met corrections

* use dictionaries to get return products in test

* add cirrus ci to run all tests on arm

* further dask work

* dask work checkpoint

* remove servicex from awkward2 tests

* get tests going again (reminder to unpin awkward once fix in place

* fix test and unpin awkward2

* fix treemaker tests

* stash work on moving nanoevents to dask

* rebase cruft

* more daskification

* tags for dask-ready schemas

* update isort

* nanoevents dask almost there, TODO: fix weakref pickling hack

* numpy limits for numba

* lookup base, CorrectedXFactory now support necessary columns analysis

* let lookup base (dask mode) deal with non-array args to functions

* daskify PackedSelection

* serialization speed improvement for lookups

* we can now use 1.0 instead of faking something in dak

* refactor _ensure_flat; PackedSelection and Weights daskified more clearly

* jetmet tools fully daskified; pins; lookup_base fixes for data-touching

* reset Weights and PackedSelection for redo; start on converting executors to dask

* daskified packed selection in a more correct way

* starting to work on daskifying treemaker

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* get the dask-nanoevents interface looking similar to the old one

* Weights is daskified; various rebase cruft for jetmet tools

* more efficient distribution strategy for heavy lookups, first try

* faster tokenization of corrections in local mode

* treemaker is daskified

* pin to awkward > 2.1.0; switch to calver

* getting closer with PHYSLITE

* delphes is probably daskified

* fix form zipping for physlite schema

* parquet nanoevents + work on physlite

* ressurect parquet nanoevents and associated tests

* now sunder for array context

* xfail tests that need it for now

* 2023.4.rc0

* more xfails, these are heisenbugs...

---------

Co-authored-by: Nick Smith <nick.smith@cern.ch>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nikolai Hartmann <nikoladze@posteo.de>
ikrommyd added a commit to ikrommyd/coffea that referenced this pull request Apr 7, 2023
commit 8b623d5
Author: iasonkrom <iason.krom@gmail.com>
Date:   Thu Apr 6 20:09:03 2023 -0500

    minor bug fixes

commit 3fe2f50
Merge: 628183d d17510b
Author: Iason Krommydas <iason.krom@gmail.com>
Date:   Thu Apr 6 01:33:47 2023 -0500

    Merge branch 'CoffeaTeam:awkward2_dev' into awkward2_dev

commit d17510b
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Apr 5 19:20:29 2023 -0500

    more xfails, these are heisenbugs...

commit b059111
Merge: 4e958e4 0633ad4
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Apr 5 18:43:56 2023 -0500

    Merge branch 'master' into awkward2_dev

commit 4e958e4
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Apr 5 18:43:10 2023 -0500

    2023.4.rc0

commit 6131b48
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Apr 5 18:41:10 2023 -0500

    xfail tests that need it for now

commit 628183d
Author: iasonkrom <iason.krom@gmail.com>
Date:   Wed Apr 5 13:40:16 2023 -0500

    variable histogramming works maybe?

commit 22b7638
Merge: e71bc80 91c2038
Author: Iason Krommydas <iason.krom@gmail.com>
Date:   Tue Apr 4 17:01:03 2023 -0500

    Merge branch 'CoffeaTeam:awkward2_dev' into awkward2_dev

commit 91c2038
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Apr 4 12:22:17 2023 -0500

    now sunder for array context

commit e71bc80
Author: iasonkrom <iason.krom@gmail.com>
Date:   Tue Apr 4 06:01:00 2023 -0500

    add reprs to PackedSelection

commit ebce87c
Author: iasonkrom <iason.krom@gmail.com>
Date:   Mon Apr 3 16:17:44 2023 -0500

    refactor N-1 and cutflow and improve interface

commit 373873e
Merge: 692cdbb 192ee34
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sat Mar 25 15:03:57 2023 -0500

    Merge remote-tracking branch 'upstream/awkward2_dev' into awkward2_dev

commit 192ee34
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Mar 25 10:21:21 2023 -0500

    ressurect parquet nanoevents and associated tests

commit 692cdbb
Merge: cb26f12 6c6d5a1
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sat Mar 25 01:53:12 2023 -0500

    Merge remote-tracking branch 'upstream/awkward2_dev' into awkward2_dev

commit 6c6d5a1
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Mar 24 14:29:49 2023 -0500

    parquet nanoevents + work on physlite

commit cb26f12
Author: iasonkrom <iason.krom@gmail.com>
Date:   Mon Mar 20 13:20:42 2023 -0500

    small typos

commit f67b8fa
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sun Mar 19 18:35:12 2023 -0500

    added docstrings to cutflow and N-1

commit b2c3fb2
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sun Mar 19 01:17:53 2023 -0500

    add histogramming to N-1 cuts

commit 03c90cb
Merge: 4c6d6e0 43b3ea7
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sat Mar 18 19:10:23 2023 -0500

    Merge remote-tracking branch 'upstream/awkward2_dev' into awkward2_dev

commit 4c6d6e0
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sat Mar 18 19:10:04 2023 -0500

    added basic cutflow and N-1 cuts

commit 43b3ea7
Merge: 0380f88 c04edc7
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Mar 18 22:32:50 2023 +0100

    Merge branch 'master' into awkward2_dev

commit 747dae5
Merge: 762a8c8 0380f88
Author: iasonkrom <iason.krom@gmail.com>
Date:   Sat Mar 18 13:23:03 2023 -0500

    Merge remote-tracking branch 'upstream/awkward2_dev' into awkward2_dev

commit 0380f88
Merge: 36bf4c6 a924d7d
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Mar 18 18:03:00 2023 +0100

    Merge pull request scikit-hep#776 from nikoladze/awkward2_dev-fix-physlite-zip

    fix form zipping for physlite schema

commit a924d7d
Author: Nikolai Hartmann <nikoladze@posteo.de>
Date:   Sat Mar 18 12:08:18 2023 +0100

    fix form zipping for physlite schema

commit 762a8c8
Author: iasonkrom <iason.krom@gmail.com>
Date:   Thu Mar 16 21:13:26 2023 -0500

    small edits to PackedSelection

commit 36bf4c6
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Mar 16 15:41:12 2023 +0100

    delphes is probably daskified

commit d3a65a5
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Mar 16 14:58:32 2023 +0100

    getting closer with PHYSLITE

commit 81cc052
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Mar 16 13:53:46 2023 +0100

    pin to awkward > 2.1.0; switch to calver

commit 0791991
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Mar 16 13:48:19 2023 +0100

    treemaker is daskified

commit e650361
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Mar 16 12:28:00 2023 +0100

    faster tokenization of corrections in local mode

commit 4ce7c3f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Mar 16 11:11:32 2023 +0100

    more efficient distribution strategy for heavy lookups, first try

commit ff251c7
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Mar 15 01:53:37 2023 +0100

    Weights is daskified; various rebase cruft for jetmet tools

commit 5e3ce4d
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Mar 14 15:26:13 2023 +0100

    get the dask-nanoevents interface looking similar to the old one

commit a539c33
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Mon Mar 13 11:34:50 2023 +0000

    [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

commit 7fabd3f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Mar 13 10:59:38 2023 +0100

    starting to work on daskifying treemaker

commit 1939324
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Mar 8 13:00:12 2023 -0600

    daskified packed selection in a more correct way

commit 79c74fd
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Mar 8 09:32:19 2023 -0600

    reset Weights and PackedSelection for redo; start on converting executors to dask

commit 0798d39
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Mar 6 18:20:38 2023 -0600

    jetmet tools fully daskified; pins; lookup_base fixes for data-touching

commit 1664e80
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Feb 28 19:28:46 2023 -0600

    refactor _ensure_flat; PackedSelection and Weights daskified more clearly

commit 06789ba
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Feb 25 16:34:08 2023 -0600

    we can now use 1.0 instead of faking something in dak

commit 617a51c
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Feb 25 16:33:36 2023 -0600

    serialization speed improvement for lookups

commit e4b398f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Feb 25 12:24:02 2023 -0600

    daskify PackedSelection

commit f4e7505
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Feb 23 11:11:28 2023 -0600

    let lookup base (dask mode) deal with non-array args to functions

commit 61f8891
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Feb 22 19:01:44 2023 -0600

    lookup base, CorrectedXFactory now support necessary columns analysis

commit 95ec250
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Feb 22 11:47:56 2023 -0600

    numpy limits for numba

commit 94f8ea8
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Feb 22 11:21:25 2023 -0600

    nanoevents dask almost there, TODO: fix weakref pickling hack

commit 95b5def
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Feb 21 20:06:16 2023 -0600

    update isort

commit 8cdc324
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Feb 21 17:25:08 2023 -0600

    tags for dask-ready schemas

commit a261d6a
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Feb 21 17:13:12 2023 -0600

    more daskification

commit bb2473e
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Feb 21 17:12:49 2023 -0600

    rebase cruft

commit dd0dac2
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Feb 21 15:02:56 2023 -0600

    stash work on moving nanoevents to dask

commit c4ab77a
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Jan 25 18:36:02 2023 -0600

    fix treemaker tests

commit 1bc9762
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Jan 25 18:17:08 2023 -0600

    fix test and unpin awkward2

commit e250cab
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Jan 25 18:06:57 2023 -0600

    get tests going again (reminder to unpin awkward once fix in place

commit 051ef1f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Jan 24 15:12:24 2023 -0600

    remove servicex from awkward2 tests

commit b150e19
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Jan 24 14:51:33 2023 -0600

    dask work checkpoint

commit fbb0556
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Jan 24 11:24:50 2023 -0600

    further dask work

commit 1013231
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Jan 6 17:42:33 2023 -0600

    add cirrus ci to run all tests on arm

commit 995220e
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Jan 5 14:18:26 2023 -0600

    use dictionaries to get return products in test

commit ef4e12e
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Jan 4 15:39:58 2023 -0600

    provide more readable names for components of met corrections

commit d199a92
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Jan 4 15:21:25 2023 -0600

    update dask(-awkward) pins, add dask-histogram, test dak in lookup_tools a bit

commit 747b457
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Jan 4 14:58:47 2023 -0600

    better implementation of handling dask in lookup_tools, rewrite of dask-aware jet tools

commit 1b541ec
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Dec 31 13:39:29 2022 -0600

    all default ak forth to True

commit b5e505d
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Dec 31 10:11:17 2022 -0600

    fix not yet released?

commit 61e2b01
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Sat Dec 31 15:55:23 2022 +0000

    [pre-commit.ci] auto fixes from pre-commit.com hooks

    for more information, see https://pre-commit.ci

commit 5f37986
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Dec 31 09:55:08 2022 -0600

    don't use awkward forth in physlite for now

commit 815cd6f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Dec 31 09:53:16 2022 -0600

    remove workaround / use ak forth by default

commit 99b23f1
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Dec 29 14:26:22 2022 -0800

    CorrectedXFactories are daskified but produces really nasty graph

commit 8265bf7
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 28 17:18:29 2022 -0800

    all individual correction components daskified

commit 74f86b9
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 28 08:47:09 2022 -0800

    jet resolution is daskified

commit c768962
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Dec 27 20:33:04 2022 -0800

    use dask in factorized jet corrector

commit 39b3da9
Author: Nick Smith <nick.smith@cern.ch>
Date:   Thu Dec 22 10:39:40 2022 -0600

    Very sneaky comma

commit a33ec28
Author: Nick Smith <nick.smith@cern.ch>
Date:   Wed Dec 21 19:22:52 2022 -0600

    Workaround uproot5 issue with @* leaves, fix TLorentzVector

commit 1992d8c
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 21 17:08:55 2022 -0600

    dask-awkward is out of alpha

commit 2905cc7
Author: Nick Smith <nick.smith@cern.ch>
Date:   Wed Dec 21 16:28:22 2022 -0600

    Unused import

commit 5cc23dc
Author: Nick Smith <nick.smith@cern.ch>
Date:   Wed Dec 21 16:24:19 2022 -0600

    Skip a step in normalizing buffers for NanoEvents

    It seems now all internal layout and index types have a __array__
    implementation for np.array() to work. And, the intermediate ak.Array()
      call was triggering from_iter, slowing things down.

commit 535a60f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 18:59:50 2022 -0600

    get local spark tests to work

commit 14f8044
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 16:53:17 2022 -0600

    pin numpy past security vulnerability

commit 846b1f5
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 16:49:20 2022 -0600

    remove errant print from physlite

commit 5d868a5
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 16:15:51 2022 -0600

    add isort with black compatibility

commit 4f6100c
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 16:01:38 2022 -0600

    allow unsecured commands for miniforge

commit 5d6e1d6
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 15:52:27 2022 -0600

    will need to figure out python 3.11 elsewhere

commit d7eadd4
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 15:24:48 2022 -0600

    python 3.11 finally?

    use miniforge instead of miniconda

    perhaps quotes?

commit 7c66e80
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 12:40:00 2022 -0600

    passes pre-commit

    add pre-commit hook instead of old lint

    correct naming

    remove more old ci

    maybe this works

    update manifest rules to pass tests

    more manifest

    more manifest

    ignore symlink

commit 1483791
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Mon Dec 12 10:12:16 2022 -0600

    expose awkward forth flag in NanoEventsFactory.from_root

    more allowed words

commit 9111f85
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sun Dec 11 12:32:56 2022 -0600

    do any of parsl/dask/spark work on windows?

    interestingly - dask functions on windows

    remove spark and parsl which assume linux

    add java to windows, remove parsl from macos

    figure out spark on windows later - seems doable

    comparison ops are hard

commit 8d9044f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Dec 9 22:15:31 2022 -0600

    uproot 5 is out of rc

commit 69333f4
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Dec 9 20:12:10 2022 -0600

    awkward2 is out of rc

commit b1f3979
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Dec 9 17:58:24 2022 -0600

    skip tests if no spark

    use importerskip to import pyspark

    remove old arrow fix

commit 9cd6b08
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Dec 9 17:47:35 2022 -0600

    de-lazify Correct*Factory in anticiptation for dask_awkward, skip spark tests while being worked on

commit b965361
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Dec 8 18:31:45 2022 -0600

    bump to java17 for spark

    black

commit 85ab153
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Dec 8 18:30:08 2022 -0600

    remove python3.8 restrictions on parsl and spark

commit 7d0e24c
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Dec 8 17:24:37 2022 -0600

    repin to spark3.3 (mapInArrow)

commit 77e43ae
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Dec 8 16:40:25 2022 -0600

    delphes schema passes tests

commit f31dbf1
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Thu Dec 8 12:51:40 2022 -0600

    physlite schema passes tests

commit ff0c364
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 7 23:22:33 2022 -0600

    try extended install extras again (except spark)

    hmm quoting?

    just dask

    add parsl

commit 9415dc5
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 7 16:42:56 2022 -0600

    treemaker schema passes tests

commit e6983e6
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 7 15:53:41 2022 -0600

    pdune schema passes tests

commit 44d03c4
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 7 15:47:32 2022 -0600

    rename auto schema file to something more obvious

commit 650ad26
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 7 15:45:57 2022 -0600

    auto schema passes tests

commit c322a3f
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Dec 7 14:30:59 2022 -0600

    _nplikes is now a private interface, need to move to backend instead

commit 8ca4afe
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Dec 2 18:09:03 2022 -0600

    use/update version pinnings in ci

    ci updates

    update rc pins to consistent versions

    remove overspecification of gh-actions

    ditto

commit a8672d5
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Dec 2 16:26:13 2022 -0600

    stash wip for dask_awkward work

    update pins

    black

    turn off noisy debugging info

commit 9b24e85
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Fri Nov 4 16:55:23 2022 -0500

    ressurect more tests, awkward is dropping behaviors in some cases

    more tests running but needs fixes in awkward as well

    preloaded works now as well

    enforce awkward2rc3

commit c8d7320
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Nov 2 18:57:22 2022 -0500

    restore all lookup tools tests that to not require nanoevents

    stash interim changes

    flake8

commit 376c9cf
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Oct 29 14:14:12 2022 -0500

    jec lookup tools fixes

commit 16a502b
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Oct 29 14:06:38 2022 -0500

    remove old btag csv parser, slow and unmaintained

commit f44b7c2
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Sat Oct 29 14:00:37 2022 -0500

    fixes for clib wrapper, analysis tools (weights)

commit 1c07c91
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Oct 26 21:36:35 2022 -0500

    lumi tools and btagging tests are back

    get fixes in awkward/uproot

    min python is 3.7 now

    3.11 everywhere.

    fix up deps (sorry spark)

    bring back extras later

    3.11 has problems

commit 942d8e5
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Oct 26 17:46:12 2022 -0500

    fix Weight, etc. pending scikit-hep/awkward#1841

commit de120e4
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Wed Oct 26 14:02:15 2022 -0500

    remove deprecated coffea.hist

commit cd6372e
Author: Lindsey Gray <lindsey.gray@gmail.com>
Date:   Tue Oct 25 17:35:11 2022 -0500

    start on updating to awkward v2 - this will require a few phases and need a transition to dask-awkward as well

    uproot5 RCs as well
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ak.type does not understand numpy.<type> style dtypes
3 participants