Skip to content

PERF: reductions #41911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 10, 2021
Merged

PERF: reductions #41911

merged 1 commit into from
Jun 10, 2021

Conversation

jbrockmendel
Copy link
Member

Discussed on today's call

@jreback
Copy link
Contributor

jreback commented Jun 10, 2021

forgot the ref issue, but this improves the relevant asvs's?

@jreback jreback added this to the 1.3 milestone Jun 10, 2021
@jreback jreback added the Performance Memory or execution speed performance label Jun 10, 2021
@jbrockmendel
Copy link
Member Author

forgot the ref issue, but this improves the relevant asvs's?

Based on the most-affected asv discussed on the call:

In [2]: ser = pd.Series([True] * 10**6)

In [3]: %timeit ser.all()
53.3 µs ± 1.85 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <-- PR
431 µs ± 7.61 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- master

@jorisvandenbossche
Copy link
Member

Sorry, I didn't see you opened a PR already (it could help to reference the original PR), and I also made one this morning: #41920

I think my fix might be a bit simpler, but didn't check in detail.

@jreback
Copy link
Contributor

jreback commented Jun 10, 2021

they are pretty similar. going to merge this one as its slightly more explict.

@jreback jreback merged commit c8e23d2 into pandas-dev:master Jun 10, 2021
@lumberbot-app
Copy link

lumberbot-app bot commented Jun 10, 2021

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
$ git checkout 1.2.x
$ git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
$ git cherry-pick -m1 c8e23d2bbb041235c021c96e9f5f8799e45a35e6
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
$ git commit -am 'Backport PR #41911: PERF: reductions'
  1. Push to a named branch :
git push YOURFORK 1.2.x:auto-backport-of-pr-41911-on-1.2.x
  1. Create a PR against branch 1.2.x, I would have named this PR:

"Backport PR #41911 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.

@jreback
Copy link
Contributor

jreback commented Jun 10, 2021

@meeseeksdev backport 1.2.x

@lumberbot-app
Copy link

lumberbot-app bot commented Jun 10, 2021

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
$ git checkout 1.2.x
$ git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
$ git cherry-pick -m1 c8e23d2bbb041235c021c96e9f5f8799e45a35e6
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
$ git commit -am 'Backport PR #41911: PERF: reductions'
  1. Push to a named branch :
git push YOURFORK 1.2.x:auto-backport-of-pr-41911-on-1.2.x
  1. Create a PR against branch 1.2.x, I would have named this PR:

"Backport PR #41911 on branch 1.2.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.

jorisvandenbossche added a commit that referenced this pull request Jun 10, 2021
jorisvandenbossche added a commit that referenced this pull request Jun 10, 2021
@jorisvandenbossche
Copy link
Member

Sorry Jeff, I don't really appreciate that (yes it was duplicate work, but I also did put time in this, then at least give some time to discuss what's the best approach), so I reverted this

@jorisvandenbossche
Copy link
Member

And sorry for my over-reaction, I shouldn't have directly reverted the merge. I was just annoyed by the lack of time to respond, but I didn't need to revert it for that.

@jorisvandenbossche
Copy link
Member

I opened a new PR with Brock's change -> #41924

@jreback
Copy link
Contributor

jreback commented Jun 10, 2021

@jorisvandenbossche this is a way over reaction

brock's pr was first and is better

but you can handle now

@jbrockmendel jbrockmendel deleted the perf-reduce branch June 10, 2021 14:40
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants