fix+perf: Benford correlation and entropy features

[abstractqqq]

1. Do not approve for now.
2. Fixed a problem in benford_correlation. (x != "") is erroneous because x is numeric. In eager mode, np.corrcoef returns a matrix so we need to extract the coefficient, not return the whole matrix. Also, moving sort after value_counts which gives us another 3-5% perf improvment.
3. Significantly improved perf for permutation entropy by utilizing parallelism provided by pl.concat_list to a greater degree.
4. I have an idea to improve perf of binned_entropy. It is not implemented yet. I will likely work on this this weekend. It is tested on integers and works as fast as Tsfresh. I mentioned in a previous issue that we were about significantly slower and this could bring the performance to the same level. So this would be a big win. The only problem now I am facing is a precision issue when the input column is float, which results in error of scale 1e-5, which can be big sometimes.

The key idea in (4) is to create the segments by floordiv, instead of using cuts, which does way more computation than needed.

  (x.floordiv((x.max() - x.min())/pl.lit(max_bin) + x.min() )).drop_nulls().value_counts().sort()
  .struct.field("counts")
  .entropy()

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
functime-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 30, 2023 8:12pm

[abstractqqq]

@topher-lo @MathieuCayssol

1. After some fiddling, binned_entropy's numerical issue seems to have gone away and we get really nice time results. It is incredible that we can get this fast using just expressions

2. For expressions, I replaced x.len() with x.count() because x.len() internally called self.count(), which is an indirection. Likely just nanosec difference. But it is a free improvement so why not.

I think it's ready to merge if no other issue come up during your review.