[Enh]: implement nw.expr.interpolate_by

[limlam96]

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

https://github.com/lvgig/tubular

Please describe the purpose of the new feature or describe the problem to solve.

We currently make use of np.interp in our weighted_quantile calculation (here), and are looking at how to convert this to narwhals - I think the most direct route for this would be to use a nw version of pl.Expr.interpolate_by, wondering if this fits into your roadmap at all?

Suggest a solution if possible.

a nw version of pl.Expr.interpolate_by

If you have tried alternatives, please describe them below.

No response

Additional information that may help us understand your needs.

No response

[MarcoGorelli]

thanks @limlam96 for the request

yes, i'd say that this is in-scope

does interpolate_by completely cover your needs?

[limlam96]

Nice :)

I think it would yeah, the rough idea would be for a df with weight column and some numeric column, where we want to interpolate the numeric column at some given weight quantiles:

• sort the df by weight, and add in a cumulative weight column
• insert rows with the wanted quantiles to this cumulative weight column, and sort by cumulative weight
• run interpolate_by on the numeric column and cumulative weight column

I think apart from interpolate_by, the rest should be achievable in narwhals

Hope that was clear - and also open to any alternative suggestions!

[skritsotalakis]

I can take this one if its free

[MarcoGorelli]

Will we run into issues with PySpark / DuckDB on this one?

I'm wondering if dataframe.interpolate_by might end up being more flexible, because then we could use one of the solutions from https://stackoverflow.com/a/35868411/4451315 ?

[skritsotalakis]

Will we run into issues with PySpark / DuckDB on this one?

I'm wondering if dataframe.interpolate_by might end up being more flexible, because then we could use one of the solutions from https://stackoverflow.com/a/35868411/4451315 ?

I took a quick look and that could be the case indeed. I am not aware of a potential downside if we use dataframe.interpolate_by.