Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integral function #455

Closed
mayku00 opened this issue Jun 23, 2022 · 9 comments
Closed

Integral function #455

mayku00 opened this issue Jun 23, 2022 · 9 comments
Assignees

Comments

@mayku00
Copy link

mayku00 commented Jun 23, 2022

Hi, I asked in the slack about how to get the function integral (like in influxDB)

The answer:
“Time weighted average def does that! Sounds like we should be exposing that! can you create an issue here: https://github.com/timescale/timescaledb-toolkit/issues? I think we have what we need in time_weight but just need to expose it the right way, so probably would be relatively quick to implement, and will just be an accessor, so if you want to, say, create continuous aggregates you can use the time_weight aggregate and when we make this accessor, you'll just be able to use it on that.”

so open is!!

@jerryxwu
Copy link
Contributor

@mayku00 thank you for the feature request! I think I have a general idea of what you are asking for. It would be great to learn your specific use case for this function. It can help us design the function better.

@mayku00
Copy link
Author

mayku00 commented Jun 24, 2022

Hi! Sure.
I have an hypertable with 3 colums: timestamp, upload_bitrate, download_bitrate. I use the table to save the bitrate that I get from a modem every x seconds.

I use the table as the historical register for the bitrate, but also I need to calculate the volume consumed in a period of time.

As I have a line/curve of values ordered by the timestamp, I calculate the volume as the area under this line/curve:

On Influx I was using: https://docs.influxdata.com/flux/v0.x/stdlib/universe/integral/

I asked for a similar function

KR

@davidkohn88
Copy link
Contributor

Thanks! Leaving as much context here is awesome because the links to Slack eventually break because we don't have full history there! So feel free to edit or add more if you think of anything. But I think time_weight('Linear', time, value) -> integral(unit, start_time, end_time) would be a good start for what you're looking for here where unit is the unit of time to use to calculate the integral, all of those could have defaults, ('1s', NULL, NULL), where null values would just use the starting and ending times seen in the data set.

@jerryxwu
Copy link
Contributor

jerryxwu commented Jul 25, 2022

Having integral built on top of time_weight definitely makes sense. A few ideas and comments:

  1. Users wanting to compute integral are more likely familiar with numeric integration lingo, so we should document 'Linear' means trapezoidal rule and LOCF means left-hand rule. Even better, allowing time_weight to accept trapezoidal as an alternative to Linear.
  2. It's unlikely that integral can accept start_time and end_time, only unit makes sense
  3. interpolated_integral(...) analogous to interpolated_average should also be implemented

@jerryxwu
Copy link
Contributor

There's another request from the community slack channel.

@jerryxwu jerryxwu assigned jerryxwu and rtwalker and unassigned jerryxwu Aug 30, 2022
@syvb
Copy link
Member

syvb commented Sep 6, 2022

Assigning myself as discussed in the standup.

syvb added a commit that referenced this issue Sep 6, 2022
syvb added a commit that referenced this issue Sep 6, 2022
syvb added a commit that referenced this issue Sep 7, 2022
@andrey-skat
Copy link

Another use case is calculating power consumption from raw time/watt data.
It can also be done with lag window function, but it can't be used in continuous aggregations.

@davidkohn88
Copy link
Contributor

That's an awesome use case, was actually just using that to explain it, essentially you want to go from kW to kWh. We'll definitely want to make sure we have interpolated_integral as well as that will be really useful there!

syvb added a commit that referenced this issue Sep 8, 2022
syvb added a commit to syvb/timescaledb-toolkit that referenced this issue Sep 9, 2022
syvb added a commit to syvb/timescaledb-toolkit that referenced this issue Sep 9, 2022
syvb added a commit that referenced this issue Sep 9, 2022
syvb added a commit that referenced this issue Sep 12, 2022
syvb added a commit that referenced this issue Sep 15, 2022
syvb added a commit that referenced this issue Sep 15, 2022
syvb added a commit that referenced this issue Sep 16, 2022
syvb added a commit that referenced this issue Sep 16, 2022
bors bot added a commit that referenced this issue Sep 20, 2022
526: Add integral function for time_weight r=Smittyvb a=Smittyvb

Implements functionality requested in #455:
- Adds an experimental `integral(tws[, unit]) -> float8` function (unit defaults to `'second'`)
- Adds an experimental `interpolated_integral(tws, start, interval, prev, next[, unit]) -> float8` function (unit defaults to `'second'`)
- Adds experimental arrow functions for `integral`/`interpolated_integral`
- Makes `trapezoidal` an alias for `linear` in the `time_weight` function

The `unit` parameter to `integral`/`interpolated_integral` is a string that specifies what time unit to use for the returned `f64`. It can be any fixed-duration unit that PostgreSQL allows in a `interval`. An alternative would be to have an `interval` be passed as the unit instead, but `interval`s can have variable length units (days and months) which might not be wanted. Also, `integral(..., 'hour')` is clearer than `integral(..., '1 hour'::interval)`.


Co-authored-by: Smitty <smitty@timescale.com>
@rtwalker
Copy link
Contributor

rtwalker commented Oct 4, 2022

Closed by #526

@rtwalker rtwalker closed this as completed Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants