Sensor abstractions.

[zelch]

This is a branch off of #16 , as the discussion has expanded somewhat.
@xxxserxxx I'm afraid that I can't comment on the wiki entry for `Sensor abstractions, so I've opened this up for general discussion about it.

One of the more interesting problems performance wise is that, at least under Linux, it is very likely that the best option is to have a single update function handle multiple different kinds of sensors.

That of course leads to a couple of design implications, though I'm frankly not sure how far it makes sense to go down this rabbit hole.

The first one is that, at least for common sensor types, I don't think that the data belongs with the widget, either for the current values or the historic values. (The metric values in most of the widgets.)

Under that approach, instead of having devices and widgets, with the widgets controlling the flow, we would have devices, data management, and widgets. Where (depending on configuration), device handlers would register that they exist, and what kinds of data they update. The data management layer would host the data structures that the device handlers populate. The widgets would pull data from the data management layer... And, er, something would manage the update frequencies and make sure that stuff happened in the right order.

And to be explicit, there would be a many to many mapping: A given device might populate multiple kinds of sensors, and a given widget might display data from multiple devices. (IE, you might have a general lmsensors driver that populates fans, temperatures, and chip voltages and current draw. And an nVidia driver that populates the same, but only for nVidia cards. And a general temperature widget that displays all of the system temperatures, regardless of the source.)

The many to many mapping makes it a little more difficult for the widgets to drive updates, but it doesn't seem like it should be horribly difficult to have that managed at a global level with checks to see if a given data source or widget needs to update at any given time.

Thoughts?

[xxxserxxx]

I'm afraid that I can't comment on the wiki entry

Ok. I need to figure out a better way of having these conversations.

One of the more interesting problems performance wise is that, at least under Linux, it is very likely that the best option is to have a single update function handle multiple different kinds of sensors.
...
The first one is that, at least for common sensor types, I don't think that the data belongs with the widget, either for the current values or the historic values. (The metric values in most of the widgets.)

Yes. This can be managed through drivers.

Take the NVidia driver, for instance (at least, for the library that's currently being leveraged). Pulling information requires an open call followed by an iteration over the list of GPUs; all sensor data is returned as a simple struct with the measurements as fields. Updating three different sensors requires performing this open+loop three times. There is still a single driver for the device; it is presented as a single physical object. The information is therefore easily cached using a simple expiration cache algorithm, and it makes most sense for the driver code to manage the device as best suited for the driver, rather than gotop trying to guess at the best algorithm.

The current version is more highly decoupled than the previous incarnation. The devices package manages the device data source. The widget package manages the data as appropriate for the widegt. I don't think this will change. A histogram widget needs a time series; a guage does not. It is incumbent on the widget to retain the data as it best sees fit. For example, let's imagine an uptime widget that shows the 10 previous reboots (a-la uprecords). It makes no sense for the reboot device to retain the records; how many records should be retained? 10? 100? 1? Maybe there's a second widget that only shows the current uptime (a-la uptime), and the user only enables that widget and doesn't enable the uprecords widget. Why does the device still retain 100 last reboots if the information isn't used? Each widget retains as much data as it is configured to show by the user; the device just provides point measurements. Consequently, widgets should manage their own data.

Under that approach, instead of having devices and widgets, with the widgets controlling the flow, we would have devices, data management, and widgets. Where (depending on configuration), device handlers would register that they exist, and what kinds of data they update. The data management layer would host the data structures that the device handlers populate. The widgets would pull data from the data management layer... And, er, something would manage the update frequencies and make sure that stuff happened in the right order.

I think this adds a level of unnecessary complexity. What you're describing is a prometheus-like workflow, where you have exporters (devices), Prometheus in the middle (your data management layer), and widgets in the front (grafana). I think if we're going to get that complex, we should just use prometheus+grafana; no need to rebuild it. gotop is top. It's getting a bit of feature creep with configurable layouts, which I felt was necessary because -- frankly -- I don't think it's possible to support all possible hardware configurations with a small fixed set of layouts; and the device abstraction was added because it was necessary to be able to support more devices without bloating the core gotop (see the wiki page about why I chose plugins). What's the value-add justification for a data management layer?

And to be explicit, there would be a many to many mapping: A given device might populate multiple kinds of sensors, and a given widget might display data from multiple devices. (IE, you might have a general lmsensors driver that populates fans, temperatures, and chip voltages and current draw. And an nVidia driver that populates the same, but only for nVidia cards. And a general temperature widget that displays all of the system temperatures, regardless of the source.)

I do think we can get there for several widgets, and without adding a data management layer. Or, rather, if we do add a management layer, it won't be as a layer but as a set of utilitity functions that device implementations can use to cache calls for optimization. Widgets already have data management, because the UI toolkit needs it to render the UIs.

The many to many mapping makes it a little more difficult for the widgets to drive updates, but it doesn't seem like it should be horribly difficult to have that managed at a global level with checks to see if a given data source or widget needs to update at any given time.

Have you dug into the code yet? Have you looked, specifically, at the new CPU device/widget separation? Also the procs widget is interesting because I doubt there's either a practical way, or value, to abstract the device from the widget.

[xxxserxxx]

I've referenced this ticket in the wiki; I'm closing the ticket for now.