PowerHog Data can contain NaN values

[ArneTR]

Currently very many API submissions for the PowerHog data fail because there are NaN values in the data.

I sent you, @ribalba, the detailed dump via email, since it contains sensitive information.

Here an excerpt from the dump to spot the location:

... 
 "gpu": {"freq_hz": 0.0, "idle_ns": 0, "idle_ratio": NaN, "dvfm_states": [{"freq": 389, "used_ns": 0, "used_ratio": 0.0}, {"freq": 486, "used_ns": 0, "used_ratio": 0.0}, {"freq": 648, "used_ns": 0, "used_ratio": 0.0}, {"freq": 778, "used_ns": 0, "used_ratio": 0.0}, {"freq": 972, "used_ns": 0, "used_ratio": 0.0}, {"freq": 1296, "used_ns": 0, "used_ratio": 0.0}], "sw_requested_state": [{"sw_req_state": "P1", "used_ns": 0, "used_ratio": 0.0}, {"sw_req_state": "P2", "used_ns": 0, "used_ratio": 0.0}, {"sw_req_state": "P3", "used_ns": 0, "used_ratio": 0.0}, {"sw_req_state": "P4", "used_ns": 0, "used_ratio": 0.0}, {"sw_req_state": "P5", "used_ns": 0, "used_ratio": 0.0}, {"sw_req_state": "P6", "used_ns": 0, "used_ratio": 0.0}], "sw_state": [{"sw_state": "SW_P1", "used_ns": 0, "used_ratio": 0.0}, {"sw_state": "SW_P2", "used_ns": 0, "used_ratio": 0.0}, {"sw_state": "SW_P3", "used_ns": 0, "used_ratio": 0.0}, {"sw_state": "SW_P4", "used_ns": 0, "used_ratio": 0.0}, {"sw_state": "SW_P5", "used_ns": 0, "used_ratio": 0.0}, {"sw_state": "SW_P6", "used_ns": 0, "used_ratio": 0.0}], "gpu_energy": 0}, "timezone": ["CET", "CEST"]}'

In general I think however the whole data has to be checked to be valida JSON before being sent.

[ArneTR]

On second notice I am not so sure that this is really the underlying problem. ORJSON can convert NaN values fine, as long as they are not a dictionary key.

See:

orjson.dumps({"asd": numpy.NaN}) 
# => {"asd": null}

[ribalba]

So I can't reproduce.

import base64
import zlib
import orjson

with open('/tmp/dump.txt', 'r', encoding='utf-8') as file:
    filecontent = file.read()

decoded_data = base64.b64decode(filecontent)
decompressed_data = zlib.decompress(decoded_data)
print(decompressed_data)
measurement_data = orjson.loads(decompressed_data.decode())
print(measurement_data)

with the dump.txt containing the data you sent me works fine. And I can't find an NaN value.

[ArneTR]

please check the newest email from yesterday afternoon

[ArneTR]

closed via green-coding-solutions/hog#29