AMR Data Validation

See also AMR CSV Parsing.

The raw CSV data requires some processing before it can be used by the Energy Sparks application. This processing happens once a day and is triggered by a cron job and rake task. This processing includes

Date format handling

Any dates in the raw data is converted in to a Date object using the feed's configured date format.

Analytics validation

Once the data has passed through this initial processing, it is passed through to the analytics code which provides a more comprehensive validation, this includes filling in any missing data, or marking when there is a long gap without any data. The data which is returned from the analytics is then stored in the amr_validated_readings table, ready for use by the application.