Skip to content

Commit

Permalink
Merge pull request #10 from DragonQ/intelligent_octopus
Browse files Browse the repository at this point in the history
Add Intelligent Octopus Go tariff support
  • Loading branch information
Yanson authored Sep 1, 2024
2 parents 1a21395 + 5610bac commit 3c34fff
Show file tree
Hide file tree
Showing 8 changed files with 827 additions and 79 deletions.
15 changes: 8 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
######### BASE Image #########
FROM python:3.12-slim as base
FROM python:3.12-slim AS base
LABEL maintainer="Iain Rauch <6860163+Yanson@users.noreply.github.com>"

ARG APP_NAME=octograph
Expand All @@ -15,20 +15,21 @@ ENV POETRY_VIRTUALENVS_CREATE=false \
RUN mkdir -p "/opt/$APP_NAME"
WORKDIR "/opt/$APP_NAME"

RUN python -m pip install --no-cache-dir --upgrade pip poetry wheel

COPY poetry.lock pyproject.toml ./
COPY app app

RUN python -m pip install --upgrade pip poetry && \
poetry install --no-dev --no-root
RUN poetry install --only main --no-root

######### TEST Image #########
FROM base as test
FROM base AS test
COPY tests tests
RUN poetry install
RUN poetry run pytest -v tests/unit

######### PRODUCTION Image #########
FROM base as production
RUN poetry install --no-dev
FROM base AS production
RUN poetry install --only main

ENTRYPOINT ["python", "/opt/octograph/app/octopus_to_influxdb.py"]
ENTRYPOINT ["python", "/opt/octograph/app/octopus_to_influxdb.py"]
47 changes: 25 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,31 +84,34 @@ From then-on, you can use the defaults `--from-date=latest` and `--to-date=tomor
Consumption data for your meters will be available, usually, at some point in the day following the consumption.
Hence, you should run the ingestion a couple of times with the configuration `--from-date=latest --to-date=yesterday` to collect yesterday's data.
If you are on an agile tariff, you might prefer to specify `--to-date=tomorrow` to get pricing data covering today and tomorrow (tomorrow's data is usually available at around 5pm).
If you are on an intelligent tariff, you should run this script at least once every 12 hours because the Octopus GraphQL API only provides information about intelligent dispatch slots that occurred during the previous 12 hours. Note that information about dispatch slots from the API might not align exactly to what appears on your bill (there is often an extra dispatch slot at the end of an EV charging period that is not reflected on your bill as a low-rate period). This is an issue with the API data, not this tool.

## Configuration

| Section | Option | Description | Default |
|------------|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|
| `influxdb` | `url` | InfluxDB connection URL | `http://localhost:8086` |
| | `org` | Influx organisation to use | `primary` |
| | `bucket` | Influx bucket to store the metrics in | `primary` |
| | `token_env_var` | Environment variable to lookup to find InfluxDB authentication token | `OCTOGRAPH_INFLUXDB_TOKEN` |
| | `included_tags` | Tags to add to the InfluxDB data points based on account and meter information (see [below](#available-tags) for list of options) | _None_ |
| | `additional_tags` | Comma-separated key=value list of tags to add to the InfluxDB data points (specify a `location` tag to use the sample dashboard) | _None_ |
| `octopus` | `account_number` | Octopus account number (something like `A-XXXXXXXX`) | |
| | `timezone` | Timezone to work in when converting dates and times (see timezone information [below](#timezone-information) | `Europe/London` |
| | `payment_method` | Pricing to select when there is more than one payment option for an agreement | `DIRECT_DEBIT` |
| | `unit_rate_low_start` | Hour of day (`0-23`) when "night" rate starts (if you have an Economy 7 meter) | `1` |
| | `unit_rate_low_end` | Hour of day (`0-23`) when "night" rate ends | `8` |
| | `gas_meter_types` | Comma-separated key=value list of meter types where key is the meter serial number and value is the type (`SMETS1` or `SMETS2`) | `SMETS1` is assumed when serial number is not found |
| | `volume_correction_factor` | Volume correction factor for `SMETS2` meters which need to be converted from cubic meters to kWh | `1.02264` |
| | `calorific_value` | Calorific value for `SMETS2` meter unit conversion to kWh | `38.8` |
| | `resolution_minutes` | Resolution of data to query and store | `30` |
| | `api_prefix` | Octopus API URI prefix | `https://api.octopus.energy/v1` |
| | `api_key_env_var` | Environment variable to lookup to find the Octopus API key | `OCTOGRAPH_OCTOPUS_API_KEY` |
| | `enable_cache` | Cache results from the Octopus API on the local filesystem (useful for development) | `/tmp/octopus_api_cache` |
| | `cache_dir` | Directory to store API responses in for caching purposes (if enabled) | `false` |
| | `included_meters` | Comma-separated list of meter serial numbers to include (useful when some meters in your account are not active) | All meters found in your account if not provided |
| Section | Option | Description | Default |
|------------|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|
| `influxdb` | `url` | InfluxDB connection URL | `http://localhost:8086` |
| | `org` | Influx organisation to use | `primary` |
| | `bucket` | Influx bucket to store the metrics in | `primary` |
| | `token_env_var` | Environment variable to lookup to find InfluxDB authentication token | `OCTOGRAPH_INFLUXDB_TOKEN` |
| | `included_tags` | Tags to add to the InfluxDB data points based on account and meter information (see [below](#available-tags) for list of options) | _None_ |
| | `additional_tags` | Comma-separated key=value list of tags to add to the InfluxDB data points (specify a `location` tag to use the sample dashboard) | _None_ |
| `octopus` | `account_number` | Octopus account number (something like `A-XXXXXXXX`) | |
| | `timezone` | Timezone to work in when converting dates and times (see timezone information [below](#timezone-information) | `Europe/London` |
| | `payment_method` | Pricing to select when there is more than one payment option for an agreement | `DIRECT_DEBIT` |
| | `unit_rate_low_start` | Hour of day (`0-23`) when "night" rate starts (if you have an Economy 7 meter) | `1` |
| | `unit_rate_low_end` | Hour of day (`0-23`) when "night" rate ends | `8` |
| | `gas_meter_types` | Comma-separated key=value list of meter types where key is the meter serial number and value is the type (`SMETS1` or `SMETS2`) | `SMETS1` is assumed when serial number is not found |
| | `volume_correction_factor` | Volume correction factor for `SMETS2` meters which need to be converted from cubic meters to kWh | `1.02264` |
| | `calorific_value` | Calorific value for `SMETS2` meter unit conversion to kWh | `38.8` |
| | `resolution_minutes` | Resolution of data to query and store | `30` |
| | `api_prefix` | Octopus API URI prefix | `https://api.octopus.energy/v1` |
| | `api_key_env_var` | Environment variable to lookup to find the Octopus API key | `OCTOGRAPH_OCTOPUS_API_KEY` |
| | `enable_cache` | Cache results from the Octopus API on the local filesystem (useful for development) | `/tmp/octopus_api_cache` |
| | `cache_dir` | Directory to store API responses in for caching purposes (if enabled) | `false` |
| | `included_meters` | Comma-separated list of meter serial numbers to include (useful when some meters in your account are not active) | All meters found in your account if not provided |
| | `intelligent_tariff_meter` | Meter serial number for intelligent import tariff (so low rate periods can be applied to this meter's consumption) | _None_ |
| | `intelligent_low_rate_hour` | Hour of day (`0-23`) to copy "off-peak" rate from for intelligent dispatch slots | `3` |

### Available Tags

Expand Down
34 changes: 34 additions & 0 deletions app/intelligent_dispatches.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
from datetime import datetime

class IntelligentDispatchItem:
start: datetime
end: datetime
charge_in_kwh: float
source: str
location: str

def __init__(
self,
start: datetime,
end: datetime,
charge_in_kwh: int,
source: str,
location: str
):
self.start = start
self.end = end
self.charge_in_kwh = charge_in_kwh
self.source = source
self.location = location

class IntelligentDispatches:
planned: list[IntelligentDispatchItem]
completed: list[IntelligentDispatchItem]

def __init__(
self,
planned: list[IntelligentDispatchItem],
completed: list[IntelligentDispatchItem]
):
self.planned = planned
self.completed = completed
170 changes: 170 additions & 0 deletions app/octopus_graphql_api_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
import json
import aiohttp
import datetime as dt
import re
from typing import Any
from intelligent_dispatches import IntelligentDispatchItem, IntelligentDispatches
from contextlib import suppress
import click

DATETIME_RE = re.compile(
r"(?P<year>\d{4})-(?P<month>\d{1,2})-(?P<day>\d{1,2})"
r"[T ](?P<hour>\d{1,2}):(?P<minute>\d{1,2})"
r"(?::(?P<second>\d{1,2})(?:\.(?P<microsecond>\d{1,6})\d{0,6})?)?"
r"(?P<tzinfo>Z|[+-]\d{2}(?::?\d{2})?)?$"
)

intelligent_dispatches_query = '''query {{
plannedDispatches(accountNumber: "{account_id}") {{
startDt
endDt
delta
meta {{
source
location
}}
}}
completedDispatches(accountNumber: "{account_id}") {{
startDt
endDt
delta
meta {{
source
location
}}
}}
}}'''

def parse_datetime(dt_str: str) -> dt.datetime | None:
"""Parse a string and return a datetime.datetime.
This function supports time zone offsets. When the input contains one,
the output uses a timezone with a fixed offset from UTC.
Raises ValueError if the input is well formatted but not a valid datetime.
Returns None if the input isn't well formatted.
"""
with suppress(ValueError, IndexError):
return dt.datetime.fromisoformat(dt_str)

if not (match := DATETIME_RE.match(dt_str)):
return None
kws: dict[str, Any] = match.groupdict()
if kws["microsecond"]:
kws["microsecond"] = kws["microsecond"].ljust(6, "0")
tzinfo_str = kws.pop("tzinfo")

tzinfo: dt.tzinfo | None = None
if tzinfo_str == "Z":
tzinfo = dt.UTC
elif tzinfo_str is not None:
offset_mins = int(tzinfo_str[-2:]) if len(tzinfo_str) > 3 else 0
offset_hours = int(tzinfo_str[1:3])
offset = dt.timedelta(hours=offset_hours, minutes=offset_mins)
if tzinfo_str[0] == "-":
offset = -offset
tzinfo = dt.timezone(offset)
kws = {k: int(v) for k, v in kws.items() if v is not None}
kws["tzinfo"] = tzinfo
return dt.datetime(**kws)

class OctopusGraphQlApiClient:
def __init__(self, api_key, timeout_in_seconds = 15):
if (api_key is None):
raise click.ClickException("API KEY is not set")

self._api_key = api_key
self._base_url = 'https://api.octopus.energy'

self._graphql_token = None
self._graphql_expiration = None

self.timeout = aiohttp.ClientTimeout(total=timeout_in_seconds)
self.api_token_query = '''mutation {{
obtainKrakenToken(input: {{ APIKey: "{api_key}" }}) {{
token
}}
}}'''

async def async_refresh_token(self):
"""Get the user's refresh token"""
if (self._graphql_expiration is not None and (self._graphql_expiration - dt.timedelta(minutes=5)) > dt.datetime.now()):
return

async with aiohttp.ClientSession(timeout=self.timeout) as client:
url = f'{self._base_url}/v1/graphql/'
payload = { "query": self.api_token_query.format(api_key=self._api_key) }
async with client.post(url, json=payload) as token_response:
token_response_body = await self.__async_read_response__(token_response, url)
if (token_response_body is not None and
"data" in token_response_body and
"obtainKrakenToken" in token_response_body["data"] and
token_response_body["data"]["obtainKrakenToken"] is not None and
"token" in token_response_body["data"]["obtainKrakenToken"]):

self._graphql_token = token_response_body["data"]["obtainKrakenToken"]["token"]
self._graphql_expiration = dt.datetime.now() + dt.timedelta(hours=1)
else:
raise click.ClickException("Failed to retrieve auth token")

async def async_get_intelligent_dispatches(self, account_id: str):
"""Get the user's intelligent dispatches"""
await self.async_refresh_token()

async with aiohttp.ClientSession(timeout=self.timeout) as client:
url = f'{self._base_url}/v1/graphql/'
# Get account response
payload = { "query": intelligent_dispatches_query.format(account_id=account_id) }
headers = { "Authorization": f"JWT {self._graphql_token}" }
async with client.post(url, json=payload, headers=headers) as response:
response_body = await self.__async_read_response__(response, url)

if (response_body is not None and "data" in response_body):
return IntelligentDispatches(
list(map(lambda ev: IntelligentDispatchItem(
parse_datetime(ev["startDt"]),
parse_datetime(ev["endDt"]),
float(ev["delta"]) if "delta" in ev and ev["delta"] is not None else None,
ev["meta"]["source"] if "meta" in ev and "source" in ev["meta"] else None,
ev["meta"]["location"] if "meta" in ev and "location" in ev["meta"] else None,
), response_body["data"]["plannedDispatches"]
if "plannedDispatches" in response_body["data"] and response_body["data"]["plannedDispatches"] is not None
else [])
),
list(map(lambda ev: IntelligentDispatchItem(
parse_datetime(ev["startDt"]),
parse_datetime(ev["endDt"]),
float(ev["delta"]) if "delta" in ev and ev["delta"] is not None else None,
ev["meta"]["source"] if "meta" in ev and "source" in ev["meta"] else None,
ev["meta"]["location"] if "meta" in ev and "location" in ev["meta"] else None,
), response_body["data"]["completedDispatches"]
if "completedDispatches" in response_body["data"] and response_body["data"]["completedDispatches"] is not None
else [])
)
)
else:
raise click.ClickException("Failed to retrieve intelligent dispatches")

return None

async def __async_read_response__(self, response, url):
"""Reads the response, logging any json errors"""

text = await response.text()

if response.status >= 400:
if response.status >= 500:
raise click.ClickException(f'Octopus Energy server error ({url}): {response.status}; {text}')
elif response.status not in [401, 403, 404]:
raise click.ClickException(f'Failed to send request ({url}): {response.status}; {text}')
return None

data_as_json = None
try:
data_as_json = json.loads(text)
except:
raise click.ClickException(f'Failed to extract response json: {url}; {text}')

if ("graphql" in url and "errors" in data_as_json):
raise click.ClickException(f'Errors in request ({url}): {data_as_json["errors"]}')

return data_as_json
Loading

0 comments on commit 3c34fff

Please sign in to comment.