HACS can fail to load when system_health_info() returns an unexpected exception #1808

lmamakos · 2021-01-12T05:57:08Z

Installation details

Description	Value
HACS version	1.9.0
Home Assistant version	2020.12.1
Installation method for HA	docker container (not supervised)

Checklist

I'm running the newest version of HACS https://github.com/hacs/integration/releases/latest
I have enabled debug logging for my installation.
I have filled out the issue template to the best of my ability.
I have read https://hacs.xyz/docs/issues
This issue is related to the backend of HACS.
This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).

Describe the issue

When HACS starts up, in async_hacs_startup() in hacs/operational/setup.py the system_health_info() function that's called can return an unexpected Exception which causes the HACS loading to fail. When this occurs, the /hacsfiles/ URL path doesn't work, so Lovelace UI elements using custom components don't display. There may be other consequences, too.

In my particular case, a yaml.parser.ParserError can be returned when the lovelace configuration YAML has severe errors within it. The cause of these YAML errors is due to a race condition of when the hass-lovelace_gen component is loaded.

I use the lovelace_gen extension to allow jinja2 templates within the lovelace configuration. Before that extension is loaded, the actual jinja2 template expressions are exposed to the YAML parser directly causing much unhappiness. Of course, once the extension has started the jinja2 templates are evaluated before the YAML parser has a chance to see them.

To be clear, this race condition isn't the bug that I'm reporting. I'm hoping that a change can be made in async_hacs_startup() to be less picky about specific exceptions that are caught. If that's done, HACS can assume that yaml mode is in effect (which is the case) and continue to successfully load. This is just a workaround since there is no dependency or sequencing apparently at work for how custom_components are loaded.

I did make this change:

--- /tmp/hacs/operational/setup.py	2020-12-14 19:18:48.000000000 -0500
+++ hacs/operational/setup.py	2021-01-11 23:24:59.573888355 -0500
@@ -125,7 +125,7 @@

     try:
         lovelace_info = await system_health_info(hacs.hass)
-    except TypeError:
+    except:
         # If this happens, the users YAML is not valid, we assume YAML mode
         lovelace_info = {"mode": "yaml"}
     hacs.log.debug(f"Configuration type: {hacs.configuration.config_type}")

which seems to work around the problem for me. I would be the first to admit this is a crude solution.

Steps to reproduce

Construct a YAML lovelace file with embedded jinja2 syntax, such as the attached:
summary.yaml.txt

I don't believe that you'd need to install the lovelace_gen extension, since the error occurs before it launches.

Debug logs

I believe this is the relevant part of the log that was generated

Logs

2021-01-11 23:13:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for alexa_media which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-01-11 23:13:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-01-11 23:13:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for lovelace_gen which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-01-11 23:13:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for browser_mod which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-01-11 23:13:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for nodered which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-01-11 23:13:26 WARNING (MainThread) [homeassistant.loader] You are using a custom integration for pyscript which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant.
2021-01-11 23:13:30 ERROR (SyncWorker_4) [homeassistant.util.yaml.loader] while parsing a flow mapping
  in "/config/lovelace/views/summary.yaml", line 56, column 18
expected ',' or '}', but got '<scalar>'
  in "/config/lovelace/views/summary.yaml", line 56, column 71
2021-01-11 23:13:30 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry  for hacs
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 73, in parse_yaml
    return yaml.load(content, Loader=SafeLineLoader) or OrderedDict()
  File "/usr/local/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 129, in compose_mapping_node
    item_key = self.compose_node(node, None)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 53, in compose_node
    node: yaml.nodes.Node = super().compose_node(parent, index)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/local/lib/python3.8/site-packages/yaml/composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "/usr/local/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/local/lib/python3.8/site-packages/yaml/parser.py", line 549, in parse_flow_mapping_key
    raise ParserError("while parsing a flow mapping", self.marks[-1],
yaml.parser.ParserError: while parsing a flow mapping
  in "/config/lovelace/views/summary.yaml", line 56, column 18
expected ',' or '}', but got '<scalar>'
  in "/config/lovelace/views/summary.yaml", line 56, column 71

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 236, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/config/custom_components/hacs/__init__.py", line 30, in async_setup_entry
    return await hacs_ui_setup(hass, config_entry)
  File "/config/custom_components/hacs/operational/setup.py", line 67, in async_setup_entry
    return await async_startup_wrapper_for_config_entry()
  File "/config/custom_components/hacs/operational/setup.py", line 91, in async_startup_wrapper_for_config_entry
    startup_result = await async_hacs_startup()
  File "/config/custom_components/hacs/operational/setup.py", line 127, in async_hacs_startup
    lovelace_info = await system_health_info(hacs.hass)
  File "/usr/src/homeassistant/homeassistant/components/lovelace/system_health.py", line 19, in system_health_info
    health_info.update(await hass.data[DOMAIN]["dashboards"][None].async_get_info())
  File "/usr/src/homeassistant/homeassistant/components/lovelace/dashboard.py", line 174, in async_get_info
    config = await self.async_load(False)
  File "/usr/src/homeassistant/homeassistant/components/lovelace/dashboard.py", line 185, in async_load
    is_updated, config = await self.hass.async_add_executor_job(
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/lovelace/dashboard.py", line 204, in _load_config
    config = load_yaml(self.path)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 62, in load_yaml
    return parse_yaml(conf_file)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 73, in parse_yaml
    return yaml.load(content, Loader=SafeLineLoader) or OrderedDict()
  File "/usr/local/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
    data = self.construct_object(node)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 120, in _include_yaml
    return _add_reference(load_yaml(fname), loader, node)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 62, in load_yaml
    return parse_yaml(conf_file)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 73, in parse_yaml
    return yaml.load(content, Loader=SafeLineLoader) or OrderedDict()
  File "/usr/local/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
    data = self.construct_object(node)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 201, in _ordered_dict
    nodes = loader.construct_pairs(node)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 155, in construct_pairs
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 232, in _construct_seq
    (obj,) = loader.construct_yaml_seq(node)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 408, in construct_yaml_seq
    data.extend(self.construct_sequence(node))
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 129, in construct_sequence
    return [self.construct_object(child, deep=deep)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 129, in <listcomp>
    return [self.construct_object(child, deep=deep)
  File "/usr/local/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 120, in _include_yaml
    return _add_reference(load_yaml(fname), loader, node)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 62, in load_yaml
    return parse_yaml(conf_file)
  File "/usr/src/homeassistant/homeassistant/util/yaml/loader.py", line 76, in parse_yaml
    raise HomeAssistantError(exc) from exc
homeassistant.exceptions.HomeAssistantError: while parsing a flow mapping
  in "/config/lovelace/views/summary.yaml", line 56, column 18
expected ',' or '}', but got '<scalar>'
  in "/config/lovelace/views/summary.yaml", line 56, column 71
2021-01-11 23:13:31 ERROR (SyncWorker_0) [homeassistant.util.yaml.loader] while parsing a flow mapping
  in "/config/lovelace/views/summary.yaml", line 56, column 18
expected ',' or '}', but got '<scalar>'
  in "/config/lovelace/views/summary.yaml", line 56, column 71
2021-01-11 23:13:31 ERROR (SyncWorker_7) [homeassistant.util.yaml.loader] while scanning for the next token
found character '%' that cannot start any token
  in "/config/lovelace/dash1/weather.yaml", line 2, column 2
2021-01-11 23:13:33 ERROR (SyncWorker_8) [homeassistant.util.yaml.loader] while parsing a flow mapping
  in "/config/lovelace/views/summary.yaml", line 56, column 18
expected ',' or '}', but got '<scalar>'
  in "/config/lovelace/views/summary.yaml", line 56, column 71

The text was updated successfully, but these errors were encountered:

hacs-bot · 2021-01-12T05:57:24Z

Make sure you have read the issue guidelines and that you filled out the entire template.

lmamakos added the issue:backend For issues with the backend/integration label Jan 12, 2021

lmamakos mentioned this issue Jan 14, 2021

Home Assistant tries to render the lovelace UI before lovelace_gen is loaded. Other custom components may fail, too. thomasloven/hass-lovelace_gen#23

Open

ludeeus mentioned this issue Jan 31, 2021

Add guard for HomeAssistantError #1855

Merged

ludeeus closed this as completed in #1855 Jan 31, 2021

github-actions bot locked and limited conversation to collaborators Aug 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HACS can fail to load when system_health_info() returns an unexpected exception #1808

HACS can fail to load when system_health_info() returns an unexpected exception #1808

lmamakos commented Jan 12, 2021 •

edited

Loading

hacs-bot bot commented Jan 12, 2021

HACS can fail to load when system_health_info() returns an unexpected exception #1808

HACS can fail to load when system_health_info() returns an unexpected exception #1808

Comments

lmamakos commented Jan 12, 2021 • edited Loading

Installation details

Checklist

Describe the issue

Steps to reproduce

Debug logs

hacs-bot bot commented Jan 12, 2021

lmamakos commented Jan 12, 2021 •

edited

Loading