Xiaomii Miio configuration for Roborock S7 not working anymore

[davinkevin]

The problem

After a complete reset of the vacuum, I tried to add it again in Home Assistant. When I followed the wizard dedicated to Xiaomii Miio I had the following error

I tried the simpler installation process and the one with IP + Token, the same error everytime in the UI.

Credentials & informations are ok, I'm able to use them in Xiaomii Cloud and in the Mii Home application (and I'm using a password manager for it) or even the python script provided in the documentation to get the token.

What version of Home Assistant Core has the issue?

2022.7.7

What was the last working version of Home Assistant Core?

Not relevant, it stopped working while using 2022.4.7 with no reason

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Xiaomii Miio

Link to integration documentation on our website

https://www.home-assistant.io/integrations/xiaomi_miio#xiaomi-mi-robot-vacuum

Diagnostics information

No response

Example YAML snippet

None, everything is done in the UI

Anything in the logs that might be useful for us?

I have some logs, but way after the error in the UI: 


2022-07-27 20:52:00 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 177, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper
    result = await method(view, request, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 109, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 277, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/xiaomi_miio/config_flow.py", line 286, in async_step_cloud
    return await self.async_step_connect()
  File "/usr/src/homeassistant/homeassistant/components/xiaomi_miio/config_flow.py", line 390, in async_step_connect
    await self.hass.config_entries.async_reload(existing_entry.entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1048, in async_reload
    unload_result = await self.async_unload(entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1035, in async_unload
    raise OperationNotAllowed
homeassistant.config_entries.OperationNotAllowed


```log
2022-07-27 20:28:00 WARNING (Thread-16 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_5] is still running at shut
    self._bootstrap_inner()
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/xiaomi_cloud_map_extractor/camera.py", line 273, in update
  File "/config/custom_components/xiaomi_cloud_map_extractor/camera.py", line 320, in _handle_map_name
  File "/usr/local/lib/python3.10/site-packages/miio/click_common.py", line 178, in _wrap
    self._fetch_info()
  File "/usr/local/lib/python3.10/site-packages/miio/integrations/vacuum/roborock/vacuum.py", line 229, in _fetch_info
    info = super()._fetch_info()
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 152, in _fetch_info
    devinfo = DeviceInfo(self.send("miIO.info"))
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 106, in send
    return self._protocol.send(
  File "/usr/local/lib/python3.10/site-packages/miio/miioprotocol.py", line 161, in send
    self.send_handshake()
  File "/usr/local/lib/python3.10/site-packages/miio/miioprotocol.py", line 65, in send_handshake
    m = MiIOProtocol.discover(self.ip)
  File "/usr/local/lib/python3.10/site-packages/miio/miioprotocol.py", line 119, in discover
    data, recv_addr = s.recvfrom(1024)

### Additional information

I'm located in France (and my region is set to France in Mii Home), so I use the `DE` cloud server like requested in the UI and [here](https://www.openhab.org/addons/bindings/miio/#country-servers).

Because the integration stopped working without any internal reason, I think the backend/API of Xiaomii Cloud has been changed. I was in 2022.4.7 and the first error I got was around July the 15th. Since this date, even after upgrade to 2022.7.7, and a reset of the vacuum, nothing changed. 

Extra information, custom integration like https://github.com/PiotrMachowski/lovelace-xiaomi-vacuum-map-card stopped working at the same moment

[probot-home-assistant]

Hey there @rytilahti, @syssi, @starkillerOG, @bieniu, mind taking a look at this issue as it has been labeled with an integration (xiaomi_miio) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

---

xiaomi_miio documentation
xiaomi_miio source
_{^{(message by IssueLinks)}}

[rytilahti]

I don't think the cloud APIs have changed so this is happening for some other reason (considering OperationNotAllowed exception), but I don't know what would be the cause.

If you want to confirm that the cloud connectivity is still working, you could use the miiocli cloud list command from python-miio to verify that: https://python-miio.readthedocs.io/en/latest/discovery.html#obtaining-tokens

[davinkevin]

I've tested it, and it's working well with the library:

If I select all instead of de, the result is the same.

Thank you for your help & support

Kevin

[davinkevin]

FYI, I'm able to connect my vacuum to Google Assistant like described in this documentation, so it means the API and my account is working correctly outside of Home Assistant.

Thank you for your help & support

[rytilahti]

It looks to me that the migration code path is doing something unexpected (homeassistant.config_entries.OperationNotAllowed exception being raised on config entry reload), but I have no idea what is wrong in there nor what would be the proper fix.

As this code was last touched by @starkillerOG over a year ago (#47955) in an unrelated change, I assume something in the core changed while this code path remained unchanged. I'll add core label to this issue in hope that someone with knowledge on what's the best approach to fix this chimes in.

[rytilahti]

Looking at the full call stack, I see some parts of it are from a custom component (File "/config/custom_components/xiaomi_cloud_map_extractor/camera.py", line 320, in _handle_map_name. Could you please deactivate all custom components to see if it starts to work again? If yes, it's possible–even if it seems odd–that the custom component has not updated the python-miio dependency that is causing the problem.

[davinkevin]

FYI, I've uninstalled everything and it's still not working. Even after the 2022.8.1 upgrade, the registration process failed.
And I don't have anything in the log part related to this 😢.

Is there any way to check the python-miio dependency version manually?

[starkillerOG]

@bdraco I think this is caused by PR #73806 that you implemented "Wait for config entry platform forwards"

This is the related part of the code:

core/homeassistant/components/xiaomi_miio/config_flow.py

Lines 355 to 373 in bfd84ba

	unique_id = self.mac
	existing_entry = await self.async_set_unique_id(
	unique_id, raise_on_progress=False
	)
	if existing_entry:
	data = existing_entry.data.copy()
	data[CONF_HOST] = self.host
	data[CONF_TOKEN] = self.token
	if (
	self.cloud_username is not None
	and self.cloud_password is not None
	and self.cloud_country is not None
	):
	data[CONF_CLOUD_USERNAME] = self.cloud_username
	data[CONF_CLOUD_PASSWORD] = self.cloud_password
	data[CONF_CLOUD_COUNTRY] = self.cloud_country
	self.hass.config_entries.async_update_entry(existing_entry, data=data)
	await self.hass.config_entries.async_reload(existing_entry.entry_id)
	return self.async_abort(reason="reauth_successful")

So the config entry is already there and I have update the entry with async_update_entry
Schould the config entry then be reloaded using self.hass.config_entries.async_reload
or does self.hass.config_entries.async_update_entry already ensure the entry is reloaded?

Now the self.hass.config_entries.async_reload throws an error.

[starkillerOG]

@bdraco the example in the dev docs does state that both hass.config_entries.async_update_entry and hass.config_entries.async_reload need to be called.
So I think the example in the dev docs will give the same homeassistant.config_entries.OperationNotAllowed error.

See here: https://developers.home-assistant.io/docs/config_entries_config_flow_handler/#reauthentication

[bdraco]

This issue is reported in 2022.7.7 but #73806 first appeared in 2022.8.x

[bdraco]

If you get a trace from 2022.8.x with the more recent line numbers to match up, I can probably get an idea whats going on.

[bdraco]

Similar issue here #52984

I'm guessing a reload is being triggered while the integration is still setting up.

We should make OperationNotAllowed explain the current state of the config entry when it throws

[starkillerOG]

@davinkevin could you try with 2022.8.x and post a stack trace from that?

@bdraco thank you very much for helping with a second look.
These OperationNotAllowed errors have apeared withouth any change to xiaomi_miio.

[starkillerOG]

@davinkevin do you happen to still have YAML configuration of the miio vacuum?

[davinkevin]

@davinkevin could you try with 2022.8.x and post a stack trace from that?

@bdraco thank you very much for helping with a second look. These OperationNotAllowed errors have apeared withouth any change to xiaomi_miio.

#75835 (comment)

I've tested that in 2022.8.1, I'll try to test on a more recent version today.

@davinkevin do you happen to still have YAML configuration of the miio vacuum?

I've never used the yaml configuration so far. I have just something to share it with Google assistant, nothing else.

[starkillerOG]

@davinkevin So on 2022.8.1 you did not see the homeassistant.config_entries.OperationNotAllowed error in the log anymore. The log was completly clean withouth anything related to xiaomi miio while you tried removing the vacuum integration and adding it back?

[davinkevin]

Yes, I do that every time I try to add the vacuum 😇

[starkillerOG]

@davinkevin still not clear to me if you did or did not see the OperationNotAllowed error in the log while on 2022.8.1.

[davinkevin]

I've upgraded to 2022.8.4 and I have this error in the UI and nothing in logs

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun home-assistant (no readiness notification)
s6-rc: info: service legacy-services successfully started
2022-08-14 17:14:44.959 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration hacs which has not been tested
 by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assist
ant

[starkillerOG]

@bdraco how is it possible there is a unknown error and still nothing in the log?
A unknown error schould always be a exception right and that schould automatically be logged right?

[bdraco]

Usually we have something like this in the config flow code

            except Exception:  # pylint: disable=broad-except
                _LOGGER.exception("Unexpected exception")

I don't see that here but I could have missed it looking on my phone

[davinkevin]

After the integration of #76714, I've upgraded to HASS 2022.9.4, and the problem is still the same 😓.

If I use the configure manually method, I have this in the log:

2022-09-17 12:15:57.766 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 82, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 178, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 73, in wrapper
    result = await method(view, request, data, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 280, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 367, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/xiaomi_miio/config_flow.py", line 326, in async_step_manual
    return await self.async_step_connect()
  File "/usr/src/homeassistant/homeassistant/components/xiaomi_miio/config_flow.py", line 390, in async_step_connect
    await self.hass.config_entries.async_reload(existing_entry.entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1062, in async_reload
    unload_result = await self.async_unload(entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1049, in async_unload
    raise OperationNotAllowed
homeassistant.config_entries.OperationNotAllowed

Let me know if there is something I can do to help on this subject

FYI, I've checked the file content ./usr/src/homeassistant/homeassistant/components/xiaomi_miio/__init__.py and the content is the same as #76714, at least about modification done in this PR.

Have a nice day everyone 👋

[bdraco]

    async def async_unload(self, entry_id: str) -> bool:
        """Unload a config entry."""
        if (entry := self.async_get_entry(entry_id)) is None:
            raise UnknownEntry

        if not entry.state.recoverable:
==>            raise OperationNotAllowed <===

        return await entry.async_unload(self.hass)

Its raising here

[bdraco]

Another guess at whats going on

core/homeassistant/components/xiaomi_miio/config_flow.py

Line 371 in bfd84ba

self.hass.config_entries.async_update_entry(existing_entry, data=data)

This code is not checking if the config entry was actually updated before reloading it. async_update_entry returns a bool if the config entry was actually changed.

We shouldn't reload if it hasn't

It looks like a yaml import will hit this path every time and reload the config entry even if nothing has changed. I expect thats happening in the middle of it being setup during startup and causing the exception when it tries to reload in the middle of being setup.

[davinkevin]

Update after upgrade to 2022.9.5:

The standard setup is still in error without anything in the console 😓.

I have more information if I follow the not recommended way using a token I've fetched using python tools before.

homeassistant.config_entries.OperationNotAllowed: The config entry XXXXX (xiaomi_miio) with entry_id 2a50e…b3ef can
not be unloaded because it is not in a recoverable state (ConfigEntryState.SETUP_IN_PROGRESS)

XXXXX is the name of my Roborock s7 and this one is configured correctly and working (everyday) in the xiaomi app.

NOTE: I've truncated the entry_id and name in the previous error message.

[bdraco]

The new error seems to confirm that #75835 (comment) is the issue

[starkillerOG]

@bdraco I made this PR #78995 in an attempt to fix this issue.
Could you please have a look at it?

[wautersj]

I've read through all the posts in this thread and am having the very same issues.

Also getting the log feedback:
The config entry XXXXX (xiaomi_miio) with entry_id XXXXXXXXX can
not be unloaded because it is not in a recoverable state (ConfigEntryState.SETUP_IN_PROGRESS)

[davinkevin]

FYI @rytilahti, this problem is still present in 2022.10.5

2022-10-28 10:27:51.851 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 82, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 180, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 73, in wrapper
    result = await method(view, request, data, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 280, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 367, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/xiaomi_miio/config_flow.py", line 326, in async_step_manual
    return await self.async_step_connect()
  File "/usr/src/homeassistant/homeassistant/components/xiaomi_miio/config_flow.py", line 390, in async_step_connect
    await self.hass.config_entries.async_reload(existing_entry.entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1083, in async_reload
    unload_result = await self.async_unload(entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1067, in async_unload
    raise OperationNotAllowed(
homeassistant.config_entries.OperationNotAllowed: The config entry Alfred (xiaomi_miio) with entry_id <REDACTED> can
not be unloaded because it is not in a recoverable state (ConfigEntryState.SETUP_IN_PROGRESS)

I have this error if I do the configuration using token. The standard one is not working without any error in the UI again… same as last time, no real change.

[starkillerOG]

@davinkevin the PR that tries to fix the problem is not included in 2022.10.5, It is already included in the beta 2022.11.0b1, and will be in HA 2022.11 in 2 weeks.

[davinkevin]

My bad, I've tested so much version I though this was in 2022.10.

So, I just have to wait for the stable 2022.11 now 😇

[bdraco]

Release is tomorrow
https://calendar.google.com/calendar/u/0/r/week/2022/11/2?eid=NWI0bnM3cGM1bG9qdHUzYTY0M3RzOGE5cHZfMjAyMjExMDIgcDA3bjk4Z28xMW9uYW1kMDhkMGttcTZqaHNAZw&ctz=America/Chicago

[davinkevin]

@bdraco I've just tested it with 2022.11.0. The result is not "good".

If I do the standard setup, I have the same error as before Unknown error occurred.

If I continue manually with the token, this time the wizard ends successfully but the integration is stuck in Initializing state:

I don't have anything relevant in log, just this entry:

2022-11-03 13:40:16.175 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Alfred' for xiaomi_miio integration not ready ye
t; Retrying in background

Someone can re-open this issue please?

[bdraco]

Please open a new issue as its a different error and likely needs a different solution.

[davinkevin]

The first error is exactly the same 🤔… but as you want 🤷