Roadmap suggestion: Suggest automation enhancement by adding sensors

[tsvi]

I suppose some automations can be improved in accuracy based on some additional inputs.

As an example, I have an automation that triggers based on the wifi BSSID my phone is connected to. Actually what I'm trying to achieve is to understand in which room I am. Obviously, the trigger won't work if I'm without my phone, or it might need some "massaging" to get it to be more exact.

If the AI could suggest some extra sensors that might give a better indication: add a camera, add a mmWave presence sensor, etc... that might be great (you might even monetize the affiliate purchase links)

[ITSpecialist111]

Thank you for your great ideas! I was wondering if we could create a small chatbot within HASS that would allow users to suggest the types of automations they would like to bring together. One challenge we face is associating each of the possible entities, which will require comprehensive intelligence across all entities.

Additionally, I've noticed that there are very few comments on each of the entities, so I might need to explore the repository for more information.

I'll take some time to think about how we can integrate these ideas. My initial thoughts involve using AI agents to gather information into a local repository, allowing AI to analyze the collected data. While this is a larger project, it is not impossible.

[ITSpecialist111]

That's not a bad idea! Let me see if if the assistant to call both - not tried that

[ITSpecialist111]

I can use the existing "assist" though to see the entities through the APIs, and with OpenAI (GPT-4), I can ask it to create the YAML for the automation based on those entities. Honestly, I don’t see much benefit in building another assistant unless I’m completely missing the point—it's been a long day, lol!

[Senbei123]

Well, your integration with the power to collect entities + the very nice wrap up style TOGETHER with the possibilieties of the Home Assistant ASSIST to start a dialog about the outcome would be very very cool.
(Anyhow, I will promote your integration on my youtube channel in germany. ;))

[ITSpecialist111]

Thank you so much! I still have many ideas for enhancing the assistance capability. My initial thoughts remain focused on creating a dedicated dashboard that would display a list of daily suggestions. It would also include a chatbot for making changes, along with additional windows or cards to showcase the up-to-date YAML code for any sensors that need to be created, as well as the automation YAML. While reusing the built-in assist is a great idea, I would appreciate having more screen space to view everything.

Another idea I shared on Reddit involves building agents into Home Assistant (HASS). The concept is that HASS doesn't fully understand what it can see from the entities, so you could provide a knowledge base along with a set of custom instructions to add more context to the integrations and entities. This could include technical schematics, details, manuals, etc. The information could either be imported or linked to from an external source.

By the way, I would love to subscribe to your channel! Could you please send me the link?

DALL-E has tried to mock this up for me ;)

[Senbei123]

Very cool ideas @ITSpecialist111 ! This could be the start of something very powerfull!

As a starting point, I wouldn't focus too much on the YAML part, because my experiance is, that those are most of the time wrong.
But the general idee which is announced by the ai are often realy nice. And figgering out the correct entities for this use case and addapting the automation is then often realy easy.

For the "context" part, I have the idea, to give the ai modell also information how often and when the entity is triggerd. Eg. a motion sensor is trigger mostly in the early morning and evening. Next to this, in the same timeperiod the bathroom light goes on. The the ai could perhaps "think" that this could belong together and suggests an automation which triggers the light based on motion in a specific period of time.

Looking forward to get more ideas and development for this automation.

You find me at https://www.youtube.com/@tristanssmartesheim and the video will be go live on Monday at 5pm cet+1.

Tristan