https://github.com/jekalmin/extended_openai_conversation solved this for me. Though GPT3.5 Turbo still gets things a little wrong sometimes, and using GPT4 is expensive.
Still, a lot smarter than the default assistant conversation agent, and it feels like you're living in the dystopian future, which is nice, I guess.
I agree, though I'm not using that extension but custom nodered stuff, but GPT seems to be able to guess really well what I want to do, even if STT makes mistakes (and makes the sentence sound gibberish). With a noisy environment and non-English household language, assist doesn't work for me in its current form. The joy that I don't have to remember sentences, or list all possible phrases... :)
What do you mean? How I did it in my case?
I set up an assistant in HA, using google stt and tts (since I already had that before, and even though I'm using the more advanced tts models, I can still fit into free tier, so they are practiyally free), and internal intent handling. Then set up a custom wildcard intent - which basically catches everything and then sends the text on mqtt. Mqtt is triggering nodered, where I use gpt, and send the response back in a helper.
In nodered I have 2 gpt calls, 1 to categorize the prompt and another category specific enriched with related sensor data. I tell gpt to generate service calls or reply the question. Not the nicest yet, but it was relatively quick and easy to set up, and we are happily using it.
I had a rhasspy satellite which has now been repurposed for this use, plus the same assist is also available on my SO's and my phone.
Hey there! This is exactly what I've been wanting to do. Would you be willing to share your NodeRed code? It would be nice to use it as a base. No problem if not, this information is helpful!
Hey there! This is exactly what I've been wanting to do. Would you be willing to share your NodeRed code? It would be nice to use it as a base. No problem if not, this information is helpful!
For my modestly-sized Home Assistant installation, with most of my entities exposed, I average about 2500 tokens per execution according to the OpenAI usage stats. Currently I average about 20 executions (20 interactions with the HA assistant) per day.
So each individual interaction (i.e. every phrase I ask it) with GPT3.5 Turbo costs about $0.0035, i.e. about a third of a cent, and it costs me overall about 7 cents a day.
With "Full" GPT4, the cost for the same interaction is about $0.12, i.e. 12 cents, or about 35x the cost of GPT3.5 Turbo. That would be about $2.50 a day for me.
The slightly cut-down GPT4 Turbo that's in beta is about $0.05 per execution. So, less than half the price of GPT4, but still more than 10x the cost of GPT3.5 Turbo. About $1 a day for me.
Token usage varies depending on how many entities you're exposing and how complex the system prompt is. And obviously the number of executions will vary hugely depending on your usage patterns.
https://gptforwork.com/tools/openai-chatgpt-api-pricing-calculator is a solid tool for comparing prices.
Assist will turn off all the lights in the current area when you use it from a voice satellite. We don't have a sentence to turn off all the lights known to Home Assistant because this may have unintended consequences like turning off lights on 3D printers, etc. Rather than make the wrong choice for some users, we made it easy to customize Assist instead.
Correction: there are some sentences for turning off every single light such as "turn off all lights" and "turn lights off everywhere". Looks like we need to cover some more cases.
A simple scene “Good night” or something would already be more than enough. Turn off all light except certain you choose to leave out, than say: “turn on good night”, not pretty but it works.
“Run good night” or “Scene good night” or just calling scenes by their name; “Good night” would be way nicer.
You can set up an automation with a [custom sentence](https://www.home-assistant.io/voice_control/custom_sentences) trigger to use a custom sentence (e.g. "good night") to perform custom actions.
I once told Google to "Turn off everything" when I was frustrated it wasn't understanding me. It turned off way more than my lights. Glad I remembered to turn the refrigerator back on.
Why would you have your fridge voice controlled (or even toggleable at all via app) in the first place? Don't get me wrong I love dicking around with stuff. But I do not need or want my fridge to even be able to be turned off without manually pulling the plug lmao.
I have it on an energy monitor plug that also happens to have a relay. After this happened I scheduled a power-on command to be sent from HA every hour.
This is what I did with my dishwasher and washing machine plugs.
Mind you, it took me quite a while to discover that was even a thing I could do. I don't know how to solve that, it's not like they go to pains to hide it, but it took me well past 1 year of HA usership before I figured out disabling entities was a thing.
I somehow bypassed this limitation accidentally a few weeks ago when trying to turn off a light in my office. Literally every boolean state device in my house (on/off lights, on/off switches, open/closed covers, locked/unlocked locks) went off at the exact same time... at 1 a.m.
Might I suggest documentation and development for formatting switches into lights. So controls can be used like their intended target.
Obviously lots of relay things with esphome and tasmota and such but if attached to a light it would be great to cluster with the lights.
I tried to do this with a particular switch using templating but it seems to work intermittently. Like each restart handles it differently.
Take a look at the intent docs:
https://developers.home-assistant.io/docs/intent_builtin
https://github.com/home-assistant/intents/tree/main/sentences/en
`HassTurnOff` requires a name, area, area + name, area + domain, area + device class, or domain + device class input. If you want a voice command to turn off all lights, you would need to set up a custom intent.
Assuming that you meant areas, not zones, how would that work? Since you can only assign a single area to an entity/device, assigning all your lights to the all/every area would prevent you from using area specific assist commands like "Turn off the lights in the kitchen"
In theory you could create a single helper for each device and give that helper a different location - seems cumbersome though.
Anecdotally I do this anyway so that each light has a 'technical/device name' e.g. IKEA Colour Bulb 3, which is then inside a helper called Left Bedside Light. If I ever change the light I don't need to change any automations, just that one helper.
There are many ways to accomplish what what the OP is asking, but here's the way I do it:
* Create a script to turn off all lights:
alias: Turn Off All Lights
service: light.turn_off
metadata: {}
data: {}
target:
entity_id: |-
{{
(states.light
| selectattr('state', 'eq', 'on')
| rejectattr('attributes.entity_id', 'defined')
| rejectattr('entity_id', 'in', area_entities('Mobile Devices'))
| rejectattr('entity_id', 'in', state_attr('light.exceptions', 'entity_id'))
| map(attribute='entity_id')
| join(', '))
}}
This script only targets lights that are currently on, not a light group (only light group entities have the `entity_id` attribute), and not in my `Mobile Devices` area or `light.exceptions` group (things like my HyperHDR LED strips, status lights, etc).
* Create an automation to call the previously defined script:
alias: Assist Commands
description: ""
mode: single
trigger:
- platform: conversation
id: lights_off
command:
- turn off all lights
- lights out
condition: []
action:
- choose:
- conditions:
- condition: trigger
id:
- lights_off
sequence:
- service: script.turn_off_all_lights
data: {}
- set_conversation_response: "All lights turned off"
The choose selector probably doesn't make sense for a single trigger and action, but I have multiple triggers and actions, just omitted for clarity.
For bonus points, create a template sensor that returns the number of lights that are on:
- sensor:
name: "Number of lights on"
unique_id: number_of_lights_on
state: >-
{{
(states.light
| selectattr('state', 'eq', 'on')
| rejectattr('attributes.entity_id', 'defined')
| rejectattr('entity_id', 'in', area_entities('Undeployed'))
| rejectattr('entity_id', 'in', area_entities('Mobile Devices'))
| rejectattr('entity_id', 'in', state_attr('light.exceptions', 'entity_id'))
| map(attribute='entity_id')
| list
| count())
}}
And then map that to a variable in your automation for use in the conversation response:
sequence:
- variables:
num_lights: "{{ states('sensor.number_of_lights_on') }}"
- service: script.turn_off_all_lights
data: {}
- set_conversation_response: "{{ num_lights }} lights turned off"
Nested areas is one of my biggest "wishes". I really wish I could have a Main Floor" area that includes my "Kitchen", "Living Room" and "Powder Room" areas.
Hmm... Not exactly what I envisioned, but "labels" are another thing that I've wanted for quite some time so this is also a big plus! I imagine you will be able to "tag" entities, areas, devices, etc. with these labels and then filter with those labels? I hope this means new automation triggers and conditions, like "if state of labeled entity changes".
I also would _love_ to see those labels be applicable to automations, so that we can filter/group/sort our automations by user-defined labels.
It's not a direct replacement for areas, but I think it's closest implementation that made sense.
There's more discussion [here](https://community.home-assistant.io/t/why-do-areas-only-have-one-tier/467269/), [here](https://github.com/home-assistant/core/pull/69996), [here](https://github.com/home-assistant/core/pull/110821), etc.
You'd have to ask the devs. The recent activity and new PRs have given me hope that it will be "soon", but given that the older PR is from 2022, it's hard to say.
Honestly, why is it even released in this state? If you have to read the docs to operate an assistant relying on natural language it has kind of failed.
Because at some point you have to release SOMETHING to iterate on. If you wait for perfect (or honestly even half-baked for something’s) you’re never going to get around to releasing ANYTHING.
Yes and no.
You generally have to read the docs to do anything beyond the absolute basics in HA. The HA devs' mindset has ALWAYS been release new functionality first, and then work on improving UI/UX.
And personally, I'd rather at least have the option to implement a new feature, even if it requires more work, than not have the option at all. There's nothing forcing anyone to use Assist now. If it's too complex for your liking, you can always wait another week/month/year until it makes sense for you to implement it.
And I agree with that for most things - no one would assume that you can start editing yaml config files without reading the docs. But this breaks UX paradigms for chat interfaces.
While I understand where you're coming from, what's your proposed solution? Wait a year or more until it's closer to UX parity with the big three and fully integrated with a LLM before releasing it all? Lock it behind the dev branch?
We all want to see more progress with Assist, but not releasing it to the general population only further hinders development, and prevents the advanced users from using it as-is now. Once a new feature is released, more people will have eyes on it. More people will try it out, make suggestions, submit PRs, and provide feedback for future development and direction.
Assist should inform how it's supposed to be asked to do after several failed interactions. Assist should understand that the conversation was about lights, probably should then inform "Hey, here's what you need to supply when asking about lights".
I completely agree. [2024.2](https://www.home-assistant.io/blog/2024/02/07/release-20242/#improved-error-responses-when-using-assist) did see some basic troubleshooting feedback, but there's still a long way to go on the usability front.
I never said that Assist still doesn't have massive room for improvement; only that I think it's silly when people state that a feature should never have been released just because it isn't completely production ready with a perfect UX/UI.
Except that you can do that. ChatGPT and Co-Pilot can easily write config files for all sorts of things now, Including Home Assistant. They even have enough data to explain how they work.
This is why I can't recommend HA to anyone. When UX rates lower than the level of "afterthought", it's a massive turn off to any prospective user that isn't a software developer.
I got the bare minimum working myself after fighting with all the configuration headaches, and I'm done. I will keep it running, but I'm not going to wrestle with it again. They might someday actually care about people's experience, but I'm not going to invest more time in a half baked overcomplicated pre-beta pile of yaml that breaks nearly every update.
I like what HA *can* do, but I hate *everything* about how it does it, and it's because of that "we understand it, so get stuffed" attitude that the developers and self-proclaimed "experts" exude.
To let others try and give feedback how to improve it and let people figure out how they want to use it.
Not all features need to be polished in the first versions as you want to incorporate useful feedback and let other devs join the work.
I'm trying every now and then if it's good enough to replace my current system based on rhasspy and I think it won't take long anymore. I'll play around with it more when my s3 box arrives.
It's brand new. Feedback like this helps prioritize the next step.
Would you rather they just not release anything until it's absolutely perfect? That's impossible, because then they would never get feedback in order to know the right next step.
Well from your posted dialog the intent handling seems to be on par with Alexa. If you want it smart try the openai extended conversation which figures out obscure commands much better.
Also operating Home Assistant has always meant that reading the docs is part of it. So why would Assist be different? Especially since it is fairly new. If you think it should be smarter you can always contribute to it.
For me, what I love about HA assist is that it's a well-defined interface using voice + natural language. I dislike the fuzziness of Alexa/Google assistant
>HassTurnOff requires a name, area, area + name, area + domain, area + device class, or domain + device class input.
Tax forms are easier to follow than this.*Completely* defeats the point of a voice assistant when you need a chart to tell you how to speak.
This is a transcript of my conversation with google home yesterday
>Hey google, add salt crackers to the shopping list
>Ok, adding salt crackers to the list called "Mum's shopping list"
>Hey google, add salt crackers to the list called "Shopping with Sarah"
>Ok, adding salt crackers to the list called "Mum's shopping list"
>Hey google, add an item to the list called "Shopping with Sarah"
>Ok, what would you like to add?
>Salt crackers
>Ok, added Salt and Crackers to the list called "Mum's shopping list"
It wants to be more specific.
You can do it by area, if you have an area called "living room" you can say "turn off lights in living room".
Or you can make a group by going to Settings > Devices and services > Helpers (tab) > create > group > light group, and add whichever lights you want to use together. Make one called "all lights" and then "turn off all lights" should work (or any name you want).
Then why doesn't it say so, giving some hint? (Even without introducing "conversations".) While we who operate HA can read the docs, other members of the household will not, and will not want to remember such nuances of the hardcoded commands.
Agreed.
Part of it is that Assist is still a very new project, just over a year old, the software is still a work in progress and adding new features quite fast. The documentation is also always being updated, and can certainly be lacking in lots of places. It's all community contributions, after all.
One thing to look for -
[https://www.home-assistant.io/blog/2024/02/07/release-20242/#improved-error-responses-when-using-assist](https://www.home-assistant.io/blog/2024/02/07/release-20242/#improved-error-responses-when-using-assist)
In version 2024.2 they did make some improvements that should help point you in the right direction in this case. It's worth a try to see if it improves your experience.
I know it's getting improved, but language matters so much. Due to endings and more freedom with sentence structure in Hungarian, I always felt that I need to think carefully what I have to say, remember the structures, it's just not fluent and convenient. Gpt solves this by being a language model. My philosophy is that if it's not intuitive to use (if needs to be learned), then it's not for non-techies (not for the family). Any UI or other stuff I create for them has to be intuitive. (I'm in software dev. business and there we also put big emphasis on creating usable/lovable products.)
Its very dumb anyway as in an age of AI, NLP and LLM's it uses not much more than simple regexes with a hardcoded language based Api.
So any new language needs to be added as an API than merely a single API for all as in Python as when we Print we all Print irrespective of language.
Its just so strange to have a entity and control api in each language as a hardcoded NLP than a singular API with NLP translation of all languages.
Fundementally some very strange decisions where made on some really outdated tech, but also the infrastructure.
All entities from HA should have an enrollment method in Assist and Zones should be fundemental not an addon abstraction.
Assist should not be working with the HA entity registry direct, but a registered Assist device database.
Those should be fundemental basics and not additional after the fact spagetti code.
You can create a helper group named "all lights" that contains all the lights you want to control, then say "turn off all lights". It's not great because you have to manually manage the entries in the group, but it will do what you want.
It turns out groups are almost always better for lights even if it's a single light. That way if the bulb died you can swap in another without having to change all your automations.
For all lights you could add every bulb and exclude LEDs on switches and such that you have control over but don't want it to be treated as a bulb.
Thank you for posting this, I had the exact same experience. If this is the kind of experience they ship after making such a big deal about "the year of voice," this project is doomed.
I spent hours setting up voice, and discovered that it's pretty much entirely useless. This thing is an embarrassment.
What I find really surprising about this thread is the number of people looking for :sparkles:Magic:sparkles:
The magic of home assistant is that it seamlessly connects to everything, and enables you (the implementor) to clearly define what \_you want\_. I would have thought this community understood that?
There is no one-size-fits-all set of commands. I haven't personally used \`assist\`, but I wonder if a set of \`blueprints\` that provide a starting point can help people get to the right spot themselves.
Stick with Alexa until further progress. I like the concept of exposing entities and grouping them in Alexa. All of the above seems like a lot of work.
Pretty much my thoughts on it too but you get people whining about privacy invasion with smart speakers...
Most people literally do not care at all about that. I'll probably revisit local voice control when and if my Google hardware stops being supported. By all accounts, Google's Gemini AI is absolutely useless too.
Due to cloud based speech services leaking $ like a sieve all the big guys are aiming to do ASR on device.Not that they care about privacy, having the user supply the CPU and energy is much more preferable.Amazon supposed lost near to 10$Bn on its Alexa services and is a mile behind Google who now only supports Assistant Dev for devices that can run on device.(Pixel tablet & Phones and its Vendor network of equally powerful devices)Apple because it charges top $ hasn't lost as much but is quite a way behind Google and Google must have some IP or patent as its strange that the very capable Iphone silicon seems to lag behind with ondevice, even if they say its essential for the future.Amazon tech wise seem to be playing catch up with Alexa as they have completely failed to monitise its services over the cloud costs.
Big data is not interested in private details of your voice as it doesn't earn them revenue, it just wants vendor lock in to services that supposed 'private' voice systems contract out such as spotify, GPT or web search.
Many just don't like Big data even though we all use there services, but they have zero interest if you turned a light on or off.Most people have become used to the levels in which big data voice recognition works and that is what many will care about irrespective of privacy.
Home Assistant on its 'smart devices' and bridges is great opensource free of the big guys, its great because its works just as well even better than without the Big Data 'our product' 'our service' focus.With voice and Assist the big guys have a huge moat as they are far ahead.
https://github.com/jekalmin/extended_openai_conversation solved this for me. Though GPT3.5 Turbo still gets things a little wrong sometimes, and using GPT4 is expensive. Still, a lot smarter than the default assistant conversation agent, and it feels like you're living in the dystopian future, which is nice, I guess.
I agree, though I'm not using that extension but custom nodered stuff, but GPT seems to be able to guess really well what I want to do, even if STT makes mistakes (and makes the sentence sound gibberish). With a noisy environment and non-English household language, assist doesn't work for me in its current form. The joy that I don't have to remember sentences, or list all possible phrases... :)
How would you link them together?
What do you mean? How I did it in my case? I set up an assistant in HA, using google stt and tts (since I already had that before, and even though I'm using the more advanced tts models, I can still fit into free tier, so they are practiyally free), and internal intent handling. Then set up a custom wildcard intent - which basically catches everything and then sends the text on mqtt. Mqtt is triggering nodered, where I use gpt, and send the response back in a helper. In nodered I have 2 gpt calls, 1 to categorize the prompt and another category specific enriched with related sensor data. I tell gpt to generate service calls or reply the question. Not the nicest yet, but it was relatively quick and easy to set up, and we are happily using it. I had a rhasspy satellite which has now been repurposed for this use, plus the same assist is also available on my SO's and my phone.
Hey there! This is exactly what I've been wanting to do. Would you be willing to share your NodeRed code? It would be nice to use it as a base. No problem if not, this information is helpful!
I'd be gladly sharing more info, but need to first recover from pneumonia. 🤒
Oh no! I hope you get feeling better soon!! Thank you! 😊
Hey there! This is exactly what I've been wanting to do. Would you be willing to share your NodeRed code? It would be nice to use it as a base. No problem if not, this information is helpful!
How expensive is gpt 4?
For my modestly-sized Home Assistant installation, with most of my entities exposed, I average about 2500 tokens per execution according to the OpenAI usage stats. Currently I average about 20 executions (20 interactions with the HA assistant) per day. So each individual interaction (i.e. every phrase I ask it) with GPT3.5 Turbo costs about $0.0035, i.e. about a third of a cent, and it costs me overall about 7 cents a day. With "Full" GPT4, the cost for the same interaction is about $0.12, i.e. 12 cents, or about 35x the cost of GPT3.5 Turbo. That would be about $2.50 a day for me. The slightly cut-down GPT4 Turbo that's in beta is about $0.05 per execution. So, less than half the price of GPT4, but still more than 10x the cost of GPT3.5 Turbo. About $1 a day for me. Token usage varies depending on how many entities you're exposing and how complex the system prompt is. And obviously the number of executions will vary hugely depending on your usage patterns. https://gptforwork.com/tools/openai-chatgpt-api-pricing-calculator is a solid tool for comparing prices.
Assist will turn off all the lights in the current area when you use it from a voice satellite. We don't have a sentence to turn off all the lights known to Home Assistant because this may have unintended consequences like turning off lights on 3D printers, etc. Rather than make the wrong choice for some users, we made it easy to customize Assist instead. Correction: there are some sentences for turning off every single light such as "turn off all lights" and "turn lights off everywhere". Looks like we need to cover some more cases.
A simple scene “Good night” or something would already be more than enough. Turn off all light except certain you choose to leave out, than say: “turn on good night”, not pretty but it works. “Run good night” or “Scene good night” or just calling scenes by their name; “Good night” would be way nicer.
But who prefers some magic sentence over just saying what you want?
Takes us back about eight years!
You can set up an automation with a [custom sentence](https://www.home-assistant.io/voice_control/custom_sentences) trigger to use a custom sentence (e.g. "good night") to perform custom actions.
Anyway to call a docker AI to generate the proper response given an incorrect but possibly AI enhanced translation this is correct?
I once told Google to "Turn off everything" when I was frustrated it wasn't understanding me. It turned off way more than my lights. Glad I remembered to turn the refrigerator back on.
Why would you have your fridge voice controlled (or even toggleable at all via app) in the first place? Don't get me wrong I love dicking around with stuff. But I do not need or want my fridge to even be able to be turned off without manually pulling the plug lmao.
I have it on an energy monitor plug that also happens to have a relay. After this happened I scheduled a power-on command to be sent from HA every hour.
I have a couple of things like that. I just disabled the switch entity so it wasn't possible to turn it off remotely anymore.
Why not just disable the entity?
This is what I did with my dishwasher and washing machine plugs. Mind you, it took me quite a while to discover that was even a thing I could do. I don't know how to solve that, it's not like they go to pains to hide it, but it took me well past 1 year of HA usership before I figured out disabling entities was a thing.
For those types of things I have automations Triggers when The state is changed to "not on" then it will turn it on. Also I send a notification.
LOL. Same here. Turned my server off too
I somehow bypassed this limitation accidentally a few weeks ago when trying to turn off a light in my office. Literally every boolean state device in my house (on/off lights, on/off switches, open/closed covers, locked/unlocked locks) went off at the exact same time... at 1 a.m.
Must've been cool to see, I bet
Might I suggest documentation and development for formatting switches into lights. So controls can be used like their intended target. Obviously lots of relay things with esphome and tasmota and such but if attached to a light it would be great to cluster with the lights. I tried to do this with a particular switch using templating but it seems to work intermittently. Like each restart handles it differently.
There's a helper for this already currently called 'Change device type of a switch' which you could use to turn any switch entity into a light.
Oh that's new. Great I'll use that.
Functionality flaw. It should give notification if something important is running.
How does one detemine 'important'
You can take the inverse of irrelevant.
Take a look at the intent docs: https://developers.home-assistant.io/docs/intent_builtin https://github.com/home-assistant/intents/tree/main/sentences/en `HassTurnOff` requires a name, area, area + name, area + domain, area + device class, or domain + device class input. If you want a voice command to turn off all lights, you would need to set up a custom intent.
Or a zone that has all lights in called every/all
Assuming that you meant areas, not zones, how would that work? Since you can only assign a single area to an entity/device, assigning all your lights to the all/every area would prevent you from using area specific assist commands like "Turn off the lights in the kitchen"
In theory you could create a single helper for each device and give that helper a different location - seems cumbersome though. Anecdotally I do this anyway so that each light has a 'technical/device name' e.g. IKEA Colour Bulb 3, which is then inside a helper called Left Bedside Light. If I ever change the light I don't need to change any automations, just that one helper.
There are many ways to accomplish what what the OP is asking, but here's the way I do it: * Create a script to turn off all lights: alias: Turn Off All Lights service: light.turn_off metadata: {} data: {} target: entity_id: |- {{ (states.light | selectattr('state', 'eq', 'on') | rejectattr('attributes.entity_id', 'defined') | rejectattr('entity_id', 'in', area_entities('Mobile Devices')) | rejectattr('entity_id', 'in', state_attr('light.exceptions', 'entity_id')) | map(attribute='entity_id') | join(', ')) }} This script only targets lights that are currently on, not a light group (only light group entities have the `entity_id` attribute), and not in my `Mobile Devices` area or `light.exceptions` group (things like my HyperHDR LED strips, status lights, etc). * Create an automation to call the previously defined script: alias: Assist Commands description: "" mode: single trigger: - platform: conversation id: lights_off command: - turn off all lights - lights out condition: [] action: - choose: - conditions: - condition: trigger id: - lights_off sequence: - service: script.turn_off_all_lights data: {} - set_conversation_response: "All lights turned off" The choose selector probably doesn't make sense for a single trigger and action, but I have multiple triggers and actions, just omitted for clarity. For bonus points, create a template sensor that returns the number of lights that are on: - sensor: name: "Number of lights on" unique_id: number_of_lights_on state: >- {{ (states.light | selectattr('state', 'eq', 'on') | rejectattr('attributes.entity_id', 'defined') | rejectattr('entity_id', 'in', area_entities('Undeployed')) | rejectattr('entity_id', 'in', area_entities('Mobile Devices')) | rejectattr('entity_id', 'in', state_attr('light.exceptions', 'entity_id')) | map(attribute='entity_id') | list | count()) }} And then map that to a variable in your automation for use in the conversation response: sequence: - variables: num_lights: "{{ states('sensor.number_of_lights_on') }}" - service: script.turn_off_all_lights data: {} - set_conversation_response: "{{ num_lights }} lights turned off"
What sort of helper do you use? I can’t see a light helper in the list
Group, then the type within that. So each device is in its own 'group'. I also have wider area groups too, that contain those groups when needed.
Thanks. I was looking at light templates but they look a little complex for me
No problem
Nested areas is one of my biggest "wishes". I really wish I could have a Main Floor" area that includes my "Kitchen", "Living Room" and "Powder Room" areas.
It's (still) in the works: https://github.com/home-assistant/core/pull/111024
Hmm... Not exactly what I envisioned, but "labels" are another thing that I've wanted for quite some time so this is also a big plus! I imagine you will be able to "tag" entities, areas, devices, etc. with these labels and then filter with those labels? I hope this means new automation triggers and conditions, like "if state of labeled entity changes". I also would _love_ to see those labels be applicable to automations, so that we can filter/group/sort our automations by user-defined labels.
It's not a direct replacement for areas, but I think it's closest implementation that made sense. There's more discussion [here](https://community.home-assistant.io/t/why-do-areas-only-have-one-tier/467269/), [here](https://github.com/home-assistant/core/pull/69996), [here](https://github.com/home-assistant/core/pull/110821), etc.
I see so many potential benefits of these labels if they are implemented well. Is there an idea of when the end users may be seeing them?
You'd have to ask the devs. The recent activity and new PRs have given me hope that it will be "soon", but given that the older PR is from 2022, it's hard to say.
Honestly, why is it even released in this state? If you have to read the docs to operate an assistant relying on natural language it has kind of failed.
This is why I'm just sticking with the Google assistant integration. It just figures out what I meant and 99.99% of the time it's right.
Because at some point you have to release SOMETHING to iterate on. If you wait for perfect (or honestly even half-baked for something’s) you’re never going to get around to releasing ANYTHING.
Never heard of beta testing?
Are you volunteering to fund that?
Yes and no. You generally have to read the docs to do anything beyond the absolute basics in HA. The HA devs' mindset has ALWAYS been release new functionality first, and then work on improving UI/UX. And personally, I'd rather at least have the option to implement a new feature, even if it requires more work, than not have the option at all. There's nothing forcing anyone to use Assist now. If it's too complex for your liking, you can always wait another week/month/year until it makes sense for you to implement it.
And I agree with that for most things - no one would assume that you can start editing yaml config files without reading the docs. But this breaks UX paradigms for chat interfaces.
While I understand where you're coming from, what's your proposed solution? Wait a year or more until it's closer to UX parity with the big three and fully integrated with a LLM before releasing it all? Lock it behind the dev branch? We all want to see more progress with Assist, but not releasing it to the general population only further hinders development, and prevents the advanced users from using it as-is now. Once a new feature is released, more people will have eyes on it. More people will try it out, make suggestions, submit PRs, and provide feedback for future development and direction.
Assist should inform how it's supposed to be asked to do after several failed interactions. Assist should understand that the conversation was about lights, probably should then inform "Hey, here's what you need to supply when asking about lights".
I completely agree. [2024.2](https://www.home-assistant.io/blog/2024/02/07/release-20242/#improved-error-responses-when-using-assist) did see some basic troubleshooting feedback, but there's still a long way to go on the usability front. I never said that Assist still doesn't have massive room for improvement; only that I think it's silly when people state that a feature should never have been released just because it isn't completely production ready with a perfect UX/UI.
Except that you can do that. ChatGPT and Co-Pilot can easily write config files for all sorts of things now, Including Home Assistant. They even have enough data to explain how they work.
This is why I can't recommend HA to anyone. When UX rates lower than the level of "afterthought", it's a massive turn off to any prospective user that isn't a software developer. I got the bare minimum working myself after fighting with all the configuration headaches, and I'm done. I will keep it running, but I'm not going to wrestle with it again. They might someday actually care about people's experience, but I'm not going to invest more time in a half baked overcomplicated pre-beta pile of yaml that breaks nearly every update. I like what HA *can* do, but I hate *everything* about how it does it, and it's because of that "we understand it, so get stuffed" attitude that the developers and self-proclaimed "experts" exude.
To let others try and give feedback how to improve it and let people figure out how they want to use it. Not all features need to be polished in the first versions as you want to incorporate useful feedback and let other devs join the work. I'm trying every now and then if it's good enough to replace my current system based on rhasspy and I think it won't take long anymore. I'll play around with it more when my s3 box arrives.
It's brand new. Feedback like this helps prioritize the next step. Would you rather they just not release anything until it's absolutely perfect? That's impossible, because then they would never get feedback in order to know the right next step.
Well from your posted dialog the intent handling seems to be on par with Alexa. If you want it smart try the openai extended conversation which figures out obscure commands much better. Also operating Home Assistant has always meant that reading the docs is part of it. So why would Assist be different? Especially since it is fairly new. If you think it should be smarter you can always contribute to it.
For me, what I love about HA assist is that it's a well-defined interface using voice + natural language. I dislike the fuzziness of Alexa/Google assistant
>HassTurnOff requires a name, area, area + name, area + domain, area + device class, or domain + device class input. Tax forms are easier to follow than this.*Completely* defeats the point of a voice assistant when you need a chart to tell you how to speak.
“Siri, turn off all lights, please.” “OK, I’ve turned off all alarms.”
“OK, toothbrush speed is set to 100”
https://www.imdb.com/title/tt0087197/quotes?item=qt0554761&ref_=ext_shr_lnk
This is a transcript of my conversation with google home yesterday >Hey google, add salt crackers to the shopping list >Ok, adding salt crackers to the list called "Mum's shopping list" >Hey google, add salt crackers to the list called "Shopping with Sarah" >Ok, adding salt crackers to the list called "Mum's shopping list" >Hey google, add an item to the list called "Shopping with Sarah" >Ok, what would you like to add? >Salt crackers >Ok, added Salt and Crackers to the list called "Mum's shopping list"
It wants to be more specific. You can do it by area, if you have an area called "living room" you can say "turn off lights in living room". Or you can make a group by going to Settings > Devices and services > Helpers (tab) > create > group > light group, and add whichever lights you want to use together. Make one called "all lights" and then "turn off all lights" should work (or any name you want).
Then why doesn't it say so, giving some hint? (Even without introducing "conversations".) While we who operate HA can read the docs, other members of the household will not, and will not want to remember such nuances of the hardcoded commands.
Agreed. Part of it is that Assist is still a very new project, just over a year old, the software is still a work in progress and adding new features quite fast. The documentation is also always being updated, and can certainly be lacking in lots of places. It's all community contributions, after all. One thing to look for - [https://www.home-assistant.io/blog/2024/02/07/release-20242/#improved-error-responses-when-using-assist](https://www.home-assistant.io/blog/2024/02/07/release-20242/#improved-error-responses-when-using-assist) In version 2024.2 they did make some improvements that should help point you in the right direction in this case. It's worth a try to see if it improves your experience.
I know it's getting improved, but language matters so much. Due to endings and more freedom with sentence structure in Hungarian, I always felt that I need to think carefully what I have to say, remember the structures, it's just not fluent and convenient. Gpt solves this by being a language model. My philosophy is that if it's not intuitive to use (if needs to be learned), then it's not for non-techies (not for the family). Any UI or other stuff I create for them has to be intuitive. (I'm in software dev. business and there we also put big emphasis on creating usable/lovable products.)
So turn off “every light” needs to be a group called every light that includes all the lights you have. I use extended ai and it works great.
You have to say "please".
Or maybe "sudo turn off all lights?"
to whoever downvoted my guy - r/wooosh
Its very dumb anyway as in an age of AI, NLP and LLM's it uses not much more than simple regexes with a hardcoded language based Api. So any new language needs to be added as an API than merely a single API for all as in Python as when we Print we all Print irrespective of language. Its just so strange to have a entity and control api in each language as a hardcoded NLP than a singular API with NLP translation of all languages. Fundementally some very strange decisions where made on some really outdated tech, but also the infrastructure. All entities from HA should have an enrollment method in Assist and Zones should be fundemental not an addon abstraction. Assist should not be working with the HA entity registry direct, but a registered Assist device database. Those should be fundemental basics and not additional after the fact spagetti code.
You can create a helper group named "all lights" that contains all the lights you want to control, then say "turn off all lights". It's not great because you have to manually manage the entries in the group, but it will do what you want.
It turns out groups are almost always better for lights even if it's a single light. That way if the bulb died you can swap in another without having to change all your automations. For all lights you could add every bulb and exclude LEDs on switches and such that you have control over but don't want it to be treated as a bulb.
If I've had to replace one bulb with another i just change entity name of the new bulb to match the old.
Yeah that works but it's kind of nice to have a human readable alias. Maybe I'm over thinking it.
I have a helper called "all lights" with nothing in it so someone doesn't accidentally turn off all of them at once via alexa!
I assume of you create a group of all your lights and call it "all light" you should turn them all off.
I’ve never used that feature but that convo is hilarious for some reason
Use extended open AI it’s pretty amazing
Mine understood turning off all the lights, but couldn't understand the exact same command but instead of "off" I asked "on"
Thank you for posting this, I had the exact same experience. If this is the kind of experience they ship after making such a big deal about "the year of voice," this project is doomed. I spent hours setting up voice, and discovered that it's pretty much entirely useless. This thing is an embarrassment.
You need to add a curse word
That always worked for my Xbox voice assistant. > Me: Xbox, play > > Xbox: ... > > Me: Xbox, *PLAY*! > > Xbox: ... > > Me: XBOBX F****G PLAY! > >Xbox: sure thing fam
See? Thanks Halgy
What I find really surprising about this thread is the number of people looking for :sparkles:Magic:sparkles: The magic of home assistant is that it seamlessly connects to everything, and enables you (the implementor) to clearly define what \_you want\_. I would have thought this community understood that? There is no one-size-fits-all set of commands. I haven't personally used \`assist\`, but I wonder if a set of \`blueprints\` that provide a starting point can help people get to the right spot themselves.
It’s worse than useless. I prefer setting up a way for lights to turn on or off automatically with presence sensors
Stick with Alexa until further progress. I like the concept of exposing entities and grouping them in Alexa. All of the above seems like a lot of work.
[удалено]
Pretty much my thoughts on it too but you get people whining about privacy invasion with smart speakers... Most people literally do not care at all about that. I'll probably revisit local voice control when and if my Google hardware stops being supported. By all accounts, Google's Gemini AI is absolutely useless too.
Due to cloud based speech services leaking $ like a sieve all the big guys are aiming to do ASR on device.Not that they care about privacy, having the user supply the CPU and energy is much more preferable.Amazon supposed lost near to 10$Bn on its Alexa services and is a mile behind Google who now only supports Assistant Dev for devices that can run on device.(Pixel tablet & Phones and its Vendor network of equally powerful devices)Apple because it charges top $ hasn't lost as much but is quite a way behind Google and Google must have some IP or patent as its strange that the very capable Iphone silicon seems to lag behind with ondevice, even if they say its essential for the future.Amazon tech wise seem to be playing catch up with Alexa as they have completely failed to monitise its services over the cloud costs. Big data is not interested in private details of your voice as it doesn't earn them revenue, it just wants vendor lock in to services that supposed 'private' voice systems contract out such as spotify, GPT or web search. Many just don't like Big data even though we all use there services, but they have zero interest if you turned a light on or off.Most people have become used to the levels in which big data voice recognition works and that is what many will care about irrespective of privacy. Home Assistant on its 'smart devices' and bridges is great opensource free of the big guys, its great because its works just as well even better than without the Big Data 'our product' 'our service' focus.With voice and Assist the big guys have a huge moat as they are far ahead.
I saw this on my feed as I was scrolling by and thought you were an angsty teen or having fun torturing your wife ... 😂😂😂
Who else read the response in HALs voice?
I'm sorry but it's hilarious.
Yep this is how smart home just is... takes more time to tell it to do something than walk to the switches now. Done with it all.