Issues with HA built-in intents [HassTurnOff]

[Thomas_S]

Hi everybody,

I’m trying to get Rhasspy to work with a HA server, to perform simple tasks like turning off/on lights, nothing crazy. I’m using a server/satellite set-up.
Now I’m stuck at testing a few basic functions. I’m not very familiar with linux or programming, so maybe I’m missing something very obvious here.
I’d be grateful if somebody could point me in the right direction.

I want to switch on and off a Zigbee plug, into with a lamp is plugged in. Switching the plug via the HA GUI works fine.
So, in HA an enitity called “living room lamp” shall be turned on/off via Rhasspy.

On the HA-server, I added “intent:” to configuration.yaml.
I’ve added the intents [HassTurnOn] and [HassTurnoff] were added to the servers configuration.ini as described in the tutorial.
In Rhasspy server’s config, the Intent andking is set to “Home Assistant”.

When I talk to the Rhasppy-satellite, the voice command works successfully triggers the intent “HassTurnOff”. But the plug is not switched.

Here’s the log from the satellite:

[DEBUG:2021-01-03 17:50:17,126] rhasspyserver_hermes: Sent 703 char(s) to websocket
[DEBUG:2021-01-03 17:50:17,123] rhasspyserver_hermes: <- NluIntent(input=‘turn off the living room lamp’, intent=Intent(intent_name=‘HassTurnOff’, confidence_score=1.0), site_id=‘default’, id=None, slots=[Slot(entity=‘name’, value={‘kind’: ‘Unknown’, ‘value’: ‘living room lamp’}, slot_name=‘name’, raw_value=‘living room lamp’, confidence=1, range=SlotRange(start=13, end=29, raw_start=13, raw_end=29))], session_id=‘default-porcupine-ed5c5c1e-c803-4f21-8817-82e5d94fd2b3’, custom_data=None, asr_tokens=[[AsrToken(value=‘turn’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘off’, confidence=1.0, range_start=5, range_end=8, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=9, range_end=12, time=None), AsrToken(value=‘living’, confidence=1.0, range_start=13, range_end=19, time=None), AsrToken(value=‘room’, confidence=1.0, range_start=20, range_end=24, time=None), AsrToken(value=‘lamp’, confidence=1.0, range_start=25, range_end=29, time=None)]], asr_confidence=None, raw_input=‘turn off the living room lamp’, wakeword_id=‘porcupine’, lang=None)
[DEBUG:2021-01-03 17:50:10,855] rhasspyserver_hermes: <- HotwordDetected(model_id=’/usr/lib/rhasspy/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/porcupine/resources/keyword_files/raspberrypi/porcupine.ppn’, model_version=’’, model_type=‘personal’, current_sensitivity=0.5, site_id=‘default’, session_id=None, send_audio_captured=None, lang=None)

Log file of Rhasspy server:

[ERROR:2021-01-03 16:50:17,140] rhasspyserver_hermes: NotFound(404)
Traceback (most recent call last):
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1863, in dispatch_request
raise request_.routing_exception
quart.exceptions.NotFound: NotFound(404)
[DEBUG:2021-01-03 16:50:17,077] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/HassTurnOff, id=822b7264-59e3-48f0-b1b8-616ef3586354)
[DEBUG:2021-01-03 16:50:17,071] rhasspyserver_hermes: <- NluIntent(input=‘turn off the living room lamp’, intent=Intent(intent_name=‘HassTurnOff’, confidence_score=1.0), site_id=‘default’, id=‘28259950-a4c7-409d-ae1a-3f487a512341’, slots=[Slot(entity=‘name’, value={‘kind’: ‘Unknown’, ‘value’: ‘living room lamp’}, slot_name=‘name’, raw_value=‘living room lamp’, confidence=1.0, range=SlotRange(start=13, end=29, raw_start=13, raw_end=29))], session_id=‘28259950-a4c7-409d-ae1a-3f487a512341’, custom_data=None, asr_tokens=[[AsrToken(value=‘turn’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘off’, confidence=1.0, range_start=5, range_end=8, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=9, range_end=12, time=None), AsrToken(value=‘living’, confidence=1.0, range_start=13, range_end=19, time=None), AsrToken(value=‘room’, confidence=1.0, range_start=20, range_end=24, time=None), AsrToken(value=‘lamp’, confidence=1.0, range_start=25, range_end=29, time=None)]], asr_confidence=None, raw_input=‘turn off the living room lamp’, wakeword_id=None, lang=None)

Thanks!!

[romkabouter]

Can you add your intent from the configuration.yaml in HA?
HA does not have entity_id’s with spaces, so “living room lamp” is invalid.

[Thomas_S]

Hi romkabouter,

I just added intent: to configuration.yaml:

# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:

# Text to speech
tts:
  - platform: google_translate

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml
intent:

Regarding the name of the enttiy: I (probably falsly) used the name of the enttiy while it should be the entity ID, I suppose?

Would this be correct to add to the sentences.ini?

[HassTurnOn]
turn on the (living room lamp):switch.on_off_plug_in_unit_3 {name}

Thanks for your patience…

[romkabouter]

I think your sentences.ini is correct indeed.

[Thomas_S]

Thanks for your response!
I restarteted HA and Rhasspy, but still doesn’t work, I’m puzzled.
Here the log of the Rhasspy server instance:

Logfile

[ERROR:2021-01-04 12:19:08,421] rhasspyserver_hermes: NotFound(404)
Traceback (most recent call last):
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1863, in dispatch_request
raise request_.routing_exception
quart.exceptions.NotFound: NotFound(404)
[DEBUG:2021-01-04 12:19:08,341] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/HassTurnOff, id=b05c1c9b-2408-4be4-8750-711961644c26)
[DEBUG:2021-01-04 12:19:08,333] rhasspyserver_hermes: <- NluIntent(input=‘turn off the switch.on_off_plug_in_unit_3’, intent=Intent(intent_name=‘HassTurnOff’, confidence_score=1.0), site_id=‘default’, id=‘249b2fc0-f386-4c01-9512-64e0e35e51aa’, slots=[Slot(entity=‘name’, value={‘kind’: ‘Unknown’, ‘value’: ‘switch.on_off_plug_in_unit_3’}, slot_name=‘name’, raw_value=‘living room lamp’, confidence=1.0, range=SlotRange(start=13, end=41, raw_start=13, raw_end=29))], session_id=‘249b2fc0-f386-4c01-9512-64e0e35e51aa’, custom_data=None, asr_tokens=[[AsrToken(value=‘turn’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘off’, confidence=1.0, range_start=5, range_end=8, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=9, range_end=12, time=None), AsrToken(value=‘switch.on_off_plug_in_unit_3’, confidence=1.0, range_start=13, range_end=41, time=None)]], asr_confidence=None, raw_input=‘turn off the living room lamp’, wakeword_id=None, lang=None)

Not sure what am I missing. I tried to following the tutorial step-by-step.
Excerpts from the relevant files:

sentences.ini: - see previous post

configuration.yaml - see previous post

profile.json

profile.json

{
    "dialogue": {
        "system": "rhasspy"
    },
    "handle": {
        "system": "hass"
    },
    "home_assistant": {
        "access_token": "",
        "api_password": "",
        "event_type_format": "rhasspy_{0}",
        "url": "http://192.168.1.34:8123/homeassistant/"
    },
    "intent": {
        "system": "fsticuffs"
    },
    "speech_to_text": {
        "system": "kaldi"
    },
    "text_to_speech": {
        "system": "nanotts"
    }
}

Rhasspy Intant Handling:

[romkabouter]

Are you running Rhasspy as an addon?
If not, you should create a token in Home Assistant and fill in inder Access Token in the Intent Handling
If you do, please the field Hass URL with http://hassio/homeassistant.
I see an 404 error in your Logfile

Which guide did your follow for creating your setup?

[Thomas_S]

Yes, running as add-on. Sorry should have mentioned that.

Set-Up:
Server: Win10 running VM with hass.io 5.9. and Rhasspy 2.5.8.1 add-in & deCONZ

Satellite: RasPi 4 with Raspian Buster Lite and Rhasspy 2.5.8.1

[romkabouter]

Ok, can you set the Hass URL to the default?

I am not that familiar with intents because I use event, maybe the name is really the name and not the enitity id or with underscores instead of spaces.

I suggest to change you living room lamp to livingroom and also your sentences.ini to
turn on the livingroom{name}

[Thomas_S]

Hey thanks again for helping, really appreciated.

I messed around again, but to no avail.

Renamed the entity to “livingroom” in HA.

Changed sentence.ini to:

[HassTurnOff]
turn off the (living room lamp):livingroom {name}

Changed profile.json to default, i.e.:

    "url": "http://hassio/homeassistant"

Added to configuration.yaml:

 http:
   base_url: localip.of.ha.server:8123

[Thomas_S]

Just tried the same set-up but with the entity_ID instead of “name”, no change.

[romkabouter]

After reading this:

Component `intent:` is needed (but undocumented in home-assistant docs) with...

Maybe both intent and intent_script are needed.

[romkabouter]

It is not working for me either.

I suggest not using the built-in intents, but create your own as per:

Rhsaspy 2.5 with Hassio built in intents

I’m just starting out exploring Rhasspy. I’ve setup Rhasspy 2.5 in a docker container on a Pi4 and a satellite docker container on a PI3. I’m trying to use Hassio’s built in intents just to turn lights on and off. I think it’s setup right but there’s no change in the lights from Hassio with an intent named HassTurnOn. If I name the intent something Hassio doesn’t recognize or I assign a value that Hassio doesn’t like I get an error in Hassio’s log for intent.py stating that so I know the in…

I am using event by the way, which trigger automations. For me that is more natural and better to debug

[Thomas_S]

Yeah, I’ll take a look at that. Built-In sounded like the easy solution, but seems not to be the case. Thanks for all the suggestions.

[romkabouter]

And if you want speech feedback, I recommend using events and creating a light from your switch
Put this in your configuration.yaml:

light:
  - platform: switch
    name: Living room
    entity_id: switch. on_off_plug_in_unit_3
    
rest_command:
  rhasspy_speak:
    url: 'http://192.168.1.34:12101/api/text-to-speech?siteId={{ siteid }}'
    method: 'POST'
    payload: '{{ payload }}' 
    content_type: text/plain

And this in automations.yaml (you might be able to do that in the frontend):

- id: '1581372525473'
  alias: EventLights
  trigger:
  - event_data: {}
    event_type: rhasspy_Lights
    platform: event
  condition: []
  action:
  - data_template:
      siteid: '{{trigger.event.data._intent.siteId}}'
      payload: OK, {{ trigger.event.data.location }} {{trigger.event.data.action}}
    service: rest_command.rhasspy_speak
  - data_template:
      entity_id: light.{{ trigger.event.data.location }}
    service_template: light.turn_{{ trigger.event.data.action }}
  mode: single

Change your sentences.ini to this:
[Lights]
turn ($actions){action} the ($locations){location} lamp

Create slot for actions (name it actions, like before the $ in your sentences):

on
off

And for locations (name it locations):

livingroom

Restart HA en set Rhasspy to sending events.

[Thomas_S]

Great idea. I’ll have to take some more time and understand more about scripting, intents etc.

[romkabouter]

I am using the above code exactly like that, you will probably find more “rhasspy_speak” rest command on this forum (copy pasted from me), but I have changed the url and added siteId
This is because when you have a satellite, the speech response also needs to go there.
When you omit the siteId, it will output the audio on your server

Good luck and I will help if you need any

[Thomas_S]

@romkabouter

rest_command:
  rhasspy_speak:
    url: 'http://192.168.1.34:12101/api/text-to-speech?siteId={{ siteid }}'
    method: 'POST'
    payload: '{{ payload }}' 
    content_type: text/plain

Question to this part of your code in the previous post: Is ‘url’ supposed to point to the satellite’s IP? Or the server’s IP? (In my concrete case .34 is the server)
I didn’t try it yet, hope to do so during this week.

[romkabouter]

ServerIP, because that is handling the responses

[Thomas_S]

Didn’t have time to work on this the last 2 weeks, but finally managed now. Triggerin HA automations by firing events works fine.

To your code I have some very, very basic questions to your post. I’ll descibe in layman terms, if I got it correctly:

Do I understand correctly that with the code you posted 2 things shuold be are achieved:
The light is actually switched and Rhasspy returns a spoken feedback, e.g. “OK, livingroom off”?

What I believe the code is doing:
By saying “turn on the livingroom lamp” the event “Lights” is triggered and sent to HA. HA then responds and sends the repsonse back to the rhasspy siteID the trigger came from, in this example ‘satellite’. This is done ist the 1st data_template section.
The 2nd data_template section toggles the switch.

If addtional lights should be integrated, the additiional entity names have to added at the bottem of this code section which then would look lile e.g.

light:
- platform: switch
name: Living room
entity_id: switch. on_off_plug_in_unit_3
entity_id: switch. on_off_plug_in_unit_2

and in sentence.ini and automations.yaml the respective event for that new entites would have to be defined addtionally. Because in your example, the event ‘rhasspy_Lights’ is explictiy for living room.

I know these questions are really basic. 'im just trying to understand if I need to define an individual event for every single possible state. I guess not because this would get very complex very fast, e.g. if “set lamp to brightness 60% and dark blue” were a single event.

[romkabouter]

@Thomas_S

Do I understand correctly that with the code you posted 2 things shuold be are achieved:
The light is actually switched and Rhasspy returns a spoken feedback, e.g. “OK, livingroom off”?

Correct

@Thomas_S

What I believe the code is doing:
By saying “turn on the livingroom lamp” the event “Lights” is triggered and sent to HA. HA then responds and sends the repsonse back to the rhasspy siteID the trigger came from, in this example ‘satellite’. This is done ist the 1st data_template section.
The 2nd data_template section toggles the switch.

Again correct

You cannot add multiple entities with with light->switch. Only one entity:

Light Switch

If you want to be able to switch more lights, simply add them to the locations slot.
The data_template uses the light.{{ trigger.event.data.location }} to determine the correct light.

So, you now have livingroom. If you want to turn on a light in the kitchen, add “kitchen” to the locations slot and add a light with name “kitchen” to Home Assistant.

@Thomas_S

'im just trying to understand if I need to define an individual event for every single possible state. I guess not because this would get very complex very fast, e.g. if “set lamp to brightness 60% and dark blue” were a single event.

No, you can add brightness and color as a slot. Your sentence would be something like

[Lights]
turn ($actions){action} the ($locations){location} lamp
set $locations){location} to $colors{color} with brightness (0…100){brightness}

See documentation here:

Training - Rhasspy

You could also create two events, on for on/off and one for color/brightness.
If you want to use 1 event, you service_template would become more complicated than light.{{}}, because you will need to build depending on what sentence is provided.
My suggestion for that case to search on the the HA forum for service_templates.

@Thomas_S

Because in your example, the event ‘rhasspy_Lights’ is explictiy for living room.

No, the Lights event is for all lights you have in your locations slot which have a corresponding name in Home Assistant.
That is what templating does, it replaces the name.

This: light.{{ trigger.event.data.location }}
can become light.livingroom or light.kitchen or any other light entity.