Issues with HA built-in intents [HassTurnOff]

Final feedback in this thread since this is way offtopic because of me already - thanks for pointing me in the right direction.
Voice command on/off of bulbs and plugs is fine via the events method.

I’ve tried to set up color chagning via a sepeate event, but only works for one entity I hardcode in HA. Somehow the location slot is not picked up by the HA event. I just don’t have the grasp of the HA automations work and heading over to HA forums.

The audio feedback isn’t working at the moment, but I havn’t taken a close look at the forum yet.

Can you post the automation? Otherwise it is a bit hard to find the error :wink:

Ok here goes. :smiley:

In this example I spoke to satellite (“sat”) to turn on an arc lamp. The arc lamp is a zigbee plug defined as light in HA and called “white_1”.

Switching the light works.

The audio feedback event in HA seems to fire fine in HA and feed info back to the server which processes the TTS.

[DEBUG:2021-01-24 10:19:45,272] rhasspyserver_hermes: -> TtsSay(text='OK, white_1 on', site_id='serv', lang=None, id='117ec17f-55e0-4b84-aad5-eac5e9ed4f60', session_id='', volume=1.0)
Full Server Log
[DEBUG:2021-01-24 10:19:45,459] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=de1f2e95-da1b-4907-ac60-86ec90a0a468)
[ERROR:2021-01-24 10:19:45,341] rhasspyserver_hermes: NotFound(404)
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1863, in dispatch_request
    raise request_.routing_exception
quart.exceptions.NotFound: NotFound(404)
[DEBUG:2021-01-24 10:19:45,317] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/serv/playBytes/117ec17f-55e0-4b84-aad5-eac5e9ed4f60, id=de1f2e95-da1b-4907-ac60-86ec90a0a468)
[DEBUG:2021-01-24 10:19:45,272] rhasspyserver_hermes: Publishing 136 bytes(s) to hermes/tts/say
[DEBUG:2021-01-24 10:19:45,272] rhasspyserver_hermes: -> TtsSay(text='OK, white_1 on', site_id='serv', lang=None, id='117ec17f-55e0-4b84-aad5-eac5e9ed4f60', session_id='', volume=1.0)
[DEBUG:2021-01-24 10:19:45,265] rhasspyserver_hermes: Sent 752 char(s) to websocket
[DEBUG:2021-01-24 10:19:45,254] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/Lights, id=7ddf37a8-9c84-4c90-8ac1-073adad75647)
[DEBUG:2021-01-24 10:19:45,245] rhasspyserver_hermes: <- NluIntent(input='switch on white_1', intent=Intent(intent_name='Lights', confidence_score=1.0), site_id='serv', id='41c651c9-c550-4a5a-afce-0c6ba36c5c6e', slots=[Slot(entity='actions', value={'kind': 'Unknown', 'value': 'on'}, slot_name='action', raw_value='on', confidence=1.0, range=SlotRange(start=7, end=9, raw_start=7, raw_end=9)), Slot(entity='locations', value={'kind': 'Unknown', 'value': 'white_1'}, slot_name='location', raw_value='arc lamp', confidence=1.0, range=SlotRange(start=10, end=17, raw_start=10, raw_end=18))], session_id='41c651c9-c550-4a5a-afce-0c6ba36c5c6e', custom_data=None, asr_tokens=[[AsrToken(value='switch', confidence=1.0, range_start=0, range_end=6, time=None), AsrToken(value='on', confidence=1.0, range_start=7, range_end=9, time=None), AsrToken(value='white_1', confidence=1.0, range_start=10, range_end=17, time=None)]], asr_confidence=None, raw_input='switch on arc lamp', wakeword_id=None, lang=None)
[DEBUG:2021-01-24 10:19:45,227] rhasspyserver_hermes: Publishing 204 bytes(s) to hermes/nlu/query
[DEBUG:2021-01-24 10:19:45,226] rhasspyserver_hermes: -> NluQuery(input='switch on arc lamp', site_id='serv', id='41c651c9-c550-4a5a-afce-0c6ba36c5c6e', intent_filter=None, session_id='41c651c9-c550-4a5a-afce-0c6ba36c5c6e', wakeword_id=None, lang=None)
[DEBUG:2021-01-24 10:19:44,302] rhasspyserver_hermes: Handling AsrTextCaptured (topic=hermes/asr/textCaptured, id=5a96629f-a13d-441e-9971-f122cb7ae8e3)
[DEBUG:2021-01-24 10:19:43,875] rhasspyserver_hermes: Publishing 71 bytes(s) to hermes/asr/stopListening
[DEBUG:2021-01-24 10:19:43,875] rhasspyserver_hermes: -> AsrStopListening(site_id='serv', session_id='2b688ffb-bda9-4574-b574-90f17ceaf082')
[DEBUG:2021-01-24 10:19:43,870] rhasspyserver_hermes: Sent 53108 byte(s) of WAV data
[DEBUG:2021-01-24 10:19:43,869] rhasspyserver_hermes: Publishing 178 bytes(s) to hermes/asr/startListening
[DEBUG:2021-01-24 10:19:43,868] rhasspyserver_hermes: -> AsrStartListening(site_id='serv', session_id='2b688ffb-bda9-4574-b574-90f17ceaf082', lang=None, stop_on_silence=False, send_audio_captured=True, wakeword_id=None, intent_filter=None)

Don’t really know what going on in sat1:

Satellite log
[ERROR:2021-01-24 10:19:45,428] rhasspyserver_hermes: 'utf-8' codec can't decode byte 0x80 in position 24: invalid start byte
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1129, in api_play_wav
    url_or_path = data.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 24: invalid start byte
[DEBUG:2021-01-24 10:19:45,314] rhasspyserver_hermes: Sent 825 char(s) to websocket
[DEBUG:2021-01-24 10:19:45,311] rhasspyserver_hermes: <- NluIntent(input='switch on arc lamp', intent=Intent(intent_name='Lights', confidence_score=1.0), site_id='sat1', id=None, slots=[Slot(entity='action', value={'kind': 'Unknown', 'value': 'on'}, slot_name='action', raw_value='on', confidence=1, range=SlotRange(start=7, end=9, raw_start=7, raw_end=9)), Slot(entity='location', value={'kind': 'Unknown', 'value': 'white_1'}, slot_name='location', raw_value='arc lamp', confidence=1, range=SlotRange(start=10, end=17, raw_start=10, raw_end=18))], session_id='sat1-porcupine_raspberry-pi-b9dfa2ae-a686-45f6-b8f8-88fc1fcf27e7', custom_data=None, asr_tokens=[[AsrToken(value='switch', confidence=1.0, range_start=0, range_end=6, time=None), AsrToken(value='on', confidence=1.0, range_start=7, range_end=9, time=None), AsrToken(value='arc', confidence=1.0, range_start=10, range_end=13, time=None), AsrToken(value='lamp', confidence=1.0, range_start=14, range_end=18, time=None)]], asr_confidence=None, raw_input='switch on arc lamp', wakeword_id='porcupine_raspberry-pi', lang=None)
[DEBUG:2021-01-24 10:19:40,981] rhasspyserver_hermes: <- HotwordDetected(model_id='/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/pvporcupine/resources/keyword_files/raspberry-pi/porcupine_raspberry-pi.ppn', model_version='', model_type='personal', current_sensitivity=0.5, site_id='sat1', session_id=None, send_audio_captured=None, lang=None)

In this guy’s post, it was solved, but he seems to play the audio on the same Rhasspy instance where the TTS is produced. I didn’t test this solution on my set-up though.

My Rhasspy instances are configured as shown below, which is most likely wrong, but I didn’t understand the documentation.

sat1:

serv:

documenation

No, not your Rhasspy setup, but your automation form Home Assistant.
The one you created for this:

I’ve tried to set up color chagning via a sepeate event, but only works for one entity I hardcode in HA.

You mention audio feedback is not working and you have hardcoded the entity in HA.
So, assuming you use automations to react on those events, please post the automation you created in Home Asssistant

Why do you post the wav to an endpoint? (the audio play on the server)
That needs to be hermes, because you want to play the audio on the sat1 right?
And maybe more sats when this is done?

The hardcode bit was only about the 2 seperate events for turn on/off and changing color not working as expected first and using an ugly hardcode soltion. But now I sorted it out in the meantime. The automations for on/off an changing colors work with templates (on/off below).

- id: '1581372525473'
  alias: OnOffLight
  trigger:
  - event_data: {}
    event_type: rhasspy_Lights
    platform: event
  condition: []
  action:
  - data_template:
      siteid: '{{trigger.event.data._intent.siteId}}'
      payload: OK, {{ trigger.event.data.location }} {{trigger.event.data.action}}
    service: rest_command.rhasspy_speak
  - data_template:
      entity_id: light.{{  trigger.event.data.location  }}
    service_template: light.turn_{{  trigger.event.data.action  }}
  mode: single

I posted wav to the endpoint only for testing if it works at all in a single satellite set-up.
Of course you’re right it should be dynamic in case of several satellites. At the moment there is only sat1, where audio should be played.

Setting server to “MQTT: internal” and audio playing: Hermes MQTT and the satellite to MQTT: internal and audio playing: aplay didn’t solve it though.

The satellite’s name has to be entered in the server’s config, right?

Can you post this as well? From your configuration. Most probably it is missing the siteid

HA’s configuation.yaml

rest_command:
  rhasspy_speak:
    url: 'http://192.168.1.34:12101/api/text-to-speech?siteId={{ siteid }}'
    method: 'POST'
    payload: '{{ payload }}' 
    content_type: text/plain

If I get Rhasspy’s concept correctly, this should make the powerful server at .34:12101 process the TTS. Satellite is at .14:12101 btw.

That command should work indeed.
It should play audio to the correct sat.

With the server setting to Hermes MQTT, the audio will be published to the broker.
Can you please check if that works? It should be on hermes/audioServer/sat1/playBytes/#
The # is an ID, coming from the session of the Dialogue Manager
When there is an incoming message, the rest_command and server are correctly setup.

Then there is only one problem left, which is the setting of Audio Playing on the sat1.
I am using a device which plays incoming audio on that topic, but maybe that does not work. I will try to see if I can check this with a Pi I have :slight_smile:

I am not sure how the sat1 Audio Playing should be set however.

It looks like it’s not sending it to sat?? The AudioPlayByter line refers to ‘serv’.

Server log, after speaking to satellite:

[DEBUG:2021-01-24 17:14:51,483] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=16d2d383-b8d4-40c5-9149-b1e5dcf27524)
[ERROR:2021-01-24 17:14:48,431] rhasspyserver_hermes: NotFound(404)
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1863, in dispatch_request
    raise request_.routing_exception
quart.exceptions.NotFound: NotFound(404)
[DEBUG:2021-01-24 17:14:48,391] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/serv/playBytes/2e1d5987-8cf7-4b2e-ba36-27daf55a962e, id=16d2d383-b8d4-40c5-9149-b1e5dcf27524)
[DEBUG:2021-01-24 17:14:48,345] rhasspyserver_hermes: Publishing 140 bytes(s) to hermes/tts/say
[DEBUG:2021-01-24 17:14:48,345] rhasspyserver_hermes: -> TtsSay(text='OK, ledstrip_1 off', site_id='serv', lang=None, id='2e1d5987-8cf7-4b2e-ba36-27daf55a962e', session_id='', volume=1.0)
[DEBUG:2021-01-24 17:14:48,329] rhasspyserver_hermes: Sent 765 char(s) to websocket
[DEBUG:2021-01-24 17:14:48,325] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/Lights, id=077322f5-e38e-4120-b63e-230c1fedf181)
[DEBUG:2021-01-24 17:14:48,314] rhasspyserver_hermes: <- NluIntent(input='switch off ledstrip_1', intent=Intent(intent_name='Lights', confidence_score=1.0), site_id='serv', id='44f15910-0d6c-40b2-93d6-e2c3a61eec1c', slots=[Slot(entity='actions', value={'kind': 'Unknown', 'value': 'off'}, slot_name='action', raw_value='off', confidence=1.0, range=SlotRange(start=7, end=10, raw_start=7, raw_end=10)), Slot(entity='locations', value={'kind': 'Unknown', 'value': 'ledstrip_1'}, slot_name='location', raw_value='strip', confidence=1.0, range=SlotRange(start=11, end=21, raw_start=11, raw_end=16))], session_id='44f15910-0d6c-40b2-93d6-e2c3a61eec1c', custom_data=None, asr_tokens=[[AsrToken(value='switch', confidence=1.0, range_start=0, range_end=6, time=None), AsrToken(value='off', confidence=1.0, range_start=7, range_end=10, time=None), AsrToken(value='ledstrip_1', confidence=1.0, range_start=11, range_end=21, time=None)]], asr_confidence=None, raw_input='switch off strip', wakeword_id=None, lang=None)
[DEBUG:2021-01-24 17:14:48,261] rhasspyserver_hermes: Publishing 202 bytes(s) to hermes/nlu/query
[DEBUG:2021-01-24 17:14:48,259] rhasspyserver_hermes: -> NluQuery(input='switch off strip', site_id='serv', id='44f15910-0d6c-40b2-93d6-e2c3a61eec1c', intent_filter=None, session_id='44f15910-0d6c-40b2-93d6-e2c3a61eec1c', wakeword_id=None, lang=None)
[DEBUG:2021-01-24 17:14:47,366] rhasspyserver_hermes: Handling AsrTextCaptured (topic=hermes/asr/textCaptured, id=3d7603f6-cf12-4279-b7be-4c483c6b3d36)
[DEBUG:2021-01-24 17:14:46,999] rhasspyserver_hermes: Publishing 71 bytes(s) to hermes/asr/stopListening
[DEBUG:2021-01-24 17:14:46,998] rhasspyserver_hermes: -> AsrStopListening(site_id='serv', session_id='d5d070d5-e209-466e-8b50-7d58d9358228')
[DEBUG:2021-01-24 17:14:46,998] rhasspyserver_hermes: Sent 56948 byte(s) of WAV data
[DEBUG:2021-01-24 17:14:46,995] rhasspyserver_hermes: Publishing 178 bytes(s) to hermes/asr/startListening
[DEBUG:2021-01-24 17:14:46,994] rhasspyserver_hermes: -> AsrStartListening(site_id='serv', session_id='d5d070d5-e209-466e-8b50-7d58d9358228', lang=None, stop_on_silence=False, send_audio_captured=True, wakeword_id=None, intent_filter=None)

Is this related?

Please fill sat1 in all “Satellite siteIds:”

I do not know if it matters, but that is what I have entered :slight_smile:

For reference, did you follow this tutorial?

And then with Remote HTTP?

If so, then I need to review your settings.
sat1:

  • All Remote HTTP should point to your server and to the correct endpoint.
  • Dialogue Management and Intent Handling should be Rhasspy and Home Assistant.

This looks ok in your previous screenshot, except Intent Handling which is not shown.

serv:

  • Dialogue Management and Intent Handling should be disabled
  • Audio Playing can be disabled

I think what is happening is that the server is handling your intent and sends it to Home Assistant.
Therefore, site_id = serv.

If you disable it and set it on sat1 to Home Assistant, the intent will be send to HA by the sat1

1 Like

Yes, I used those tutorials with the Remote HTTP method.

I now switched to external MQTT, cheked all settings and syntaxes again…and it’s working! :smile: : I am thrilled to bits. Love it! Thanks to everybody creating this for all of us.

1 Like

Hi @Thomas_S and @romkabouter,

I am very pleased to have stumbled on this post, and thanks for all the detail. I had recently posted my issue (which is more or less the same as the way you, @Thomas_S, first describe your own) in a similarly named thread “Rhsaspy 2.5 with Hassio built in intents” (you referred to this, @romkabouter).

I will look into the EVENTS alternative as described here. Thanks again.

I do wonder though: If the built in Hassio intents are known not to be working, should they be reviewed, corrected and updated in the Rhasspy documentation? Or perhaps removed in favour of a rather more comprehensive explanation of implementation using Events? I am motivated by the fact that it took me many days to get the various components in Rhasspy and Hassio set up correctly and to achieve having entities pulled into Rhasspy. Following this with several days trying and failing to make the builtin HASS intents work before finding this thread. I just feel that others following us down this road would benefit from an update to the Rhasspy documents(?) Perhaps this is under way.

Yeah I hoped that giving a rather verbose description of everything I did would help other people who find this thread.

About the doc - well the way I see it is that this project is generously made available to everybody for free by people doing this in their spare time. And the project IMO is still aimed at more experience people, as you also can tell by many very technical posts on here which I usually can’t follow. So I kind of understand that updating the “end-user” tutorials are maybe not the top priorty for the folks.
Also, I’m not sure that it’s clear that the intents aren’t working - maybe the problem is somewhere else.

Just fyi: for me the built in intents in home assistant are working just fine…

The names are the entity names not the entity id

[HassTurnOn]
zet de (woonkamerverlichting | staande lamp | garageverlichting | bureauverlichting | dressoir){name} (aan){state}

[HassTurnOff]
zet de (woonkamerverlichting | staande lamp | garageverlichting | bureauverlichting | dressoir ){name} (uit|af){state}

In my configuration.yaml I just added:

intent: 
intent_script: !include intents.yaml    

an example intent script:

GetTime:
  speech:
    text: Het is nu {{ now().strftime("%H %M") }}
1 Like

Hi @Thomas_S… I agree with you. The work that people are doing here is absolutely incredible. One day I hope I might get up to speed and contribute something useful, but I do feel like a novice at the moment for sure. In my experience and as you have written, documentation is often the last to be updated. Having said that, the Rhasspy (and Hassio) docs are in fact very comprehensive and a lot of care has clearly been taken already to help the new comer get started. So thanks to all for the hard work.

I notice @tipofthesowrd has also replied stating that he has HASS builtin intents working. Thanks for posting that @tipofthesowrd as HASS builitin intents seems like a very clever solution, and I may have another go as I do like it as an approach.

The good news is that I managed to get the Event form working pretty much immediately!

My setup is Rhasspy within HASSIO on a Raspberry Pi. For my simple proof of concept test: I restored the HASSIO system baclk to a point of just after the Rhasspy install and initial choice of configuration entries as per docs. in Rhasspy I then enabled the intent handling as Home Assistant, and chose the Send Events option. No tokens or any other entry changes. In Hassio I created an automation with trigger of type Event and event_id rhasspy_ChangeLightState. The automation Action was kept simple to turn on a specific light to 100% brightness. No changes to the HASSIO configuration.yaml at all (no “intent:” entry, no “auth_providers:”).

The sentences was left “as is” from install of Rhasspy. It worked! Lights tuned on with a simple command of “turn on the dining lights”, and so I’m pretty happy (under statement!) with the Event solution!