Rhasspy and HA communicating?

ah, I’m still a bit confused how Rhasspy sends out intents. Does it broadcast them over MQTT and via the HA Intent API or HA Event API?

There is one case where I might need Node Red to interact with Rhasspy to stream radio stations to my Chromecast. I haven’t figured out if I can do that through my current RadioStation-selector-in-HA-to-Chromecast setup.

I’ll give that a watch later on.

Question about HTTP vs MQTT satellite/base setup. I noticed that in the MQTT setup the Dialog Management is disabled on the Satellite and set to Rhasspy on the Base. However in the HTTP setup the Satellite is set to Rhasspy and the Base is disabled.

I’m curious why that is?

The dialogue manager’s job is to catch the wake word detection and automatically engage ASR/NLU to do the typical voice assistant loop (wake -> ASR -> NLU -> TTS).

When using MQTT, this loop is managed by the base station because it can send messages to each satellite saying start/stop ASR, NLU, etc. Most people choose to do ASR/NLU on the base anyways, so the satellites end up mostly just running wake and audio playback.

With HTTP, the satellites all think they’re running everything. The ASR/NLU services look like real services, but quietly use Rhasspy’s HTTP API to do their work. So the satellite still needs the dialogue manager to engage those “services” and pass messages between them.

Communicating intents to HA just means using a specific HTTP endpoint that’s enabled by adding intent: to your configuration.yaml file. It’s nice because it directly triggers intent_script instead of you having to handle events.

So with HTTP setup I am seeing the events firing in HA and in the base log I am seeing the TTS being passed back but it’s being sent to the Base and not the originating Site-ID

[DEBUG:2020-11-13 16:54:09,909] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=fc988ead-23a5-4b94-98cd-54d55391f683)
[DEBUG:2020-11-13 16:54:06,885] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/Rhasspy-Base/playBytes/7a36b8b7-903a-43bc-a555-a11ff6803da2, id=fc988ead-23a5-4b94-98cd-54d55391f683)
[DEBUG:2020-11-13 16:54:06,120] rhasspyserver_hermes: Publishing 145 bytes(s) to hermes/tts/say
[DEBUG:2020-11-13 16:54:06,120] rhasspyserver_hermes: -> TtsSay(text=‘The temperature is 6.7 degrees’, site_id=‘Rhasspy-Base’, lang=None, id=‘7a36b8b7-903a-43bc-a555-a11ff6803da2’, session_id=’’)

You’re further along than I am - but this post may help:

The TTS rest payload in HA might need to be adjusted based on a siteId variable (hopefully it is exposed as well in the event), I’ll have to play with this in the next couple days and see if I can get it set properly.

The TtsSay should be sent to the site id from the intent, which should have ultimately come from the satellite. Can you follow the MQTT messages from wake -> ASR -> NLU and see where the breakdown is?

Please also open a Github issue if this looks like a bug :slight_smile:

I’m using HTTP now instead of MQTT as I wasn’t able to get MQTT working correctly.

Is there a way for me to get the info you are looking for?

I’m thinking that the siteId is just not being set in HA, as there is no siteId sent back in the url:

This should be able to be modified with the siteId variable in the url. I’ll play around with it tonight.

Edit: Quoted wrong message initially, apologies.

I looked at the JSON received by HA and then on the Rhasspy Base and the SiteID is being set to ‘Rhasspy-Base’
The JSON on the Satellite shows the SiteID as ‘Kitchen-Sat’.

JSON From HA Events (Listen to Events)
Event 1 fired 4:54 PM:

{
“event_type”: “rhasspy_Weather”,
“data”: {
“_text”: “what’s the temperature”,
“_raw_text”: “what’s the temperature”,
“_intent”: {
“input”: “what’s the temperature”,
“intent”: {
“intentName”: “Weather”,
“confidenceScore”: 1
},
“siteId”: “Rhasspy-Base”,
“id”: “c66e3907-4377-4c08-8ea0-24ec1440559a”,
“slots”: [],
“sessionId”: “c66e3907-4377-4c08-8ea0-24ec1440559a”,
“customData”: null,
“asrTokens”: [
[
{
“value”: “what’s”,
“confidence”: 1,
“rangeStart”: 0,
“rangeEnd”: 6,
“time”: null
},
{
“value”: “the”,
“confidence”: 1,
“rangeStart”: 7,
“rangeEnd”: 10,
“time”: null
},
{
“value”: “temperature”,
“confidence”: 1,
“rangeStart”: 11,
“rangeEnd”: 22,
“time”: null
}
]
],
“asrConfidence”: null,
“rawInput”: “what’s the temperature”,
“wakewordId”: null,
“lang”: null
}
},
“origin”: “REMOTE”,
“time_fired”: “2020-11-13T21:54:06.076811+00:00”,
“context”: {
“id”: “c02709fa25fa11eb828c83e2d7432685”,
“parent_id”: null,
“user_id”: “1f9a5bb9dc1a4c21a8783d9775948449”
}
}

I’m experiencing the same issue - I’m not seeing any JSON attribute being sent to HA to determine the satellite siteId. I think this will be a necessary property to get HA to send back TTS to the proper device.

I’ve switched to HA intents on the base and MQTT on the satellite, and it looks like the base does not handle the intent when it receives the message - is this a bug?

When I transmitted the information via HTTP from the satellite, the base did actually handle the intent and send it to HA.

Base config (I also have the FrontRoom Satellite siteId entered under Speech to Text, Text to Speech, and Dialogue Management):

Satellite config:

Rhasspy normally doesn’t listen to messages from the various satellites but we have to specify in the base.

I’m not sure this is the correct answer - the documentation doesn’t show the remote siteId placed in the base siteId:

The documentation says that the satellites should be put under the applicable areas here:

Having said that, putting the satellite siteId in that place indeed does pass through the intent, but it plays back on the base rather than passing through to the satellite, so it is still quasi-broken this way…

It seems that when the base receives the intent from the satellite it changes the siteid to the Base’s siteID.

That’s what I’m seeing as well -
The base receives the JSON payload from the satellite, and that payload shows the satellite siteId proper, but when the base sends that payload to Home Assistant, the siteId in the JSON payload is changed to the base.

I don’t remember where I had read it since the first rhasspy satellite working I did it more than 6 months ago. but as I remember once put the siteId on top is the same if you put it on every text field, so I just put the siteId on top and I left empty the other.

I opened an Issue on Github for this issue

I opened a githug issue.
Let me know what other information you need from me

1 Like

Hi, I know this is an old thread but am having an issue with Rhasspy addon. Satellite works fine sending the speech to the base. Base translates well what I say and tries to connect to HA intent but it fails with 404. I am not sure what to put in the HASS part of the addon, am using SSL with HA and I put the URL of https://xxxx.duckdns.org:8123 in the URL field in Intenet. I also tried the IP address and it also failed

[WARNING:2021-03-19 17:53:34,289] rhasspydialogue_hermes: No session for id 1819bbd9-9247-411d-a69f-95c237516f1d. Dropping recognition.
[DEBUG:2021-03-19 17:53:34,291] rhasspyhomeassistant_hermes: https://caha.duckdns.org:8123/api/intent/handle
[ERROR:2021-03-19 17:53:34,375] rhasspyhomeassistant_hermes: handle_home_assistant_intent
Traceback (most recent call last):
  File "/usr/lib/rhasspy/rhasspy-homeassistant-hermes/rhasspyhomeassistant_hermes/__init__.py", line 169, in handle_home_assistant_intent
    response.raise_for_status()
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 946, in raise_for_status
    headers=self.headers)
aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url=URL('https://xxxxx.duckdns.org:8123/api/intent/handle')
[ERROR:2021-03-19 17:53:34,376] rhasspyhomeassistant_hermes: handle_intent
Traceback (most recent call last):
  File "/usr/lib/rhasspy/rhasspy-homeassistant-hermes/rhasspyhomeassistant_hermes/__init__.py", line 88, in handle_intent
    assert response_dict, f"No response from {self.url}"
AssertionError: No response from https://xxxx.duckdns.org:8123
[ERROR:2021-03-19 17:53:34,377] asyncio: SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x7fafa42fd860>
transport: <_SelectorSocketTransport fd=12 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/lib/python3.7/asyncio/sslproto.py", line 207, in feed_ssldata
    self._sslobj.unwrap()
  File "/usr/lib/python3.7/ssl.py", line 767, in unwrap
    return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)