Rhasspy and HA communicating?

I have Rhasspy setup with a Satellite and Base, and I would like to connect them with Home Assistant.

I’ve been working on trying to ask Rhasspy the temperature, Rhasspy hand off to HA and have HA respond. I’m using the intents option right now.

I can see the Satellit handing things off to the base. I can see the base identifing the intent and publishing to websocket but then I get nothing back.
So I’m not sure if HA isn’t getting the intent, it’s not doing anything, or it’s responding but Rhasspy isn’t doing it’s thing.

So I thought maybe if I try to control a light through Rhasspy and HA that it might be easier to sort where the issue it.

I figured it might easier too, well I think I was wrong. I saw there was once was a RGB light example in the docs at: https://rhasspy.readthedocs.io/en/latest/tutorials/#rgb-light-example
but it seems that the example has been removed.

I have no idea what the HA intent should look light if there is an example I could see that would be great.

I send intents to Home Assistant as events,

If you want to do it this way, you’ll need to create an access token in HA under your username > Long-Lived Access Tokens.

Rhasspy example config:

You can test that the API is accessible in your container, for the above example:

curl -X GET -H "Authorization: Bearer [TOKEN]" -H "Content-Type: application/json" http://192.168.0.46:8123/api/

And it should come back with

{"message": "API running."}

Example of a rhasspy sentence:

[Weather]
how's the weather [outside]
what's the temperature [outside]
how (hot | cold) is it

Check the payload in Home Assistant under Developer Tools > Events > Listen To Events > rhasspy_Weather, then Talk/Text the command “how cold is it outside” to rhasspy, then you should get something like this:

You’ll need to set up Home Assistant to output text to your Rhasspy for TTS, example in configuration.yaml:

rest_command:
  tts:
    url: 'http://[RHASSPYHOST]:12101/api/text-to-speech'
    method: 'POST'
    content_type: text/plain
    payload: '{{payload}}'

Then a simple temperature reading automation example is as follows:

- alias: "Temperature Check"
  trigger:
    platform: event
    event_type: rhasspy_Weather
  action:
     - service: rest_command.tts
       data_template:
         payload: "The temperature is {{ state_attr('weather.home', 'temperature') }} degrees" 

With any luck, it should work for you. This is a super basic example, I’ve just started playing with Rhasspy and it’s HA integration today, so I’m learning as well.

Thanks for the detailed information.

I checked to make sure the HA API was open and accessable from inside the Rhasspy container as you suggested and I got the “API running” response.

I setup the [Weather] sentence on my Rhasspy base, checged from Intents to Events and updated the HA Token to the one I used to test with.
I setup the rest_command and automation in HA and then restarted the server.
I went to Dev Tools, Events, and started listening to rhasspy_Weather. I spoke my command, and nothing happened on the HA side of things.

I assume that my base isn’t actually connecting with HA.
In my base log I see

[DEBUG:2020-11-11 23:09:21,781] rhasspynlu_hermes: Publishing 623 bytes(s) to hermes/intent/Weather
This log entry was pulled from the container log as the Rhasspy GUI log viewer doesn’t have anything newer than '2020-11-11 22:00:40,996"

But in my Satellite log I see (My Sat is still set to GMT instead of EST)

[DEBUG:2020-11-12 04:09:22,105] rhasspyserver_hermes: Sent 432 char(s) to websocket
[DEBUG:2020-11-12 04:09:22,091] rhasspyserver_hermes: Sent 432 char(s) to websocket
[DEBUG:2020-11-12 04:09:22,073] rhasspyserver_hermes: Sent 432 char(s) to websocket
[DEBUG:2020-11-12 04:09:22,065] rhasspyserver_hermes: Sent 432 char(s) to websocket

I checked my debug node in Node-Red and found that at 23:09 this was sent out over MQTT on the “hermes/intent/Weather” Topic:

“{“input”: “how cold is it”, “intent”: {“intentName”: “Weather”, “confidenceScore”: 1.0}, “siteId”: “Kitchen-Sat”, “id”: null, “slots”: [], “sessionId”: “Kitchen-Sat-porcupine-912aacb5-19f3-40a2-ac7e-ad4b3e208ba4”, “customData”: null, “asrTokens”: [[{“value”: “how”, “confidence”: 1.0, “rangeStart”: 0, “rangeEnd”: 3, “time”: null}, {“value”: “cold”, “confidence”: 1.0, “rangeStart”: 4, “rangeEnd”: 8, “time”: null}, {“value”: “is”, “confidence”: 1.0, “rangeStart”: 9, “rangeEnd”: 11, “time”: null}, {“value”: “it”, “confidence”: 1.0, “rangeStart”: 12, “rangeEnd”: 14, “time”: null}]], “asrConfidence”: null, “rawInput”: “how cold is it”, “wakewordId”: “porcupine”, “lang”: null}”

When I started with Rhasspy I tried for a time using Home Assistant to handle intents, both “intents” and “events”, with rather mixed success. In the end I tried NodeRed instead and found it generally more reliable, although still a bit prone to the odd non-action. It works off the MQTT messages sent by Rhasspy (hermes) as you mentioned. It’s a different style of programming of course, and takes a bit of getting used to, but I’ve generally got on better with NR than HA as the intent-handler.
It may be worth considering. YMMV of course. :wink:

Is there a good guide to Rhasspy and Node-Red? The docs seem to contain bits but stop short of providing full details and they seem out of date?
I’m doubly confused when in intent handling there are 4 options
Disabled, Home Asssitant, Remote HTTP, and Local Command.

Yet in Node-Red I can see the intent information is being broadcast over hermes/intent via MQTT.

I guess I’m looking how to properly use intents and slots, and how to send data back to Rhasspy to acknowledge the request.
eg. Playing Now, or I’ve turned on the light, or Excuse me, there is someone at the door.

I recall seeing a couple of examples of the bare basics in the Rhasspy docs, or it might have been this forum. I never did find much, I was lucky enough to have dabbled before with NodeRed so I guess I made it up by trial and error from there. If you’re interested, when I have a bit more time, I’ll post an example.

I’m using 2.5.5 in a master/satellite setup. I have intent handling set to Disabled on the master, and I think it’s the same on the satellite from memory (it’s down at the moment so I’d need to check). Anyway, NodeRed watches the hermes mqtt messages on the broker and takes the appropriate action according to the name of the intent. It’s worth noting that many of the actions are implemented by service calls in HA via a webhook connecton, and many send speech to the satellite using tts.

Hope this helps, and let me know if you’d like more info.

What does home_assistant in profile.json look like?
It should look something like:

  "home_assistant": {
        "access_token": "YOURACCESSTOKEN",
        "handle_type": "event",
        "url": "http://HOMEASSISTANTIP:8123"
    },

You might want to create a new access token in Home Assistant, put it in the applicable rhasspy configuration area, and ensure you save the changes and restart.

This is how the final process should work:

I had a few flows setup in Node Red that I was using way back when Snips was still a thing.
I had a flow that would start, stop, pause radio streams to my Chromecast speakers. I could also control the volume.
The other flows I had Node Red passing or pulling info to and from Home Assistant.

I haven’t used Node Red in quite a while so I quite rusty and can’t quite remember how the flows work :slight_smile:

I checked my bast profile.json and it looks like your example.

I tried the steps you show in your ‘video’ and I see the base recognizing the intent “Weather” but I don’t see anything in Home Assistant listening to the event “rhasspy_Weather”

I just ran a test and tried typing the 'How cold is it" into the base GUI and I saw the event trigger in HA.
I tried typing the 'How cold is it" into the satellite gui again and it still did not trigger the event in HA but the base did recognize the intent “Weather” and I can see the JSON when I click the ‘Show JSON’ button on the base

This indicates that the event does actually transmit properly from your base,

This indicates that your satellite is probably not set up properly. I set up a satellite according to this:

And then, on the satellite, passing “how cold is it” via recognize (with Handle checked) passed it to the base which then passed it to home assistant. It sounds most likely like an configuration issue regarding the satellite…

It’s brief, but I do have a short video demonstration where I connect Rhasspy and NodeRED: https://youtu.be/IsAlz76PXJQ?t=538

When using intents in HA, you at least need something like this in your configuration.yaml file:

intent:
# Empty

intent_script:
    NameOfTheIntent:
        # See https://www.home-assistant.io/integrations/intent_script/
    

Is there an advantage to using http over MQTT? I have my base and satellite setup using the https://rhasspy.readthedocs.io/en/latest/tutorials/#shared-mqtt-broker guide.

I’m not convinced it’s a problem with the Satellite config as the base recognizes Intents for both typed and spoken commands.

But it seems the base doesn’t pass the Satellite based Intents along, however it is passing Intents for typed commands issued directly to it.

I’ll happily use http if it will work though

I use MQTT for many devices, but I couldn’t get HA’s intent integration working quickly - so I just went event based (http).

I don’t think there’s any disadvantage one way or another, they both can be consumed by HA, it just depends on your use case (and whatever method you can get to work).

MQTT is great if you need multiple devices to possibly subscribe to an event. In my case, rhasspy will only ever be consumed directly by HA, so http based won’t be a disadvantage for me.

I stopped trying to use Intents to communicate with HA and switched to Events but I’m perhaps misusing the term Intent(s) to describe the actionable bit gathered from the spoken command.

ah, I’m still a bit confused how Rhasspy sends out intents. Does it broadcast them over MQTT and via the HA Intent API or HA Event API?

There is one case where I might need Node Red to interact with Rhasspy to stream radio stations to my Chromecast. I haven’t figured out if I can do that through my current RadioStation-selector-in-HA-to-Chromecast setup.

I’ll give that a watch later on.

Question about HTTP vs MQTT satellite/base setup. I noticed that in the MQTT setup the Dialog Management is disabled on the Satellite and set to Rhasspy on the Base. However in the HTTP setup the Satellite is set to Rhasspy and the Base is disabled.

I’m curious why that is?

The dialogue manager’s job is to catch the wake word detection and automatically engage ASR/NLU to do the typical voice assistant loop (wake -> ASR -> NLU -> TTS).

When using MQTT, this loop is managed by the base station because it can send messages to each satellite saying start/stop ASR, NLU, etc. Most people choose to do ASR/NLU on the base anyways, so the satellites end up mostly just running wake and audio playback.

With HTTP, the satellites all think they’re running everything. The ASR/NLU services look like real services, but quietly use Rhasspy’s HTTP API to do their work. So the satellite still needs the dialogue manager to engage those “services” and pass messages between them.

Communicating intents to HA just means using a specific HTTP endpoint that’s enabled by adding intent: to your configuration.yaml file. It’s nice because it directly triggers intent_script instead of you having to handle events.

So with HTTP setup I am seeing the events firing in HA and in the base log I am seeing the TTS being passed back but it’s being sent to the Base and not the originating Site-ID

[DEBUG:2020-11-13 16:54:09,909] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=fc988ead-23a5-4b94-98cd-54d55391f683)
[DEBUG:2020-11-13 16:54:06,885] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/Rhasspy-Base/playBytes/7a36b8b7-903a-43bc-a555-a11ff6803da2, id=fc988ead-23a5-4b94-98cd-54d55391f683)
[DEBUG:2020-11-13 16:54:06,120] rhasspyserver_hermes: Publishing 145 bytes(s) to hermes/tts/say
[DEBUG:2020-11-13 16:54:06,120] rhasspyserver_hermes: -> TtsSay(text=‘The temperature is 6.7 degrees’, site_id=‘Rhasspy-Base’, lang=None, id=‘7a36b8b7-903a-43bc-a555-a11ff6803da2’, session_id=’’)