Help configuring Rhasspy with Home Assistant

Help
1

Hi All,

I’ve read a post or two here, multiple documents and tutorials but I can’t figure out how to get Rhasspy working with Home Assistant 2.0.

My configuration:

RPI 4
Home Assistant Full OS
Rhasspy add-on
Disregard mic at this time, I’m working with the text based intents from the Rhasspy home and will worry about a mic once I get the manually typed intents working

Can anyone provide a COMPLETE guide for configuring Rhasspy? I keep running in to inconsistent guidance, likely because many seem to use Raspian with docker for HA, and I can’t figure out where to go next.

If anyone is willing to literally hold my hand through the configuration, I’ll be happy to draft a complete document of those steps to post to the public.

Rather than post what I’ve done, and where things aren’t working, I want to start over from the beginning to ensure that none of the changes I’ve made interfere with instructions I’ve been given.

I can attest, I was able to configure Rhasspy to understand an intent “turn on living room lamp” and I was able to configure an automation in HA that turned on a switch which will control a lamp. As far as I can tell, I never got Rhasspy to correctly communicate with HA. I see no means of testing if messages are being successfully sent to HA, or if they’re being received by HA.

Your assistance is most appreciated. If there is another post in these forums with this information, please feel free to link it and I’ll happily review, but I have been unable to find a clear guide.

2

Hi MagicTomato! I think the Rhasspy Docs are very good (among the best I’ve seen on open source software) but both Rhasspy and HA have a bit of steep learning curve. Take a look at this section of the docs:

https://rhasspy.readthedocs.io/en/latest/intent-handling/

You’ll see that there are several ways of integrating Rhasspy and HA, at the “top” level, you can use the HA API or you can use MQTT. If you want to use the HA API, you’ll need to change your “Intent Handling” section in Rhasspy’s settings (the “gears” icon") to enter the information on HA instance. (IP address, the “long-lived access token” you generate in HA) If you want to use MQTT, you’ll need to add:

snips:

to your configuration.yaml in HA.
Once you decide on whether you want to use the API or MQTT, you’ll need to decide whether to use “Intents” or “Events”.
Personally, I like to use MQTT, I use a program called MQTTfx to monitor what Rhasspy is publishing on MQTT when I building a new command.
I’m very far from an expert, so there might be folks who will jump in with more details (or correct me if I’ve gotten anything incorrect :slight_smile: .
I’d suggest the following (just so you can get something to work (and boy, the first time it does, it feels really cool!)
Unfortunately, you’ll need to set up MQTT broker on your raspberry, which is a little complicated. Here’s a good tutorial on how to do it:

https://randomnerdtutorials.com/how-to-install-mosquitto-broker-on-raspberry-pi/

Next, add the MQTT integration to HA:

https://www.home-assistant.io/integrations/mqtt/

Next, add the “snips:” component to HA by adding it to your configuration.yaml (reboot HA afterwards).

Finally, again in your configuration.yaml, add this:

intent:
intent_script:
  ChangeLightState:
    action:
      - service_template: 'switch.turn_{{state}}'
        data_template:
          entity_id: switch.{{ name | replace(" ","_") }}

Watch the spacing in your configuration.yaml, YAML is very picky about spacing!
Since you have an automation with a switch, and Rhasspy has a “ChangeLightState” sentence as a default, this should work for you. Hope this helps!

1 Like
3

This will help: Beginners guide to use Rhasspy with HomeAssistant

Or the wiki on github Home · rhasspy/rhasspy Wiki · GitHub

@JeffC you do not have to add snips: to the config :wink:

2 Likes
4

Hi all,

Thank you for the replies. I have been busy an unable to attend to this project since your replies. I will follow up soon!

MT

5

As always, thanks for the help romkabouter! (another thing for my list of things to clean up :grinning: )

6

Hello all,

I have followed the guides provided and I am still stuck.

I have successfully configured the MQTT broker and I can confirm that both HA and Rhasspy are successfully publishing messages.

I have also configured an automation which is listening for the event rhasspy_ChangeLightState. Rhasspy is also properly translating the typed command “turn on living room lamp” and the JSON generated reflects “ChangeLightState”.

While monitoring events in the dev tools of HA, I can confirm when I fire the “ChangeLightState” or “rhasspy_ChangeLightState” it will trigger my automation. I tested both events to ensure it worked each way.

While monitoring events and triggering the command from Rhasspy, I get no updates in the event listener. It seems as though Rhasspy is not sending the commands to HA. They are being published to the MQTT broker.

I am using the events configuration under intent handling.

Send events to Home Assistant (/api/events) Events will be named rhasspy_INTENT_NAME"

I have tried every variation of my HomeAssistant URL that I can imagine. I have also generated a new access token multiple times.

Can someone confirm the URL format required for the intent handling field? Or anything else I may have missed.

7

Hi Magic Tomato, Can you post your yaml for the automation? The reason I’m asking is that your automation can act on a MQTT message, or it can be triggered by the API integration. For example, here’s my automation to set my thermostat for a MQTT message:

- id: '1644611952238'
  alias: Thermostat
  description: MQTT
  trigger:
  - platform: mqtt
    topic: hermes/intent/SetTemp
  condition: []
  action:
  - service: climate.set_temperature
    data:
      temperature: '{{ trigger.payload_json.slots[0].value.value }}'
      device_id: 0155e09182c90748aba59850bee83189
8

@JeffC You have saved the day!

I copied your format here and Rhasspy now toggles the Z-Wave controlled switch I have configured. I will post YAML below for review. My question now is, how do I define trigger variables. I’ll paste the MQTT message that is sent to the broker from Rhasspy. I can’t figure out how I configure the automation to listen specifically for “living room lamp” vs “garage light” which were both defaults in Rhasspy. I also can’t figure out how to get it to acknowledge the light state variable in the command “on” vs “off”.

I removed unique ID data as I’m not aware what could be used maliciously if I posted publicly.

YAML:

alias: Rhasspy
description: ‘’
trigger:

  • platform: mqtt
    topic: hermes/intent/ChangeLightState
    condition: []
    action:
  • service: switch.toggle
    data: {}
    target:
    device_id: ***
    entity_id: switch.smart_switch_7
    mode: single

MQTT:

{“input”: “turn living room lamp on”, “intent”: {“intentName”: “ChangeLightState”, “confidenceScore”: 1.0}, “siteId”: “default”, “id”: “", “slots”: [{“entity”: “name”, “value”: {“kind”: “Unknown”, “value”: “living room lamp”}, “slotName”: “name”, “rawValue”: “living room lamp”, “confidence”: 1.0, “range”: {“start”: 5, “end”: 21, “rawStart”: 5, “rawEnd”: 21}}, {“entity”: “state”, “value”: {“kind”: “Unknown”, “value”: “on”}, “slotName”: “state”, “rawValue”: “on”, “confidence”: 1.0, “range”: {“start”: 22, “end”: 24, “rawStart”: 22, “rawEnd”: 24}}], “sessionId”: "”, “customData”: null, “asrTokens”: [[{“value”: “turn”, “confidence”: 1.0, “rangeStart”: 0, “rangeEnd”: 4, “time”: null}, {“value”: “living”, “confidence”: 1.0, “rangeStart”: 5, “rangeEnd”: 11, “time”: null}, {“value”: “room”, “confidence”: 1.0, “rangeStart”: 12, “rangeEnd”: 16, “time”: null}, {“value”: “lamp”, “confidence”: 1.0, “rangeStart”: 17, “rangeEnd”: 21, “time”: null}, {“value”: “on”, “confidence”: 1.0, “rangeStart”: 22, “rangeEnd”: 24, “time”: null}]], “asrConfidence”: null, “rawInput”: “turn living room lamp on”, “wakewordId”: null, “lang”: null}

9

I would also like to know how to use the API option for intent processing, since it seems to be better integrated with Rhasspy’s intents. However, I am pleased to get the two communicating via MQTT.

10

Woo hoo! Isn’t it cool when things work. Couple of things, first, when you post code, enclose the code segment in three back tics(`), e.g. start a new line with three back tics, hit return, paste your code, hit return, and on the last line, three back tics. This will format your code so it is easier to read. Second, here are a couple of sites that I’ve found very useful in the Rhasspy/HA world:

https://jsonpathfinder.com/

The first will help you find that value that you’re looking for in JSON and how to get to the value. The second will help you make sure you’re writing valid YAML. (As I said, YAML is very picky about spaces, and I can’t tell you how many time I’ve wondered why something isn’t working and it turns out I’ve left out a space, or put in one too many).

With regard to your question about trigger variables, this is really getting into HA rather than Rhasspy (and you should sign up for HA’s Discord channel, they’re a good group, and helpful Discord). Anyway, you could certainly set up individual automations for your devices and device states (on/off, etc.), but I wouldn’t recommend it. What I’d recommend you do instead is use templates. Templates will allow you to parse out from the JSON of a MQTT message what action HA should perform on what entity. Here’s an example (not tested, I have to get some work done, but should work)

- id: '1657823581836'
  alias: MQTT Example
  description: ''
  trigger:
  - platform: mqtt
    topic: hermes/intent/ChangeLightState
  condition: []
  action:
  - service_template: switch.turn_{{state}}
    data:
      entity_id: switch.{{ name | replace(" ","_") }}
  mode: single

What this should do is when HA gets a MQTT message from Rhasspy, with the topic ChangeLightState, it will evaluate the JSON to determine in you want on/off, and what device you want to take the action on. by doing it this way, you only have to create the one automation in HA, rather than multiple automations for each device. As I said earlier, Rhasspy and HA have a bit of a learning curve, and I hope I’ve pointed you in the right direction. I think once you’ve built one working loop (Rhasspy to HA and back to Rhasspy if necessary) the second will be easier, and so forth!

11

@JeffC

I clicked the Discord link you provided but it doesn’t seem to be working.

I agree I do not want to set up an individual command set and automation for each device I intend to deploy, as there will be at least 3 lights but likely many others.

I tried to copy your YAML example directly in to an automation but it didn’t work. Was I supposed to change anything to make it unique to my environment? I also tried manually entering my device ID in the {{ }}.

If you think this would best be handled by the HA public forums etc, I have no problem moving my conversation to those platforms.

The message found in my MQTT monitor is below, the JSON sent by Rhasspy is below that. I still don’t grasp which data within the published MQTT message is can be filtered by HA.

{“input”: “turn living room lamp on”, “intent”: {“intentName”: “ChangeLightState”, “confidenceScore”: 1.0}, “siteId”: “default”, “id”: “", “slots”: [{“entity”: “name”, “value”: {“kind”: “Unknown”, “value”: “living room lamp”}, “slotName”: “name”, “rawValue”: “living room lamp”, “confidence”: 1.0, “range”: {“start”: 5, “end”: 21, “rawStart”: 5, “rawEnd”: 21}}, {“entity”: “state”, “value”: {“kind”: “Unknown”, “value”: “on”}, “slotName”: “state”, “rawValue”: “on”, “confidence”: 1.0, “range”: {“start”: 22, “end”: 24, “rawStart”: 22, “rawEnd”: 24}}], “sessionId”: "”, “customData”: null, “asrTokens”: [[{“value”: “turn”, “confidence”: 1.0, “rangeStart”: 0, “rangeEnd”: 4, “time”: null}, {“value”: “living”, “confidence”: 1.0, “rangeStart”: 5, “rangeEnd”: 11, “time”: null}, {“value”: “room”, “confidence”: 1.0, “rangeStart”: 12, “rangeEnd”: 16, “time”: null}, {“value”: “lamp”, “confidence”: 1.0, “rangeStart”: 17, “rangeEnd”: 21, “time”: null}, {“value”: “on”, “confidence”: 1.0, “rangeStart”: 22, “rangeEnd”: 24, “time”: null}]], “asrConfidence”: null, “rawInput”: “turn living room lamp on”, “wakewordId”: null, “lang”: null}

{
    "entities": [
        {
            "end": 7,
            "entity": "state",
            "raw_end": 7,
            "raw_start": 5,
            "raw_value": "on",
            "start": 5,
            "value": "on",
            "value_details": {
                "kind": "Unknown",
                "value": "on"
            }
        },
        {
            "end": 24,
            "entity": "name",
            "raw_end": 24,
            "raw_start": 8,
            "raw_value": "living room lamp",
            "start": 8,
            "value": "living room lamp",
            "value_details": {
                "kind": "Unknown",
                "value": "living room lamp"
            }
        }
    ],
    "intent": {
        "confidence": 1,
        "name": "ChangeLightState"
    },
    "raw_text": "Turn on living room lamp",
    "raw_tokens": [
        "Turn",
        "on",
        "living",
        "room",
        "lamp"
    ],
    "recognize_seconds": 0.16049231899978622,
    "slots": {
        "name": "living room lamp",
        "state": "on"
    },
    "speech_confidence": 1,
    "text": "turn on living room lamp",
    "tokens": [
        "turn",
        "on",
        "living",
        "room",
        "lamp"
    ],
    "wakeword_id": null
}
12

In the last post you have two slots: “name” and “state”
Those are available as properties on trigger.event.data.

So at template {{ trigger.event.data.name }} will become “living room lamp” and {{ trigger.event.data.state }} will become “on”.

You mention listening in HA to the events does not work, is that still the case?
Can you post your pofile? Be sure to format it for readablility.
You can find it under the “Advanced” option in the dropdown menu

You do not need to get to the HA forum, we can help you just fine if not better :wink:

13

Hey @romkabouter sorry for the delay, I got busy with work and other home projects.

Can you elaborate on your comment about slots? I had a grasp around this when I last messaged but its been a long 8 days and I’m drawing a blank on what to do with that information.

As for events, yes, that’s correct. I have not been able to confirm HA is receiving the events sent by Rhasspy.

Here is the profile you requested. I removed the password.

{
    "dialogue": {
        "system": "rhasspy"
    },
    "handle": {
        "system": "hass"
    },
    "home_assistant": {
        "handle_type": "event",
        "url": "http://homeassistant.local/"
    },
    "intent": {
        "system": "fsticuffs"
    },
    "microphone": {
        "system": "pyaudio"
    },
    "mqtt": {
        "enabled": "true",
        "host": "homeassistant.local",
        "password": "",
        "site_id": "Main",
        "username": "homeassistant"
    },
    "sounds": {
        "system": "aplay"
    },
    "speech_to_text": {
        "system": "pocketsphinx"
    },
    "text_to_speech": {
        "system": "espeak"
    },
    "wake": {
        "system": "porcupine"
    }
}
14

You can read more about slots here:

https://rhasspy.readthedocs.io/en/latest/training/#sentencesini

Slots are basically the important parts of your command, like the what and the where.

It might be best to change the URL to the IP adress:

    "home_assistant": {
        "access_token": "yourlongaccesstoken",
        "handle_type": "event",
        "url": "http://<ipadress_homeassistant>:8123"
    }

The http://homeassistant.local did not work in my network., also be sure to create and use a token.
While the Addon should use the token, without a token it was not working for me.

15

@romkabouter I changed my config to target the IP address:8123. I also added a new long token.

I navigate to the developer tools and listen for “Rhasspy_ChangeLightState” or “ChangeLightState” and never get a hit when running “Turn on living room lamp” from Rhasspy. To confirm, I am checking the Handle checkbox on Rhasspy when sending the command.

16

Possibly does not matter, but the event is rhasspy_ChangeLightState. Strange issue, I will post my profile here shortly

17

@romkabouter

I have no words…

I swear I tried multiple case options, including the above, and it didn’t work. Well now it does.

I can now toggle the automation/switch with Rhasspy using events.

However, it still isn’t interpreting the state “on” vs “off” and simply toggles the switch to the next state.

I speculate this is within my automation configuration.

If so, this is drifting in to HA territory.

alias: Rhasspy (Duplicate)
description: ''
trigger:
  - platform: event
    event_type: rhasspy_ChangeLightState
condition: []
action:
  - service: switch.toggle
    data: {}
    target:
      entity_id: switch.living_room_lamp
mode: single
18

Ohh, finally one I can answer :wink:

Instead of “switch.toggle” you want switch.turn_on or switch.turn_off.

Fortunately you can use the slot data in templates, as JeffC mentioned in post #10

  action:
  - service_template: switch.turn_{{state}}
    data:
      entity_id: switch.{{ name | replace(" ","_") }}

Well worth the effort to get your head around HA templating … though as usual the HA documentation is not easy for newbies :frowning:

19

Cool :slight_smile:

About the toggle: what @donburch already mentioned you must use turn_on or turn_off. Just using {{state}} will probably not work, it need the event

service_template: switch.turn_{{trigger.event.data.state}}

and for the entity_id: entity_id: switch.{{ trigger.event.data.name | replace(" ","_") }}

Checking the wiki for details might help

Apart from the fact you use a switch, the code you need is there :slight_smile: