Rhasspy, Respeaker and Openhabian/Home Assistant - Most easy setup?

own_rich · May 4, 2021, 3:06am

Hello, I hope you guys are feeling good. I am looking for some beginner advices on getting started with my home automation setup. I am interested in voice communicating with my house. I bought a raspberry pi 4 couples months ago and am trying to push my knowledge in different context and I need some help to get a good core setup for home automation specifically. I do not mind to have openhab or home assistant but I would like to be able to use my Respeaker and have also Rhasspy. I have been able to setup Openhabian and Rhasspy but I didn’t find much information on how to make both instance communicate together. I believe Rhasspy was more developed with Home assistant integration. I tried to install HA os and Rhasspy and it worked, although I can not use my Respeaker because I did not find how to install the drivers… I followed by trying to install raspberry os and HA with docker but ended without add on tab… Lastly, I follow some deprecated tutorial on installing supervised
on raspberry os or Debian 10 buster but never ended with the intended result.

Is there a way to achieve my goal and if yes, can you please give me the direction to follow. Thank you very much, hope you guys have a great one.

romkabouter · May 4, 2021, 8:27am

Welcome

Rhasspy is working great with Home Assistant
For Home Assisant, here is a comparison table:

On a raspberry Pi you must either:

Install Home Assistant OS, you then have addons but cannot install drivers for the Respeaker
Install Docker and Home Assistant Container. No addons, but you can install Rhasppy as a container as well and be able to install drivers for the Respeaker

own_rich · May 4, 2021, 11:32am

Thank you very much for your answer. I will try to install Docker HA and docker Rhasspy.
Will they communicate well and easy together just like a regular setup? Will I need to do some twisting to get it to work? Thank you and stay safe.

romkabouter · May 4, 2021, 12:35pm

You need to create a long live token in HA and set that in the intent handling. But that is a simple step

own_rich · May 4, 2021, 12:49pm

Thank you for the quick answer. Everything is working fine thank you, this was the solution. I have added my HA localhost ip address and port and also added a access token. My last question is where can I find information on how to program everything? Example: I have the porcupine wake word working with Respeaker and I ask “what is the time” and Rhasspy give a gettime intent. Can I return answers like : It is x o clock? Basically, it checks the time zone of HA and return the time? How will I program everything. There is probably documentation on that but a simple example would help me understand the general functioning. It looks like a very powerful setup. Privacy oriented and open source, you guys rock. Thanks

romkabouter · May 4, 2021, 12:53pm

I use events to send to Home Assistant.
These can be used in automations, the automations can be anything you want

If you check on my postst, you will probably find some examples whch I have posted here earlier

own_rich · May 4, 2021, 1:19pm

Perfect thank you. I will be looking in the documentation and your history to find more example. In automation in HA, I do not see where to call the service current time. Also to send the answer thru the speaker, you have to create an other event to send back a message to Rhasspy. Is that it?

romkabouter · May 4, 2021, 2:31pm

There is no service for current time in Home Assistant, but you can use a data_template or something like that.

No, you can publish a message to MQTT.

This is my lights event (in automations.yaml)

- id: '1581372525473'
  alias: EventLampen
  trigger:
  - event_data: {}
    event_type: rhasspy_Lights
    platform: event
  condition: []
  action:
  - data_template:
      entity_id: light.{{ trigger.event.data.location }}
    service_template: light.turn_{{ trigger.event.data.action }}
  - service: mqtt.publish
    data:
      topic: hermes/dialogueManager/endSession
      payload_template: '{"sessionId": "{{trigger.event.data._intent.sessionId}}",
        "text": "Okee, {{ trigger.event.data.location }} {% if trigger.event.data.action
        == "on" %}aan{% else %}uit{%  endif %}"}'
  mode: single

The automation is triggered by the rhasspy_Lights event and does two things:

turn the light on the location on or off (slots are location and action)
Send a response to Rhasspy via hermes/dialogueManager/endSession

The second action has a sessionID and a text, with which Rhasspy closes the session and does a speech-to-text on the text provided.

arpagor62970 · May 4, 2021, 6:39pm

The easiest method is to use MQTT
and write a script that will react according to the Intents
1 - create a broker with the IP of Rhasspy
2 - create a Bridge with the brocker
3 - create one or more channels
4 - create a link item string which will receive the topic

1

3
4

and finally a rules to process the topic received in the string

Arpagor

romkabouter · May 4, 2021, 8:35pm

Your last image is the same as the one before that.
And yes, your route is also good.

Depends on what your home automation system you want to use I guess.
So I suggest to try both and see what fits you

own_rich · May 6, 2021, 9:01pm

Thank you very much. I was able to setup my Z-wave stick and have a smart plug and I lamp.
I created an intent “BedroomLamp” in the sentences. I am able to see it on Node red. I have selected Send events to Home Assistant (/api/events) in intent handling with my localhost ip and port to HA. I have also created a Long-Lived Access Tokens and added it in the rhasspy config too. I then created an automation in HA that triggers on rhasspy_BedroomLamp to turn it on. The automation itself is working but I am not able to get the voice command to turn on the lamp even though I see the intent on Node red. Any idea?

romkabouter · May 6, 2021, 9:13pm

Can you post some logfiles from Rhasspy and your automation? There must be a mismatch somewhere.
What do you see in Node Red?

own_rich · May 6, 2021, 9:27pm

5/6/2021, 4:23:54 PM[node: d7f94fdd.9b5028](http://192.168.1.75:1880/#)msg : Object

object

intent: object

name: "BedroomLamp"

confidence: 1

entities: array[0]

slots: object

empty

text: "turn on the bedroom lamp"

raw_text: "turn on the bedroom lamp"

tokens: array[5]

0: "turn"

1: "on"

2: "the"

3: "bedroom"

4: "lamp"

raw_tokens: array[5]

0: "turn"

1: "on"

2: "the"

3: "bedroom"

4: "lamp"

wakeword_id: "default"

siteId: "default"

sessionId: "default-default-e43b3e7d-c6a2-491c-9922-f68fe3ed615e"

customData: "default"

wakewordId: "default"

lang: null

_session: object

type: "websocket"

id: "edb45b25.695228"

_msgid: "9ed2675c.b5c498"

this is on Node-red

id: '1620213779441'
alias: rhasspy_BedroomLamp
description: ''
trigger:
  - platform: event
    event_type: rhasspy_BedroomLamp
    context: {}
condition: []
action:
  - type: turn_on
    device_id: 1a1f7a2d0b43cfaad6845786ef34306a
    entity_id: switch.zwave_plug_2_channel_plugin_outlet
    domain: switch
mode: single

HA

romkabouter · May 7, 2021, 10:43am

I think you send Intents, not Events. Can you post a screenshot of your Rhasspy setting?

own_rich · May 7, 2021, 5:55pm

Thanks for the help.

romkabouter · May 7, 2021, 7:49pm

Looks ok, but I still think there is a connection problem

You can check if an event is fired on the developer tab in Home Assistant:

If you press “START LISTENING”, Home Assistant starts listening. Then try to turn on the lamp and the event should be listed below.
If it is, something might be wrong with the automation, if it is not something is wrong in Rhasspy or in the connection

Also, create a MQTT in in Node Red with topic hermes/intent/# and let the node flow to a debug output.
Check what’s logged.
The make things easier, you can enter a text in Rhasspy, check “handle” and press “Recognize”"

own_rich · May 8, 2021, 1:46am

Perfect, I did exactly as you mentioned in the first screenshot. When I fire the event from HA, the lamp opens. It is when I use rhasspy and respeaker that I can not get it to work it seems. I do not know exactly how to create MQTT debug message. Should I import special json from hamburger button like we do to catch intent from rhasspy tutorial?

Thank you for your time.

romkabouter · May 8, 2021, 6:49am

No, just do this in a Node Red flow:

Connect it to the broker your Rhasspy is connected to.

You said fire the event from HA, do you mean from Rhasspy or HA? My second screenshot is Rhasspy, you should try that to check if events are send from Rhasspy

own_rich · May 8, 2021, 11:00am

Oh sorry. I though you meant to fire the event directly from the developer tab in HA. This made the lamp open. Although, when I listen to rhasspy_BedroomLamp and use Rhasspy to try to launch that even, it doesn’t work. You say to connect it to a MQTT broker, which instance/port my MQTT broker exactly? I do not know much about all of these tools. Most are things I’ve never used. Sorry x)

I think the issue is the connection between Rhasspy and HA or the fact that HA doesn’t receive the event trigger from Rhasspy.

romkabouter · May 8, 2021, 2:25pm

You used Node Red right? That is wat you have said earlier
That is why I thought you knew how to use it.

Let’s take it a step back. How did you setup MQTT in Rhasspy?
If internal, you must use the ip of Rhasspy and the internal port (12183 per default)
If external, you the ipaddress and port of that broker.

If you have the addons tab in Home Assistant, I suggest installing the MQTT addon and use that as broker

own_rich · May 9, 2021, 12:58am

Ok perfect. Yes MQTT is setup internal in Rhasspy. Should it looks like this? It says disconnected so it has probably an issue. my setup is a raspberry pi with everything docker installed. No add-on tab in HA.

romkabouter · May 9, 2021, 7:39am

Yes, I see it is disconnected indeed
I am not sure about the port, please try 1883 as well

Can post a screen shot what is coming out of the debug from the websocket? The text you pasted earlier was a bit hard to read. I can compare that with my out and that should be the same

own_rich · May 10, 2021, 9:59am

Debug websocket image

Port 1883 test, not working I believe

romkabouter · May 10, 2021, 10:43am

OK, then you port was correct and it is a bit strange there is no message.

I will check the websocket messages on my end

own_rich · May 10, 2021, 5:04pm

Thank you for the technical help. Much appreciated. Rhasspy seems like a very great and powerful tool for home automation. I hope we can find a solution together. Have great one

romkabouter · May 10, 2021, 7:35pm

Since your ws is connected to /api/events, I think it should send events.
When I switched to intents, no response was visible on the ws endpoint (it also says so in the settings)

What is on the connection tab of the serversettings for the MQTT node?
In your picture, the Server is just a name. Might be depending on the nodered version.

own_rich · May 11, 2021, 3:05pm

Here is the connection tab

I had to install a zwave to mqtt socket to be able to use my Zwave usb key. It might be the reason the connection isn’t working. I am not sure so I will just give you the detail so you have more information.

romkabouter · May 11, 2021, 3:21pm

Your port should be the default internal port 12183, is it still disconnected after that?

own_rich · May 11, 2021, 5:07pm

No… not connecting on neither port

romkabouter · May 11, 2021, 6:16pm

hmmm, that is strange.
Is there anything in the logs files? Maybe a mosquitto is already running or something.

own_rich · May 12, 2021, 1:01am

Where is the logs file? Do you mean on a specific instance (Rhasspy, node red) or logs from the raspberry? I do not know exactly what mosquitto is. I haven’t downloaded any MQTT I believe. Is it done automatically with rhasspy ?

romkabouter · May 12, 2021, 7:28am

Soryr, yes the Rhasspy instance: docker logs rhasspy I assume

own_rich · May 13, 2021, 11:47pm

Thank you for keeping up with me. Here is what I have in logs:

romkabouter · May 14, 2021, 6:38am

Ah ok, that looks like an error

Can you post your profile.json? You can find it under advanced:

I scrollt back a little and I see in your home assistant url no http:// in front of the ip address.
Together with the invalid url message, this might be the problem.

Your URL should be http://192.168.1.75:8123, if that is the IP address of you Home Assistant instance.

own_rich · May 15, 2021, 7:24pm

The issue was that the http:// was missing in my Rhasspy, HA config like you mentionned. Thank you very much. I will be able to explore and start configuring my setup. I’m very hyped. Hope you have a great day and thank you for putting time with me. Since I have you, I will go ahead and ask a last easy question. I was interested by Rhasspy and HA because of the decentralized and privacy preserving ideology. I just want to use my home automation setup without worrying about data leaks or being listened too. Is it possible and a better practice to use my setup completely offline? I don’t want to overkill but want the best privacy preserving and decentralized setup possible. Thank you again

romkabouter · May 15, 2021, 8:48pm

Possible yes, with Rhasspy and Home Assistant everything can be offline.
Best practice? That is a matter of opinion and personal preference

own_rich · May 22, 2021, 11:02am

Ok perfect thanks, if it is connected to internet, is there any operating security risks?
Also, let say I want to add a TTS answer after my open bedroom light command, how would I proceed?
I have used the tutorial with node red to setup the TTS tab. It is working. I would like to add specific TTS command to specific intent. Can you please give me the small step to do this? Thank you again sir.

romkabouter · May 22, 2021, 3:01pm

There are a number of methods, but I have 1 that I use.

I have set Rhasspy to send events (not intents). That option is available under the Intent Handling setting:

Whenever you make a command, Rhasspy sends events in with the name rhasspy_<Intentname>
So if you have a [Lights] intent in your sentences, an event is fired with the name rhasspy_Lights.

Create an automation which is fired an that event. See my example here:

romkabouter · May 22, 2021, 3:03pm

What do you exactly mean with “it” in this sentence?

own_rich · May 23, 2021, 11:26am

Wait, we just setup up the intents handling to make my automation get trigged and if I want to add an answer I have to change my whole setup??

romkabouter · May 23, 2021, 8:16pm

You already have an automation? I think I missed that.
What does your automation look like?

romkabouter · May 23, 2021, 9:20pm

I think this topic was a bit long.
I gave the automation already somewhere at the start, including an answer send back.

So now I am confused about the above quote.
What is it that you want to achieve?

own_rich · May 24, 2021, 2:09am

Basically, now that I am able to send events and trigger an automation with the wake word. I would like to receive a vocal answer when the automation as been executed. How would I proceed exactly? Example : Wake word Turn on bedroom light - Bedroomlight intent send rhasspy_Bedroomlight event in HA. At this point, I’d like to receive an answer from the speakers. “Bedroom light has been opened” How can I proceed exactly? Thanks

romkabouter · May 24, 2021, 6:26am

My example automation does exactly that.

romkabouter:

- id: '1581372525473'
  alias: EventLampen
  trigger:
  - event_data: {}
    event_type: rhasspy_Lights
    platform: event
  condition: []
  action:
  - data_template:
      entity_id: light.{{ trigger.event.data.location }}
    service_template: light.turn_{{ trigger.event.data.action }}
  - service: mqtt.publish
    data:
      topic: hermes/dialogueManager/endSession
      payload_template: '{"sessionId": "{{trigger.event.data._intent.sessionId}}",
        "text": "Ok, {{ trigger.event.data.location }} {{trigger.event.data.action}}"}'
  mode: single

The text part in the payload_template does that.

You do not have to use Bedroomlight as intent by the way. You can use Lights.
The service_template knows by the slot location what light you want and by the slot action if it needs to be on or off

So you need sentences with slots and create those slots.
Check here for examples

own_rich · May 25, 2021, 11:23am

Ok perfect thank you. It means I should put this specific config in my automation HA tab right?

I do not find mqtt.publish in the tab. Is it normal?

romkabouter · May 25, 2021, 12:50pm

No, it should be there. Is MQTT setup in your HA? (I assume yes)
You can always edit the automation in yaml (either via the three dots or open up automations.yaml)

own_rich · May 25, 2021, 1:04pm

No. I haven’t setup MQTT in HA I believe. This probably means I should go in integration and add MQTT. Also, I have the docker raspberri pi setup.
I did go there but can’t seem to connect to it. take a look:

I tried different ports, with or without the last url character. I might have missed a step somewhere. It is a very extensive setup and I am not a professional. Just trying to learn the DIY. Thanks for the help. I appreciate the time you put in helping people.

romkabouter · May 25, 2021, 1:27pm

Your broker should just be your ip address, your first port is ok (12183)

own_rich · May 25, 2021, 1:31pm

Ok thanks. Do I have to download mosquito or something?

romkabouter · May 25, 2021, 1:48pm

No, Rhasspy is providing that in your setup (ip address + internal rhasspy port)

It is not actually the most ideal setup my opinion
I would

Install moquitto
Use that mosquitto as external mqtt in Rhasspy
Setup the MQTT integration in HA with that mosquitto

own_rich · May 25, 2021, 1:51pm

It doesn’t seem to work…

own_rich · May 25, 2021, 1:56pm

Ok perfect. I will try this. Thank you will update.

romkabouter · May 25, 2021, 1:56pm

Is Rhasspy running? If not, then that is the issue

If it is, maybe it is a good idea to install mosquitto like I suggest

own_rich · May 25, 2021, 9:04pm

Yes. Everything was open and functional but couldn’t connect to the internal rhasspy MQTT. I did like you suggested and it works. I see the MQTT publish. I am not sure what to add or how to configure to receive an answer even with your provided example. Should I copy the entire payload_template line just like yours ?

Should I download and install https://github.com/rhasspy/rhasspy-dialogue-hermes for this to work?

romkabouter · May 25, 2021, 10:09pm

Yes, but I suggest using a text editor on automations.yaml to verify if it is correct.

No, you already got that if you are using docker Rhasspy

own_rich · May 25, 2021, 10:12pm

Perfect, aftter you published MQTT in hermes/dialogueManager/endSession, Rhasspy will automatically output the text thru the configured speaker. Is that right? Thanks

romkabouter · May 26, 2021, 6:44am

To the siteId which made the reqeust.

Also, my example assumes you have location and action as slots, with location being the entity_id and the action being on or off.
I do not know your slots and sentences, so be ware of this.

I see in your screenshot you have 1 specific device, but with my example it is more generic.

own_rich · May 26, 2021, 11:11am

Ok thank you. Can you please just show me an example to make my specific example working?
I just want to have an answer to this specific command. “Ok done”. After that, I will probably be able to find a better framework which will allow me to open, close and chose different locations in one big automation instance. If I can get this one to work, I think I will be able to expand on that.

romkabouter · May 26, 2021, 12:58pm

Just put “Ok done” in the payload field of the mqtt.publish. Payload Template is not needed

own_rich · May 27, 2021, 3:35pm

Ok thanks. I had tried that but it doesn’t seem to work. I do not receive any audio output from the speaker.

romkabouter · May 27, 2021, 6:59pm

Does the wakeword feedback sound work?
If it does not, the problem lies with there.
What are your Rhasspy settings and can you post Rhasspy logging?

Please recap on the hardware you currently use in your setup.

own_rich · May 28, 2021, 4:02pm

Yes. The wakeword feedback sound work. I know when it start and stop. I just don’t receive the payload answer. Here is the log

Thank you have a great day.

romkabouter · May 28, 2021, 5:00pm

The last line shows an error. Your payload is incorrect. Please post screenshot or the yaml

own_rich · May 28, 2021, 6:10pm

Like this?

romkabouter · May 28, 2021, 6:44pm

It should be json like this '{"text":"Ok, done"}'

own_rich · May 28, 2021, 6:55pm

Ok perfect. I did exactly as you mention and still doesn’t receive the text audio.
Here is the log after the updated payload :

romkabouter · May 28, 2021, 8:42pm

Just look at the error. You also need the aessionid, see my example.
Ik think you understood me incorrectly when I said Just put ok, done.

Paste met example in your payloadtemplate, but replace the text part.

romkabouter · May 31, 2021, 8:46am

Like this, assuming you have the automation triggered on event

romkabouter:

    service: mqtt.publish
    data:
      topic: hermes/dialogueManager/endSession
      payload_template: '{"sessionId": "{{trigger.event.data._intent.sessionId}}",
        "text": "Ok, done"}'