Rhasspy demo with HassIO

Hi there,

I have created a small demo for people wanting to start with Rhasspy.
In this case, I am using Hassio and use the Rhasspy Addon, setting the language to english which is the default.

In the settings of Rhasspy, I have enabled MQTT and pointed the ip address to Hassio.
The siteID is set to matrixvoice, because I use my Matrix Voice Audio Streamer as a remote microphone. These settings are maybe not what other peolple need, but this demo is independent of it.

First, I have two persons in my Home Assistant, see screenshot below:

I have configured the following slots in Rhasspy:


Paul and Susan are person in Hassio, but Xander and Florence are not.

The sentences in Rhasspy:

$person is corresponding with the slot and is sent as and event to Hassio as slotname person

Now, I needed to setup Hassio to respond to the questions like: “Where is paul”.
When a specific name is asked for, the automation has to lookup the person and report back the state.
Also, when a person is not found or at an unknown state, this will be reported as well.

When the question is asked where everybody is, the automation loops over the persons in Hassio and reports their state.

Below screenshot is the automation, a payload is created which is posted to the rest_command.rhasspy_speak:

The rest_command.rhasspy_speak looks like the screenshot below and is in configuration.yaml:


The command posts the payload to the text-to-speech api of rhasspy.

In the video:
Paul = me, I am at home
Susan = my better half, no device tracker is attached, so the state is “unknown”
Xander: not existant person in Home Assistant

Another video link (dutch) with Rhasspy controlling the brightness and using default TTS of Home Assistant (but multiroom with mopidy and snapserver/snapclient
This showing that Rhasspy also supports Dutch

My MQTT streamer repo:

4 Likes

VERY much appreciated, dank je wel !!

Could you please embed your automation definition as yaml text, so it can be copied and updated on my own system??

Looks cool! To bad the Matrix voice has no speakers. I am also thinking of buying one but i am not that good with 3d modeling and it has no real cases yet.

Hi @romkabouter
Did you try snowboy? Have you any issues with it?

It has aan audio Jack and speaker connector. The streamer outputs audio over that.

yes and no. The question sounds like you have?

  • id: ‘1556640424576’
    alias: Locations
    trigger:
    • event_data: {}
      event_type: rhasspy_Locations
      platform: event
      condition: []
      action:
    • data_template:
      payload: >
      ‘{% if trigger.event.data.person == “everybody” %}
      {% for p in states.person %}
      {% if loop.last and not loop.first %}and {% endif %}
      {% if p.state == “unknown” %}
      I do not know where {{p.attributes.friendly_name.split()[0]}} is
      {% else %}
      {{p.attributes.friendly_name.split()[0]}} is{% if p.state == “home” %} at{% endif %} {{p.state}}
      {% endif %}
      {% if not loop.first %}, {% endif %}
      {% endfor %}
      {% else %}
      {% set response %}
      {% for p in states.person if p.attributes.friendly_name.split()[0] | lower == trigger.event.data.person | lower %}
      {% if p.state == “unknown” %}
      I do not know where {{p.attributes.friendly_name.split()[0]}} currently is
      {% else %}
      {{p.attributes.friendly_name.split()[0]}} is{% if p.state == “home” %} at{% endif %} {{p.state }}
      {% endif %}
      {% else %}
      I am sorry, but I can’t seem to find {{trigger.event.data.person}} in the system
      {% endfor %}
      {% endset %}
      {{ response }}
      {% endif %}’
      service: rest_command.rhasspy_speak
  • id: ‘1556640424575’
    alias: Rolluiken
    trigger:
    • event_data: {}
      event_type: rhasspy_Covers
      platform: event
      condition: []
      action:
    • data_template:
      payload: Dat is goed, ik doe het {{ trigger.event.data.whichcover
      }} rolluik {{ trigger.event.data.actiontype }}
      service: rest_command.rhasspy_speak
  • id: ‘1557060131078’
    alias: MatrixVoiceBrightnessChange
    trigger:
    • entity_id: input_number.matrix_brightness
      platform: state
      condition: []
      action:
    • alias: ‘’
      data_template:
      payload: ‘{“brightness”:{{ trigger.to_state.state | int }}}’
      retain: true
      topic: matrixvoice/everloop
      service: mqtt.publish
  • id: ‘1557080306626’
    alias: MatrixVoiceBrightnessSetValue
    trigger:
    • platform: mqtt
      topic: matrixvoice/everloop
      condition: []
      action:
    • alias: ‘’
      data_template:
      entity_id: input_number.matrix_brightness
      value: ‘{{ trigger.payload_json.brightness }}’
      service: input_number.set_value
  • id: ‘1570370359402’
    alias: Lampen
    trigger:
    • event_data: {}
      event_type: rhasspy_Lights
      platform: event
      condition: []
      action:
    • data_template:
      payload: Dat is goed, ik zet de lamp in de {{ trigger.event.data.location }} {{ trigger.event.data.actiontype
      }}
      service: rest_command.rhasspy_speak

Hmm, paste did not go to well, I am trying to edit on my phone.

Thanks for the reply!
I had an issue, but it was not caused by Snowboy, but by pyAudio
So it works pretty well. I’m testing Porcupine now - in the Docs on rhasspy site it’s marked as excellent =)

Yes, but I do not like the hotword. Works pretty good as well.
I want to create some custom hotwords for snowboy, but did not have time yet

I found Porcupine uses more CPU resources when waiting for a command, so I’m using snowboy again =)

FYI, there are some other pretrained words for both systems


https://github.com/Picovoice/porcupine/tree/master/resources/keyword_files/raspberrypi

Here a better paste of my automation in the demo

- id: '1556640424576'
  alias: Locations
  trigger:
  - event_data: {}
    event_type: rhasspy_Locations
    platform: event
  condition: []
  action:
  - data_template:
      payload: >
        '{% if trigger.event.data.person == "everybody" %}
           {% for p in states.person %}
             {% if loop.last and not loop.first %}and {% endif %}
               {% if p.state ==  "unknown" %}
               I do not know where {{p.attributes.friendly_name.split()[0]}} is
               {% else %}
               {{p.attributes.friendly_name.split()[0]}} is{% if p.state == "home" %} at{% endif %} {{p.state}}
               {% endif %}
             {% if not loop.first %}, {% endif %}
           {% endfor %}
         {% else %}
           {% set response %}
               {% for p in states.person if p.attributes.friendly_name.split()[0] | lower == trigger.event.data.person | lower  %}
                   {% if p.state ==  "unknown" %}
                   I do not know where {{p.attributes.friendly_name.split()[0]}} currently is
                   {% else %}
                   {{p.attributes.friendly_name.split()[0]}} is{% if p.state == "home" %} at{% endif %} {{p.state }}
                   {% endif %}
               {% else %}
                   I am sorry, but I can't seem to find {{trigger.event.data.person}} in the system
               {% endfor %}
           {% endset %}
           {{ response }}
         {% endif %}'
    service: rest_command.rhasspy_speak
1 Like

Many, many thanks!! :grinning:

I have now started implementing WakeNet.

This is the concept working

With this, I can do WakeWord detection on device so there is no constant audio stream over the network.
Hotwords are limited however, but still a nice feature

https://www.espressif.com/en/news/ESP-Skainet_Released

2 Likes

I didn’t know this was possible on the ESP32, cool!

Any plans for some case for the matrix?

Yes, I have ordered some 4ohm 3W speakers. After that I will try to create some casing for both speaker and device. Ultimately using at as a google/amazon type of device.

I am also working on using it as a BT speaker, but have issue with WiFi and BT together.

2 Likes

Let me know when you have some finished product. Interested in it, seems like a very nice “open” replacement for all those echo dots and home nest smart speakers. Haven’t seen much yet.

I will, but don’t expect anything soon :wink:

Wakeword on device!

Pretty slow and only Alexa is working at the moment, but this is because there is only Alexa or chinese models

2 Likes