Rhasspy, MQTT, node-red, proxmox, raspberry pi3, nuc - Part 1

First I’d like to say how great Rhasspy is.
I have only been playing with it for a couple of weeks but it is the easiest install and configuration I could imagine. And the documentation is impressive.

So down to my system.

To start with I have a fairly mature HA system that comprises the following hardware:

  • main system is on an Asus Tinker Board with 2GB RAM. This runs node-red, openhab for Z-Wave devices, Influxdb and Grafana for charting and node-sonos-http-api and socos for sonos
  • A raspberry pi running deconz for Zigbee devices
  • A sonoff rfbridge433 with Tasmota for rf remote controls (using only as a receiver at the moment)
  • Broadlink rmpro mini for IR
  • Daikin Airbridge for HVAC control
  • Unifi Controller and nmap for presence detection - mobile phones only at present
  • Yeelight light strips
  • Xiaomi gateways - mainly replaced by deconz but still used for status lighting
  • Sonoff ifan02 for ceiling fans with Tasmota
  • A nuc with proxmox with a container for the Rhasspy server.
  • A raspberry pi as a Rhasspy satellite

Node-red is the heart of the system which made integration with Rhasspy a breeze through MQTT.

In the couple of weeks I’ve been playing with it I have so far managed to integrate:

  • 73 Lights, 72 power plugs/wall sockets, 11 fans, 21 cover devices (shutters, curtains and skylights)
  • Report on the status of 60 door and window sensors
  • Arm/disarm devices
  • Control groups of items, like saying “Turn on all lights in John’s Room” or set all lights to red.
  • Report who is home
  • Ask for the status of almost any device, like Is John’s light on (and get a yes no answer) or What is the status of John’s light, door, window etc
  • Ask what is the weather and get the Dark Sky summary for my area
  • Ask things like the time, date, temperature (from Dark Sky), the temperature/humidity in any room in the house.
  • Control my Sonos system, ie Play/Pause, volume control, etc, select tracks by Artist name, Playlist, Favourites and also ask questions such as the current artist name or track details
  • Add things to a shopping list and have them read back

to be continued…


Part 2

For Rhasspy server I am using a container configured on a 3rd Generation i3 Nuc.
I have allocated 10GB for memory which it sometimes needed during training, but with a bit of tweaking it doesn’t seem to need it any more.
It is setup to use MQTT, webrtcvad, kadi and fsticuffs. Everything else is diabled.
The container is running Ubuntu 19.04 and Rhasspy is installed using the .venv method.
It also has an instance of node-red running.

For the client I have only tested so far on a Raspberry Pi 3b+ with a Docker installation.
The microphone is a PS3eye.
This also has MQTT. It uses pocketsphinx for wake word and remote STT and Intent recognition back to the server.
The SiteId is set with the same name as the Room name as defined in node-red. This is also the prefix used in my system for all devices in that room.

The HA node-red instance is set up to subscribe to the MQTT intents and then handles the processing.
It sends the TTS through the Sonos speaker via node-sonos-http-api to the speaker in the room with the client. node-sonos-http-api has a say function that will pause playing music and resume after the response is sent.


Part 3

In node-red I have written a function that goes through all the devices it knows about and generates a number of slot lists based on the type of information and/or device.
It then publishes the list to the MQTT broker on the Rhasspy server (I use mosquitto) where the local node-red instance writes them out to the slots folder and subdirectories, so any change in the HA system will be passed on to Rhasspy. If any changes are detected, node-red then uses the API to retrain the system and then restarts the clients.

Training currently takes about 6 1/2 minutes.

Things to do:

  • HVAC commands
  • Replace pocketsphinx for Wake Word
  • Alarms and Timers
  • more, tbd :grinning:

That is a huge system! Good work

Impressive, thanks for sharing this and inspiring others!

Very cool :sunglasses:

Is this on 2.4?

Yes, I have been playing with 2.5 in a test config but have been having trouble with my Pi Zero and respeaker combo.

I have moved the wakeword to Snowboy to reduce false positives.
I also implemented MaryTTS locally on the Rhasspy server.

1 Like