Rhasspy, MQTT, node-red, proxmox, raspberry pi3, nuc - Part 1

Impressive, thanks for sharing this and inspiring others!

Very cool :sunglasses:

Is this on 2.4?

Yes, I have been playing with 2.5 in a test config but have been having trouble with my Pi Zero and respeaker combo.

I have moved the wakeword to Snowboy to reduce false positives.
I also implemented MaryTTS locally on the Rhasspy server.

1 Like

its nice and great…
i am new to Rhasspy and want to ask you about can we use rhasspy for home automation using only one raspberry pi ?

thanks in advance…

Honestly it would depend on the size of you system and how complicated you want to make it.

I might not be the right person to ask as my Rhasspy system is very large and complex, and would never run on a Pi. The rest of my home automation is split on an Asus Tinkerboard for node-red, influxdb, grafana and Openhab (which is only used for its z-wave binding and nothing else) and a Pi for deConz for Zigbee, although I think the Tinkerboard would probably have enough power to run this as well.

If you are talking just Rhasspy on a single Raspberry Pi 4 as a master and then satellites in rooms it may be able to handle it.
It might also work combined with Home Assistant or the like on a single Pi if your system is fairly small.

In my case I have about 80-90 sensors, 131 lights/light strips/powerpoints/plugs etc, 6 shutters, 6 curtains, 9 ceiling fans to control. Plus I have sonos speakers and my entire list of albums, artists, playlists and favourites imported as slot lists, plus my previous grocery orders as a list of things I can add to a shopping list. So mine is certainly way too large for a Pi of any sort.

I’m pretty sure there are a fair number of people on this forum running this on Raspberry Pi’s though.

I am currently running rhasspy, FHEM and node-red on my Pi4 with 4gb RAM which runs great. It also runs MaryTTS and a mosquitto mqtt broker.
I would recommend using atleast a pi4 with 2GB of RAM. If you want to run other things besides rhasspy aswell.

thanks …
i have completed rhasspy setup with docker in pi 3B+ successfully,
can we get recognized text intent output of Rhasspy in pi terminal for controlling gpio pins ?

thanks in advance …

If you know some Python, that would be fairly straightforward. For GPIO access, use one of:

And then you can let your Python program react to your intents with one of:

Let me know if you need some guidance.

You can also use nodered if your not the biggest python fan :see_no_evil: There you can listen for the intents with mqtt and switch gpio too but with nearly no programming needed. only prerequisite is that you shouldn’t fully hate javascript :wink:

thanks @koan and @JGKK
First, I’ll try if i got stuck I’ll get to you @koan
and thanks again…

hey @koan, i tried its been long time but i would like to ask for your guidance in command intent handler on same topic since its long time,
i remind you topic is to control pi gpio with rhasspy intents ?

thanks in advance … :grinning:

Well, you first have to define the intent you want to have recognized. How does the intent look like?

i want to set sentences as intent ( like turn on lights , lights please ) and other stuff and by intent i mean STT output of rhasspy …
and based upon that ( speech to text output ) wanna control gpio…

can we see any code you produced on this hughe project? I mean I am totally interested who you solved the shopping list skill.

Hi guys, I as well am wondering how to control the GPIO pins with Rhasspy. Very novice user and would appreciate the guidance! I have an 8 relay board connected to RPi 4 and would like to be able to control each relay individually with simple voice commands, offline and hardwired with no additional server or mobile app required. Is this possible to have everything required locally on the RPi (32gb card) and hardwired via GPIOs to the relay board for use in a garage? Really hoping for a fully local, disconnected, hardwired solution! Cheers :slight_smile:

And btw, wow amazing project @jrb5665!:ok_hand: definitely the most involved and ambitious I’ve come across. Very excited about the potential here with Rhasspy!

As you would gather there is a lot of code :slight_smile:
For the shopping the real key is populating the slot list file for words it will accept then sentences includes the following

[Shopping]
add ($shopping/shopping) [and] [($shopping/shopping)] to [(my|the)] shopping [list]
(remove|delete|cancel):delete ($shopping/shopping) [and] [($shopping/shopping)] from [(my|the)] shopping [list]
clear [(my|the)] shopping [list]
(list|whats in|what is in):list [(my|the)] shopping [list]

to generate the list of words I do the following:

  1. Setup a rule in my email to send the confirmation emails from Coles to a folder called Coles.
  2. In node-red use the email node to watch that folder
  3. Split the emails into individual lines
  4. Use a switch node to throw away unwanted lines
  5. I feed that through a function node with the following code to create a list in the flow context. It looks for the start of the items based on the content “ORDER SUMMARY” and the end when it finds “Estimated Total”
    if (msg.payload.includes(")"))
    {
    return;
    }
    if (msg.payload.includes(“ORDER SUMMARY”))
    {
    flow.set(“done”, false);
    return;
    }
    if (msg.payload.includes(“Estimated total”))
    {
    flow.set(“done”, true);
    return;
    }
    if (flow.get(“done”) === true)
    {
    return;
    }
    var prevordered = flow.get(“previousOrders”) || [];
    if (prevordered.includes(msg.payload))
    {
    return;
    }
    prevordered.push(msg.payload);
    flow.set(“previousOrders”, prevordered);
    return msg;

the output is then sent through a filter node that waits for 5 seconds of no more messages and then triggers the list of words to be created and sent to a nodered instance on the Rhasspy server via mqtt.

The code to create the word list is:
var words = [];

flow.get(“previousOrders”).forEach(item =>
{
var newitem = item.replace(/[^a-zA-Z0-9 ]/g, “”).replace(/[A-Z0-9]/g, function(x){return " " + x}).replace(/ */g," “).trim().toLowerCase();
newitem.split(” ").forEach(word =>
{
[“kg”, “g”, “cm”, “m”, “s”, “mg”, “ml”, “l”, “pm”].forEach(unit =>
{
if (word.endsWith(unit) && !isNaN(word.replace(unit, “”)))
{
word = word.replace(unit, “”);
}
})
if (isNaN(word) && (word.length > 2) && !words.includes(word))
{
words.push(word);
}
});
});
flow.set(“shoppingWords”, words.sort());

return {topic: “slotlist/shopping/shopping”, payload: words.sort().join(";;")};

On the rhasspy server I have a node subscribing to mqqt messages slotlist/# which receives any lists and if the list has changed it saves it into the slots folder structure and uses the api to trigger a re-training. Every now and then I have to manually apply unidentified words but given I populated the list with a couple of years worth of orders initially, most of the things we buy are already there.

Then in main nodered instance if it sees the shoppinglist intent it processes accordingly.

Similar logic is also used for all my slot lists including the lists of all my home automation devices, music playlist, favourites, album and artist lists which are extracted from Sonos using socos and a bash script on my home automation machine.

I hope this helps. Please let me know if you need more specific details in any area

1 Like

Hi everybody,

I´m very new to Rhasspy. But it is a very interesting tool and I like it.

@jrb5665 I like what you have done with it.
Especially the things with Dark Sky. Can you help me to do the same?
And how is your progress with Alarms and Timers?

Unfortunaly I´m not very familar to programming. And need maybe a lot of Help.

Thank you in advance

Hi @Becks89 and welcome,

I’m happy to help where I can.
As far as the weather from Dark Sky, I will just preface in saying that since Dark Sky has been sold I have moved to the openweathermap (OWM) service for my data, although that doesn’t actually matter to the rest of the implementation as it is just data coming into the system

I contributed a message or two in the thread
https://community.rhasspy.org/t/rhasspy-can-tell-you-the-weather/671
which might be a good place for you to start as there a few ways to approach the same issue discussed there.

For the alarms and timers, I’ve been pretty busy with other things since I posted the original message so other than having node-red receive and identify the intents I haven;t progressed this any further as it will require me to rewrite my existing schedule handler in my node-red code and I haven’t had time to design what I want it to look like.

If you have specific questions I will be happy to share what I have but as you can probably gather it is quite large and complex and there is a fair bit if javascript incorporated into node-red functions.

You haven’t really given any details of what you already have in place, maybe if you share a bit about what you already have or plan to implement I can target my answers a but better

@jrb5665 Thank you for your answer.
I will have a look in the thread you mentioned.

To my system:
I have a RaspberryPi with Rhasspy running in an Docker-Container, which should be my Voice Assistant.
A second one is the “Brain” of my smarthome. There is openhab running on it.
The two systems are communicating over MQTT with each other.
For example I can turn on some lights. Or I can ask after the temerature in my rooms.
My plan is to get Rhasspy working like a real Voice Assistant. For example set a alarm or a timer, or say how the weather is outside or at the next day.

Well this community is a great place to get ideas and solutions

I used to use openhab but moved to an almost entirely node-red based system a while back.
I still have openhab but only for the excellent Z-Wave binding that Mark Jackson built.

I found it to be resource hungry particularly with the combination of Java and the need to scale to the size my system had grown to. This left rules slow and unreliable, although I think the new rule engine there may have made a difference, but I already had replace that part with node-red anyway, so just continued to migrate everything else.

With what I had already done to standardise naming of everything in my system it made it fairly easy (requires coding) to just have node-red generate slot lists for Rhasspy to cover all my home automation.
I send them via MQTT to a node-red instance on the Rhasspy server and once there they are written out to files and retraining initiated with an API call.

The main issue I have struck there is that in generating the amount of entries I have in the slot lists and the number of sentences needed to cover a fairly natural way of controlling them all, it takes about 8 minutes to train my server on a proxmox container with 8GB RAM, 4 vCPU on an AMD 6 core/12 thread CPU. It also sometimes leads to a few more incorrect intent resolutions, like “Add Cranberries” could end up with additional tracks in the current queue on the Sonos or it could end up with an entry in the shopping list, although that specific case hasn’t happened.

I have toyed with the idea of maybe breaking this up a bit into a number of smaller instances and then using multiple wake words with it, maybe saying “Hey Sonos” to control the music, “Hey Shopping” to manage the shopping list, etc. I’d also like to add my Plex library of movie, tv shows and anime into voice control, but at the moment I’m sure that would just be too much.
So then it would be a bit more like the old sci-fi shows where the characters would just say “Lights” or “Music”, etc with no wake word and the computer would just know what they want. You would just speak to the “device” you want the action from.

This would also allow for the possibility of having some very specific controls, like the home automation along side something very free-form like grocery lists or adhoc questions, like “Hey Wiki, What’s the height of Mount Everest” or other wiki queries