Rhasspy, MQTT, node-red, proxmox, raspberry pi3, nuc - Part 1

Part 3

In node-red I have written a function that goes through all the devices it knows about and generates a number of slot lists based on the type of information and/or device.
It then publishes the list to the MQTT broker on the Rhasspy server (I use mosquitto) where the local node-red instance writes them out to the slots folder and subdirectories, so any change in the HA system will be passed on to Rhasspy. If any changes are detected, node-red then uses the API to retrain the system and then restarts the clients.

Training currently takes about 6 1/2 minutes.

Things to do:

  • HVAC commands
  • Replace pocketsphinx for Wake Word
  • Alarms and Timers
  • more, tbd :grinning:
4 Likes

That is a huge system! Good work

Impressive, thanks for sharing this and inspiring others!

Very cool :sunglasses:

Is this on 2.4?

Yes, I have been playing with 2.5 in a test config but have been having trouble with my Pi Zero and respeaker combo.

I have moved the wakeword to Snowboy to reduce false positives.
I also implemented MaryTTS locally on the Rhasspy server.

1 Like

its nice and great…
i am new to Rhasspy and want to ask you about can we use rhasspy for home automation using only one raspberry pi ?

thanks in advance…

Honestly it would depend on the size of you system and how complicated you want to make it.

I might not be the right person to ask as my Rhasspy system is very large and complex, and would never run on a Pi. The rest of my home automation is split on an Asus Tinkerboard for node-red, influxdb, grafana and Openhab (which is only used for its z-wave binding and nothing else) and a Pi for deConz for Zigbee, although I think the Tinkerboard would probably have enough power to run this as well.

If you are talking just Rhasspy on a single Raspberry Pi 4 as a master and then satellites in rooms it may be able to handle it.
It might also work combined with Home Assistant or the like on a single Pi if your system is fairly small.

In my case I have about 80-90 sensors, 131 lights/light strips/powerpoints/plugs etc, 6 shutters, 6 curtains, 9 ceiling fans to control. Plus I have sonos speakers and my entire list of albums, artists, playlists and favourites imported as slot lists, plus my previous grocery orders as a list of things I can add to a shopping list. So mine is certainly way too large for a Pi of any sort.

I’m pretty sure there are a fair number of people on this forum running this on Raspberry Pi’s though.

I am currently running rhasspy, FHEM and node-red on my Pi4 with 4gb RAM which runs great. It also runs MaryTTS and a mosquitto mqtt broker.
I would recommend using atleast a pi4 with 2GB of RAM. If you want to run other things besides rhasspy aswell.

thanks …
i have completed rhasspy setup with docker in pi 3B+ successfully,
can we get recognized text intent output of Rhasspy in pi terminal for controlling gpio pins ?

thanks in advance …

If you know some Python, that would be fairly straightforward. For GPIO access, use one of:

And then you can let your Python program react to your intents with one of:

Let me know if you need some guidance.

You can also use nodered if your not the biggest python fan :see_no_evil: There you can listen for the intents with mqtt and switch gpio too but with nearly no programming needed. only prerequisite is that you shouldn’t fully hate javascript :wink:

thanks @koan and @JGKK
First, I’ll try if i got stuck I’ll get to you @koan
and thanks again…

hey @koan, i tried its been long time but i would like to ask for your guidance in command intent handler on same topic since its long time,
i remind you topic is to control pi gpio with rhasspy intents ?

thanks in advance … :grinning:

Well, you first have to define the intent you want to have recognized. How does the intent look like?

i want to set sentences as intent ( like turn on lights , lights please ) and other stuff and by intent i mean STT output of rhasspy …
and based upon that ( speech to text output ) wanna control gpio…

can we see any code you produced on this hughe project? I mean I am totally interested who you solved the shopping list skill.

Hi guys, I as well am wondering how to control the GPIO pins with Rhasspy. Very novice user and would appreciate the guidance! I have an 8 relay board connected to RPi 4 and would like to be able to control each relay individually with simple voice commands, offline and hardwired with no additional server or mobile app required. Is this possible to have everything required locally on the RPi (32gb card) and hardwired via GPIOs to the relay board for use in a garage? Really hoping for a fully local, disconnected, hardwired solution! Cheers :slight_smile:

And btw, wow amazing project @jrb5665!:ok_hand: definitely the most involved and ambitious I’ve come across. Very excited about the potential here with Rhasspy!

As you would gather there is a lot of code :slight_smile:
For the shopping the real key is populating the slot list file for words it will accept then sentences includes the following

[Shopping]
add ($shopping/shopping) [and] [($shopping/shopping)] to [(my|the)] shopping [list]
(remove|delete|cancel):delete ($shopping/shopping) [and] [($shopping/shopping)] from [(my|the)] shopping [list]
clear [(my|the)] shopping [list]
(list|whats in|what is in):list [(my|the)] shopping [list]

to generate the list of words I do the following:

  1. Setup a rule in my email to send the confirmation emails from Coles to a folder called Coles.
  2. In node-red use the email node to watch that folder
  3. Split the emails into individual lines
  4. Use a switch node to throw away unwanted lines
  5. I feed that through a function node with the following code to create a list in the flow context. It looks for the start of the items based on the content “ORDER SUMMARY” and the end when it finds “Estimated Total”
    if (msg.payload.includes(")"))
    {
    return;
    }
    if (msg.payload.includes(“ORDER SUMMARY”))
    {
    flow.set(“done”, false);
    return;
    }
    if (msg.payload.includes(“Estimated total”))
    {
    flow.set(“done”, true);
    return;
    }
    if (flow.get(“done”) === true)
    {
    return;
    }
    var prevordered = flow.get(“previousOrders”) || [];
    if (prevordered.includes(msg.payload))
    {
    return;
    }
    prevordered.push(msg.payload);
    flow.set(“previousOrders”, prevordered);
    return msg;

the output is then sent through a filter node that waits for 5 seconds of no more messages and then triggers the list of words to be created and sent to a nodered instance on the Rhasspy server via mqtt.

The code to create the word list is:
var words = [];

flow.get(“previousOrders”).forEach(item =>
{
var newitem = item.replace(/[^a-zA-Z0-9 ]/g, “”).replace(/[A-Z0-9]/g, function(x){return " " + x}).replace(/ */g," “).trim().toLowerCase();
newitem.split(” ").forEach(word =>
{
[“kg”, “g”, “cm”, “m”, “s”, “mg”, “ml”, “l”, “pm”].forEach(unit =>
{
if (word.endsWith(unit) && !isNaN(word.replace(unit, “”)))
{
word = word.replace(unit, “”);
}
})
if (isNaN(word) && (word.length > 2) && !words.includes(word))
{
words.push(word);
}
});
});
flow.set(“shoppingWords”, words.sort());

return {topic: “slotlist/shopping/shopping”, payload: words.sort().join(";;")};

On the rhasspy server I have a node subscribing to mqqt messages slotlist/# which receives any lists and if the list has changed it saves it into the slots folder structure and uses the api to trigger a re-training. Every now and then I have to manually apply unidentified words but given I populated the list with a couple of years worth of orders initially, most of the things we buy are already there.

Then in main nodered instance if it sees the shoppinglist intent it processes accordingly.

Similar logic is also used for all my slot lists including the lists of all my home automation devices, music playlist, favourites, album and artist lists which are extracted from Sonos using socos and a bash script on my home automation machine.

I hope this helps. Please let me know if you need more specific details in any area

1 Like

Hi everybody,

I´m very new to Rhasspy. But it is a very interesting tool and I like it.

@jrb5665 I like what you have done with it.
Especially the things with Dark Sky. Can you help me to do the same?
And how is your progress with Alarms and Timers?

Unfortunaly I´m not very familar to programming. And need maybe a lot of Help.

Thank you in advance

Hi @Becks89 and welcome,

I’m happy to help where I can.
As far as the weather from Dark Sky, I will just preface in saying that since Dark Sky has been sold I have moved to the openweathermap (OWM) service for my data, although that doesn’t actually matter to the rest of the implementation as it is just data coming into the system

I contributed a message or two in the thread
https://community.rhasspy.org/t/rhasspy-can-tell-you-the-weather/671
which might be a good place for you to start as there a few ways to approach the same issue discussed there.

For the alarms and timers, I’ve been pretty busy with other things since I posted the original message so other than having node-red receive and identify the intents I haven;t progressed this any further as it will require me to rewrite my existing schedule handler in my node-red code and I haven’t had time to design what I want it to look like.

If you have specific questions I will be happy to share what I have but as you can probably gather it is quite large and complex and there is a fair bit if javascript incorporated into node-red functions.

You haven’t really given any details of what you already have in place, maybe if you share a bit about what you already have or plan to implement I can target my answers a but better