Nice work, but in all honesty I think the people you mention are not going to set this up either.
I hope you can prove me wrong though, because it might help a lot of people struggling with the HA integration 
1 Like
I’m hoping once I get the base of it working enough and I have satellite support in (which is more about creating an easy plug 'n play solution), I can make it installable as a hass.io addon. So folks can just install it there, and then follow instructions on getting a satellite setup on a pi zero w or an esp32 and they’ll be good to go.
I was also thinking about making it distributable as an OS that you can just flash to a pi. But that has a lot more complications, like how do you create an OS image and keep it up to date. Maybe that’ll come even further down the line!
1 Like
Having it as an addon is good idea! Keep us posted
1 Like
Well, it’s Christmas !
I think there is different profiles using Rhasspy :
- the really beginner : he has various tutorials to play with Rhasspy OR with HA
- the experimented makers : the doc of HA and Rhasspy are quite exhaustive to make interactions
- the intermediate… like me : it is hard and your job would fill this lack ! Tuto aren’t enough explicits to achieve a fully functional installation.
I’d would like to set up my own project (something you don’t find in “all in one” solution), so your help should help me to achieve my goal.
After a first glimpse at your home page, I have some feedback :
- the only wake word available is “Jarvis” ? Should i train my own (raven) ?
- A big thanks for the integration of respeaker HAT !
- Another challenge… Easy bluetooth connection for speakers ?
- Perhaps, you should add a reminder of docker configuration needed for HA (supervised I suppose), and Rhasspy ?
Great job !
Glad you like it - Thanks for the feedback!
To address some of the concerns:
- Currently only Jarvis is available. It uses Porcupine since it works the best OOTB. In theory you can override the rhasspy config and use something else instead, but it would require a bit more setup. This actually made me think about having the
rhasspy_profile.jsonmerge with the default settings instead of straight override. That way people can customize a section, but still get the other defaults. I’ll add it to the backlog! - I think a lot of folks use the ReSpeaker hat, so I added some directions as to how it could work! I should probably get one and try it out at some point.
- For bluetooth speakers, you’ll probably need to connect it to the main pi/computer running Home Intent and then set the sounds device settings correctly. Eventually I want users to be able to select it interactively via a UI, but that’ll come a bit later.
- And yes, I should probably update the installation page with how exactly installation is supported right now and that it require some docker knowledge to get it going (for now at least!)
Thanks Again!
After reading that you are dealing with respeaker hats without having one on your own, I took a look at it. First off, the rhasspy tutorial on that is not the best place to send ppl because right now it has you get the drivers from the official github with is almost never up to date because the devs do next to nothing to update for newer kernels. There is a guy that tries to keep everything current here that the devs merge into their repro from time to time which is more current. For now he supports any kernel from 5.9 to 5.11 with the v5.9 branch, and it seems 5.12 broke something again so there is a separate branch for that. Any problems, even with that branch, can be directed to the official github since hintak is the one providing almost all active support there anyway. Right now the official repro seems to be up to date, but I am pretty sure that it will not be after a few kernel updates.
I would advise linking both the official and the hintak repro and advising ppl to use the official repro as long as they use RaspiOS and the pi ppl haven’t gotten to kernel 5.12. Running debian on the pi or any other distro, I would advise HinTak. I haven’t tested 64bit compatability with either repro, but I know HinTak actually uses 64bit debian so hopefully it works on RaspiOS, too.
The next thing is, using those drivers you should have support for the 2-mic hat, the 4-mic square hat, the 4-mic linear hat and the 6-mic hat and it seems they added a new 4-mic square hat from the pictures on the readme. This also means that setting the mic to plughw:CARD=seeed2micvoicec,DEV=0 is a bad idea, since that only works for the 2-mic hat. You should instead advice to set it to the device that has seeed in the name because all of them should have that in common.
Also put a warning somewhere that every kernel update can break respeaker drivers, either because the kernel got to new or because you need to switch the repro fork to a newer version for any mic that isn’t the 2-mic hat. For best user experience I would suggest you ask users to exclude kernel updates in their package manager once they have the mic working if they don’t know how to or don’t want to manually check if the newer kernel version is supported yet. Actually, suggest they do so anyway and only undo this if they want to upgrade the kernel and have either checked if their branch is still compatible or they uninstalled it before the upgrade to install the new branch. Otherwise you will get quite a few linux beginners that did a system update and suddenly the mic broke.
The next thing I would recommend is adding a small tutorial on how to use alsamixer to adjust the mic gain and level and how to save that so the driver won’t override after reboot. The default settings for this are pretty bad in most cases and make using the mic for rhasspy hard. Gerneral advice is turn the volume up to something between 70 and 87 and the gain to 50-80, depending on how far away you plan to use the mic from. They unhelpfully removed this and the command for saving the config from their readme a while back, but I don’t see any commit that fixes this or saves automatically. The command to save is sudo alsactl --file=/etc/voicecard/ac108_asound.state store for the 4-mic square and line as well as the 6-mic, the 2-mic differs and uses sudo alsactl --file=/etc/voicecard/wm8960_asound.state store. Which file to save in depends on the chip used on the mic array, so if they ever add something with a different chip, this is where one needs to look.
Respeaker mics normally come with inbuilt leds, so it would be a good thing to either integrate hermes-led-control into your project or at least linking to it so ppl with respeaker hats can use their leds also.
And with this writeup I think you understand why the general advice in this forum is don’t get a respeaker that isn’t a 2-mic, because you get less work and at least the same quality with literally everything else. I personally still use respeaker 4-mic-square (or more correct, a clone of it) but I had quite a few headaches with it over the 2 or so years I used this. They do their job and the inbuild leds are nice to have but the official support is less than optimal and if you can get anything that is supported by the kernel without extra drivers you are far better off.
3 Likes
Cool! I think fans of ha might persist with this if satellites were easy to add, as it then can become part of a whole house automation.
The android client means being able to use old mobiles and tablets in multiple locations without going into setting up satellite hardware etc. An interface that adds clients would simplify this and make it more attractive to ha users as an alternative or supplement to big voice assistants.
Just means having a way of setting additional port numbers in the docker config and site IDs for each client and the relevant port number.
Oh wow, that is a lot more work than I figured! Thanks for the writeup. For now I might just put a warning on the docs about some of the issues you’ve outlined and link to your post. I may end up revisiting adding full support for the ReSpeaker later in the future, but it seems like quite a bit of work to get it going.
Again, thanks for calling this out!
Thanks Daenara, it mostly echoes my findings and adds some missing pieces for me.
I understand that HA and Rhasspy are trying to be all things to all people, supporting any combination of hardware and choice of modules at each stage of the voice processing. Great job … but means there is no one tutorial, not even one clear path for beginners to follow. Add that FOSS documentation tends to be technical, and it is left to out-of-date YouTube videos and blog posts to provide the user level. Too many things a beginner has to get their head around, in order to get something running.
I have got a satellite RasPi3 with reSpeaker 4-mic HAT and Rhasspy working with a base RasPi4 with Home Assistant OS and Rhasspy add-on. It’s not hard - once you know what you’re doing. Adding HermesLedControl and MQTT broke it, so try again.
Currently wanting to add another satellite which will be RasPi Zero W with reSpeaker 2-mic HAT, Rhasspy and HermesLedControl. This seems like such an obvious combination that I am surprised there isn’t already a beginners level tutorial; or it been packaged as a commercial product (like the RasPi & KODI media centres). … so I am documenting my build step by step.
I am already using HinTak’s drivers, but will have to add your warning about newer kernel versions (and research how to prevent them). I’ve used alsamixer in testing, but need to save and restore settings. Adding HermesLedControl seems to want shared MQTT - another thing to learn 
I am working on that, time is however limited.
Currently i try to get an rhasspy satellite to work with the hass.io rhasspy and mqtt addon.
I found out, installing hermesledcontrol using the install script also installs the respeaker driver, mosquitto and some more, so you dont have to do it yourself.
But i do not get the leds to light at all (wrong mqtt settings?)
I had an working satellite which stopped audio output after plugging the audio jack out and in again while running. I dont get it to say anything after that.
I tried to update the OS and rhasspy docker to 2.5.11
After that i had sound again but with a very bad high noise.
Then my SD card got in an write only state without informing me (looked like i could write/flash/… but do not really change anything).
I made a new card with raspios lite, copied ~/.config from the old one, installed hlc with driver/mosquitto. and run
$ docker run -d -p 12101:12101 \
--name rhasspy \
--restart unless-stopped \
-v "$HOME/.config/rhasspy/profiles:/profiles" \
-v "/etc/localtime:/etc/localtime:ro" \
--device /dev/snd:/dev/snd \
rhasspy/rhasspy \
--user-profiles /profiles \
--profile de
Same terrible noise.
After changing it to
$ docker run -d -p 12101:12101 -p 12183:12183 \
--name rhasspy \
--restart unless-stopped \
-v "$HOME/.config/rhasspy/profiles:/profiles" \
-v "/etc/localtime:/etc/localtime:ro" \
--device /dev/snd:/dev/snd \
rhasspy/rhasspy \
--user-profiles /profiles \
--profile de
surprisingly the noise disappeared.
But it seems something is missing for mqtt. I do not want tu use the internal rhasspy mqtt but the one from homeassistant. So i should not need port 12180 but the one from profile.json should be used.
No idea why its not working.
found the problem: https://github.com/project-alice-assistant/HermesLedControl/issues/112
import logging had to be added to the top of /home/pi/hermesLedControl_v2.0.15/models/LedPattern.py
So, after a couple of months, Home Intent now has the ability to work with an external rhasspy instance, and supports the ability to be translated!
If you want to help out with translations, feel free to put in an issue on GitHub. We have some notes in the docs on how to translate as well!
Do you mean multiple Rhasspy Satellites ? Now that I have 3 Rhasspy Satellites, and a Home Assistant OS/Rhasspy add-on as Base … I can’t imagine much point to a home voice assistant that only works in one room.
I think I have a similar configuration, Home Assistant OS with MQTT and Rhasspy add-ons - though I have not needed to do anything with Docker. I have setup 3 Rhasspy Satellites using RasPi OS Lite, Rhasspy’s Debian install (no point adding Docker on a dedicated Satellite) with External MQTT and HermesLEDcontrol.
There was a couple of undocumented additional steps required which I found in various online sources; and even more work-arounds to get HLC working. Since I already had Rhasspy configured and working I passed on letting HLC reinstall anything.
You may be missing the MQTT Port number. HLC defaults to using the MQTT details from Rhasspy’s profile.json, and Rhasspy’s web page shows the correct MQTT port number - but for some reason it is only written to the profile file if changed. And HLC’s author didn’t know to look through Rhasspy’s program code to find where the default MQTT Port is determined to be 12183 or 1883.
BTW, since the 2021-10-30-raspios-bullseye release, there is another workaround needed for Rhasspy, and several more for HLC. I think it may be time to make a Rhasspy Satellite IMG file using the common RasPi Zero W / reSpeaker 2-mic HAT / Rhasspy pre-configured with External MQTT and HLC. After all, once a satellite is running, there’s not much point updating it until a major improvement is available.
So, the external Rhasspy setting lets Home Intent to be used as an intent handler and to setup slots/sentences. The rest of Rhasspy would be managed by the user, so if they have satellites up and running, then that should continue working (there have been a couple of bug reports with satellites, but I’m working on it).
Apart from that, I am working on getting satellites natively supported in the managed Rhasspy instance within Home Intent. So users can just install Home Intent and then setup satellites and connect them.
And the long term is to get my Rhasspy Satellite integrated. I haven’t had enough time to work on it as of late, but I had made some plans around it.
My objective was to fast-track new users to a working, extensible HA voice assistant without all the experimenting, dead ends, lessons and learning curves I went through. More specifically:
- Really basic tutorial with lots of screen shots to make it foolproof for a beginner
- provide some background to the different technologies used
- limit it to a satellite comprising Raspberry Pi (Zero, 3A, 3B or Zero 2) with reSpeaker HAT (not the best hardware, but seems most common)
- build a full working satellite comprising Rhasspy with External MQTT, and HermesLEDcontrol
However this then also requires configuration at the base station (which has not been so strictly defined):
- install of base station HA MQTT, Rhasspy add-on configuration is required
- discuss setting up sentences (in Base Rhasspy) and intents in Home Assistant
- I am now personally using (and much prefer) node-RED instead of HA intents, scripts and automations- but that’s probably a whole new tutorial.
When I got to 30 pages I realised this wasn’t going to fit into a forum post … and maybe I should take a break and re-evaluate.
Well it’s been a few months, and time to come back to this. I’m tossing up whether to
a. publish it as a document in .docx or PDF format (best formtting, but not flexible viewing),
b. publish as a post on my personal blog,
c. producing pre-configured base and satellite .img files would make the text shorter, but require them to be regularly updated.
d. Split into 2 documents - with sentences and intents, distinct from hardware/software installation
e. or wondering if there’s another better option.
This looks great. I’ve tried using Rhaaspy a few times and always give up because I can’t ever import all of my lights/switches/covers/rooms properly due to name formatting for one reason or another, and I’ve spent hours trying to figure it out with no success. I don’t have the time nor desire to manually enter all of my entities. How do you get around issues like this?
A big thing that would keep me from using this is the lack of custom wakewords.
This also concerned me:
…but the interface is in english, why can’t I set it to “en”?
Since Home Intent integrates with Home Assistant, I just pull all the entities using the API and configure them for the pre-generated sentences.
The language defaults to english, which is why you don’t see en mentioned there. I’ll update the docs indicating the default is english. Sorry for the confusion.
This seems amazing! I just finished installing rhasspy and I love it but there is close to none examples or documentation in how to make it work with home assistant efficiently. Right know I have Rhasspy in an individual Pi and connect it to home assistant with the MQTT. Can I update that rhasspy with home intent and it will continue to work with my separate instance of HA or does it has to be in the same build?
Thank you and congratulations for such an amazing project! I am extremely hoping in can grow bigger!