Error and microphone problems

Hi All,

I have two problems which drive me nuts. I have the rhasspy add-on installed on my rpi 4 in home assistant. I have a pi zero 2W which has rhasspy installed on it. I have (currently) one anker power conf microphone and speaker combo. I can attach it through USB to both devices. Both devices see the speaker/microphone combo because I can select it from the drop-down menu. But there is no response on the micro/speaker combo. I don’t understand why it doesn’t pick up anything nor plays anything while I can select the device.

My second problem is the “error” and “timeout error” I get very often. As far as I understand it should be possible on my pi zero 2w but also on my pi 4.

I would love to get this working especially now synthesiam will join nabu casa. Any solutions or suggestions? Everything will be appreciated!

Hi Popi,

The Rhasspy add-on which is installed on your Home Assistant RasPi4 … was it a recent install of Rhasspy Junior (Current version: 0.2.2); or Rhasspy Assistant (Current version: 2.5.11) ?

You say that your Anker microphone connects via USB, and is detected and displayed in the Audio Input and Output drop-downs, which is good,

Next step depends very much on whether you are running the Junior or full version of Rhasspy. No point confusing you with the wrong instructions, so let us know which.

Where are you seeing these errors ? What are the actual error messages ? What should be possible ?

Hi,

Thanks for the quick reply. I have the full version of the rhasspy add on version 2.5.11.

The errors don’t give (me at least) much clues. It’s just a red bar beneath which states error or Timeout error.

I tried to change the language (Dutch to English) as well to see if there is a problem with the Dutch language. But other then that Kaldi does function in English the same errors occurs.

See screenshot. The word “hello” is not spoken.

Gave up on the pi zero 2w. Maybe should go for a pi 4 2gb or something similar. Pi3 is not available here.

OK; full Rhasspy 2.5.11.

I have a RasPi Zero W (not even the model 2) and it works OK as a satellite (though noticeably slower to respond than my RasPi3A+ satellite) - but I would not even try to run the more CPU-intensive modules on it.

I see you have 2 equally viable options:

  1. Since Rhasspy on your RasPi 4 detects the USB audio device, you can use the RasPi 4 for HA and Rhasspy add-on, and your audio device - all on the one machine. Simpler for testing, but limited to using voice commands in the room where the RasPi 4 is located.

  2. If you want multi-room voice control you could also go the base + satellites route now since you already have two RasPis. It uses your RasPi4 with HA and the Rhasspy add-on (called the base) to provide the more CPU-intensive services and action your commands.
    You use the RasPi Zero 2 W as a Rhasspy satellite by connecting your audio device, installing full Rhasspy, but configuring only audio input, audio output and wakeword - everything else it sends to your RasPi 4 Rhasspy add-on,

They both have advantages and disadvantages. I know I got confused when reading the Rhasspy documentation because - while it’s all in there - it’s not so clear which parts are relevant to what configuration.

Next step is to decide whether you want to go the all-in-one or base+satellite mode ?

log

Yeah, I don’t often bother looking at that log because it doesn’t say much :frowning:
On your RasPi 4 you can go into the Rhasspy add-on and look at the “log” tab there … to be overwhelmed by the sheer number of apparently meaningless messages - it’s a famine or feast :wink: However I have found useful information hiding in there.

Thanks @donburch. I think I will go the sattelite route and buy an extra speaker/microphone combi. Raspi4 is in the livingroom so not usefull for my bedroom/office. Might buy the anker powerconf s330 that works on the pi4 so hopefully also on a pi zero 2w.

So on the satelite it’s just mqtt-> external
Audio recording on, wake word on, audio playing? Not intent handling → homeassistant?

Popi do not buy more hardware, at least until you are happy with your current RasPi 4 and RasPi Zero 2 working as you want.

OK, base + satellite install.

This is a pretty classic client-server configuration. The satellite (client) devices detect the audio, and coordinate the processing, calling the appropriate modules (services) on the local or base machine to perform particular tasks. The satellite machine is in charge of the overall process.

base machine

On the base machine you already have Home Assistant and the Rhasspy add-on.

  • If you don’t already you should install HA’s MQTT add-on to allow multiple machines and apps to talk to each other (the Internal MQTT option only allows the rhasspy modules on that machine to interact).
  • If you are like me you have probably tried fiddling with all the different settings; but there is actually not much required. My base machine’s Rhasspy settings are:
{
    "dialogue": {
        "satellite_site_ids": "sat-0, sat-1, sat-2, sat-3B, muscle",
        "system": "rhasspy"
    },
    "intent": {
        "fsticuffs": {
            "fuzzy": false
        },
        "satellite_site_ids": "sat-0,sat-1,sat-2,sat-3B, muscle",
        "system": "fsticuffs"
    },
    "mqtt": {
        "enabled": "true",
        "host": "192.168.1.98",
        "password": "redacted",
        "site_id": "base",
        "username": "rhasspy"
    },
    "speech_to_text": {
        "kaldi": {
            "min_confidence": "0"
        },
        "satellite_site_ids": "sat-0,sat-1,sat-2,sat-3B, muscle",
        "system": "kaldi"
    },
    "text_to_speech": {
        "satellite_site_ids": "sat-0,sat-1,sat-2,sat-3B, muscle, announce",
        "system": "larynx"
    }
}
  • Note that only some of the sections are used, and mostly with default settings. Other sections are disabled.
  • satellite_site_ids are very important, as this tells the base station which satellite machines it should provide that service for. I called mine “base”, “sat-1”, “sat-2” because I have no imagination, and I was unsure which room I will end up putting them.
  • I have chosen to use IP addresses, and have created a separate user in HA for the MQTT service to use … probably not necessary.

satellite machine

On a satellite you only need the minimum to run Rhasspy.
For my RasPi Zero I installed Raspberry Pi OS lite version, then a Debian Rhasspy install - without Docker because honestly I didn’t want to tackle that learning curve yet; and the satellite will only be running the one task so minimal complexity.
On your regular PC simply browse to the satellite machine’s web interface on port 12101 (eg http://192.168.1.92:12101/) to see Rhasspy’s interface.

It’s the same user interface as the base, which can be confusing, but you can plug in your audio device, configure the Audio input and output.
You could use the base for listening for the wake word - but that will send all the sound over the network - so local wake word detection is probably better.


Note that I have not used Intent Handling - which is because I am using node-RED add-on in Home Assistant to listen for the messages and action them. You will want to set Intent Handling to your machine running Home Assistant

Wow, that’s probably enough for now. Time for me to locate that tutorial I wrote way back, to see what else I did (particularly when I used Intents).

Thx I will start comparing my config to yours.

Found it. I had tried to distil all the knowledge I had learned; and provide a detailed tutorial of every aspect including:

  • installing the reSpeaker 2-mic HAT for RasPi
  • troubleshooting audio problems
  • configuring base machine
  • configuring Rhasspy on the satellite
  • adding HermesLedControl to flash the LEDs on the reSpeaker as a visual indicator
  • running Rhasspy on the satellite as a service (so it doesn’t need to be manually started every reboot)
  • Telling Rhasspy what commands to expect (setting up sentences)
  • setting up intents within Home Assistant
  • actioning those intents in Home Assistant

… and when I got to 30 pages I was wondering if I had bitten off more than I could chew :wink: Should I really be covering so much, and in so much detail ? Obviously I should split it in to manageable sections - but where ?

Also I’m a ‘first principles’ guy - I like to know why, and what the other options are; so maybe I was giving too much general info and background. Heck, the first 7 pages were introduction :wink:

Would you like the whole 30 page document, or just the relevant notes as you get to them ?

All off them would be nice. I am keen to learn as much as possible

I am new to github … think I have uploaded the file there as donburch888/Add Rhasspy local voice control.docx at main · donburch888/donburch888 · GitHub

I ran out of steam, particularly when trying to anticipate all the troubleshooting that could be possible - given that I was learning it myself at the time

Got it, thank you very much! Will be an interesting read. :smile:

@donburch it is improving with the help of your guide! Currently working on the base station pulseaudio server configured and states working. Intents do get recognize but nothing is spoken by the speaker yet. Also wakeword is not working yet but I am further then I have ever beencin this proces. Thanks for all your help!

Popi, I believe standard Rhasspy should play a sound to let you know that Rhasspy has recognised the wakeword, and another to indicate that it has stopped listening. I am concerned that you are not hearing those, and that Wakeword is not recognised. have you verified that the audio is working ? Not sure whether my notes on troubleshooting the audio will be as much help for your USB device.

Pleas let me know any question or problems with my document, as I really should get back and polish it off for publication - and it needs to work for a new user.

I gave up on the Anker Powerconf the drivers for Linux just dont’ work well, in fact so badly I got a full refund from Anker who where superb.
I got mine when it was released and spoke for some time with there tech support.

But also the xmos chip it was built on I don’t rate as I also have a Anker Powerconf C300 2mic cam and its absolutely glorious for farfield and clarity, which is so weird as the powerconf speaker/mic so isn’t.

The powerconf would often need a 2-3 sec wakeup so would cut output and various other problems but also its farfield wasn’t that great.
Conference speaker mics work in a different way that what you need really for a smart assistant as they don’t focus on a particular voice and beam to the most predomiant noise and voice if they have vad.

There is a huge hole in available hardware and also audio engineering from Rhasspy and every body dodges this point of honesty.
The best solution is likely one of the 2mic hats and then its not that great but at least you only spent about $10.

Well that’s is the strange thing. Saying my wakeword doesn’t work but the tv did wake it up yesterday evening. The button wake up does give the “listening sound”, records what I say (I can see the intent) and then gives the “off” sound. I am considering if I just should switch to English instead of Dutch and hope that when the rhasspy developer works for HA Dutch will eventually work. Kaldi doesn’t work in Dutch for example. My guess is that it does work considering wake up en speech do work but I haven’t found the correct working settings yet.

Edit: I just used wakeword “grasshopper” and the speaker woke up and the intent got properly recognized. I didn’t get de date back as I asked. So it should be able to become a working setting.

@donburch progress update. HA Base rhasspy is running in Dutch wakeword works, intents get recognized but no speech response. Longer sentences don’t get recognized, do you have the same?

Satelite rhasspy installed through docker (had trouble with the other installation) it’s running, speaker get’s recognized so now I need to adjust and get the intents working.

@donburch satelite is working but it gives me double the speech. So just one bleep but the text message get’s spoken twice? HA Base still not working but we are making progress. :slight_smile:

Oh Popi now I am totally confused about what is working and what is not !

Can you please post the current configuration for both your base and satellite Rhasspys. The regular GUI is too many images to post - but the summary can be found under the “Avdvanced” option of the drop-down menu (as shown here)
Screenshot from 2022-11-19 13-18-23

When pasting here, use the </> Preformatted Text button to keep the indentation and formatting.