Rhasspy, AEC, voice-engine, pulse audio and bluetooth speaker

I have Rhasspy and Mopidy running on the same raspberry pi. I have a Respeaker 4-mic for audio input and I use bluetooth to connect to a mobile speaker. Rhasspy is used to take in commands and send them to home assistant. Based on the command home assistant will turn on/off lights or direct Mopidy to play music. Without any music playing Rhasspy works great directing home assistant. However tell Mopidy, via Rhasspy and home assistance, to play music greatly impacts Rhasspy ability to hear. My system uses pulse audio. I installed ec from voice-engine and tried to get it working but when using a bluetooth speaker there isn’t a hardware device to supply in the ec command. Any insight into how to make ec work with a bluetooth speaker or some other means of implenting AEC would be greatly appreciated. Without some form of AEC Rhasspy use in home automation seems limited. Thanks for any suggestions.

EC uses the very tight timings of the single clock of a sound card that can record & playback so they are exactly in sync.

So no its not going to work.

So does this mean there is no AEC solution if you’re going to use a bluetooth speaker? ReSpeaker has the Mic Array v2.0 that indicates it can communicate with speaker via bluetooth. Anyone have any insight if it might be a solution for providing mic in, AEC with a bluetooth speaker?

Pulse audio has a AEC solution but for some reason the webrtc tanslation to pulseaudio on Arm has a very low threshold but it can latency adjust between 2 separate devices but fails with relatively low levels of echo but does completely cancel.

Get a soundcard that can play & record with a 3.5mm bluetooth transmitter?

I’m also considering integrating snapcast. Considering snapcast is all about auto syncing across multiple systems, is there anyway to tie it into the solution and get the desired timing?

I haven’t used a separate sound card with a raspberry pi but I suspect someone makes one. Actually that’s what I think the Respeaker V2 device act like.

I thought about that and snapcast is great as it uses network time to sync but the speexdsp needs even tighter sync than that.

I did a bit of a git repo on an ‘easy’ install for ec as you also need a loopback to stop ec halting after the first ec.

Respeaker 4-mic hat or https://wiki.seeedstudio.com/ReSpeaker_Mic_Array_v2.0/

As likely you should be able to get it to work but you need to set the latncy correct.

Have a read in the above repo I never did work out the multi channel version its also not cancelation but attenuation but does quite a good job irrespective of loudness as opposed to the pulseaudio webrtc-aec

Setup a vanilla install minus rhasspy on the host and play as you will prob find it works and then it will come much clearer then move to rhasspy.

Also move your speaker if bluetooth further away

thanks I’ll have a look

1 Like

I’ve been playing around with different option for AEC with directly connected speakers. I enabled the pulse echo cancellation module and did some testing with both the PS3 Eye and the Respeak 4-mic array for Raspberry Pi. The AEC seemed to clean up the audio, however the audio is at a very low level. I wasn’t successful in getting the cleaned up mic data to a level that allowed Rhasspy to preform as I’d expect. In general I was underwhelmed by both of these solutions. I suspect with more time I might set the right parameter to gain some improvement. I then received and did testing with Seeed Studio ReSpeaker Mic Array v2.0. There are two versions of firmware for this product, a 6 channel and a 1 channel version. With the 6 channel firmware I had some challenges and wasn’t getting much better performance than the previous two options. I then loaded the 1 channel firmware version and what a difference. With low noise in the room, I can stand 20 to 30 feet away from the mic array and get excellent performance from Rhasspy. I can play music through the system and the AEC provided voice is excellent, allowing Rhasspy to correctly interrupt my commands. This doesn’t give me the bluetooth option I desire but it really puts the platform on par with the google and amazon devices. I’ll update if I figure out how to get similar performance using a bluetooth speaker.

1 Like

Can you not stick a bluetooth transmitter on the 3.5mm output of the respeaker?

1st one on ebay I found

But with a bit of a search you will prob find a ‘key fob’ sized one.
Dunno if the latency plays havoc with the AEC though, Aptx?

I found the same with Pulse webRtc_Aec on the Pi which seems to fail at much lower volume than X86 for some reason.
In fact it was so low I just thought it was broke until one time I tried with a lot lower volume.

The Speex EC from VoiceEN works quite well but debian lags behind Alsa and uses an old RC of SpeexDsp and even then never did get the Alsa-plugin version working.
But https://github.com/voice-engine/ec does work quite well even though the install can make eyes roll and uses the Speexdsp lib but dunno why the alsa-plugin doesn’t work.

I’ll probably look at adding in a bluetooth transmitter, but I expect that to kill the respeaker AEC. Hopefully the board does some buffering and has a parameter to adjust latency.

I tried using a bluetooth transmitter off the ReSpeaker V2.0 to a bluetooth speaker. The device I used was only supposed to introduce 40ms of latency. I didn’t have any means to check the latency. This delay was enough to trash the ReSpeaker echo cancellation.

1 Like

Good to know, oh well was worth a try.