Speaker Cancellation

Just to update older thread, there is 48kHz firmware for Respeaker Mic Array v2 available. So AEC is now also available for 48kHz playback/recording.

Thanks. I know of it and tested it a few months back to no avail. AEC is pretty inexistant and audio playback quality suffers from strange saturation issues.

Seed has stated on their forum that the playback quality / AEC issue can not be solved because of the limited capabilities of the XMOS chip.

@litinoveweedle Hi :wave:. Can you confirm that the Respeaker 48KHz 1 channel firmware have a working AEC? Did you notice audio quality issues? Thanks :blush:

@fastjack On the topic of EC and hardware.
Got another Syba USB and again seems to be fake with a c-media mono mic chipset than the stereo via envy.

Also 2x I2S mics and a cheap china hdmi2hdmi+audio seems to work well.

Hello, sorry for late reply. The sound with 48kHz is little bit better than 16kHz, but is is far from perfect, there are some strange artifacts. AEC seems to be working but is far from perfect, especially when playing loud music.

I did checked all available MICs arrays, but all seems to be based on same chip (xmos XVF-3000), with same 16kHz limitation. I can only hope for some new MIC array with xmos XVF-3500 to arrive soon. :-o

@litinoveweedle

Could you try and help this guy out a Mic wise the Anker Anker PowerConf seems to be an amazing piece of kit.

For some reason aplay -Dplughw:1 -r16000 -f S16_LE file_example_WAV_10MG.wav works perfectly but playback from Rhasspby is cutting at the start and I am unsure why.

But might be of interest as they are a pretty new product and if aplay -Dplughw:1 -r16000 -f S16_LE file_example_WAV_10MG.wav works fine from the cli then I can not see it being the hardware.

Only thing I haven’t done is suggest creating an asound.conf defining sample rates and channels maybe summing them.

Sort of hard work remote and blind, I might take the plunge and buy one.

I’m using a ReSpeaker 4 Mic Array with a raspberry pi 3b+ and cannot run ec with “./ec -i plughw:CARD=seeed4micvoicec,DEV=0 -o plughw:CARD=seeed4micvoicec,DEV=0 -d 200”

It just says
cannot open audio device plughw:CARD=seeed4micvoicec,DEV=0 (No such file or directory)

Anyone have a respaeker 4 mic and got this working?

My arecord -L:
null
Discard all samples (playback) or generate zero samples (capture)
jack
JACK Audio Connection Kit
pulse
PulseAudio Sound Server
eci
eco
default
Playback/recording through the PulseAudio sound server
usbstream:CARD=b1
bcm2835 HDMI 1
USB Stream Output
usbstream:CARD=Headphones
bcm2835 Headphones
USB Stream Output
sysdefault:CARD=seeed4micvoicec
seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0
Default Audio Device
dmix:CARD=seeed4micvoicec,DEV=0
seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0
Direct sample mixing device
dsnoop:CARD=seeed4micvoicec,DEV=0
seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0
Direct sample snooping device
hw:CARD=seeed4micvoicec,DEV=0
seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0
Direct hardware device without any conversions
plughw:CARD=seeed4micvoicec,DEV=0
seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec.1-003b-0
Hardware device with all software conversions
usbstream:CARD=seeed4micvoicec
seeed-4mic-voicecard
USB Stream Output

Dunno try aplay -l and see if you can get it to work with the index number instead plughw:2 or whatever it will be

This is my aplay -l:
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
Subdevice #7: subdevice #7
card 1: b1 [bcm2835 HDMI 1], device 0: bcm2835 HDMI 1 [bcm2835 HDMI 1]
Subdevices: 4/4
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
card 2: Headphones [bcm2835 Headphones], device 0: bcm2835 Headphones [bcm2835 H eadphones]
Subdevices: 4/4
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3

I might be wrong, but doesn’t this only list playback devices, not mics? The respaeker isn’t there anyway

You are missing card 0 from your cut and paste but presume thats the loopback.
But that looks like the respeaker is not installed or installed but not found.

PS is this docker or just host install?

If Im not mistaken, you are using the normal 4mic Respeaker Board that doenst have an audio out. So you will not find it in your aplay.
In this case, your sound output is probably routed through the 3.5mm from the PI itself.

1 Like

Yeah, the respeaker is not listed with aplay -l. Does that mean I cannot use the echo cancelling program on my mic?
It’s Raspbian installed with the Rhasspy .deb, so I guess host?

Yeah it uses the clock sync on in and out to match the AEC, AEC works only with cards or USB that both play and record.

I sort of hate that 4 mic I have one in a box and doh it was late at night as always going on about how they are whole load of nothing with a pixel ring.
Nice Pixel ring but otherwise pretty useless.

Have a word with @JGKK as think he now also about a USB dongle or the https://www.raspberrypi.org/products/iqaudio-codec-zero/ looks pretty amazing if it ever gets stocked.

Without high speed DSP beamformingn, algs multiple omnidirectional mics are useless as even when summed they can create 1st order filters and even can be even detrimental than just pointless.
The are just more of the same so you have a mc and a pixel ring and the 4 mic is just one of those products they make cause they can because they sell even if they know they are pretty pointless.

Alright, thank you so much for explaining.
I think I’ll buy a ReSpeaker Mic Array v2.0 instead as it actually has EC and noise suppression built-in. Hopefully it will filter out loud music and background noise well.

A unidirectional electret mic has an element of natural AEC as well as directionality that with the addition of AEC probably with the hardware we have makes for the best MIC for a builder unless your going to spend a lot of $ on a DSP high end mic.

A USB cheap sound card is a few $ and with a MAX9814 mic module provides a double dose of AGC that can give really good far results on a budget.
I will see if I can get @JGKK to comment as always chanting this mantra but maybe he will give a honest review on a recent test he has done.

It would be awesome if a MAX9814 microphone with the EC program could be almost as good as a ReSpeaker Mic Array v2.0. Would love to try it out sometime to see.
I found this guide which seems very helpful for beginners.

The Respeaker Mic Array is not that great from other reviews, I have a Anker Powerconf which is a Xmos chip above the Respeaker and its OK but like all it fails when noise=near & voice=far.
noise=near & voice=far happens far often than you would expect in a domestic situation such as hifi, radio and TV to even the washing machine and is very dependent on the placement of a single device.
AEC can cope with the noise it plays but 3rd party noise has no input and is a problem for all.

Its better to have multiple devices and use the stream with the best KW hit confidence as with correct placement it is likely to have more chance of getting nearer to voice=near & noise=far.

Just a shame rhasspy doesn’t support this or multiple instances of KWS mics in a single device as the unidirectional can face out at different angles.

https://www.scan.co.uk/products/enermax-ap001e-dreambass-usb-soundcard-plus-earphones-genie-with-integrated-80-hz-plus6-db-bass-boos
Is prob the best usb sound card on the market for price as S243LE cards at 48kHz tend to be prosumer studio prices but £7 from scan.
Its stereo and you can phantom power electret mics with it.
So you can add another if rhasspy does ever do miltiple kws instance.

Mono mic cards I will give examples from raspberry shops but you can find them cheaper on ebay.

Or the CM108 module seems to be popular.

unidirectional electrets with good sensitivity need some sourcing but

Which is me as other you end up buying x25 from aliexpress as those seemed to have really good sensitivity.

Max9814 the ones with the onboard ldo regulator seem to be better for SNR

They end up about the same price as a 2mic or approx half the 2mic raspberry codec zero HAT, but generally I think they are better.

https://speechbrain.github.io/ is a speech toolkit that comes with beamforming algs its currently going through a stage beta / bug testing and should go public soon.
I have not looked enough to test but prob will as if it will run then multi-mic does become valid.

1 Like

Hey folks,

Did anyone try AEC on Respeaker Core v2? They supply librespeaker with DSP algorithms from 3rd-party provider Alango. Specifically, there’s a VepAecBeamformingNode which should do AEC automatically (according to docs). But I’m wondering how? It’s not really clear how to supply an output audio stream to this library to be able to correctly apply AEC. When I tried to play a random music file while active listening, it didn’t seem like AEC works at all. As it’s a C++ library, I guess music should be played from code and somehow plugged into librespeaker node chain. Note that Alango delegates tech support to Seeed Studio and Seeed Studio doesn’t reply for a couple of years neither on GitHub nor on their forum / email.

P.S. I already have a code that successfully does WWD, NS, AGC, DOA and BF. Not perfect, but good enough to use this board for common smart home tasks. The only important thing left is the acoustic echo canceller. Any thoughts/suggestions would be greatly appreciated.

I think the respeaker blurb if not is very close to snakeoil as yeah the Alango EC wasn’t free and you just have an API for it.
https://github.com/voice-engine/ec works pretty well in fact it seems the only EC that works on a PI level hardware as the pulseaudio webrtcec only works on really low levels and then fails.

Its based on SpeexDSP you have to have audio in/out on the same card and a bit of a hack with piping through a loopback adapter but the Respeaker Core does that not have hardware one?
If not you can just modprobe a kernel one.
On raspi os speexdsp & speex are missing from asound2-plugins as for some reason its an old version.
But instructions here.

I did some simple one-liner helper scripts here

Its echo attenuation but works well enough that barge-in should work.

PS that Respeaker Core v2 is some where between a Zero & 3 in oomph?
If your doing something yourself have a look at Googles state of the art KWS for tensorflow-lite
https://github.com/StuartIanNaylor/google-kws runs about 70% on a Zero so should work well on a Core2 but is heavily optimized for 64bit and a Pi3 will likely greatly outperform running TFL on Aarch64

Without beamforming algs the mic array on that board is a whole lot of pointless even when you sum its likely to be detrimental as the will form 1st order high pass filters based on the distance apart.
You prob should use a single channel.
I don’t think its got the Ooomf to run ODAS.

1 Like

Thanks for the extensive reply!

According to specs, Respeaker Core v2 has 8-channels ADCs for 6-microphone array and 2 loopbacks (hardware loopback).

The main problem I see with an external EC is the actual integration to my code. librespeaker already does beamforming, wake word detection, noise suppression, automatic gain control, etc for me. I can’t simply run an external service that does AEC separately. It should be tightly coupled to the existing chain of operations.

As far as I understood, the above EC writes the output to the file. And librespeaker works with mics directly. Well, technically it can read a 8-channels wav-file as an input. So theoretically, if the above EC had top processing priority, it could do AEC and save the output to wav, which could be used as an input for librespeaker in the next step. However, I’m not really sure about the real-time performance of such an approach, as it would lead to constant I/O operations against SD Card. Ideally, this EC code should behave as a middleware w/o saving intermediate results to the file. But I’m not sure if it’s even technically possible.