Respeaker 2 Hat - Buster - Rhasspy + HA as service install + picotts + snips wakeword

I have the Respeaker 4 mic array (real name is Respeaker mic array v2.0) and it is pretty good.

The AEC and beam forming claerly do the job and I can get the wakeword detected (the one from Snips though) from more than 5 meters away while playing music near the mic array.

Note that the Respeaker Mic Array v2 firmware limits the audio playback quality to 16Khz (their 48Khz firmware is pretty bad due to XMOS capacity limitations and the AEC is inexistant).

I’ve heard good things regarding the sensitivity of the Respeaker 2 mic hat but it does not provides AEC or any audio algorithms. For AEC you’ll have to implement it yourself using something like Pulseaudio.

If you plan to play music via your assistant, the Respeaker mic array v2 is a very good device.

If not, I think the 2 mic hat should be pretty good.

Hope this helps.

1 Like

have you the MicArray usb version? I’m confusing about the names :slight_smile:

I tried snowboy and I can definitively say that with respeaker 2 it’s much more precise than porcupine.
I suggest it with the same hardware.
Thanks you

I have the Respeaker Mic Array v2.0 (USB) and the Respeaker 4 mics Raspberry Hat.

Does Respeaker Mic Array v2.0 (USB) work out of the box on raspbian or does it need driver installation?
I bought one I will receive it in the next days :slight_smile:

No driver installation is needed. It works out of the box as both an ALSA input and output device.

Enjoy :blush:

1 Like

Fastjack did you modify webrtcvad default parameters for respeaker mic array v2.0?
Are you using a satellite configuration?

I did not modify the default Respeaker config.

Though I have not yet tested the full Rhasspy chain (mic->wakeword->asr->nlu->intent->tts->speaker).

I got good results with the snips software.

For now, I’m concentrating on the ASR and NLU part as this seems like the most complex stuff to get right.

Once I get satisfied with the ASR and NLU results I’ll move on the wakeword.

:blush:

ok thanks you :slight_smile:

Sorry to resurrect this thread, but I’m trying to get the same setup (snips-satellite (I like snips’ hotword detection) and rhasspy handling the rest (I dislike the rest of modern-day snips)).

Do you run the snips part on the same raspy as the rhasspy software? I’m using the ps3 eye mic and run into “arecord: main:828: audio open error: Device or resource busy” errors when I have snips-satellite running.

Any idea how to solve that?

Hey, do you have to use it as an output device in order for the AEC and the rest of its magic to work? Can I use for output, for instance, a regular lineout from the board?

The AEC needs the audio ouput to remove it from the input signal.

Sending the playback to 2 outputs at the same time is quite problematic due to clock synchronisation and latency issue between audio devices (I’m not savy enough to get past that).

So, If one send the output through Raspberry’s line out device, Respeaker won’t be able to work or will perform like it doesn’t have AEC enabled, right?

Yes, you are correct. Respeaker needs the playback audio data to be able to remove it from input audio data.

I was thinking of trying to get one myself, but this is a cold shower. How does the 16Khz audio sound like? Is it obvious? Is it at least stereo?

The Respeaker Mic Array v2 outputs stereo in 16Khz 16bits. You can ear the difference between 16Khz and 48Khz when playing music through earphones. The sound is like muffled. To me, It is not obvious when playing through loudspeakers. For a vocal assistant that can play radio stream and Spotify it’ll do for me. I prefer a lesser audio quality but a great voice interactivity.

I tried Seeed’s 48Khz firmware that improves the playback quality (with strange « shhh » sounds that are not acceptable though) and the AEC is almost non existant. I notified them on their forum and their response was that the XMos chip is not capable of processing AEC at this sample rate.

If I have enough time, I’ll give PulseAudio AEC module another go because I’d really like to not be dependent on a specific Mic in the future.

Hope this helps. :slight_smile:

Is it an option to output the audio both to the respeaker and your normal speakers, so the XMos Chip can do it’s processing at 16kHz, while you listen to it with a higher sample rate?

I was not able to make it work unfortunately. I doubt it is possible.

Hm ok. Yeah at least for me that makes the respeaker pretty useless, since there is no way I will listen to all my music with 16kHz for the AEC. Especially since the respeaker has quite some latency with all the preprocessing and USB communication already :sweat_smile:
I guess I will give Pulseaudio AEC with a faster mic using I2S a try when I find the time aswell then.

You can also try Seeed software AEC for a lighter solution using ALSA instead of the heavier PulseAudio service.

I was not able to make it work correctly either but I did not put a lot of effort into it though.

I wonder if something like this could be incorporated directly into Rhasspy audio input chain using a python c++ binding.

What do you think ?

I will try it out when I have some time. When it works well I can absolutely create python bindings using the Python C Api so we can integrate it better into rhasspy.