I tried to send the pulseaudio guy an email, but I guess it went to the spam folder. I would like to beg him/they to leave it there. It does wonders to the audio, even though it also distorts a little the voice.
Ok, I’m quite pleased now with my current Raspberry Pi4 + pulseaudio configuration. For anyone reading this, I’m posting my findings and workarounds for a great microphone capture experience and reasonable audio quality.
Overview:
- Pulseaudio is the main sound architecture. I use an alsa configuration only to tell alsa utils to use the pulse devices so that I get automatic audio mixing, converting and adjusting plus pulseaudio filters.
- Rhasspy uses arecord to get the sounds and it is fine because it is aactually using pulse behind.
- The 4-mic playstation eye is the source of sounds and the 4 mics are processed with the currently playing audio so that there is echo cancelation and noise removal from the voice input. Beamforming is also working to an extant, but it causes a little distortion to the resulting voice. In my humble tests, the results were better with it turned off.
- The way it works is: you load an echo cancellation module which gets the currently playing audio AND the sound from the mics to do its magic. BUT, all applications MUST use the module’s created SOURCE for all voice capture AND the module’s SINK for all sound output. The quirk I’ve found with this is that I cound’t find a way to force the echo cancellation module’s sink to be stereo. So I had to use a virtual sink (the workaround).
- In the end the audio quality is as good as your speakers and audio hardware (mine is the default Raspberry Pi Soc + a good old stereo system). Maybe in the future I’ll get another audio output device.
The workaround is simple:
- Create a virtual sink and let the echo cancellation module think that is the output sink;
- Load the echo cancellation module and tell it to read from the 4-mic playstation eye source and also read from the virtual sink (yes, it reads from a sink). The module creates a new source (aec_source) and a new sink (this sink is unfortunatelly only a mono channel, called aec_sink).
- Load a combine module which creates a new sink that accepts high quality stereo sound and sends it to two places: A) the real stereo speakers; B) The aec_sink (downmixing the sound only for processing).
- Tell the applications (MPD, Rhasspy, etc) to use the sink crated in 3 for all audio output and the aec_source to read any audio.
When I find out how to post the script here I do it.
You probably need to do it in the docker version as Rhasspy is heavily docker oriented.
I tried pulseaudio AEC with a PS3eye and really you need to post some comparison wavs of with-aec/no-aec as for me there was litttle difference on a Pi with PS3eye and 3.5mm Pi output.
I did some tests with the supposedly lesser quality Speexdsp with respeaker 2mic that produced noticeable AEC results that wasn’t that noticeable via PS3eye and Pulseaudio.
Its clock drift between PS3eye and Pi 3.5mm but could never get pulseaudio to run with the respeaker 2mic drivers as presume it is better but could never prove.
PS extant doesn’t mean it works. The default beam setting if ommited like often it is starts at 90’ right of the mic. That is some beamforming in action that is 90 degrees out of phase and it makes liitle tangible difference to recorded audio.
Its already been removed upstream from Webrtc and next pulseaudio will also follow suit.
There is no DoA input or feedback in the code and its absolutely impossible for any beamforming to work without (directional of angle) to beamform to unless fixed position, but hey it is extant until next version.
I am always strugging to get my PS Eye working. Every now and then it stops to work and I usually have a hard time to get it back. Sometimes it works as “default”, sometimes as “USB Camera-B4.09.24.1 Multichannel” audio input. HA detects it but it is always a pain… 
When it works, it is not a very good performance, taking advantage of noise cancellation and so on, with the 4 mic array (I believe).
I found this thread with a lot of fancy configurations that would probably give me more reliabilty, less struggling and also improve its performance. But you know what you are doing, what files you are changing and so on. I only use the UI till the moment.
So, I would like to know if there is some place, orientation, where I could learn exactly what to do to solve the issues that I mentioned above… I See command lines but I don´t know where it should be applied.
Thanks to anyone that could show me the path.
Using: Proxmox > Ubuntu 18.04.3 > Docker
The PS3 eye is a 4 channel omni-directional broadside array where the clever stuff was in software inside the PS3-Eye.
It never had windows or linux drivers and was reversed engineered and is hacky.
It can not even do a alsactl store without error.
The errors you see have never been resolved in what now is almost 2 decades and even if it did the closed source of the algs in the PS3 have never been available apart when used with a PS3.
You have a broadside array when you sum your channels its dependent on the SNR and sensitivity of each electret which nobody knows.
Even the wiki is incorrect guess work as https://en.wikipedia.org/wiki/PlayStation_Eye and
The PlayStation Eye microphone array operates with each channel processing 16-bit samples at a sampling rate of 48 kilohertz, and a signal-to-noise ratio of 90 decibels.
Is a total WTF as considering decibels is a logarithmic scale you will be hard pushed to find any microphone that goes past 70db SNR! I guess the silicon could but doubt the result is anywhere near.
So it a omndirectional broadside array that has attenuation on the sides and also acts as an 12db? (because of 4 mics summed) Low pass filter from audio on the side.
Its all in the above app note but no just because its an array means you have the antenna but you don’t have the control software and there isn’t any available opensource for the Pi.
It has zero noise cancellation its just a mic array, just another microphone without the software that was in the PS3.