2x Matrix Voice ESP32

Hello, I’m not sure if I made a mistake in the configuration or if Rhasspy doesn’t support it. Following situation:
I have installed Rhasspy on a Raspi Pi 3B I am using an external MQTT server and two Matrix Voice ESP32 with Matrix Voice ESP32-MQTT audio streamers from Romkabouter as microphone satellite.

I want to use the two audio streamers, i.e. two microphones for two separate rooms. I now have a Rhasspy that listens via MQTT to the micophones. I have adapted the satellite site ids in Rhasspy and MQTT IDs. Both Matrix Voice are running and sending with their own topics but unfortunately I can only register one audio streamer at a time on Rhasspy. Both do not work together even if I insert them into the satellite ID in Rhasspy with separate commas.

Do I have to activate something in Rhasspy to recognize both streamers or do I need a separate Rhasspy installation for each streamer?

many thanks

This needs to be fixed in the streamer, it uses a hardcode clientID for the connections to MQTT
As soon as you connect the second device, the first will get kicked out

You need to change the “MatrixVoiceAudio” and “MatrixVoice” in the MQTT connections.

search for in the streamer code:
syncClient.setClientId("MatrixVoice");
and
if (audioServer.connect("MatrixVoiceAudio", MQTT_USER, MQTT_PASS)) {

Flash one, change the id’s and flash the second. This should fix your issue

Hello romkabouter,
thank you very much for the answer. Yes, I have already done that. The streamers run wonderfully with these adjustments. But I can’t get them both together in Rhasspy.
But I found “syncClient.setClientId” twice in the code and adapted it. It seemed to me to be right…

ok, that’s strange then indeed.

Can you share some screenshot from the settings and some logs?

Okay, let’s rule out that I did nothing wrong with the audio streamers. I have made the following adjustments:
MatrixVoiceAudioServer.cpp
Line 19 and 20
'-DSITEID="matrixvoice71"' '-DMODELID="computer71"'
The rest I also added, Wifi, MQTT etc.

platformio.ini
Line 425
asyncClient.setClientId("matrixvoice71");
Line 434
if (audioServer.connect("matrixvoice71Audio", MQTT_USER, MQTT_PASS)) {
Line 1406
asyncClient.setClientId("matrixvoice71");

At the second Matrix Voice I swapped the 71 for 72. Is that right or have I forgotten something?
What do you need the logs and screen shots from?

Yes that seems right (but I think you have swapped platformio.ini and the cpp file in your text), but since you mention both devices stream ok on the MQTT broker, the problem must be somewhere in Rhasspy.

So, therefore we might get some info from the Rhasspy log and settings

Good evening,

sorry for the late reply. Oh yes that’s right because I mixed up the platformio.ini and MatrixVoiceAudioServer.cpp :sweat_smile:

So my Rhasspy settings are as follows:
Pi 3B, Rhasspy 2.4.20 (Version 2.5 does not run stable with my streamers, not even in single mode. This is another problem which I have to solve separately)

Rhasspy Settings

Side ID are both entered.

The Hermes protocol is selected for Audio Input.

Here is an excerpt from my Advanced
{ "microphone": { "system": "hermes" }, "mqtt": { "enabled": true, "host": "192.168.1.60", "site_id": "matrixvoice71,matrixvoice75" }, "sounds": { "aplay": { "device": "default:CARD=ALSA" } }, "wake": { "pocketsphinx": { "threshold": 3.1622776601683794e-28 }, "snowboy": { "apply_frontend": true, "model": "snowboy/computer.umdl", "sensitivity": "0.4" }, "system": "snowboy" } }

And here the log when I call something, in this case the time.
[INFO:18040198] quart.serving: 192.168.1.60:59410 POST /api/text-to-speech 1.1 200 28 3307615 [DEBUG:18040164] InboxActor: -> stopped [DEBUG:18040144] EspeakSentenceSpeaker: speaking -> ready [DEBUG:18036967] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=ALSA'] [DEBUG:18036964] EspeakSentenceSpeaker: ready -> speaking [DEBUG:18036893] EspeakSentenceSpeaker: ['espeak', '-v', 'de', '--stdout', 'Es ist 19Uhr und 38 Minuten.'] [DEBUG:18036815] SnowboyWakeListener: loaded -> listening [DEBUG:18036813] DialogueManager: ready -> asleep [INFO:18036810] DialogueManager: Automatically listening for wake word [DEBUG:18036807] DialogueManager: handling -> ready [DEBUG:18036806] HermesMqtt: Published intent to hermes/intent/GetTime [DEBUG:18036804] WebSocketObserver: {"intent": {"name": "GetTime", "confidence": 1.0}, "entities": [], "text": "wie sp\u00e4t ist es", "raw_text": "wie sp\u00e4t ist es", "recognize_seconds": 0.004081788000007691, "tokens": ["wie", "sp\u00e4t", "ist", "es"], "raw_tokens": ["wie", "sp\u00e4t", "ist", "es"], "wav_seconds": 0.0, "transcribe_seconds": 0.0, "speech_confidence": 0.1493115175976181, "wakeId": "snowboy/computer.umdl", "siteId": "matrixvoice71,matrixvoice75", "slots": {}} [DEBUG:18036802] DialogueManager: recognizing -> handling [DEBUG:18036800] DialogueManager: {'intent': {'name': 'GetTime', 'confidence': 1.0}, 'entities': [], 'text': 'wie spät ist es', 'raw_text': 'wie spät ist es', 'recognize_seconds': 0.004081788000007691, 'tokens': ['wie', 'spät', 'ist', 'es'], 'raw_tokens': ['wie', 'spät', 'ist', 'es'], 'wav_seconds': 0.0, 'transcribe_seconds': 0.0, 'speech_confidence': 0.1493115175976181, 'wakeId': 'snowboy/computer.umdl', 'siteId': 'matrixvoice71,matrixvoice75'} [DEBUG:18036792] DialogueManager: decoding -> recognizing [DEBUG:18036789] DialogueManager: wie spät ist es (confidence=0.1493115175976181) [DEBUG:18036786] PocketsphinxDecoder: wie spät ist es [DEBUG:18036785] PocketsphinxDecoder: Transcription confidence: 0.1493115175976181 [DEBUG:18036783] PocketsphinxDecoder: Decoded WAV in 1.2460699081420898 second(s) [DEBUG:18035540] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=ALSA', '/usr/share/rhasspy/etc/wav/beep_lo.wav'] [DEBUG:18035534] PocketsphinxDecoder: rate=16000, width=2, channels=1. [DEBUG:18035530] DialogueManager: awake -> decoding [DEBUG:18035526] WebrtcvadCommandListener: listening -> loaded [DEBUG:18035520] WebrtcvadCommandListener: Voice command finished [DEBUG:18033679] WebrtcvadCommandListener: Voice command started [DEBUG:18033189] SnowboyWakeListener: listening -> loaded [DEBUG:18033178] WebrtcvadCommandListener: loaded -> listening [DEBUG:18033158] WebrtcvadCommandListener: Will timeout in 30 second(s) [DEBUG:18033157] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=ALSA', '/usr/share/rhasspy/etc/wav/beep_hi.wav'] [DEBUG:18033152] DialogueManager: asleep -> awake [DEBUG:18033151] DialogueManager: Awake! [DEBUG:18033149] SnowboyWakeListener: Hotword(s) detected: ['snowboy/computer.umdl']

As mentioned before, only the first entry of the Side ID is evaluated as microphone input. Everything after the comma is not considered. Let’s see if we can get Rhasspy to listen to the second streamer.
:smile:

Ah, you are using 2.4
Well, as you already have noticed, that does not work correctly.

Rhasspy is listening to the topic
hermes/audioServer/matrixvoice71,matrixvoice75/audioFrame.
This is not correct and should be

hermes/audioServer/matrixvoice71/audioFrame
and
hermes/audioServer/matrixvoice75/audioFrame

So there is the problem :slight_smile:

I suggest solving the problem with 2.5, which is able to listen correctly to multiple device.
Haven’t tried it however, but in 2.4 is will certainly not work

Mmmh then I guess I have no other choice than to update then I’ll get to work. Are you sure that it works with the 2.5 and two audio streamers?

I only have 1 but when I enter: “matrixvoice,server” in the siteid of Rhasspy, the matrixvoice keeps working.
Also, when I enter “server,matrixvoice”, the matrixvoice keeps working.

Yes it works with version 2.5.7, but it runs slower, is currently more error-prone in language recognition and just crashed. :exploding_head: 2.4.20 ran much more stable for me. Thanks for the help, now I will see if I can tune 2.5 to get the performance of 2.4. I wish you a nice evening.

Maybe first try and get the same results with only 1 sat. 2.5 runs fine on my server, but I do not run a pi3b

Good evening,

unfortunately it still does not work out the way I imagine it. It is also possible that I have configured something wrong because I do not see through some settings. At the moment it runs like this. Both streamers transmit with their own topic. So far good. If they are both active and in separate rooms they are not recognized by Rahsspy. If one is switched off, so only one is running, it will be recognized, even with the correct site ID.
I am not sure if I have entered the IDs correctly in the settings. There are different menus and so I entered them:
siteid: rhasspyserver,matrixvoice71,matrixvoice75
Wake Word= Satellite siteIds: matrixvoice71,matrixvoice75
Dialogue Managemt= Satellite siteIds: matrixvoice71,matrixvoice75

For all other menu items I have not stored siteIDs. Is this correct or do I have to add anything?

1 thing I notice is that you have 75 in your siteID, but 72 is your code
I do not think that is the cause, but still…

I think the siteID should be enough, you do not have to explicitly mentions them in the wakeword and dialogue.
Maybe I can come up with some device to test this as well

I changed the IDs from 72 to 75 afterwards. With 71 and 75 also the IDs on the streamers were uploaded. Sorry for the misunderstanding I forgot to mention.

No problem, I could not imagine this to be the issue :wink: