2x Matrix Voice ESP32

Hello, I’m not sure if I made a mistake in the configuration or if Rhasspy doesn’t support it. Following situation:
I have installed Rhasspy on a Raspi Pi 3B I am using an external MQTT server and two Matrix Voice ESP32 with Matrix Voice ESP32-MQTT audio streamers from Romkabouter as microphone satellite.

I want to use the two audio streamers, i.e. two microphones for two separate rooms. I now have a Rhasspy that listens via MQTT to the micophones. I have adapted the satellite site ids in Rhasspy and MQTT IDs. Both Matrix Voice are running and sending with their own topics but unfortunately I can only register one audio streamer at a time on Rhasspy. Both do not work together even if I insert them into the satellite ID in Rhasspy with separate commas.

Do I have to activate something in Rhasspy to recognize both streamers or do I need a separate Rhasspy installation for each streamer?

many thanks

This needs to be fixed in the streamer, it uses a hardcode clientID for the connections to MQTT
As soon as you connect the second device, the first will get kicked out

You need to change the “MatrixVoiceAudio” and “MatrixVoice” in the MQTT connections.

search for in the streamer code:
syncClient.setClientId("MatrixVoice");
and
if (audioServer.connect("MatrixVoiceAudio", MQTT_USER, MQTT_PASS)) {

Flash one, change the id’s and flash the second. This should fix your issue

Hello romkabouter,
thank you very much for the answer. Yes, I have already done that. The streamers run wonderfully with these adjustments. But I can’t get them both together in Rhasspy.
But I found “syncClient.setClientId” twice in the code and adapted it. It seemed to me to be right…

ok, that’s strange then indeed.

Can you share some screenshot from the settings and some logs?

Okay, let’s rule out that I did nothing wrong with the audio streamers. I have made the following adjustments:
MatrixVoiceAudioServer.cpp
Line 19 and 20
'-DSITEID="matrixvoice71"' '-DMODELID="computer71"'
The rest I also added, Wifi, MQTT etc.

platformio.ini
Line 425
asyncClient.setClientId("matrixvoice71");
Line 434
if (audioServer.connect("matrixvoice71Audio", MQTT_USER, MQTT_PASS)) {
Line 1406
asyncClient.setClientId("matrixvoice71");

At the second Matrix Voice I swapped the 71 for 72. Is that right or have I forgotten something?
What do you need the logs and screen shots from?

Yes that seems right (but I think you have swapped platformio.ini and the cpp file in your text), but since you mention both devices stream ok on the MQTT broker, the problem must be somewhere in Rhasspy.

So, therefore we might get some info from the Rhasspy log and settings

Good evening,

sorry for the late reply. Oh yes that’s right because I mixed up the platformio.ini and MatrixVoiceAudioServer.cpp :sweat_smile:

So my Rhasspy settings are as follows:
Pi 3B, Rhasspy 2.4.20 (Version 2.5 does not run stable with my streamers, not even in single mode. This is another problem which I have to solve separately)

Rhasspy Settings

Side ID are both entered.

The Hermes protocol is selected for Audio Input.

Here is an excerpt from my Advanced
{ "microphone": { "system": "hermes" }, "mqtt": { "enabled": true, "host": "192.168.1.60", "site_id": "matrixvoice71,matrixvoice75" }, "sounds": { "aplay": { "device": "default:CARD=ALSA" } }, "wake": { "pocketsphinx": { "threshold": 3.1622776601683794e-28 }, "snowboy": { "apply_frontend": true, "model": "snowboy/computer.umdl", "sensitivity": "0.4" }, "system": "snowboy" } }

And here the log when I call something, in this case the time.
[INFO:18040198] quart.serving: 192.168.1.60:59410 POST /api/text-to-speech 1.1 200 28 3307615 [DEBUG:18040164] InboxActor: -> stopped [DEBUG:18040144] EspeakSentenceSpeaker: speaking -> ready [DEBUG:18036967] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=ALSA'] [DEBUG:18036964] EspeakSentenceSpeaker: ready -> speaking [DEBUG:18036893] EspeakSentenceSpeaker: ['espeak', '-v', 'de', '--stdout', 'Es ist 19Uhr und 38 Minuten.'] [DEBUG:18036815] SnowboyWakeListener: loaded -> listening [DEBUG:18036813] DialogueManager: ready -> asleep [INFO:18036810] DialogueManager: Automatically listening for wake word [DEBUG:18036807] DialogueManager: handling -> ready [DEBUG:18036806] HermesMqtt: Published intent to hermes/intent/GetTime [DEBUG:18036804] WebSocketObserver: {"intent": {"name": "GetTime", "confidence": 1.0}, "entities": [], "text": "wie sp\u00e4t ist es", "raw_text": "wie sp\u00e4t ist es", "recognize_seconds": 0.004081788000007691, "tokens": ["wie", "sp\u00e4t", "ist", "es"], "raw_tokens": ["wie", "sp\u00e4t", "ist", "es"], "wav_seconds": 0.0, "transcribe_seconds": 0.0, "speech_confidence": 0.1493115175976181, "wakeId": "snowboy/computer.umdl", "siteId": "matrixvoice71,matrixvoice75", "slots": {}} [DEBUG:18036802] DialogueManager: recognizing -> handling [DEBUG:18036800] DialogueManager: {'intent': {'name': 'GetTime', 'confidence': 1.0}, 'entities': [], 'text': 'wie spät ist es', 'raw_text': 'wie spät ist es', 'recognize_seconds': 0.004081788000007691, 'tokens': ['wie', 'spät', 'ist', 'es'], 'raw_tokens': ['wie', 'spät', 'ist', 'es'], 'wav_seconds': 0.0, 'transcribe_seconds': 0.0, 'speech_confidence': 0.1493115175976181, 'wakeId': 'snowboy/computer.umdl', 'siteId': 'matrixvoice71,matrixvoice75'} [DEBUG:18036792] DialogueManager: decoding -> recognizing [DEBUG:18036789] DialogueManager: wie spät ist es (confidence=0.1493115175976181) [DEBUG:18036786] PocketsphinxDecoder: wie spät ist es [DEBUG:18036785] PocketsphinxDecoder: Transcription confidence: 0.1493115175976181 [DEBUG:18036783] PocketsphinxDecoder: Decoded WAV in 1.2460699081420898 second(s) [DEBUG:18035540] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=ALSA', '/usr/share/rhasspy/etc/wav/beep_lo.wav'] [DEBUG:18035534] PocketsphinxDecoder: rate=16000, width=2, channels=1. [DEBUG:18035530] DialogueManager: awake -> decoding [DEBUG:18035526] WebrtcvadCommandListener: listening -> loaded [DEBUG:18035520] WebrtcvadCommandListener: Voice command finished [DEBUG:18033679] WebrtcvadCommandListener: Voice command started [DEBUG:18033189] SnowboyWakeListener: listening -> loaded [DEBUG:18033178] WebrtcvadCommandListener: loaded -> listening [DEBUG:18033158] WebrtcvadCommandListener: Will timeout in 30 second(s) [DEBUG:18033157] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=ALSA', '/usr/share/rhasspy/etc/wav/beep_hi.wav'] [DEBUG:18033152] DialogueManager: asleep -> awake [DEBUG:18033151] DialogueManager: Awake! [DEBUG:18033149] SnowboyWakeListener: Hotword(s) detected: ['snowboy/computer.umdl']

As mentioned before, only the first entry of the Side ID is evaluated as microphone input. Everything after the comma is not considered. Let’s see if we can get Rhasspy to listen to the second streamer.
:smile:

Ah, you are using 2.4
Well, as you already have noticed, that does not work correctly.

Rhasspy is listening to the topic
hermes/audioServer/matrixvoice71,matrixvoice75/audioFrame.
This is not correct and should be

hermes/audioServer/matrixvoice71/audioFrame
and
hermes/audioServer/matrixvoice75/audioFrame

So there is the problem :slight_smile:

I suggest solving the problem with 2.5, which is able to listen correctly to multiple device.
Haven’t tried it however, but in 2.4 is will certainly not work

Mmmh then I guess I have no other choice than to update then I’ll get to work. Are you sure that it works with the 2.5 and two audio streamers?

I only have 1 but when I enter: “matrixvoice,server” in the siteid of Rhasspy, the matrixvoice keeps working.
Also, when I enter “server,matrixvoice”, the matrixvoice keeps working.

Yes it works with version 2.5.7, but it runs slower, is currently more error-prone in language recognition and just crashed. :exploding_head: 2.4.20 ran much more stable for me. Thanks for the help, now I will see if I can tune 2.5 to get the performance of 2.4. I wish you a nice evening.

Maybe first try and get the same results with only 1 sat. 2.5 runs fine on my server, but I do not run a pi3b

Good evening,

unfortunately it still does not work out the way I imagine it. It is also possible that I have configured something wrong because I do not see through some settings. At the moment it runs like this. Both streamers transmit with their own topic. So far good. If they are both active and in separate rooms they are not recognized by Rahsspy. If one is switched off, so only one is running, it will be recognized, even with the correct site ID.
I am not sure if I have entered the IDs correctly in the settings. There are different menus and so I entered them:
siteid: rhasspyserver,matrixvoice71,matrixvoice75
Wake Word= Satellite siteIds: matrixvoice71,matrixvoice75
Dialogue Managemt= Satellite siteIds: matrixvoice71,matrixvoice75

For all other menu items I have not stored siteIDs. Is this correct or do I have to add anything?

1 thing I notice is that you have 75 in your siteID, but 72 is your code
I do not think that is the cause, but still…

I think the siteID should be enough, you do not have to explicitly mentions them in the wakeword and dialogue.
Maybe I can come up with some device to test this as well

I changed the IDs from 72 to 75 afterwards. With 71 and 75 also the IDs on the streamers were uploaded. Sorry for the misunderstanding I forgot to mention.

No problem, I could not imagine this to be the issue :wink:

Hello romkabouter,

I am still trying to get the two MatrixVoice satellites to work together. Unfortunately still without success. There were in the meantime some update with many new features but I do not get them both to run. I have now the following software status:
Rhasspy = 2.5.9
ESP32-Rhasspy-Satellite = 7.4
Eclipse Mosquitto = 2.0.9
In the settings the SiteID = matrixvoice75,matrixvoice71
It is still not possible to run both MatrixVoice satellites at the same time. Somehow they interfere with each other even though the settings.ini entries are different. I have discovered the following in the mqtt log:
MatrixVoice satellite “matrixvoice71” is connected to Rhasspy. Everything works. Now I switch the MatrixVoice satellite “matrixvoice75” to it and I can see the following from the mqtt log:

1616859234: Client matrixvoice75 has exceeded timeout, disconnecting.
1616859236: Client c057ff22099b422cb59b9ebb838e126e disconnected.
1616859240: Client matrixvoice75Audio has exceeded timeout, disconnecting.
1616859372: Client matrixvoice71 has exceeded timeout, disconnecting.
1616859372: Client matrixvoice71Audio has exceeded timeout, disconnecting.
1616859408: Client auto-231E9760-08HG-C54F-8AE7-07143E49CF48 has exceeded timeout, disconnecting.
1616859438: Client auto-0D0243A-3855-F900-73F5-69EB187FE7 has exceeded timeout, disconnecting.
1616859438: Client auto-7A5A3CE-A65D-57D7-6767-7F20664EC4 has exceeded timeout, disconnecting.
1616859438: Client auto-87249A7-5480-3ORD-724F-B454AA5267 has exceeded timeout, disconnecting.
1616859438: Client auto-B7F437D-AJDD-A26B-0D1E-78F4277F2E has exceeded timeout, disconnecting.
1616859438: Client auto-F097035-8ES4-C00F-7143-5C337549AC has exceeded timeout, disconnecting.
1616859438: Client auto-6210301-MWSW-67E2-2F19-F1783B83AB has exceeded timeout, disconnecting.
1616859438: Client auto-BD711CF-7476-2F6D-F55D-BEEA5D51C5 has exceeded timeout, disconnecting.

And these entries repeat themselves over and over again. But it’s not that no device can be reached via mqtt. Everything works and responds, but unfortunately not Rhasspy. Only after I have disconnected the second satellite, Rhasspy is running again.

I can’t figure it out and I’m running out of ideas. Can there be a setting that I have overlooked? :thinking:

That is strange indeed. I do not own two device, so I cannot reproduce.
I will try with my MatrixVoice and M5Echo and rename them matrixvoice71 and 75. If that gives the same error, the error is probably in Rhasspy and not in the esp32 code.

Maybe try a shorter name? Maybe somehow satid’s get shortened internally leaving two “matrixvo” or something :smiley:
That is really a guess, so I will try with my two devices

I have now given the two matrix Voice satelitten another, smaller name and flashed them this time via ota. Unfortunately the same result. :pensive:
Can I somehow find out if it is due to rhasspy. In the log files I see I can not see anything.

I will change the MQTT server next and run it together with rhasspy in Docker. Maybe that will help.

The change to a new server unfortunately also had no success, the same behavior. :triumph:
I have recorded the logfile in MQTT.fx. There you can see quite well how the connections are constantly interrupted and reconnected:


First I have only one Matrix Voice in operation and everything is recognized by Rhasspy then I add the second one. If I then deactivate again Rahsspy reacts again. Or is that normal? :thinking: