Rebranded the Matrix Voice to esp32-rhasspy-satellite

Looks like it can not find git.

The solution is to reinstall Platform.io extension.
Something under Windows is weird with path to git.

Described in this forum
https://community.platformio.org/t/git-library-download/17000/10

1 Like

I now build and upload your code to a m5echo. plattform.io shows several errors or warnings while uploading but at the end it says “success”.
Is there an easy way for testing m5echo that it is working correctly?

I can access the webserver on the m5.
I have an external mqtt server, configured in the m5 and rhasspy server.
I have wakeword detection working with the android app.
I get no response from the m5.
When it is connected to my WiFi, the LED is blue.
When i push the button, nothing changes.
The rhasspy log shows nothing when pushing the m5 Button or speaking hot word.

what is the usual way using the m5? wake word in m5 settings is on remote. when it is connected to WiFi what should be the usual behavior? should i see something in the rhasspy log when pushing the button? do i have to speak something to get a log entry? can i look somewhere at the external mqtt server for some action? is there an topic to subscribe to get some information? at the moment i dont know where to start debugging…

Yes, check the broker on the hermes/audioServer/<siteID>/audioFrame topic.
Where siteID is what you called it.

If you can acces the webserver, the flashing worked :slight_smile:
Also, if you have it connected to the pc, open up the terminal and press the reset button.
Several messages should be printed there, like which state is being entered.

1 Like

That is the usual way, local hotword is not working.
What are your settings in Rhasspy?

Start by checking the terminal and check the hermes/audioServer/<siteID>/audioFrame topic.
That way, you can verify that audio stream is working or not.

Oh, you fast in responding :wink:
5 minutes ago i got it working. The problem was the wifi connection over my repeater downstairs. I dont know why i can arrive the website on the m5 but the udp stream was not working. When it connects over the access point in my Living room everything works fantastic, it is realy great, very fast response.

When i use the same siteId for the rhasspy app and the m5, i can use the app for looking at the intent recognition. This is great for testing.

This is my profile in rhasspy, maybe it helps someone…

{
“dialogue”: {
“satellite_site_ids”: “default”,
“system”: “rhasspy”
},
“intent”: {
“satellite_site_ids”: “default”,
“system”: “fsticuffs”
},
“microphone”: {
“system”: “hermes”
},
“mqtt”: {
“enabled”: “true”,
“host”: “192.168.0.30”,
“password”: “your password”,
“username”: “your username”
},
“sounds”: {
“system”: “hermes”
},
“speech_to_text”: {
“satellite_site_ids”: “default”,
“system”: “pocketsphinx”
},
“text_to_speech”: {
“satellite_site_ids”: “default”,
“system”: “espeak”
},
“wake”: {
“precise”: {
“sensitivity”: “0.9”,
“udp_audio”: “172.17.0.4:20000:default”
},
“satellite_site_ids”: “default”,
“system”: “precise”
}
}

Ok great.

If you want Rhasspy to act as a server and the M5 as the one and only satellite, you can set the siteid of Rhasspy and the M5 the same

If you need more satellites, Satellite SiteID’;s should be a comma separated list of siteID’s in all Rhasspy server fields.
Rhasspy siteID should be called different, for which I suggest “server” or something like that.

1 Like

I just got my audio kit and I managed to install this and get it working. I do have a few questions thought:

  1. What does each button do? I did not manage to find an explanation for that yet
  2. Is there a way to disable led D1, D2 and D3, or reduce the brightness? I use the device in the same room I sleep and it is waaay to bright
  3. I there a (at least somewhat) easy way to record audio with this to train my hotword model on? Right now it is a bit hit and miss. I know this has not much to do with this project for ESP32 but I thought others might have ran into this issue before.

I do not own an Audio Kit and did not create that device.
The best thing you can do is create an issue on github and tag de developer :slight_smile:

With regards to point 3, that is indeed beyond the scope of this project there might be other projects which record audio with the Audio Kit.

Hi all,

I am new to this forum but extremely enthusiastic about this project. I have three Matrixvoice devices and used Snips in the past. As Snips is no longer available to the public, I wanted to test Rhasspy. I cloned romkabouter’s repo and were able to setup Rhasspy (as docker container) and uploaded the project to two of the (Matrixvoice) satellites (aim is to use them without a raspberry). I have two questions:

  1. I want to use little speakers and print a case for every satellite. Is it possible to output audio from the satellites by using the speaker output? I have seen there is a variable in “ESP32-Rhasspy-Satellite/tree/master/PlatformIO/src/device.h” file and I published the setting via MQTT to the satellite: {“amp_output”:“0”}. Unfortunaltely, I only hear strange noise when hotword is detected (either by amp or jack). What could I do to have a nice sound output?

  2. Did anyone use Matrixvoice satellites with speakers and has a case (3D) printed?

Thanks in advance. All the best,
Flo

1: Yes, there is a setting for speaker/headphone. Use speaker. You can set it in the webUI as well. What is the samplerate of your audio?
2: I do not :wink:

Hi @romkabouter,

  1. I have found this setting and I hear an output. Unfortunately it is “only” noise. Where do I find the samplerate? I do not output any own messages, just the “hotword detection” is recognized. I have Rhasspy as docker container and Matrix Voice satellites (without raspberries). What do I have to choose as “Text to speech”? Is there any other setting to be made?

  2. I’ll build an own case and share it with you.

Regards,
Flo

You can find the files under “Sounds”, but the project should resample to the format the Matrix Voice supports.
You can set any text to speech, but Audio Playing should be HermesMQTT

Also, each satId (different per device) should be set on all the “Satellite siteIds” boxes.
Server should be a different siteID (base/server/whatever)
But when you hear the hotword detection you probably already have set that correctly

Yes, please share your case!

Hi,
after setting up my M5 I get this Errors:

Blockquote
Guru Meditation Error: Core 0 panic’ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC : 0x400efaa1 PS : 0x00060f30 A0 : 0x800d2338 A1 : 0x3ffeb340
A2 : 0x00000000 A3 : 0x3ffeb3b0 A4 : 0x00000200 A5 : 0x3ffeb37c
A6 : 0x00000064 A7 : 0x00000000 A8 : 0x00000001 A9 : 0x0000000b
A10 : 0x3ffb4940 A11 : 0x3ffeb35f A12 : 0x00000000 A13 : 0x00000008
A14 : 0x00060f23 A15 : 0x00000000 SAR : 0x00000000 EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000014 LBEG : 0x00000000 LEND : 0x00000000 LCOUNT : 0x00000000
ELF file SHA256: 0000000000000000
Backtrace: 0x400efaa1:0x3ffeb340 0x400d2335:0x3ffeb370 0x400d3658:0x3ffeb3a0 0x40089f82:0x3ffeb820
#0 0x400efaa1:0x3ffeb340 in i2s_read at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/driver/i2s.c:1352
#1 0x400d2335:0x3ffeb370 in M5AtomEcho::readAudio(unsigned char*, unsigned int) at src/StateMachine.hpp:19
#2 0x400d3658:0x3ffeb3a0 in I2Stask(void*) at src/StateMachine.hpp:19
#3 0x40089f82:0x3ffeb820 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)
Rebooting…

Any Ideas?

That is a strange error, are you using the latest release?

Yes its the last Version: 7.4

Because the ESP32 runs wireless networking on Core0 it usually wise to have little code running there as if networking + custom code is too much you garner the above errors.

Would seem like you have too much on core0 at a certain point.

What software are you using because in the code there is:

M5AtomEcho::readAudio(uint8_t *data, size_t size) {

Is see esp32-arduino-lib-builder, what is that?

I don’t think this is the issue. The asyncMQTT is running on core1 so only the stream task is extra on core0. It has a priority of 3, this should not cause this issue.