Rhasspy - ESP32 Satellite base station setup

Help
1

So im just getting in Rhasspy, and im trying to use a M5 Atom Echo with “esp32-rhasspy-satellite” image.

The web server for the esp32 gives me just options to connect MQTT and “remote hotword detection”. I presume that means it just send its audio via MQTT to the base station for processing.

However nothing ive done so far has been sent to the base station.

image
image581×708 29 KB

These are my settings on the Rhasspy server (HA Addon)

Any idea where im going wrong?

2

Yes indeed, What are your settings for MQTT on esp32 sat?
And are they the same as the server?

3

So im using the Moquitto Broker Addon from HA. Credentials im using are just my HA user, so both the Rhasspy Addon and Atom Echo are pointed at that broker with the same credentials.

Is there an MQTT topic i can sub to with a client so i can see whats happening from that satellite? Maybe i should test with the button first but tbh im struggling to even get this working, although on Rhasspy when i press the button on the echo, i see “Intent unknown” appear on the test page where you would normally see the command that was last spoken.

4

Here is my config from Rhasspy.

{
    "dialogue": {
        "satellite_site_ids": "satellite",
        "system": "rhasspy"
    },
    "handle": {
        "satellite_site_ids": "satellite",
        "system": "hass"
    },
    "home_assistant": {
        "access_token": "LONGLIVETOKEN",
        "handle_type": "event",
        "url": "http://homeassistant.local:8123/"
    },
    "intent": {
        "satellite_site_ids": "satellite",
        "system": "fsticuffs"
    },
    "microphone": {
        "system": "hermes"
    },
    "mqtt": {
        "enabled": "true",
        "host": "192.168.0.6",
        "password": "XXXXXXXXXX",
        "site_id": "server",
        "username": "Tom"
    },
    "sounds": {
        "system": "hermes"
    },
    "speech_to_text": {
        "satellite_site_ids": "satellite",
        "system": "kaldi"
    },
    "text_to_speech": {
        "nanotts": {
            "language": "en-GB",
            "volume": "1"
        },
        "satellite_site_ids": "satellite",
        "system": "nanotts"
    },
    "wake": {
        "pocketsphinx": {
            "keyphrase": "mother"
        },
        "porcupine": {
            "keyword_path": "terminator_linux.ppn"
        },
        "satellite_site_ids": "satellite",
        "system": "porcupine"
    }
}
5

Your profile looks good.

You can sub to hermes/audioServer/<siteId>/audioFrame, there should be loads of messages.
Those are all small wave chunks send to the server, that topic should also be in the log of the server

Reference here: Reference - Rhasspy
Wiki here: Home · rhasspy/rhasspy Wiki · GitHub

6

So i can see the audio coming over the MQTT topic but im not sure Rhasspy is picking it up for some reason. Ive got this in the logs after trying to use button press, and then turning on hotword. No speech is detected.

[DEBUG:2022-08-14 23:08:51,782] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:08:51,779] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:08:29,718] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:08:29,718] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:07:59,091] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:07:59,090] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:06:05,752] rhasspyserver_hermes: Sent 262 char(s) to websocket
[DEBUG:2022-08-14 23:06:05,751] rhasspyserver_hermes: Sent 262 char(s) to websocket
[ERROR:2022-08-14 23:05:54,644] rhasspyserver_hermes: 
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1624, in api_play_recording
    await core.play_wav_data(wav_bytes)
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 755, in play_wav_data
    handle_finished(), messages(), message_types
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 995, in publish_wait
    result_awaitable, timeout=timeout_seconds
  File "/usr/lib/python3.7/asyncio/tasks.py", line 423, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2022-08-14 23:05:54,644] rhasspyserver_hermes: Publishing 43 bytes(s) to hermes/asr/toggleOn
[DEBUG:2022-08-14 23:05:54,644] rhasspyserver_hermes: -> AsrToggleOn(site_id='server', reason=<AsrToggleReason.PLAY_AUDIO: 'playAudio'>)
[DEBUG:2022-08-14 23:05:54,643] rhasspyserver_hermes: Publishing 43 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2022-08-14 23:05:54,643] rhasspyserver_hermes: -> HotwordToggleOn(site_id='server', reason=<HotwordToggleReason.PLAY_AUDIO: 'playAudio'>)
7

No, if you press that on the server, it will not listen to the satellite.

The M5 Atom Echo has a very low sound level sadly, to check what is recorded you can run this python script on a computer. You have to make some adjustments like server and such but i will record a couple of seconds from the audio and save it to a wave file. That way you can verify of the device is correctly sending data.