Audio output from satellite. (In current config It outputs on basestation speaker)

First, since it’s my first post, I’d like to say HY!

Here is my situation,
I have a working rhasspy instance using docker on a proxmox VM, this will be my basestation (since it had more resources). I’d like to add a satellite (for my bedroom)
The satellite uses a PiZero2W and a Respeaker2Mic HAT.

I got the audio part working, (PiOs + Drivers).
I installed rhasspy using docker.

After configuration, I can wake up the satellite and issue commands, the commands are sent to the base and executed properly.

Now here is my issue, the response is played on the base station speakers instead of the satellite speaker.

As example:

  1. Saying the satellite wakeup word, I can hear the confirmation chime (from the satellite speaker)
  2. Asking the satellite “what’s the time?” I can see the request (web interface) and that it’s forwarded to the base station.
  3. Then the base station says what’s the time.

logs (from the example above) :

Satelite (sat1)

[DEBUG:2022-11-27 18:07:58,835] rhasspyserver_hermes: Sent 455 char(s) to websocket
[DEBUG:2022-11-27 18:07:58,827] rhasspyserver_hermes: <- NluIntent(input='what time is it', intent=Intent(intent_name='GetTime', confidence_score=1.0), site_id='sat1', id=None, slots=[], session_id='sat1-porcupine_raspberry-pi-58b452ce-75b8-47e8-885f-98db89e3fc4f', custom_data='porcupine_raspberry-pi', asr_tokens=[[AsrToken(value='what', confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value='time', confidence=1.0, range_start=5, range_end=9, time=None), AsrToken(value='is', confidence=1.0, range_start=10, range_end=12, time=None), AsrToken(value='it', confidence=1.0, range_start=13, range_end=15, time=None)]], asr_confidence=None, raw_input='what time is it', wakeword_id='porcupine_raspberry-pi', lang=None)
[DEBUG:2022-11-27 18:07:57,775] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/sat1/playBytes/c570916f-3e1a-4261-973e-9d71bf387087, id=a09e79a2-911c-4557-bafd-9948e5899964)
[DEBUG:2022-11-27 18:07:55,373] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/sat1/playBytes/e59a8e90-3f48-462b-9140-43d645f9b40b, id=a09e79a2-911c-4557-bafd-9948e5899964)
[DEBUG:2022-11-27 18:07:55,333] rhasspyserver_hermes: <- HotwordDetected(model_id='/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/pvporcupine/resources/keyword_files/raspberry-pi/porcupine_raspberry-pi.ppn', model_version='', model_type='personal', current_sensitivity=0.5, site_id='sat1', session_id=None, send_audio_captured=None, lang=None, custom_entities=None)

Base station (base)

[DEBUG:2022-11-27 18:08:01,325] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=6fb8c777-f22b-4292-8de9-2f0d6920b682)
[DEBUG:2022-11-27 18:07:59,249] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/base/playBytes/94c5f27e-31f4-4e8d-aa5e-e723aed4093c, id=6fb8c777-f22b-4292-8de9-2f0d6920b682)
[DEBUG:2022-11-27 18:07:58,769] rhasspyserver_hermes: Publishing 138 bytes(s) to hermes/tts/say
[DEBUG:2022-11-27 18:07:58,769] rhasspyserver_hermes: -> TtsSay(text='The time is 18 7', site_id='base', lang=None, id='94c5f27e-31f4-4e8d-aa5e-e723aed4093c', session_id='', volume=1.0)
[DEBUG:2022-11-27 18:07:58,766] rhasspyserver_hermes: TTS timeout will be 30 second(s)
[DEBUG:2022-11-27 18:07:58,750] rhasspyserver_hermes: Sent 367 char(s) to websocket
[DEBUG:2022-11-27 18:07:58,749] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/GetTime, id=656f3b47-d3c8-4279-904a-40ef1743b313)
[DEBUG:2022-11-27 18:07:58,697] rhasspyserver_hermes: Publishing 268 bytes(s) to hermes/nlu/query
[DEBUG:2022-11-27 18:07:58,697] rhasspyserver_hermes: -> NluQuery(input='what time is it', site_id='sat1', id='e18eb494-ada3-4d22-b497-83f77c6c7fe2', intent_filter=None, session_id='e18eb494-ada3-4d22-b497-83f77c6c7fe2', wakeword_id=None, lang=None, custom_data=None, asr_confidence=None, custom_entities=None)
[DEBUG:2022-11-27 18:07:57,717] rhasspyserver_hermes: Handling AsrTextCaptured (topic=hermes/asr/textCaptured, id=b7702cae-c118-4d36-9bed-33dad531b63b)
[DEBUG:2022-11-27 18:07:57,591] rhasspyserver_hermes: Publishing 71 bytes(s) to hermes/asr/stopListening
[DEBUG:2022-11-27 18:07:57,591] rhasspyserver_hermes: -> AsrStopListening(site_id='sat1', session_id='dca49ab0-f57d-4db1-a643-b76f6c10a0c4')
[DEBUG:2022-11-27 18:07:57,590] rhasspyserver_hermes: Sent 40540 byte(s) of WAV data
[DEBUG:2022-11-27 18:07:57,589] rhasspyserver_hermes: Publishing 178 bytes(s) to hermes/asr/startListening
[DEBUG:2022-11-27 18:07:57,588] rhasspyserver_hermes: -> AsrStartListening(site_id='sat1', session_id='dca49ab0-f57d-4db1-a643-b76f6c10a0c4', lang=None, stop_on_silence=False, send_audio_captured=True, wakeword_id=None, intent_filter=None)

And the configurations

Satelite

{
    "dialogue": {
        "satellite_site_ids": "sat1",
        "system": "rhasspy",
        "volume": "0.5"
    },
    "intent": {
        "remote": {
            "url": "http://192.168.100.171:12101/api/text-to-intent"
        },
        "system": "remote"
    },
    "microphone": {
        "arecord": {
            "device": "default:CARD=seeed2micvoicec"
        },
        "pyaudio": {
            "device": "0"
        },
        "system": "arecord"
    },
    "mqtt": {
        "site_id": "sat1"
    },
    "sounds": {
        "aplay": {
            "device": "default:CARD=seeed2micvoicec",
            "volume": "0.5"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "remote": {
            "url": "http://192.168.100.171:12101/api/speech-to-text"
        },
        "satellite_site_ids": "sat1",
        "system": "remote"
    },
    "text_to_speech": {
        "remote": {
            "url": "http://192.168.100.171:12101/api/text-to-speech"
        },
        "satellite_site_ids": "sat1",
        "system": "remote"
    },
    "wake": {
        "porcupine": {
            "keyword_path": "porcupine_raspberry-pi.ppn"
        },
        "system": "porcupine"
    }
}

Base Station

{
    "dialogue": {
        "satellite_site_ids": "base,sat1",
        "system": "rhasspy"
    },
    "intent": {
        "satellite_site_ids": "sat1",
        "system": "fsticuffs"
    },
    "microphone": {
        "arecord": {
            "device": "dmix:CARD=CameraB409241,DEV=0"
        },
        "pyaudio": {
            "device": "0",
            "siteId": "base,sat1"
        },
        "system": "pyaudio"
    },
    "mqtt": {
        "site_id": "base"
    },
    "sounds": {
        "aplay": {
            "device": "default"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "satellite_site_ids": "base,sat1",
        "system": "vosk"
    },
    "text_to_speech": {
        "larynx": {
            "vocoder": "vctk_small"
        },
        "satellite_site_ids": "base,sat1",
        "system": "larynx"
    },
    "wake": {
        "porcupine": {
            "keyword_path": "jarvis_linux.ppn"
        },
        "satellite_site_ids": "base,sat1",
        "system": "porcupine"
    }
}

I’ve tried scouring the forum and the dockumentation but I’ve been unable to find what’s wrong or misconfigured. :frowning: .

Thanks in advance for your help.

Hi and welcome,

It looks as though both your satellite and base are set to use their own Internal MQTT services, and you are using “remote http://” to call the modules on the base machine … as per the documentation. I can see how control is transferred to the base machine … but not how control (and the spoken reply) is sent back to the satellite. I guess the “remote Http” module appends the satellite’s IP address when calling the base ? Satellite siteId is given, but by definition Internal MQTT doesn’t know about any other machine.

Its been a while since I set up my own base+satellite system, so my memory is rusty about why I changed to use External MQTT to communicate between all my rhasspy machines - possibly because of this.
If you want to give this a try, it shouldn’t be too difficult.

  1. If you are using Home Assistant, install the “Mosquitto Broker” in the HA Add-ons.
  2. I chose to setup a separate HA user for rhasspy MQTT to use, but You can use an existing HA user in the next step.
  3. Change configuration on both your Base and Satellite use the same External MQTT. Everything else in your configuration files looks ready.

How are you going with this ?

@donburch first of all, thank you for the suggestion and the follow-up question. Sadly Work kept me busy as hell and, until today I didn’t get the chance to tinker.
Sadly I’m not bearing good news. Even after switching to MQTT, I get the same behavior.

[DEBUG:2022-12-05 13:10:04,089] 
rhasspyserver_hermes: <- NluIntent(input='what time is it',
                    intent=Intent(intent_name='GetTime', 
                    confidence_score=1.0), site_id='sat1', 
                    id=None, slots=[], 
                    session_id='sat1-bumblebee_raspberry-pi-65303184-0c7a-45e2-9e28-70d74ffc0450', 
                    custom_data='bumblebee_raspberry-pi', asr_tokens=[[AsrToken(value='what', confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value='time', confidence=1.0, range_start=5, range_end=9, time=None), AsrToken(value='is', confidence=1.0, range_start=10, range_end=12, time=None), AsrToken(value='it', confidence=1.0, range_start=13, range_end=15, time=None)]], 
                    asr_confidence=1.0, 
                    raw_input='what time is it', 
                    wakeword_id='bumblebee_raspberry-pi', 
                    lang=None)
[DEBUG:2022-12-05 13:10:00,099] 
rhasspyserver_hermes: <- HotwordDetected(model_id='/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/pvporcupine/resources/keyword_files/raspberry-pi/bumblebee_raspberry-pi.ppn', 
                    model_version='', 
                    model_type='personal', 
                    current_sensitivity=0.5, 
                    site_id='sat1', 
                    session_id=None, 
                    send_audio_captured=None, 
                    lang=None, 
                    custom_entities=None)

As I can see the request is sending the proper satellite ID (site_id=‘sat1’)
But the log on base seems to ignore it ( TtsSay(text=‘The time is 13 10’, site_id=‘base’,…) )

[DEBUG:2022-12-05 13:10:06,540] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=b0ee01d8-4b5c-4aab-a922-beedf7ab8475)
[DEBUG:2022-12-05 13:10:04,517] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/base/playBytes/a3200f3d-1043-4240-a98f-124d48fb85fe, id=b0ee01d8-4b5c-4aab-a922-beedf7ab8475)
[DEBUG:2022-12-05 13:10:04,080] rhasspyserver_hermes: Publishing 139 bytes(s) to hermes/tts/say
[DEBUG:2022-12-05 13:10:04,080] rhasspyserver_hermes: -> TtsSay(text='The time is 13 10', site_id='base', lang=None, id='a3200f3d-1043-4240-a98f-124d48fb85fe', session_id='', volume=1.0)
[DEBUG:2022-12-05 13:10:04,079] rhasspyserver_hermes: TTS timeout will be 30 second(s)
[DEBUG:2022-12-05 13:10:04,071] rhasspyserver_hermes: Sent 455 char(s) to websocke

Just in case here are the new settings

Base

{
    "dialogue": {
        "satellite_site_ids": "sat1",
        "system": "rhasspy"
    },
    "intent": {
        "satellite_site_ids": "sat1",
        "system": "fsticuffs"
    },
    "microphone": {
        "arecord": {
            "device": "dmix:CARD=CameraB409241,DEV=0"
        },
        "pyaudio": {
            "device": "0"
        },
        "system": "pyaudio"
    },
    "mqtt": {
        "enabled": "true",
        "host": "192.168.100.125",
        "password": "############",
        "site_id": "base",
        "username": "RhaspyMQTT"
    },
    "sounds": {
        "aplay": {
            "device": "default"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "satellite_site_ids": "sat1",
        "system": "vosk"
    },
    "text_to_speech": {
        "larynx": {
            "vocoder": "vctk_small"
        },
        "satellite_site_ids": "sat1",
        "system": "larynx"
    },
    "wake": {
        "porcupine": {
            "keyword_path": "jarvis_linux.ppn"
        },
        "satellite_site_ids": "sat1",
        "system": "porcupine"
    }
}

Satellite

{
    "dialogue": {
        "satellite_site_ids": "sat1",
        "system": "hermes",
        "volume": "0.5"
    },
    "intent": {
        "system": "hermes"
    },
    "microphone": {
        "arecord": {
            "device": "default:CARD=seeed2micvoicec",
            "udp_audio_port": "12202"
        },
        "pyaudio": {
            "device": "0"
        },
        "system": "arecord"
    },
    "mqtt": {
        "enabled": "true",
        "host": "192.168.100.125",
        "password": "############",
        "site_id": "sat1",
        "username": "RhaspyMQTT"
    },
    "sounds": {
        "aplay": {
            "device": "default:CARD=seeed2micvoicec",
            "volume": "0.5"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "satellite_site_ids": "sat1",
        "system": "hermes"
    },
    "text_to_speech": {
        "system": "hermes"
    },
    "wake": {
        "porcupine": {
            "keyword_path": "bumblebee_raspberry-pi.ppn",
            "udp_audio": "12202"
        },
        "system": "porcupine"
    }
}

THANKS again for your time :slight_smile:

L.E.
While scouring the interwebs for new ideas I found a post from you dated 2021 in which you say that your walkthrough for setting up a Rhasspi satellite has around 60 pages :slight_smile: ( link )
L.E. - 2
In a different post I even found the file :smiley: ( just for reference link )
I must say the document is very well written, very detailed, and full of useful information. Big Thanks for the effort.
The only significant difference that may have some impact is the fact that even on the satellite I’m using Rhasspy in a docker container. If time will allow me I’ll install the Debian version.
Also, I’ve manually added the MQTT port in the config - as per your instructions.

L.E. 3
I’ve noticed that by using the web interface - Speak button, I can get audio output on the satellite speaker. Here’s the satellite log :slight_smile:

[DEBUG:2022-12-05 15:50:32,273] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=b54b9851-2b6b-4099-9c2c-4e95c3ebc4a3)
[DEBUG:2022-12-05 15:50:30,581] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/satellite/playBytes/85f4a2d6-075c-496e-9928-60b551e3d992, id=b54b9851-2b6b-4099-9c2c-4e95c3ebc4a3)
[DEBUG:2022-12-05 15:50:30,519] rhasspyserver_hermes: Publishing 144 bytes(s) to hermes/tts/say
[DEBUG:2022-12-05 15:50:30,518] rhasspyserver_hermes: -> TtsSay(text='test on sattelite', site_id='satellite', lang=None, id='85f4a2d6-075c-496e-9928-60b551e3d992', session_id='', volume=1.0)
[DEBUG:2022-12-05 15:50:30,514] rhasspyserver_hermes: TTS timeout will be 30 second(s)

Even so, if I use the Recognize button I get the audio output on the base speaker. Again, some more satellite logs.

[DEBUG:2022-12-05 15:59:12,481] rhasspyserver_hermes: Publishing 23 bytes(s) to rhasspy/handle/toggleOn
[DEBUG:2022-12-05 15:59:12,480] rhasspyserver_hermes: -> HandleToggleOn(site_id='satellite')
[DEBUG:2022-12-05 15:59:12,470] rhasspyserver_hermes: Sent 372 char(s) to websocket
[DEBUG:2022-12-05 15:59:12,467] rhasspyserver_hermes: Sent 372 char(s) to websocket
[DEBUG:2022-12-05 15:59:12,460] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/GetTime, id=0e5a5f61-4d35-493c-99a4-a5fda005a6da)
[DEBUG:2022-12-05 15:59:12,459] rhasspyserver_hermes: <- NluIntent(input='what time is it', intent=Intent(intent_name='GetTime', confidence_score=1.0), site_id='satellite', id='08e1245a-9d76-4b42-b9f3-a7c6aeb5a961', slots=[], session_id='08e1245a-9d76-4b42-b9f3-a7c6aeb5a961', custom_data=None, asr_tokens=[[AsrToken(value='what', confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value='time', confidence=1.0, range_start=5, range_end=9, time=None), AsrToken(value='is', confidence=1.0, range_start=10, range_end=12, time=None), AsrToken(value='it', confidence=1.0, range_start=13, range_end=15, time=None)]], asr_confidence=None, raw_input='what time is it', wakeword_id=None, lang=None)
[DEBUG:2022-12-05 15:59:12,353] rhasspyserver_hermes: Publishing 273 bytes(s) to hermes/nlu/query
[DEBUG:2022-12-05 15:59:12,352] rhasspyserver_hermes: -> NluQuery(input='what time is it', site_id='satellite', id='08e1245a-9d76-4b42-b9f3-a7c6aeb5a961', intent_filter=None, session_id='08e1245a-9d76-4b42-b9f3-a7c6aeb5a961', wakeword_id=None, lang=None, custom_data=None, asr_confidence=None, custom_entities=None)
[DEBUG:2022-12-05 15:59:12,347] rhasspyserver_hermes: Publishing 23 bytes(s) to rhasspy/handle/toggleOff
[DEBUG:2022-12-05 15:59:12,345] rhasspyserver_hermes: -> HandleToggleOff(site_id='satellite')