Using external MQTT not working?

Help
1

I started using a single node setup (one Rhasspy, internal MQTT) and everything worked fine, so now I wanted to use a multi node setup (two Rhasspy, external MQTT - Mosquitto) but it seems like the MQTT connection isn’t working, but I do not really get what’s wrong :thinking:

I am using the Docker image rhasspy/rhasspy:2.5.11

Base Rhasspy

{
    // ...
    "mqtt": {
        "enabled": "true",
        "host": "<host>",
        "password": "<password>",
        "port": "1883",
        "site_id": "base",
        "username": "rhasspy"
    },
    // ...
}

Satellite Rhasspy

{
    // ...
    "mqtt": {          
        "enabled": "true",
        "host": "<host>",
        "password": "<password>",
        "port": "1883",  
        "site_id": "satellite",
        "username": "rhasspy"
    },
    // ...
}

I use the following command to see what these Rhasspy’s are doing:

$ mosquitto_sub -h <host> -p <port> -u rhasspy -P <password> -d -t "hermes/#" -t "rhasspy/#"

But… There is nothing… :man_shrugging: When I switch to get all data, by using -t "#" instead, I only see messages from my other applications (e.g. homeassistant) but none from my Rhasspy nodes.

So I had a look into the Mosquitto log and found some of these entries:

# ...
1647274202: Socket error on client <unknown>, disconnecting.
1647274202: Socket error on client <unknown>, disconnecting.
# ...

As all other applications work fine with the Mosquitto server, I expect these messages to be caused by Rhasspy :thinking:

I also had a look into logs of the Rhasspy nodes:

Base Rhasspy

Starting up...
Using virtual environment at /usr/lib/rhasspy/.venv
DEBUG:rhasspysupervisor:Namespace(debug=True, docker_compose='', local_mqtt_port=12183, mosquitto_path='mosquitto', profile='de', supervisord_conf='supervisord.conf', system_profiles=None, user_profiles=PosixPath(
'/data/profiles'))
DEBUG:rhasspysupervisor:Loading profile de (user=/data/profiles, system=None)
DEBUG:rhasspyprofile.profile:Loading /usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/profiles/de/profile.json
DEBUG:rhasspyprofile.profile:Loading /data/profiles/de/profile.json
DEBUG:rhasspysupervisor:Generating supervisord conf
DEBUG:rhasspysupervisor:Microphone disabled (system=dummy)
DEBUG:rhasspysupervisor:Speakers disabled (system=dummy)
DEBUG:rhasspysupervisor:Wake word disabled (system=dummy)
DEBUG:rhasspysupervisor:Intent handling disabled (system=dummy)
DEBUG:rhasspysupervisor:Dialogue disabled (system=dummy)
DEBUG:rhasspysupervisor:Wrote /data/profiles/de/supervisord.conf
Starting services...
Using virtual environment at /usr/lib/rhasspy/.venv
Using virtual environment at /usr/lib/rhasspy/.venv
2022-03-14 15:30:40,621 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to
avoid this message.
2022-03-14 15:30:40,626 INFO supervisord started with pid 181
[DEBUG:2022-03-14 15:30:41,055] rhasspyserver_hermes: Namespace(certfile=None, host='0.0.0.0', keyfile=None, local_mqtt_port=12183, log_format='[%(levelname)s:%(asctime)s] %(name)s: %(message)s', log_level='DEBUG'
, mqtt_host=None, mqtt_password=None, mqtt_port=None, mqtt_username=None, port=12101, profile='de', set=[], system_profiles=None, url_root='', user_profiles='/data/profiles', web_dir='/usr/lib/rhasspy/rhasspy-serv
er-hermes/web')
[DEBUG:2022-03-14 15:30:41,110] rhasspyprofile.profile: Loading /usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/profiles/de/profile.json
[DEBUG:2022-03-14 15:30:41,111] rhasspyprofile.profile: Loading /data/profiles/de/profile.json
[DEBUG:2022-03-14 15:30:41,111] rhasspyprofile.profile: Loading default profile settings from /usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/profiles/defaults.json
[DEBUG:2022-03-14 15:30:41,112] rhasspyserver_hermes: Starting core
[DEBUG:2022-03-14 15:30:41,112] rhasspyserver_hermes: Connecting to <host>:1883 (retries: 0/10)
[DEBUG:2022-03-14 15:30:41,114] rhasspyserver_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/hotword/+/detected
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/intent/#
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/nlu/intentNotRecognized
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to rhasspy/asr/base/+/audioCaptured
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/audioServer/base/audioSummary
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/hotword/+/detected
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/nlu/intentNotRecognized
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/audioServer/base/audioSummary
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2022-03-14 15:30:41,115] rhasspyserver_hermes: Subscribed to rhasspy/asr/base/+/audioCaptured
[DEBUG:2022-03-14 15:30:41,116] rhasspyserver_hermes: Subscribed to hermes/intent/#
[INFO:2022-03-14 15:30:41,116] rhasspyserver_hermes: Started
[DEBUG:2022-03-14 15:30:41,116] rhasspyserver_hermes: Starting web server at http://0.0.0.0:12101
Running on 0.0.0.0:12101 over http (CTRL + C to quit)
2022-03-14 15:30:41,631 INFO spawned: 'intent_recognition' with pid 209
2022-03-14 15:30:41,639 INFO spawned: 'speech_to_text' with pid 210
2022-03-14 15:30:41,645 INFO spawned: 'text_to_speech' with pid 211
# ...

Satellite Rhasspy

# ...
[DEBUG:2022-03-14 16:28:01,890] rhasspydialogue_hermes: Subscribed to hermes/nlu/intentNotRecognized
[DEBUG:2022-03-14 16:28:01,890] rhasspydialogue_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2022-03-14 16:28:01,890] rhasspydialogue_hermes: Subscribed to hermes/dialogueManager/startSession
[DEBUG:2022-03-14 16:28:01,891] rhasspydialogue_hermes: Subscribed to hermes/hotword/+/detected
[DEBUG:2022-03-14 16:28:01,937] rhasspywake_porcupine_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:28:01,938] rhasspywake_porcupine_hermes: Subscribed to hermes/hotword/toggleOff
[DEBUG:2022-03-14 16:28:01,938] rhasspywake_porcupine_hermes: Subscribed to hermes/audioServer/satellite/audioFrame
[DEBUG:2022-03-14 16:28:01,938] rhasspywake_porcupine_hermes: Subscribed to rhasspy/hotword/getHotwords
[DEBUG:2022-03-14 16:28:01,939] rhasspywake_porcupine_hermes: Subscribed to hermes/hotword/toggleOn
[DEBUG:2022-03-14 16:28:01,956] rhasspyspeakers_cli_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:28:01,957] rhasspyspeakers_cli_hermes: Subscribed to rhasspy/audioServer/setVolume
[DEBUG:2022-03-14 16:28:01,957] rhasspyspeakers_cli_hermes: Subscribed to hermes/audioServer/satellite/playBytes/#
[DEBUG:2022-03-14 16:28:01,957] rhasspyspeakers_cli_hermes: Subscribed to rhasspy/audioServer/getDevices
[DEBUG:2022-03-14 16:28:01,958] rhasspyspeakers_cli_hermes: Subscribed to hermes/audioServer/toggleOn
[DEBUG:2022-03-14 16:28:01,958] rhasspyspeakers_cli_hermes: Subscribed to hermes/audioServer/toggleOff
[DEBUG:2022-03-14 16:28:02,098] rhasspymicrophone_pyaudio_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:28:02,099] rhasspymicrophone_pyaudio_hermes: Subscribed to hermes/audioServer/toggleSummaryOn
[DEBUG:2022-03-14 16:28:02,099] rhasspymicrophone_pyaudio_hermes: Subscribed to hermes/audioServer/toggleSummaryOff
[DEBUG:2022-03-14 16:28:02,099] rhasspymicrophone_pyaudio_hermes: Subscribed to rhasspy/audioServer/getDevices
[DEBUG:2022-03-14 16:28:02,534] rhasspyremote_http_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:28:02,535] rhasspyremote_http_hermes: Subscribed to hermes/asr/stopListening
[DEBUG:2022-03-14 16:28:02,535] rhasspyremote_http_hermes: Subscribed to hermes/asr/toggleOff
[DEBUG:2022-03-14 16:28:02,535] rhasspyremote_http_hermes: Subscribed to hermes/audioServer/satellite/audioFrame
[DEBUG:2022-03-14 16:28:02,536] rhasspyremote_http_hermes: Subscribed to hermes/asr/toggleOn
[DEBUG:2022-03-14 16:28:02,536] rhasspyremote_http_hermes: Subscribed to hermes/asr/startListening
[DEBUG:2022-03-14 16:28:02,536] rhasspyremote_http_hermes: Subscribed to hermes/audioServer/satellite/+/audioSessionFrame
[DEBUG:2022-03-14 16:28:02,543] rhasspyremote_http_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:28:02,543] rhasspyremote_http_hermes: Subscribed to hermes/nlu/query
[DEBUG:2022-03-14 16:28:02,543] rhasspyremote_http_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:28:02,543] rhasspyremote_http_hermes: Subscribed to hermes/tts/say
[ERROR:2022-03-14 16:28:36,765] rhasspyserver_hermes:
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1313, in api_train
    result = await core.train()
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 462, in train
    timeout_seconds=self.training_timeout_seconds,
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 995, in publish_wait 
    result_awaitable, timeout=timeout_seconds
  File "/usr/lib/python3.7/asyncio/tasks.py", line 423, in wait_for                                          
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2022-03-14 16:30:01,620] rhasspyserver_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:30:01,620] rhasspyserver_hermes: Subscribed to hermes/hotword/+/detected
[DEBUG:2022-03-14 16:30:01,621] rhasspyserver_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2022-03-14 16:30:01,621] rhasspyserver_hermes: Subscribed to hermes/nlu/intentNotRecognized
[DEBUG:2022-03-14 16:30:01,622] rhasspyserver_hermes: Subscribed to hermes/intent/#
[DEBUG:2022-03-14 16:30:01,622] rhasspyserver_hermes: Subscribed to hermes/audioServer/satellite/audioSummary
[DEBUG:2022-03-14 16:30:01,623] rhasspyserver_hermes: Subscribed to rhasspy/asr/satellite/+/audioCaptured
[DEBUG:2022-03-14 16:30:01,899] rhasspydialogue_hermes: Connected to MQTT broker
[DEBUG:2022-03-14 16:30:01,899] rhasspydialogue_hermes: Subscribed to hermes/dialogueManager/endSession
[DEBUG:2022-03-14 16:30:01,900] rhasspydialogue_hermes: Subscribed to hermes/dialogueManager/continueSession
[DEBUG:2022-03-14 16:30:01,900] rhasspydialogue_hermes: Subscribed to hermes/intent/#
[DEBUG:2022-03-14 16:30:01,901] rhasspydialogue_hermes: Subscribed to hermes/tts/sayFinished
[DEBUG:2022-03-14 16:30:01,901] rhasspydialogue_hermes: Subscribed to hermes/audioServer/satellite/playFinished
[DEBUG:2022-03-14 16:30:01,901] rhasspydialogue_hermes: Subscribed to hermes/dialogueManager/configure
[DEBUG:2022-03-14 16:30:01,902] rhasspydialogue_hermes: Subscribed to hermes/nlu/intentNotRecognized
[DEBUG:2022-03-14 16:30:01,902] rhasspydialogue_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2022-03-14 16:30:01,902] rhasspydialogue_hermes: Subscribed to hermes/dialogueManager/startSession
[DEBUG:2022-03-14 16:30:01,903] rhasspydialogue_hermes: Subscribed to hermes/hotword/+/detected
# ...

Maybe someone can help me with getting this working :slightly_smiling_face:

2

Seems like all requests to the API endpoints (/api/...) take forever :thinking:

(I’ve used the “Copy as cURL” in Chrome to get the request)

$ time curl -vvv 'http://localhost:12101/api/microphones?_=1647270008538'   -H 'Connection: keep-alive'   -H 'sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="97", "Chromium";v="97"'   -H 'Accept: */*'   -H 'Content-Type: application/json'   -H 'X-Requested-With: XMLHttpRequest'   -H 'sec-ch-ua-mobile: ?0'   -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36'   -H 'sec-ch-ua-platform: "Linux"'   -H 'Sec-Fetch-Site: same-origin'   -H 'Sec-Fetch-Mode: cors'   -H 'Sec-Fetch-Dest: empty'   -H 'Referer: http://localhost:12101/settings'   -H 'Accept-Language: en-US,en;q=0.9'   --compressed
*   Trying 127.0.0.1:12101...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 12101 (#0)
> GET /api/microphones?_=1647270008538 HTTP/1.1
> Host: localhost:12101
> Accept-Encoding: deflate, gzip, br
> Connection: keep-alive
> sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="97", "Chromium";v="97"
> Accept: */*
> Content-Type: application/json
> X-Requested-With: XMLHttpRequest
> sec-ch-ua-mobile: ?0
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36
> sec-ch-ua-platform: "Linux"
> Sec-Fetch-Site: same-origin
> Sec-Fetch-Mode: cors
> Sec-Fetch-Dest: empty
> Referer: http://localhost:12101/settings
> Accept-Language: en-US,en;q=0.9
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 
< content-type: text/html; charset=utf-8
< content-length: 14
< date: Mon, 14 Mar 2022 19:56:52 GMT
< server: hypercorn-h11
< 
* Connection #0 to host localhost left intact
TimeoutError: 
real    0m30.017s
user    0m0.009s
sys     0m0.000s
3

Wasn’t able to get it working with an external MQTT broker (Hermes MQTT) so I switched to HTTP (Remote HTTP) which works fine :wink:

It’s really weird that you can find 1000+ posts mentioning the issue with external MQTT brokers, but there is no real solution for it :man_shrugging:

4

what docs/directions did you use to setup Rhasspy with remote HTTP?

5

Initially used Server with satellites - Remote MQTT broker and switched to Server with satellites - Remote HTTP.

6

Hi Jan, I am pleased you got your satellite working with remote HTTP.

For anyone else with the same issue, I also had a bit of trouble setting up External MQTT.
My issue was that it seemed that I needed to specify a MQTT username/password (which I note that you have added to your config files); but adding a login to the MQTT config created problems for the built-in ‘homeassistant’ user elsewhere in HA.

I ended up removing logins from the Mosquitto configuration; and instead creating Home Assistant config/users for ‘homeassistant’ and ‘Rhasspy’