External MQTTS Broker - 8883 using TLS?

Hi folks,

Does Rhasspy support MQTTS? For example, to connect to my external MQTT broker I need to specify the --capath option:

mosquitto_pub -h mqtt.mydomain.com -t “test” -m “hello world” -u username -p 8883 -P ‘secret’ --capath /etc/ssl/certs/

I have added /etc/ssl/certs to the certificate path in the webui, however none seems to get passed when looking at the logs.

Thanks!

I believe that in rhasspy 2.5 this function is not available because looking at the GitHub repository in the file TODO.md is present MQTT TLS support

Thanks for reporting this, it’s a bug indeed. I fixed this now in rhasspy-hermes 0.3.1. I’ll ask @synesthesiam to rebuild the Docker image with the fixed code.

@razzo04 MQTT TLS support has been added only recently, so the TODO file wasn’t updated. I removed the MQTT TLS support now in the TODO file.

1 Like

I’ve tried this with the docker image that was released a couple of days ago and it doesn’t seem to be working - not sure if the fix @koan mentions has landed yet?

My docker image just seems to hang - max out the cpu on my pi0. although i can see some sort of activity on the MQTT server with MQTT expklorer.

I have checked the documentation link but i am not 100% confident i have the right settings in the gui.
Should the ca_certs box be a cert or a folder? ‘certs’ implies folder but i’ve tried both.
Do we need anything in the certfile and keyfile? (I assume these 2 settings are for a client cert)
Should we be using .pem or .crt or does it not matter?

Would be interested to know if anyone has this working and what setting they have!

I verified this with the latest Docker image, TLS connections don’t work indeed. It seems the tls_ca_certs configuration option that you enter in the web interface doesn’t get picked up. If I put the following configuration manually in the “mqtt” section of profile.json, the connection works:

        "tls": {
            "enabled": true,
            "ca_certs": "/etc/ssl/private/rootCA.pem"
        },

You need to enter the path to the CA certificate itself, not a directory.

Can you confirm that this works for you with the manual change?

By the way, the certfile and keyfile are for client certificates indeed, we should probably clarify this in the web interface.

Wait, it doesn’t… I get this message repeatedly in the logs:

rhasspy    | [DEBUG:2020-07-04 13:09:26,871] rhasspyserver_hermes: Connecting to pi-red.home:8883 (retries: 0/10)

Looking at the code of rhasspy-server-hermes/__init__.py, it doesn’t seem to support TLS yet, am I seeing that right, @synesthesiam?

thanks @koan for looking at this. i’ll give it another go when you get the bottom of it!

Ah, that’s right! :man_facepalming: Everything else uses rhasspy-hermes, which has TLS support. The web server needed special MQTT handling, so it uses custom code and I forgot to add TLS.

I’ve added GitHub issue

This is another area where the encapsulation of so much that probably doesn’t need to in Rhasspy code of a small community and ever increasing bloat confuses me.

Why a universal 3rd party TLS is not employed when and where needed without need for code inclusion or support as many know bemuses me.

https://www.stunnel.org/

But hey.

Hello @synesthesiam and @koan ,

Thank you for the nice work (including support of TLS :slight_smile:)

I think there is a bug in type assignment for “version” when calling rhasspy-hermes :

With 2.5.1 and that configuration,

“tls”: {
“ca_certs”: “/etc/ssl/local/ca.pem”,
“certfile”: “/etc/ssl/local/certs/cert.pem”,
“enabled”: true,
“keyfile”: “/etc/ssl/private/local/key.pem”,
“version”: 2
}

rhasspyserver_hermes is happy :

[DEBUG:2020-07-18 22:44:02,726] rhasspyserver_hermes: Loading MQTT TLS settings
[DEBUG:2020-07-18 22:44:02,726] rhasspyserver_hermes: MQTT TLS enabled: {'ca_certs': '/etc/ssl/local/certs.pem', 'certfile': '/etc/ssl/local/certs/cert.pem', 'keyfile': '/etc/ssl/private/local/key.pem', 'cert_reqs': <VerifyMode.CERT_REQUIRED: 2>, 'tls_version': 2, 'ciphers': None}
[DEBUG:2020-07-18 22:44:02,727] rhasspyserver_hermes: Starting core
[DEBUG:2020-07-18 22:44:02,727] rhasspyserver_hermes: Connecting to host (retries: 0/10)
[DEBUG:2020-07-18 22:44:02,740] rhasspyserver_hermes: Connected to MQTT broker

but rhasspy-hermes isn’t :

Traceback (most recent call last):
File “main.py”, line 4, in
File “rhasspy/main.py”, line 56, in main
File “rhasspy-asr-kaldi-hermes/rhasspyasr_kaldi_hermes/main.py”, line 26, in main
File “rhasspy-asr-kaldi-hermes/rhasspyasr_kaldi_hermes/main.py”, line 231, in run_mqtt
File “rhasspy-hermes/rhasspyhermes/cli.py”, line 82, in connect
File “paho/mqtt/client.py”, line 815, in tls_set
File “ssl.py”, line 399, in new
TypeError: an integer is required (got type str)
2020-07-18 22:44:04,992 INFO exited: speech_to_text (exit status 255; not expected)

because it is called with ‘2’ (note the quotes that are added here whereas they are not added in rhasspyserver_hermes nor in the profile) that is considered as a string, instead of 2 as an integer :

rhasspyasr_kaldi_hermes: (…) tls_keyfile=‘key.pem’, tls_version=2,

Note that if I don’t specify “version” in the profile, it is rhasspy-server-hermes that cannot connect whereas rhasspy-hermes is happy :upside_down_face:.

Thanks for looking into this!

This turned out to be a subtle interaction between rhasspy-hermes, rhasspy-profile and rhasspy-server-hermes. I fixed this in:

This is published now as Rhasspy Hermes 0.3.2.

This should be published as Rhasspy Profile 0.3.1. @synesthesiam can you add a PyPI credential to the GitHub project and upload this version manually to PyPI?

This should be published as Rhasspy Server Hermes 2.5.3. @synesthesiam can you take a look at the credentials in GitHub? The package can’t be uploaded to PyPI because of an invalid token: https://github.com/rhasspy/rhasspy-server-hermes/runs/886629824?check_suite_focus=true

1 Like

Thank you @koan !

I’m using the deb package and I don’t know how this stuff (in particular rhasspyhermes/cli.py) will come to my local installation.
Do I have to wait for a new deb package ? Or is there some kind of dynamic update that can be managed by the currently installed package ?

It’s safest if you wait until the Rhasspy 2.5.2 package with these fixes is published, and you install it then. If you want to try it earlier, you can always apply these fixes manually, but I haven’t used the deb package yet so I don’t know exactly where the files are installed, probably /usr/lib/rhasspy or something like that.

Hum, I don’t know neither how the deb package is done, but it seems to me that the python scripts are compiled into some kind of binary as I cannot find “cli.py” nowhere on my system.

Do you know if there is an automated nightly / snapshot build of deb package ? it would be nice to quickly test bugfixes and to be able to quickly test / revert versions on “early testers” end-users systems.

Thank you for the 2.5.5 deb package :slight_smile:

I confirm that TLS for MQTT is all right now !

Cheers

Thanks for sorting this out. I can also confirm that it works with the 2.5.4 and 2.5.5 docker image.

using these settings in the profile

"mqtt": {
    "enabled": "true",
    "host": "mymqtthost.mynetwork.local",
    "password": "xyz123",
    "port": "8883",
    "site_id": "my_site_id",
    "tls": {
        "ca_certs": "/profiles/en/certs/ca.crt",
        "certfile": "${RHASSPY_PROFILE_DIR}/certs/clientcert.crt",
        "enabled": true,
        "keyfile": "${RHASSPY_PROFILE_DIR}/certs/clientkey.key"
    },
    "username": "my_username"
}

A couple of observations:-

  1. The settings cannot be set in the web gui, you have to edit the profile file directly, but i think this is known already.
  2. i can use ${RHASSPY_PROFILE_DIR} quite happily in the 2 client cert settings, but if i use it in the ca_certs settings i get a ‘File not found’ error, so i have to put in the full path.
  3. i am no real wiz with wireshark, but i can see TLS1.2 comms between a rhasspy instance on my pc and the MQTT box. My talent runs out at this point!

Because Matrix voice ceases to work as microphone from rhasspy 2.5 and above I’m not able to upgrade to 2.5.

If anybody has a similar problem that’s preventing him or her from upgrading to 2.5 I’ve configured a workaround using the bridge mode of Mosquitto. Basically you setup a plain Mosquitto server that forwards all messages to the encrypted one.

root@linux3:/etc/mosquitto# cat conf.d/bridge.conf
#See http://www.steves-internet-guide.com/mosquitto-bridge-configuration/ for more info
connection bridge-01
address mqtt.yourdomain:8883
bridge_cafile /etc/ssl/certs/some_root_cert_base64.crt
topic # out 0
topic # in 0
root@linux3:/etc/mosquitto#