Announcing to multiple satelites using MQTT

I am sure I saw discussion on making an announcement - getting all satellites to say a specified phrase - but haven’t been able to find it or think what other keywords to use :frowning:

Is this something other people are doing with Rhasspy ?

I have found suggestions using http, but I am using external (HA addon) MQTT server and node-RED automations. I have basically copied working Template and MQTT Out nodes, and altered to call MQTT-out with:

topic of hermes/tts/say
msg.payload of {“text”:“Hello. This is a test”,“siteId”:“base, sat-0, sat-1, sat-2, sat-3B”}

It looks Ok in node-RED debugger, but no sound.

Looking at other forum posts about the MQTT playByes topic, I realised that the text I want to say needs to be parsed through Text-to-Speech (Larynx running on my server named ‘base’) then sent to the satellites … yet only one siteId is being given. As you can see above I added ‘base’, but still no difference.

I would appreciate any assistance.

I think a comma separated list does not work. Most probably you need to publish the text to tts/say for every satellite. If you are familiar to Node Red, that should not be to hard I guess.

OK, That was it, thanks !
I probably got confused with Rhasspy using a comma separated list for Satellite siteIds, and assumed it was a MQTT thing.

I got side-tracked thinking that the satellite has the Audio Out but is not running the Text-To-Speech module; forgetting that it calls the server for all those other modules. My bad. :frowning:

So far so good … but there is a noticeable delay between each satellite starting to speak.

I guess that each satellite in turn is calling my base station to translate the same text into audio before playing the audio ? Would it be feasible to translate the text to speech once, and then send to multiple satellites ?

Looking at the documentation, I see that hermes/tts/say does the TTS and then automatically calls playBytes to speak the message.
I have also seen other posts about using hermes/audioServer//playBytes/ to send a WAV file to Rhasspy.
BUT I didn’t notice any MQTT topic to do the Text-To-Speech and return a WAV file - which is essentially the first half of hermes/tts/say ? Is hermes/tts/say actually calling an undocumented routine to return the WAV file, which it then feeds to the playBytes call ?

I can always work around by recording my own messages into WAV files, and using playBytes to speak them; but this (a) limits the messages which will be available, (b) provides inconsistent voices coming from Rhasspy, and (c) is more work for me to do :wink:

No, it says so in the reference here: Reference - Rhasspy

tts/say automatically sends playBytes. How it creates that wave data is just an internal method and indeed not specified in the documentation. Just like a lot of other internal methods are not specifically listed.
What you could do is create a flow in Node Red, and send the tts to an virtual sat “announce”.
Then, in Node Red listen to the topic hermes/audioServer/announce/playBytes/# and when a message comes in, copy that message to hermes/audioServer/base/playBytes, hermes/audioServer/sat-0/playBytes etc.

That way, there should be hardly any delay.

Thank you for confirming that there is not an existing undocumented MQTT topic I can use easily.

And even more for suggesting that work-around :slight_smile:
I am still very new to node-RED, but you you made it sound easy !

Try this flow

[{"id":"ba2b0ba8d821941f","type":"tab","label":"Announce","disabled":false,"info":"","env":[]},{"id":"aea796dba6c75f14","type":"mqtt in","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/announce/playBytes/#","qos":"2","datatype":"buffer","broker":"dbcaffe2.f8f59","nl":false,"rap":true,"rh":0,"inputs":0,"x":250,"y":160,"wires":[["e89a5061723859e5","f6d9d9900eee380b"]]},{"id":"e89a5061723859e5","type":"mqtt out","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/server/playBytes","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"dbcaffe2.f8f59","x":630,"y":80,"wires":[]},{"id":"a9e1c761a8523989","type":"mqtt out","z":"ba2b0ba8d821941f","name":"","topic":"hermes/tts/say","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"dbcaffe2.f8f59","x":320,"y":320,"wires":[]},{"id":"9b1cd23dd29d83ca","type":"debug","z":"ba2b0ba8d821941f","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":610,"y":440,"wires":[]},{"id":"5e491ede9a3a52f3","type":"inject","z":"ba2b0ba8d821941f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"text\":\"Dit is een test\",\"siteId\": \"announce\"}","payloadType":"json","x":130,"y":320,"wires":[["a9e1c761a8523989"]],"icon":"node-red/arrow-in.png"},{"id":"f6d9d9900eee380b","type":"mqtt out","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/office/playBytes","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"dbcaffe2.f8f59","x":590,"y":240,"wires":[]},{"id":"60a4ef15c9e1300e","type":"mqtt in","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/office/playBytes/#","qos":"2","datatype":"buffer","broker":"dbcaffe2.f8f59","nl":false,"rap":true,"rh":0,"inputs":0,"x":257,"y":511,"wires":[["9b1cd23dd29d83ca"]]},{"id":"858a1bf3effe97bd","type":"mqtt in","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/server/playBytes/#","qos":"2","datatype":"buffer","broker":"dbcaffe2.f8f59","nl":false,"rap":true,"rh":0,"inputs":0,"x":210,"y":420,"wires":[["9b1cd23dd29d83ca"]]},{"id":"dbcaffe2.f8f59","type":"mqtt-broker","name":"HA MQTT","broker":"192.168.43.54","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]

There is an inject button sending a test to tts/say with siteId announce
The fake siteId announce must be added to the siteId’s field on TextToSpeech in Rhasspy settings

The audio is then send to announce/playBytes and copied to server and office. The debyg shows no delay at all.
You need to change some things for the correct MQTT connection and siteId’s, but that should be no problem

Thanks for that flow.
For anyone else following this later, I also had to append a requestID onto the playBytes topics.

In my open-plan kitchen/living room the two satellites (a RasPi Zero with reSpeaker 2-mic HAT, and a RasPi 3A with reSpeaker 4-mic HAT) play at the same time - but not simultaneously. Result is a bit confusing. It also doesn’t help the WAF when she’s not expecting an announcement :wink:

So (1) I will try to prefix the announcement with a chime.WAV file (or siren if appropriate), and (2) try delaying the start of the second message so they aren’t playing at once.

Wow, that really turned it on its head ! Interesting interplay of multiple threads (through the node-RED flows) but threads sometimes overwriting msg. variables.

In practice the logic and processing effort to play chime.wav then message in turn to 3 satellites … would have been better to just use tts/say 3 times. Even storing the chime payload in a global variable doesn’t help much because the message payload has to be saved and restored.

I have ended up adding a 1 second pause between each satellite when copying from announce; and a 3 second delay after calling playBytes for the chime.wav file, and calling tts/say for the announcement text. Everything seems to be coming out in correct order … though I think it’s actually limited by the speed my RasPi 4 is processing the node-RED flow.

OK I have tidied it up and added a couple of features:

  • plays a chime.wav before any announcement
  • uses a list of Satellite IDs to send announcement to - easy to update for any number of satellites
  • adjustable delays in case you have multiple satellites within audio range - to reduce chance that humans will be confused by hearing two nearby units at the same time

[{"id":"ba2b0ba8d821941f","type":"tab","label":"Announce","disabled":false,"info":"","env":[]},{"id":"aea796dba6c75f14","type":"mqtt in","z":"ba2b0ba8d821941f","name":"capture from announce","topic":"hermes/audioServer/announce/playBytes/#","qos":"2","datatype":"buffer","broker":"3e24a26f3fe6c572","nl":false,"rap":true,"rh":0,"inputs":0,"x":120,"y":400,"wires":[["ce246cc18ef5c971"]]},{"id":"9b1cd23dd29d83ca","type":"debug","z":"ba2b0ba8d821941f","name":"message to satellite","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":640,"y":720,"wires":[]},{"id":"5e491ede9a3a52f3","type":"inject","z":"ba2b0ba8d821941f","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"this is a test","payloadType":"str","x":110,"y":140,"wires":[["4e091364f9e43010"]],"icon":"node-red/arrow-in.png"},{"id":"60a4ef15c9e1300e","type":"mqtt in","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/sat-3B/playBytes/#","qos":"2","datatype":"buffer","broker":"3e24a26f3fe6c572","nl":false,"rap":true,"rh":0,"inputs":0,"x":210,"y":760,"wires":[["9b1cd23dd29d83ca"]]},{"id":"858a1bf3effe97bd","type":"mqtt in","z":"ba2b0ba8d821941f","name":"","topic":"hermes/audioServer/sat-0/playBytes/#","qos":"2","datatype":"buffer","broker":"3e24a26f3fe6c572","nl":false,"rap":true,"rh":0,"inputs":0,"x":210,"y":700,"wires":[["9b1cd23dd29d83ca"]]},{"id":"db0189b52e760b58","type":"comment","z":"ba2b0ba8d821941f","name":"Make an announcement","info":"The MQTT topic hermes/tts/say converts a text message to a WAV file and automatically uses playBytes to play it HOWEVER sometimes we want a message to be converted by tts/say once and relayed to multiple satellites simultaneously. \n\nThis method uses a virtual rhasspy satellite called \"announce\" to do the tts, and the second sequence capturtes the playBytes from \"announce\" and redirects it to the real satellites. \n\nBefore using, add \"announce\" to the list of Satellite IDs in the Text to Speech section on your Rhasspy base station. \nI also found a chime.wav file, which I copied to the media folder accessible through the HA samba add-on. \n\nComment:\nThis technique does significantly reduce the latency - but the announcements are still not sufficiently simultaneous (and probably made worse by my using different hardware for my satellites).  In practice, multiple satellites in my open-plan livingroom-Kitchen area resulted in humans hearing multiple satellites speaking at once, and having trouble understanding the message. \n\nI also found that a sudden unexpected loud voice could be a bit frightening, so decided to prefix all announcements with a chime sound.  \n\nTo call:\nI intend to link call this flow, passing in the message text in msg.payload.  Nothing will be passed back (largely because I expect that some threads will still be running)\n\nTiming:\nTrying to reduce overlapping speech yet minimise delays has resulted in me adding and fine tuning dalays.  I have placed my open-plan livingroom and kitchen at oposite ends of the list of satellites, whereas speech in the study can happily overlap. \nI don't mind the chimes overlapping, since they are only to get humans' attention. \n\nI suspect the timings are very dependant on the assortment of hardware used for base and satellites, and of course having multiple Rhasspy satellites within audio range.  You will probably need to fiddle with this flow to get the best performance for you. \n\nThe flow does:\n - save the message text (since payload gets used for everything)\n - play the chime.wav file to all the satellites.  \n - delay to allow all chimes to play before we overwrite the msg.payload variable with the announcement\n - say the announcement message to the \"announce\" satelliite.\n - the second sequence captures the MQTT announce/playBytes topic, and copies the payload to each real satellite in turn. A tiny delay is added to ensure that two satellites withina audio range aren't talking at once. \n\nSince I currently only have 3 satellites a hardcoded nodes are appropriate. If you have more satellites, a different technique can loop through a list of satellites. \n","x":120,"y":60,"wires":[]},{"id":"6b471e9f113fe359","type":"comment","z":"ba2b0ba8d821941f","name":"copy announcement to all satellites (almost) simultaneously","info":"The announcement was converted from text to a WAV file sent to the virtual \"announce\" device. \nWe capture the playBytes sent to \"announce\" and copy it to playBytes for the actual satellites.\n\nThis sequence uses a list of satellite IDs (defined in the second node), making it easy to add as many satellites as you wish by simply updating the list. \n\nI have added a slight pause so the satellites aren't all talking at once. In practice, I think the delays only need to be long enough that the CPU which node-RED is running on queues up the multiple threads in the desired sequence. Consequently you will want to adjust to find values that suit your own hardware and layout. \n","x":230,"y":360,"wires":[]},{"id":"2030c88ef5f29f88","type":"comment","z":"ba2b0ba8d821941f","name":"debugging - what was received by satellite","info":"This sequence allows us to see what was actually sent to the specified satellites' playBytes topics ... only for debugging.\n","x":180,"y":660,"wires":[]},{"id":"ab8b38455fa77e9c","type":"debug","z":"ba2b0ba8d821941f","name":"message to \"announce\"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":790,"y":200,"wires":[]},{"id":"072fb3d5db4407c6","type":"template","z":"ba2b0ba8d821941f","name":"message to \"announce\"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"{\n  \"text\": \"{{message}}\",\n  \"siteId\": \"announce\"\n}","output":"json","x":450,"y":240,"wires":[["be58e7e8bbab9c70","ab8b38455fa77e9c"]]},{"id":"be58e7e8bbab9c70","type":"mqtt out","z":"ba2b0ba8d821941f","name":"announce to all satellites","topic":"hermes/tts/say","qos":"2","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"3e24a26f3fe6c572","x":730,"y":240,"wires":[]},{"id":"586c65bfc8e2d971","type":"mqtt out","z":"ba2b0ba8d821941f","name":"send chime to all satellites","topic":"hermes/audioServer/announce/playBytes/chime","qos":"1","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"3e24a26f3fe6c572","x":730,"y":160,"wires":[]},{"id":"b5855a16dbe22493","type":"comment","z":"ba2b0ba8d821941f","name":"chime before announcement","info":"I found a free chime.wav on the internet and copied it to my HA server; in my case to the \"media\" folder exposed by the HA Samba server.  \n","x":440,"y":120,"wires":[]},{"id":"b8d914a8fd4e21bd","type":"file in","z":"ba2b0ba8d821941f","name":"","filename":"../media/chime.wav","format":"","chunk":false,"sendError":false,"encoding":"none","allProps":false,"x":430,"y":160,"wires":[["586c65bfc8e2d971","ab8b38455fa77e9c","482324372441f92b"]]},{"id":"482324372441f92b","type":"stoptimer","z":"ba2b0ba8d821941f","duration":"2","units":"Second","payloadtype":"num","payloadval":"0","name":"delay till chimes have finished","x":250,"y":240,"wires":[["072fb3d5db4407c6"],[]]},{"id":"c270845faec29dd8","type":"split","z":"ba2b0ba8d821941f","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":690,"y":420,"wires":[["65c1379ce9d8da33","7e7f2f651fbd2164"]]},{"id":"96d9aca3cb87ac08","type":"delay","z":"ba2b0ba8d821941f","name":"send at 1 second intervals","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"5","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":410,"y":540,"wires":[["db96e59311df63a3"]]},{"id":"1b09bca4c164832b","type":"mqtt out","z":"ba2b0ba8d821941f","name":"","topic":"","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"3e24a26f3fe6c572","x":950,"y":500,"wires":[]},{"id":"ce246cc18ef5c971","type":"change","z":"ba2b0ba8d821941f","name":"save message, load list of satellites","rules":[{"t":"set","p":"message","pt":"msg","to":"payload","tot":"msg"},{"t":"set","p":"payload","pt":"msg","to":"[\"sat-1\",\"sat-3B\",\"sat-0\"]","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":420,"y":400,"wires":[["c270845faec29dd8","ed57b9d034699070"]]},{"id":"0a1686e7045538a8","type":"change","z":"ba2b0ba8d821941f","name":"set  payload","rules":[{"t":"set","p":"payload","pt":"msg","to":"message","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":790,"y":500,"wires":[["1b09bca4c164832b"]]},{"id":"ed57b9d034699070","type":"debug","z":"ba2b0ba8d821941f","name":"before split","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":770,"y":360,"wires":[]},{"id":"7e7f2f651fbd2164","type":"debug","z":"ba2b0ba8d821941f","name":"after split","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":860,"y":420,"wires":[]},{"id":"db96e59311df63a3","type":"template","z":"ba2b0ba8d821941f","name":"set topic","field":"topic","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"hermes/audioServer/{{payload}}/playBytes/12340","output":"str","x":620,"y":500,"wires":[["0a1686e7045538a8"]]},{"id":"50905872f501126f","type":"link in","z":"ba2b0ba8d821941f","name":"Rhasspy_announce","links":[],"x":35,"y":100,"wires":[["4e091364f9e43010"]]},{"id":"4e091364f9e43010","type":"change","z":"ba2b0ba8d821941f","name":"save the message","rules":[{"t":"set","p":"message","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":190,"y":100,"wires":[["b8d914a8fd4e21bd"]]},{"id":"65c1379ce9d8da33","type":"switch","z":"ba2b0ba8d821941f","name":"delay if not chimes","property":"topic","propertyType":"msg","rules":[{"t":"eq","v":"hermes/audioServer/announce/playBytes/chime","vt":"str"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":190,"y":500,"wires":[["db96e59311df63a3"],["96d9aca3cb87ac08"]]},{"id":"3a612f384b74989e","type":"comment","z":"ba2b0ba8d821941f","name":"process one request at a time","info":"","x":200,"y":460,"wires":[]},{"id":"3e24a26f3fe6c572","type":"mqtt-broker","name":"HA MQTT broker add-on","broker":"192.168.1.98","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]
1 Like