Pause between TTS spoken sentences or MQTT messages

I am attempting to have a lamp turn on when I wake up and my rhasspy speaker tell me the weather and my calendar events. I have everything worked out except I am trying to add a pause to in the sentence I create to send over to TTS for the speaker to say.

this is the chain of events and how I want them to happen:
turn on lamp (MQTT message to home assistant)
Pause 2 secoinds
Speakers starts and says
“Good morning”
pause .5 seconds
“the weather is ####”
pause for 1 second
“your calendar events are #####”

The issue I am having is that its an MQTT message to turn on the lamp, then more MQTT messages for the pieces of the sentence. I cant get the pauses to work, All of the pauses happen, then everything runs one right after another. So in my example above what really ends up happening is:

Pause for 3.5 seconds
turn on lamp
“Good morning” “the weather is ####” “your calendar events are #####”

its like the MQTT messages all wait until the end of my script to run. I have tried using functions and also have tried threading. Nothing I have tried so far would allow me to get my speaker to say something and then let my script continue.

So I am not sure if its an MQTT issue or TTS issue. but since turning on the lamp is not a TTS thing, I am leaning towards it being an MQTT thing

My relevant code is below.

import json
import datetime
import paho.mqtt.client as mqtt
import websocket
import requests
import time
import paramiko
import sys
import random
import schedule
import os
from pathlib import Path
from threading import Thread

def on_connect(client, userdata, flags, rc, properties):
	"""Called when connected to MQTT broker."""
	client.subscribe("hermes/intent/#")
	client.subscribe("hermes/nlu/intentNotRecognized")
	print("Connected. Waiting for intents.")


def on_disconnect(client, userdata, flags, rc):
	"""Called when disconnected from MQTT broker."""
	client.reconnect()


def on_message(client, userdata, msg):
	"""Called each time a message is received on a subscribed topic."""
	nlu_payload = json.loads(msg.payload)

	if msg.topic == "hermes/nlu/intentNotRecognized":
		sentence = "I dont know, you figure it out"
		print("Recognition failure")
	else:

		site_id = nlu_payload["siteId"]
		try:

			def pirateWeather(day):
				info = requests.get('https://api.pirateweather.net/forecast/***********')
				pirate = json.loads(info.text)
				summary = pirate["daily"]["data"][day]["summary"]
				summary = summary.replace("(","").replace(")","").replace('-','').replace("\u2013"," to ").replace("in.","inches")
				highTemp = round(pirate["daily"]["data"][day]["temperatureHigh"])
				highTemp = f"\"{highTemp}\""
				lowTemp = round(pirate["daily"]["data"][day]["temperatureLow"])
				lowTemp = f"\"{lowTemp}\""
				precip = pirate["daily"]["data"][day]["precipProbability"]
				precip = int(precip*100)
				precip = f"\"{precip}\""
				summaryCurrent = pirate["currently"]["summary"]
				temp = round(pirate["currently"]["temperature"])
				temperature = f"\"{temp}\""
				apTemp = round(pirate["currently"]["apparentTemperature"])
				apparentTemp = f"\"{apTemp}\""
				return summary, highTemp, lowTemp, precip, summaryCurrent, temperature, apparentTemp
				
			def todayDate():
				def ordinal_suffix(day):
					if 3 < day < 21 or 23 < day < 31:
						return 'th'
					else:
						return {1: 'st', 2: 'nd', 3: 'rd'}[day % 10]

				today = datetime.date.today()
				date_string = today.strftime('%A, %B %-d{suffix}, %Y')
				total = date_string.format(suffix=ordinal_suffix(today.day))
				return total
				
			def calEvents():
				date  = datetime.datetime.now()
				date = date.strftime("%Y-%m-%d")
				url = "**********"
				headers = {
					"Authorization": "Bearer **************",
					"content-type": "application/json",
				}
				response = requests.get(url, headers=headers)
				if (response.json() == []):
					response = "You have nothing on your calendar today"
				else:
					response = response.json()[0]['summary']
					response = "On your calendar today you have, "+response
				return response
				
			
			if ("WakeUp" == nlu_payload["intent"]["intentName"]):
				client.publish("aha/a4e57c9fcee7/mqttOutlet/cmd_t", 'ON')	
				
				forecast = pirateWeather(0)
				date = todayDate()
				events = calEvents()
			
				sentence = "Good Morning, Today is "+date+", The forecast is "+forecast[0]+" with a high of "+forecast[1]+", a low of "+forecast[2]+", and a "+forecast[3]+" percent chance of precipitation, "+events

			
		except Exception as e: 
			print(e)
			sentence = "I seem to be experiencing a malfunction, call my creator Derek"

	site_id = nlu_payload["siteId"]
	client.publish("hermes/tts/say", json.dumps({"text": sentence, "siteId": site_id}))

# Create MQTT client and connect to broker
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set(username="rhasspy",password="Brixton04")
client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message
client.connect("172.21.4.209", 1883)
client.loop_forever()

Would 't help to use a tool like MQTT Explorer. (Or maybe done something like that)
To monitor the MQTT messages. And even make your own.
So to find out if it is MQTT or TTS related. Or both :wink:
It’s hard to be smart.

I am not very familiar with python, but I don’t see any delays in the code you posted. In fact all of your speaking appears to be in the one command:

sentence = "Good Morning, Today is “+date+”, The forecast is “+forecast[0]+” with a high of “+forecast[1]+”, a low of “+forecast[2]+”, and a “+forecast[3]+” percent chance of precipitation, "+events

I assume you want to split it into 3 separate client.publish() calls with the delays between.

1 Like

Sorry I should have put the code that I tried. I have tried what you are suggesting. So like this

import json
import datetime
import paho.mqtt.client as mqtt
import websocket
import requests
import time
import paramiko
import sys
import random
import schedule
import os
from pathlib import Path
from threading import Thread

def on_connect(client, userdata, flags, rc, properties):
	"""Called when connected to MQTT broker."""
	client.subscribe("hermes/intent/#")
	client.subscribe("hermes/nlu/intentNotRecognized")
	print("Connected. Waiting for intents.")


def on_disconnect(client, userdata, flags, rc):
	"""Called when disconnected from MQTT broker."""
	client.reconnect()


def on_message(client, userdata, msg):
	"""Called each time a message is received on a subscribed topic."""
	nlu_payload = json.loads(msg.payload)

	if msg.topic == "hermes/nlu/intentNotRecognized":
		sentence = "I dont know, you figure it out"
		print("Recognition failure")
	else:

		site_id = nlu_payload["siteId"]
		try:

			def pirateWeather(day):
				info = requests.get('https://api.pirateweather.net/forecast/***********')
				pirate = json.loads(info.text)
				summary = pirate["daily"]["data"][day]["summary"]
				summary = summary.replace("(","").replace(")","").replace('-','').replace("\u2013"," to ").replace("in.","inches")
				highTemp = round(pirate["daily"]["data"][day]["temperatureHigh"])
				highTemp = f"\"{highTemp}\""
				lowTemp = round(pirate["daily"]["data"][day]["temperatureLow"])
				lowTemp = f"\"{lowTemp}\""
				precip = pirate["daily"]["data"][day]["precipProbability"]
				precip = int(precip*100)
				precip = f"\"{precip}\""
				summaryCurrent = pirate["currently"]["summary"]
				temp = round(pirate["currently"]["temperature"])
				temperature = f"\"{temp}\""
				apTemp = round(pirate["currently"]["apparentTemperature"])
				apparentTemp = f"\"{apTemp}\""
				return summary, highTemp, lowTemp, precip, summaryCurrent, temperature, apparentTemp
				
			def todayDate():
				def ordinal_suffix(day):
					if 3 < day < 21 or 23 < day < 31:
						return 'th'
					else:
						return {1: 'st', 2: 'nd', 3: 'rd'}[day % 10]

				today = datetime.date.today()
				date_string = today.strftime('%A, %B %-d{suffix}, %Y')
				total = date_string.format(suffix=ordinal_suffix(today.day))
				return total
				
			def calEvents():
				date  = datetime.datetime.now()
				date = date.strftime("%Y-%m-%d")
				url = "**********"
				headers = {
					"Authorization": "Bearer **************",
					"content-type": "application/json",
				}
				response = requests.get(url, headers=headers)
				if (response.json() == []):
					response = "You have nothing on your calendar today"
				else:
					response = response.json()[0]['summary']
					response = "On your calendar today you have, "+response
				return response
				
			
			if ("WakeUp" == nlu_payload["intent"]["intentName"]):
				client.publish("aha/a4e57c9fcee7/mqttOutlet/cmd_t", 'ON')	
				time.sleep(2)
				forecast = pirateWeather(0)
				date = todayDate()
				events = calEvents()
			
				sentence1 = "Good Morning"
                client.publish("hermes/tts/say", json.dumps({"text": sentence1, "siteId": site_id}))
                time.sleep(1)
                sentence2 = "Today is "+date+", The forecast is "+forecast[0]+" with a high of "+forecast[1]+", a low of "+forecast[2]+", and a "+forecast[3]+" percent chance of precipitation "
                client.publish("hermes/tts/say", json.dumps({"text": sentence2, "siteId": site_id}))
                time.sleep(2)
                sentence3 = events
                client.publish("hermes/tts/say", json.dumps({"text": sentence3, "siteId": site_id}))
			
		except Exception as e: 
			print(e)
			sentence = "I seem to be experiencing a malfunction, call my creator Derek"

	site_id = nlu_payload["siteId"]
	client.publish("hermes/tts/say", json.dumps({"text": sentence, "siteId": site_id}))

# Create MQTT client and connect to broker
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set(username="**********",password="**********")
client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_message = on_message
client.connect("172.21.4.209", 1883)
client.loop_forever()

when trying the above. I would trigger the WakeUp intent, it would do all the sleeps first. so there would be a 5 second pause( the sum of all the pauses) then turn on the light and start saying the sentences one after another with no pauses in between

I doubt that python is executing all the time.sleep()s first.
My guess is maybe the MQTT module is effectively buffering the client.publish() statements, rather than sending them immediately your program is executing them.

Looking at MQTT limitations i am reminded that MQTT messages are not guaranteed, with QoS being significant. In particular

This means that messages passed to publish() may be lost. This could be mitigated 
by taking care that all messages passed to publish() have a corresponding on_publish() 
call or use wait_for_publish.

I found other comments including this suggesting that client.loop_forever() may be having an undesired effect.

I am only new to Python and MQTT myself, so already beyond my expertise. Good luck working it out.