Rhasspy TTS Repeats Last Sentence(same as repeating current sentence twice?!) On Jetson Nano

tombonee · December 6, 2020, 7:46pm

I’m using 2.5.8 from docker container and running on Jetson Nano using Ubuntu Bionic Beaver. I only have one issue the TTS sentence is repeated twice. I found it looks like maybe last sentence is turned on as I get 2 aplay calls with two different PIDs for one say call. There is one say command in the log for each intent. Any way I can turn that off or set the value to off for repeat last sentence. Thanks in advance.

tombonee · December 6, 2020, 8:05pm

I forgot the log for this.

[DEBUG:2020-12-06 15:04:14,072] rhasspyserver_hermes: -> NluQuery(input=‘what is the temperature outside’, site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, intent_filter=None, session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, wakeword_id=None, lang=None)

[DEBUG:2020-12-06 15:04:14,073] rhasspyserver_hermes: Publishing 220 bytes(s) to hermes/nlu/query

020-12-06 15:04:14,078] rhasspynlu_hermes: <- NluQuery(input=‘what is the temperature outside’, site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, intent_filter=None, session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, wakeword_id=None, lang=None)

[DEBUG:2020-12-06 15:04:14,080] rhasspynlu_hermes: -> NluIntentParsed(input=‘What is the temperature outside’, intent=Intent(intent_name=‘OutdoorTemperature’, confidence_score=1.0), site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, slots=[], session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’)

[DEBUG:2020-12-06 15:04:14,081] rhasspynlu_hermes: Publishing 249 bytes(s) to hermes/nlu/intentParsed

[DEBUG:2020-12-06 15:04:14,086] rhasspynlu_hermes: -> NluIntent(input=‘What is the temperature outside’, intent=Intent(intent_name=‘OutdoorTemperature’, confidence_score=1.0), site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, slots=[], session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, custom_data=None, asr_tokens=[[AsrToken(value=‘What’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘is’, confidence=1.0, range_start=5, range_end=7, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=8, range_end=11, time=None), AsrToken(value=‘temperature’, confidence=1.0, range_start=12, range_end=23, time=None), AsrToken(value=‘outside’, confidence=1.0, range_start=24, range_end=31, time=None)]], asr_confidence=None, raw_input=‘what is the temperature outside’, wakeword_id=None, lang=None)

[DEBUG:2020-12-06 15:04:14,087] rhasspynlu_hermes: Publishing 822 bytes(s) to hermes/intent/OutdoorTemperature

[DEBUG:2020-12-06 15:04:14,135] rhasspyserver_hermes: <- NluIntent(input=‘What is the temperature outside’, intent=Intent(intent_name=‘OutdoorTemperature’, confidence_score=1.0), site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, slots=[], session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, custom_data=None, asr_tokens=[[AsrToken(value=‘What’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘is’, confidence=1.0, range_start=5, range_end=7, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=8, range_end=11, time=None), AsrToken(value=‘temperature’, confidence=1.0, range_start=12, range_end=23, time=None), AsrToken(value=‘outside’, confidence=1.0, range_start=24, range_end=31, time=None)]], asr_confidence=None, raw_input=‘what is the temperature outside’, wakeword_id=None, lang=None)

[DEBUG:2020-12-06 15:04:14,135] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/OutdoorTemperature, id=f929447c-ba07-44e4-a8f7-3e64eed87d44)

[DEBUG:2020-12-06 15:04:14,138] rhasspyserver_hermes: Sent 451 char(s) to websocket

[DEBUG:2020-12-06 15:04:14,140] rhasspyhomeassistant_hermes: <- NluIntent(input=‘What is the temperature outside’, intent=Intent(intent_name=‘OutdoorTemperature’, confidence_score=1.0), site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, slots=[], session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, custom_data=None, asr_tokens=[[AsrToken(value=‘What’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘is’, confidence=1.0, range_start=5, range_end=7, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=8, range_end=11, time=None), AsrToken(value=‘temperature’, confidence=1.0, range_start=12, range_end=23, time=None), AsrToken(value=‘outside’, confidence=1.0, range_start=24, range_end=31, time=None)]], asr_confidence=None, raw_input=‘what is the temperature outside’, wakeword_id=None, lang=None)

[DEBUG:2020-12-06 15:04:14,143] rhasspydialogue_hermes: <- NluIntent(input=‘What is the temperature outside’, intent=Intent(intent_name=‘OutdoorTemperature’, confidence_score=1.0), site_id=‘default’, id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, slots=[], session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, custom_data=None, asr_tokens=[[AsrToken(value=‘What’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘is’, confidence=1.0, range_start=5, range_end=7, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=8, range_end=11, time=None), AsrToken(value=‘temperature’, confidence=1.0, range_start=12, range_end=23, time=None), AsrToken(value=‘outside’, confidence=1.0, range_start=24, range_end=31, time=None)]], asr_confidence=None, raw_input=‘what is the temperature outside’, wakeword_id=None, lang=None)

[WARNING:2020-12-06 15:04:14,144] rhasspydialogue_hermes: No session for id e9cd0c33-a157-425c-b821-155940eafdd5. Dropping recognition.

[DEBUG:2020-12-06 15:04:14,146] rhasspyhomeassistant_hermes: https://172.17.0.1:8123/api/intent/handle

[DEBUG:2020-12-06 15:04:14,460] rhasspyserver_hermes: -> TtsSay(text=‘It is currently 30.6 degrees fahrenheit outside’, site_id=‘default’, lang=None, id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=’’, volume=1.0)

[DEBUG:2020-12-06 15:04:14,461] rhasspyserver_hermes: Publishing 172 bytes(s) to hermes/tts/say

[DEBUG:2020-12-06 15:04:14,465] rhasspytts_cli_hermes: <- TtsSay(text=‘It is currently 30.6 degrees fahrenheit outside’, site_id=‘default’, lang=None, id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=’’, volume=1.0)

[DEBUG:2020-12-06 15:04:14,466] rhasspytts_cli_hermes: [‘flite’, ‘-o’, ‘/dev/stdout’, ‘-voice’, ‘awb’, ‘-t’, ‘It is currently 30.6 degrees fahrenheit outside’]

[DEBUG:2020-12-06 15:04:14,793] rhasspytts_cli_hermes: Got 122764 byte(s) of WAV data

[DEBUG:2020-12-06 15:04:14,794] rhasspytts_cli_hermes: -> AudioPlayBytes(122764 byte(s)) to hermes/audioServer/default/playBytes/0dd6f85c-8abf-423d-9c2d-ecddf5504eae

[DEBUG:2020-12-06 15:04:14,795] rhasspytts_cli_hermes: Waiting for play finished (timeout=4.085)

[DEBUG:2020-12-06 15:04:14,797] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/default/playBytes/0dd6f85c-8abf-423d-9c2d-ecddf5504eae, id=b41fb2fa-60f3-4f73-a074-d6307e052842)

[DEBUG:2020-12-06 15:04:14,798] rhasspyspeakers_cli_hermes: <- AudioPlayBytes(122764 byte(s))

[DEBUG:2020-12-06 15:04:14,798] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘plughw:CARD=tegrahda,DEV=3’]

[DEBUG:2020-12-06 15:04:18,832] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’)

[DEBUG:2020-12-06 15:04:18,833] rhasspyspeakers_cli_hermes: Publishing 99 bytes(s) to hermes/audioServer/default/playFinished

[DEBUG:2020-12-06 15:04:18,836] rhasspydialogue_hermes: <- AudioPlayFinished(id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’)

[DEBUG:2020-12-06 15:04:18,837] rhasspytts_cli_hermes: <- AudioPlayFinished(id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’)

[DEBUG:2020-12-06 15:04:18,839] rhasspytts_cli_hermes: -> TtsSayFinished(site_id=‘default’, id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=’’)

[DEBUG:2020-12-06 15:04:18,839] rhasspytts_cli_hermes: Publishing 84 bytes(s) to hermes/tts/sayFinished

[DEBUG:2020-12-06 15:04:18,842] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=b41fb2fa-60f3-4f73-a074-d6307e052842)

[DEBUG:2020-12-06 15:04:18,842] rhasspydialogue_hermes: <- TtsSayFinished(site_id=‘default’, id=‘0dd6f85c-8abf-423d-9c2d-ecddf5504eae’, session_id=’’)

[DEBUG:2020-12-06 15:04:18,855] rhasspyhomeassistant_hermes: -> TtsSay(text=‘It is currently 30.6 degrees fahrenheit outside’, site_id=‘default’, lang=None, id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, volume=None)

[DEBUG:2020-12-06 15:04:18,855] rhasspyhomeassistant_hermes: Publishing 209 bytes(s) to hermes/tts/say

[DEBUG:2020-12-06 15:04:18,858] rhasspytts_cli_hermes: <- TtsSay(text=‘It is currently 30.6 degrees fahrenheit outside’, site_id=‘default’, lang=None, id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’, volume=None)

[DEBUG:2020-12-06 15:04:18,859] rhasspytts_cli_hermes: [‘flite’, ‘-o’, ‘/dev/stdout’, ‘-voice’, ‘awb’, ‘-t’, ‘It is currently 30.6 degrees fahrenheit outside’]

[DEBUG:2020-12-06 15:04:19,190] rhasspytts_cli_hermes: Got 122764 byte(s) of WAV data

[DEBUG:2020-12-06 15:04:19,190] rhasspytts_cli_hermes: -> AudioPlayBytes(122764 byte(s)) to hermes/audioServer/default/playBytes/f04eb642-64e1-4d28-a331-483ef06146b7

[DEBUG:2020-12-06 15:04:19,191] rhasspytts_cli_hermes: Waiting for play finished (timeout=4.085)

[DEBUG:2020-12-06 15:04:19,194] rhasspyspeakers_cli_hermes: <- AudioPlayBytes(122764 byte(s))

[DEBUG:2020-12-06 15:04:19,195] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘plughw:CARD=tegrahda,DEV=3’]

[DEBUG:2020-12-06 15:04:23,211] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘f04eb642-64e1-4d28-a331-483ef06146b7’)

[DEBUG:2020-12-06 15:04:23,212] rhasspyspeakers_cli_hermes: Publishing 99 bytes(s) to hermes/audioServer/default/playFinished

[DEBUG:2020-12-06 15:04:23,215] rhasspydialogue_hermes: <- AudioPlayFinished(id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘f04eb642-64e1-4d28-a331-483ef06146b7’)

[DEBUG:2020-12-06 15:04:23,215] rhasspytts_cli_hermes: <- AudioPlayFinished(id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘f04eb642-64e1-4d28-a331-483ef06146b7’)

[DEBUG:2020-12-06 15:04:23,218] rhasspytts_cli_hermes: -> TtsSayFinished(site_id=‘default’, id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’)

[DEBUG:2020-12-06 15:04:23,218] rhasspytts_cli_hermes: Publishing 120 bytes(s) to hermes/tts/sayFinished

[DEBUG:2020-12-06 15:04:23,222] rhasspydialogue_hermes: <- TtsSayFinished(site_id=‘default’, id=‘f04eb642-64e1-4d28-a331-483ef06146b7’, session_id=‘e9cd0c33-a157-425c-b821-155940eafdd5’)

tombonee · December 7, 2020, 6:19am

I think this may be the issue. How do I change this timeout ? Too close to actually time in takes to complete.

[DEBUG:2020-12-06 15:04:19,191] rhasspytts_cli_hermes: Waiting for play finished (timeout=4.085)

tombonee · December 7, 2020, 12:23pm

Unless I missed some options I believe what is needed to resolve this is the following profile options:

timeout multiplier - the calculated timeout is multiplied by this real number value. This allows for differences in systems where the calculated timeout will not be long enough.
retry - set this bool to true or false. If set to true and for example the timeout occurs the play process will be repeated. If set to false the play process will not be repeated.

I could see more of these for other processes so the user could customize based on their systems timings.

I can help get this done just let me know what to do.

Thanks.

tombonee · December 8, 2020, 8:17pm

I have switched from intents to events. THe time out and repeating last sectence are now fixed. I only have one audio issue remaining when using flite. Appears the last word or last second is repeated. Any ideas ? Thanks.

synesthesiam · December 11, 2020, 7:55pm

The changes would need to happen in three places:

In rhasspy-speakers-cli-hermes a new command-line option needs to added and passed into the TtsHermesMqtt object. There’s a a field named finished_timeout_extra that’s added to the timeout (currently hard-coded).
In rhasspy-profile a new setting for the timeout should be added to defaults.json
In rhasspy-supervisor the profile setting needs to get translated into the command-line argument

Do you think a multiplier is enough? We could add an option to disable the timeout, but some logic can get messed up if the “finished” events never make it back (due to a crash or something).