TLDR: How do I implement continuous mode in Rhasspy ie speak multiple commands one after the other without having to repeat the wakeword after each one?
I am trying to implement continuous mode ie:
I say “Porcupine, enter continuous mode” and then I can say as many commands as I require, without having to say the wakeword before each one and then once I am done I say “exit continuous mode”.
My use case is for example navigating Netflix. Once in the Netflix app on my TV, I want to be able to do something like say “right, right, down, right, down, left, select, play” and my selection will start playing (I can control my TV via a Broadlink RM 2 Pro).
I have the MQTT broker that I use for Home Assistant set as an external MQTT broker in Rhasspy. I have successfully created some other automations using hermes/dialogueManager/startSession
and hermes/dialogueManager/endSession
(to mute my speakers when saying a rhasspy command and to cancel a command respectively.)
From reading other posts on here I think I understand what I need to do and I have come up with the following.
Rhasspy Intents (note that I use Events to interact with Home Assistant):
[EnterContinuousMode]
enter{action} continuous{mode} mode
[ExitContinuousMode]
exit{action} continuous{mode} mode
[ContinuousModeNavigation]
actions = (right | left | up | down | select | play) {action}
<actions>
Home Assistant Automations:
- alias: 'Enter Continuous Mode'
trigger:
platform: event
event_type: rhasspy_EnterContinuousMode
event_data:
action: 'enter'
mode: 'continuous'
action:
- service: 'mqtt.publish'
data_template:
payload_template: '{"intentFilter": ["enter continuous mode","exit continuous mode", "right", "left", "up", "down", "select", "play"], "sessionId": "{{ session_id }}", "siteId":"lounge" }'
topic: 'hermes/dialogueManager/continueSession'
- alias: 'Exit Continuous Mode'
trigger:
platform: event
event_type: rhasspy_ExitContinuousMode
event_data:
action: 'exit'
mode: 'continuous'
action:
- service: 'mqtt.publish'
data_template:
payload_template: '{ "intentFilter": "exit continuous mode", "sessionId": "{{ session_id }}", "siteId":"lounge" }'
topic: 'hermes/dialogueManager/endSession'
- alias: 'Continuous Mode Right'
trigger:
platform: event
event_type: rhasspy_ContinuousModeNavigation
event_data:
action: 'right'
action:
- service: switch.toggle
entity_id: switch.tv_right
- service: 'mqtt.publish'
data_template:
payload_template: '{ "intentFilter": "right", "sessionId": "{{ session_id }}", "siteId":"lounge" }'
topic: 'hermes/dialogueManager/continueSession'
- alias: 'Continuous Mode Select'
trigger:
platform: event
event_type: rhasspy_ContinuousModeNavigation
event_data:
action: 'select'
action:
- service: switch.toggle
entity_id: switch.tv_ok
- service: 'mqtt.publish'
data_template:
payload_template: '{ "intentFilter": "select", "sessionId": "{{ session_id }}", "siteId":"lounge" }'
topic: 'hermes/dialogueManager/continueSession'
So if I understood the other posts correctly, I can limit the intents that work with the Enter Continuous Mode
automation using intentFilter
and as long as I end each of my navigation automations (eg Continuous Mode Select
) with another continueSession
command then I should be able to daisy chain any commands that I require together?
So with the above intents and automations in place, if I say “Porcupine, enter continuous mode”, Rhasspy is able to successfully recognise that specific intent and Home Assistant says that the Enter Continuous Mode
automation has been successfully triggered in the logbook, however if I then say “right” or “select”, then the second command is not registered.
I opened up MQTT Explorer and when I do the above this is what I see under the dialogueManager:
sessionStarted:
{"sessionId": "lounge-grasshopper_raspberry-pi-7822xxxx-xxxx-41da-a670-xxxxxxx24d74", "siteId": "lounge", "customData": "grasshopper_raspberry-pi", "lang": null}
continueSession:
{"intentFilter": ["enter continuous mode","exit continuous mode", "right", "left", "up", "down", "play", "select"], "sessionId": "", "siteId":"lounge" }
and then after a few seconds I get this:
sessionEnded:
{"termination": {"reason": "timeout"}, "sessionId": "lounge-grasshopper_raspberry-pi-7822xxxx-xxxx-41da-a670-xxxxxxx24d74", "siteId": "lounge", "customData": "grasshopper_raspberry-pi"}
I think what the above is telling me is that I am not successfully passing the sessionId
between my commands so basically when I say “Porcupine, enter continuous mode”, Rhasspy is looking for an intent with a matching sessionId
and then times out when it does not receive one. I am very new to MQTT and automations in general so I may have misunderstood.
Can anyone please advise me how to get this working? How do I pass the sessionId
between commands?
Is what I am doing logical or is there a better way of doing this?
Thanks