Thanks for that confirmation. One more question, the MQTT is built-in, not a separate server, right?
Yes. Rhasspy 2.5 runs an internal instances of mosquitto
when you don’t have it connected to an external MQTT broker.
how do you implement the continue session in 2.5pre ? in homeassistant i made the following intent scripts:
Volume:
action:
- service: 'mqtt.publish'
data_template:
payload_template: '{"intentFilter": "Again", text: "ok" , sessionId": {{ session_id }}, "customData": {{ volume }} }'
topic: 'hermes/dialogueManager/continueSession'
retain: true
- service: 'freebox_player.remote'
data_template:
code: '{{ volume }}'
Again:
action:
- service: 'persistent_notification.create'
data_template:
message: 'test {{custom_data}}'
the mqtt topic seems ok
hermes/dialogueManager/continueSession
{"intentFilter": "Again", text: "ok" , sessionId": salon-jarvis-a08864d4-b919-4b42-8e0c-3b960c910360, "customData": vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc }
but it seems rhasspy doesn t listen
Your JSON has syntax errors. Check it again.
indeed thank you for your answer indeed i had a few mistakes
fixed it but still it’s not working
here’s the updated code:
Volume:
action:
- service: 'mqtt.publish'
data_template:
payload_template: '{"intentFilter": "Again", text: "ok" , "sessionId": "{{ session_id }}", "customData": "{{ volume }}" }'
topic: 'hermes/dialogueManager/continueSession'
retain: true
- service: 'freebox_player.remote'
data_template:
code: '{{ volume }}'
Again:
action:
- service: 'persistent_notification.create'
data_template:
message: 'test {{custom_data}}'
hermes/dialogueManager/continueSession
{"intentFilter": "Again", text: "ok" , "sessionId": "salon-jarvis-012e3c9a-0649-4292-a06d-cca3edf9fa1b", "customData": "vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc" }
I’m assuming you set “Rhasspy” as dialogue manager in the settings.
I also recommend you use rhasspy-watch to watch what is happening on your instance (or you could just subscribe to the MQTT topics, but rhasspy-watch is cooler ).
yes on the server and Hermes MQTT on the satelite
ill install rhasspy-watch thank you for the suggestion didn’t know about it
[2020-05-11 13:01:42] [hotword] detected on site salon, for model jarvis
[2020-05-11 13:01:42] [hotword] was asked to toggle itself 'off' on site salon
[2020-05-11 13:01:42] [Asr] was asked to toggle itself 'off' on site salon
[2020-05-11 13:01:43] [hotword] was asked to toggle itself 'on' on site salon
[2020-05-11 13:01:43] [Asr] was asked to toggle itself 'on' on site salon
[2020-05-11 13:01:43] [Dialogue] session with id salon-jarvis-707bec44-3da7-4cae-8719-ed4268e589b9 was started on site salon.
[2020-05-11 13:01:43] [hotword] was asked to toggle itself 'off' on site salon
[2020-05-11 13:01:43] [Asr] was asked to listen on site salon
[2020-05-11 13:01:44] [Asr] captured text 'monte le son' in 1.3529847989993868s on site salon
[2020-05-11 13:01:44] [hotword] was asked to toggle itself 'off' on site salon
[2020-05-11 13:01:44] [Asr] was asked to toggle itself 'off' on site salon
[2020-05-11 13:01:45] [hotword] was asked to toggle itself 'on' on site salon
[2020-05-11 13:01:45] [Asr] was asked to toggle itself 'on' on site salon
[2020-05-11 13:01:45] [Asr] was asked to stop listening on site salon
[2020-05-11 13:01:45] [hotword] was asked to toggle itself 'on' on site salon
[2020-05-11 13:01:45] [Nlu] was asked to parse input 'monte le son'
[2020-05-11 13:01:45] [Nlu] Detected intent Volume with confidence score 1.0 for input 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc le son
[2020-05-11 13:01:45] [Asr] was asked to stop listening on site salon
[2020-05-11 13:01:45] [Dialogue] session with id salon-jarvis-707bec44-3da7-4cae-8719-ed4268e589b9 was ended on site salon. Reason: nominal
with customData :
jarvis
[2020-05-11 13:01:45] [hotword] was asked to toggle itself 'on' on site salon
indeed the session seems to be closed before i send the continuesession or / it s not detected at all
There is something strange in your log.
I dont’ understand why
[2020-05-11 13:01:45] [Nlu] Detected intent Volume with confidence score 1.0 for input 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc le son
Normally, input
in this line has to be the capture text : monte le son
example :
[2020-05-10 13:31:40] [Nlu] was asked to parse input 'quelle heure est-il'
[2020-05-10 13:31:40] [Nlu] Detected intent AskTime with confidence score 1.0 for input 'quelle heure est-il
[2020-05-10 13:31:40] [Nlu] Intent AskTime with confidence score 1.0 on site bureau
And you have not send a continue session, otherwize, you’ll have :
[2020-05-10 13:31:40] [Nlu] was asked to parse input 'quelle heure est-il'
[2020-05-10 13:31:40] [Nlu] Detected intent AskTime with confidence score 1.0 for input 'quelle heure est-il
[2020-05-10 13:31:40] [Nlu] Intent AskTime with confidence score 1.0 on site bureau
[2020-05-10 13:31:40] [Dialogue] was ask to continue session with id bureau-jarvis-e72da4c0-d1ac-4c5e-bd0f-1c054e499851 by saying 'tu es sûr de vouloir l'heure ?. '
with customData :
[...blablabla..]
there is a strange transfo between “monte le son” and “vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc le son”
Strange…
You can switch to json format by adding --outputFormat raw
to rhasspy watch. You’ll see the messages themselves.
Ced
This line in my log means that something was sent on hermes/intent/AskTime
You don’t have this line in your log.
can you switch in raw mode and check if you have a line like this :
[2020-05-11 13:01:45] hermes/intent/Volume - {'input': [blabla]...
that s because of my rhasspy sentence
[Volume]
(plus:vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc | monte:vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc | augmente:vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc | descend:vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec, | baisse:vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec | moins:vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec,vol_dec | coupe:mute | remet:mute ){volume} [le] ( fort | volume | son )
some words are replaced by values
theres a lot of vol_inc because it increase volume by 1% and that not enough
in raw mode
[2020-05-11 21:10:29] hermes/hotword/jarvis/detected - {'modelId': 'jarvis', 'modelVersion': '', 'modelType': 'personal', 'currentSensitivity': 0.6, 'siteId': 'salon', 'sessionId': None, 'sendAudioCaptured': None}
[2020-05-11 21:10:29] hermes/hotword/toggleOff - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:29] hermes/asr/toggleOff - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:30] hermes/hotword/toggleOn - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:30] hermes/asr/toggleOn - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:30] hermes/dialogueManager/sessionStarted - {'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354', 'siteId': 'salon', 'customData': 'jarvis'}
[2020-05-11 21:10:30] hermes/hotword/toggleOff - {'siteId': 'salon', 'reason': 'dialogueSession'}
[2020-05-11 21:10:30] hermes/asr/startListening - {'siteId': 'salon', 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354', 'stopOnSilence': True, 'sendAudioCaptured': True, 'wakewordId': 'jarvis', 'intentFilter': None}
[2020-05-11 21:10:32] hermes/asr/textCaptured - {'text': 'monte le son', 'likelihood': 1, 'seconds': 1.588414803991327, 'siteId': 'salon', 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354', 'wakewordId': None, 'asrTokens': None}
[2020-05-11 21:10:32] hermes/hotword/toggleOff - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:32] hermes/asr/toggleOff - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:33] hermes/hotword/toggleOn - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:33] hermes/asr/toggleOn - {'siteId': 'salon', 'reason': 'playAudio'}
[2020-05-11 21:10:33] hermes/asr/stopListening - {'siteId': 'salon', 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354'}
[2020-05-11 21:10:33] hermes/hotword/toggleOn - {'siteId': 'salon', 'reason': 'dialogueSession'}
[2020-05-11 21:10:33] hermes/nlu/query - {'input': 'monte le son', 'siteId': 'salon', 'id': None, 'intentFilter': None, 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354', 'wakewordId': 'jarvis'}
[2020-05-11 21:10:33] hermes/nlu/intentParsed - {'input': 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc le son', 'intent': {'intentName': 'Volume', 'confidenceScore': 1.0}, 'siteId': 'salon', 'id': None, 'slots': [{'entity': 'volume', 'value': {'kind': 'Unknown', 'value': 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc'}, 'slotName': 'volume', 'rawValue': 'monte', 'confidence': 1.0, 'range': {'start': 0, 'end': 79, 'rawStart': 0, 'rawEnd': 5}}], 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354'}
[2020-05-11 21:10:33] hermes/intent/Volume - {'input': 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc le son', 'intent': {'intentName': 'Volume', 'confidenceScore': 1.0}, 'siteId': 'salon', 'id': None, 'slots': [{'entity': 'volume', 'value': {'kind': 'Unknown', 'value': 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc'}, 'slotName': 'volume', 'rawValue': 'monte', 'confidence': 1.0, 'range': {'start': 0, 'end': 79, 'rawStart': 0, 'rawEnd': 5}}], 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354', 'customData': None, 'asrTokens': [[{'value': 'vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc,vol_inc', 'confidence': 1.0, 'rangeStart': 0, 'rangeEnd': 79, 'time': None}, {'value': 'le', 'confidence': 1.0, 'rangeStart': 80, 'rangeEnd': 82, 'time': None}, {'value': 'son', 'confidence': 1.0, 'rangeStart': 83, 'rangeEnd': 86, 'time': None}]], 'asrConfidence': None, 'rawInput': 'monte le son', 'wakewordId': 'jarvis'}
[2020-05-11 21:10:33] hermes/dialogueManager/endSession - {'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354'}
[2020-05-11 21:10:33] hermes/asr/stopListening - {'siteId': 'salon', 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354'}
[2020-05-11 21:10:33] hermes/dialogueManager/sessionEnded - {'termination': {'reason': 'nominal'}, 'sessionId': 'salon-jarvis-69e3ffc0-2fcf-4069-ae64-1c48f1544354', 'siteId': 'salon', 'customData': 'jarvis'}
[2020-05-11 21:10:33] hermes/hotword/toggleOn - {'siteId': 'salon', 'reason': 'dialogueSession'}
Ok, so, first thing, it seems there’s a bug in human view of Rhasspy watch. I’ll have to check that. It’s strange that you don’t have this in human view mode :
[Nlu] Intent Volume with confidence score 1.0 on site bureau with slots...
But the most important, it’s you don’t have continue session in your log.
You have only : hermes/dialogueManager/endSession
The endSession is sent by your handling intent app.
I don’t know home assistant enough to help you.
Ced
well since i never got the Intent handling setting (in rhasspysettings) working i m relying on the snips integration from home assistant to handle everything with hermes
that setting isn t working at all
but when i set to disabled and add “snips:” to my config in home assistant it works
Thanks to @lilbuh . The bug in Rhasspy-watch is solved.
But after analyse of MQTT message, there’s no continueSession sent.
I think it’s a problem between Rhasspy and HomeAssistant. Maybe a bad parameter somewhere ?
Ced
Hi,
Do any of you have any examples of successfully using continueSession
with Rhasspy 2.5 to achieve the original poster’s aim or something similar? If so then could you please share an example here?
I have set up multiple intents and thought that I would be able to daisy chain them together however my sessionId
is never passed between intents, it is always an empty string.
Thanks
If you’re using rhasspy-hermes-app, you can use this example for inspiration:
Note that there’s still a bug that prevents the
custom_data
field from being passed around (https://github.com/rhasspy/rhasspy-nlu-hermes/issues/3), but the basic principle of continuing a session and hence having multiple intents being recognized with the same session ID works.
Hi @koan,
Thanks for the heads up about the Hermes app, it looks really promising, nice work!
If I am honest, I am in a bit of a rabbit hole already with getting my head around how MQTT works in Rhasspy, the snips API reference, HA automations and templating. I am just kind of curious if what I am trying to do is even possible in Rhasspy or if I’m going to have to get my head around python and then use Hermes app instead.
Basically I have an intent called [ContinuousMode] which is as follows:
[ContinuousMode]
(enter|exit){action} continuous{mode} mode
The corresponding HA automation is:
- alias: 'Enter Continuous Mode'
trigger:
platform: event
event_type: rhasspy_ContinuousMode
event_data:
action: 'enter'
mode: 'continuous'
action:
- service: 'mqtt.publish'
data_template:
payload_template: '{"intentFilter": ["ContinuousMode,"ContinuousModeNavigation"], "sessionId": "{{ session_id }}", "siteId":"lounge" }'
If I open up MQTT Explorer and say “Grasshopper, enter continuous mode”, saying my wakeword starts a new session that contains a sessionId
as follows:
{"sessionId": "lounge-grasshopper_raspberry-pi-2xxxxaff-xxxx-4f9f-xxxx-a7e5xxxxx186", "siteId": "lounge", "customData": "grasshopper_raspberry-pi", "lang": null}
Saying the intent phrase launches a separate session, this one contains no sessionId
:
{"intentFilter": ["ContinuousMode","ContinuousModeNavigation"], "sessionId": "", "siteId":"lounge" }
and then a number of seconds later, the session that was triggered by the wakeword is terminated:
{"termination": {"reason": "timeout"}, "sessionId": "lounge-grasshopper_raspberry-pi-2xxxxaff-xxxx-4f9f-xxxx-a7e5xxxxx186", "siteId": "lounge", "customData": "grasshopper_raspberry-pi"}
So as far as I can tell, a new session is created by the wakeword (which has a sessionId
), then another new separate session is started by my intent, but this has no sessionId
and hence can’t be linked to the first.
Then the session started by the wakeword ends and the session started by my intent disappears to never be seen again.
I would have expected that saying the wakeword followed by an intent name would have linked those 2 sessions and then I could continue.
Does anyone know where I am going wrong?
Thanks
I don’t use Rhasspy with Home Assistant automations, so I can’t really pinpoint the exact error, but I think you’re doing something wrong with the flow of messages. I don’t see you sending a hermes/dialogueManager/continueSession
message, for instance. Did you forget to set the MQTT topic in the mqtt.publish
action?
The (relevant part of the) message flow of a session is:
- Wake word detected, session started
- Intent recognized, continue the session
- Intent recognized, continue the session
- …
- Intent recognized, end the session
hermes/dialogueManager/continueSession
and hermes/dialogueManager/EndSession
are different response messages to a recognized intent. During this whole flow, the session ID stays the same. After you end a session, the next wake word recognition starts a new session, with a new session ID.
Hi @koan, Apologies, I had just failed to copy paste the topic
line into my question.
I have now worked out how to achieve what I wanted: How to Enable Continuous Mode
Thanks for your help