Prompt for missing slot

JeffC · January 24, 2021, 8:57pm

Howdy, finally got Rhasspy, HA & MaryTTS working together, super impressed!!! I’ve got ChangeLightState set up as an Intent in HA, and can turn on/off my living room lamp. I’m wondering is there is way to have a “conversation” between Rhasspy & HA if an utterance is missing a slot; e.g. if I were to say “Turn on the lamp” rather than “Turn on the living room lamp” to have Rhasspy to HA prompt “Which light?” and subsequently fill in the slot for HA to act on. I think this would be helpful reducing the number to sentences that need to be trained & recognized, as well as to a user friendliness factor. Thanks!

RaspiManu · January 24, 2021, 10:06pm

This feature is currently a work in progress and it’s not ready to use yet. If you are an experienced programmer, you could try writing your own code for this function.

daywalker1180 · January 25, 2021, 9:53pm

Unfortunately I have no know-how about HA, due to not using it.

But take a look at the topic

about a possibility to develop your own intent handlers in a straightforward way using Python.

Especially the methods ContinueSession and EndSession could be of interest for you.

I use it in the way that if some slot value is missing, I first try to extract the information myself (out of the siteId mainly) and directly then carry out the action using EndSession; otherwise if that does not suceed - I simply ask the user, using ContinueSession.

JeffC · January 25, 2021, 11:40pm

Thank you, I’ll take a look. (ps, love the username!)

koan · January 26, 2021, 8:02am

Coincidentally I just added an example app yesterday of continuing a session in rhasspy-hermes-app:

github.com

rhasspy/rhasspy-hermes-app/blob/master/examples/continue_session.py

"""Example app that shows how to continue a session."""
import logging

from rhasspyhermes.nlu import NluIntent

from rhasspyhermes_app import ContinueSession, EndSession, HermesApp

_LOGGER = logging.getLogger("ContinueApp")

app = HermesApp("ContinueApp")


@app.on_intent("Yes")
async def yes(intent: NluIntent):
    """The user confirms."""
    if intent.custom_data == "TurnOffLight":
        response = "OK, turning off the light"
    elif intent.custom_data == "TurnOnLight":
        response = "OK, turning on the light"
    else:

This file has been truncated. show original

However, this is an easier case: this app just asks for a confirmation (yes/no). Note that currently the custom_data attribute doesn’t seem to work yet, but maybe it’s not needed in your case.

So in your intent handler you try to extract the slot value with something like https://github.com/rhasspy/rhasspy-hermes-app/issues/12. If there are missing slots, continue the session with a question for the missing slot. Then in the intent handler fired by telling the slot, again try to extract the slot value. If it’s there, you have all the information you need.

However, I’m not sure what’s currently the best way to forward to Home Assistant the original intent with the slot filled in by the later intent. Snips had a hermes/nlu/partialQuery message for a follow up query to extract a specific slot given an intent, which should solve this, but Rhasspy doesn’t support it yet. So for now, everything you try would be a bit of a hack

JeffC · January 26, 2021, 7:46pm

Thanks Koen! good to see you involved with rhasspy, I remember you from snips land and was always impressed with your work.