Hey,
I am currently customising the rhasspysnips_nlu_hermes component in order to create a voice assistant which can understand commands without a wake word based on the nlu recognition confidence and pass them to hass.
As I am doing the ASR transcription externally and call the intent recognition via the HTTP API I disable the hasshandle in order to check for the confidence first.
However once the threshold is reached I need to enable the hasshandle via MQTT again before creating and publishing the NLUIntentParsed and NLUIntent.
As I am not as experienced with MQTT i was wondering how to properly utilise the rhasspyhermes.handle.HandleToggleOn(site_id) method in order to enable the handeling via hass before publishing the intent objects. How can I access the core.site_id (rhasspyserver_hermes/init.py) value from within the rhasspysnips_nlu_hermes init.py?
Sorry if the explanation is kinda fuzzy I’m trying my best…
Thanks in Advance 
shedz
I’ve got another question:
As it is possible to pass intentfilter via the http api i am trying to include another variable which is needed in the customised snipshermes nlu component.
As various functions are called until the data reaches there I’ve tried to include define the variable for passing in:
- rhasspyserver_hermes/init.py .recognize_intent()
- rhasspyserver_hermes/main.py api_text_to_intent(),text_to_intent()
- rhasspyhermes/nlu.py NLUQuery()
- rhasspysnips_nlu_hermes/init.py handle_query()
However while execution I get a TypeError unexpected keyword argument implicit (Name of my bool variable) which leads me to believe that I missed a spot of where to include the variable.
Thanks in advance for any assistance
@synesthesiam any hints on where to start looking ?
As google leads me to believe the problem might be laying in the Message(**kwargs) inheritance of NLUQuery as only specific calls instantiating NLUQuery (all from the custom snipsNLU component) will include the extra variable. I tried as much as I could and am at the end of my rope.
Sorry for the spam 
Hi @shedz, hopefully I can help. If I understand your overall goal, would it be better just to have a confidence threshold setting in the Snips NLU Hermes service? In other words, if the intent confidence is below the threshold, an “intent not recognized” message is emitted instead.
Can you post the code for this? Thanks.
Thanks for your quick response
First of all: My system is constantly listening and utilises deepspeech for open transcription. Rhasspy is integrated after the recognition and gets called via the HTTP API /text-to-intent usually with hasshandle set to true.
Two cases of activation exist:
- The user speaks the hotword and the command: In this case the system should respond even if the command was not recognized which is already covered by the current rhasspysnips hermes component. (I already changed the response there to include the confidence computed by snips instead of always returning 1.0)
- The user mentions an implicit command (“It’s really dark in here”) without hotword. In this case the system should only react if the confidence is high enough. Therefore the command should not be passed to HAss otherwise as it would be annoying to be constantly interrupted by the assistant saying “Sorry I did not catch that”.
So my further thinking was to pass a bool along with the “intenttext” like done with the hasshandle (rhasspy.adress/api/text-to-intent?hasshandle=true&implicit=true) and integrate it into the function calls and NLUQuery in order to call it in the rhasspysnips_nlu_hermes
What i changed:
- class NluQuery(Message)(rhasspy-hermes/rhasspyhermesnlu.py: Inserted implicit:bool and docs for it):
- recognize_intent() in rhasspy-server-hermes/rhasspyserver_hermes/_init_.py (Included implicit in fun def and NLUQuery initialization
3. function api_text_to_intent(*1) and text_to_intent_dict()(*2) in rhasspy-server-hermes/rhasspyserver_hermes/_main_.py (Included implicit http handle in (*1) and in function def and await core.recognize_intent() (*2)):
(*1)
(*2)
4. and finally class NLUHermesMqtt(HermesClient).handle_query() (included the condition
When i try to convert any text to intent i get this error:
My guess is this has something to do with how NLUQuery inheritance or me forgetting a spot where to include the parameter. I tried to include it on various locations as well but i do not believe i.e. the Dialogue Hermes class has something to do with this interaction ?
Sorry for the varying format and thanks so much for your assistance!
It seems like rhasspy-server-hermes may not be using your modified rhasspy-hermes code. Depending on how you set up your development environment, it may have placed an existing version of rhasspy-hermes into your virtual environment (or on your system) and is using that instead of your code.
You could quickly confirm this by putting a syntax error somewhere in your rhasspy-hermes code and see if it errors out on that or on the missing implicit keyword arg.
Also, very cool sounding project 
I’m curious how well DeepSpeech does with this job.
I was actually baffled by the performance on my pi4. I don’t need any kind of hotword detection when utilizing a hotword like porcupine… The overlapping with porcupine ppns is no coincidence… Dont need porcupine in the current setup though.
As for your solution there is but one thing to say: 
I was handing local forks to the docker container supposedly overwriting the original components but had a typo in rhasspyhermes …
Once i get the whole thing running I’ll make a post in show us and make it available via repository
While we’re at it: What is the best way to enable the hass handle from within rhasspysnips_nlu_hermes init py after receiving sufficient confidence, before publishing the NLUIntents?
I disable after receiving implicit:
I can import HandleToggleOn from rhasspyhermes.handle however is there a way to publish messages from NluHermesMqtt(HermesClient)?
Yep, just use self.publish(HandleToggleOn(...))
If you have an async generator, like an async function with yield, you can use await self.publish_all(generator_fun()) to do it asynchronously.
Thanks so much It’s slowly coming along. Now it passes it along to hass only after breaching the threshold.
However if the intent is not recognized Rhasspy plays an error sound which should not play unless explicitly activated. Can you point me to where the NLUIntentNotRecognized.topic is handled and the sound is played?
I would just integrate an optional implicit-bool in the NLUIntentNotR… class and check before the sound is played.
Sorry for asking so many questions…
Sure, that happens over in rhasspy-dialogue-hermes (dialogue manager)
Not a problem