Help with Rhasspy+Home Assistant using intent

[jokerigno]

Hi,

I just started to use rhasspy and have some question. I successfully added external broker and token to my HA instance.

Now I’m struggling managing intent to turn on/off open/close “things” in HA.

This is my sentences.ini

[HassTurnOn]
accendi (lampadario | striscia led| lampada scrivania | lampada simonasalvo | lampada fiore | piantana | razzo | jack lamp) {name} {stato}

[HassTurnOff]
spegni (lampadario | striscia led| lampada scrivania | lampada simonasalvo | lampada fiore | piantana | razzo | jack lamp) {name} {stato}

(using friendly name, not entity_id)

On “intent handling” Send intents to Home Assistant (/api/intent/handle) is flagged.

On HA I added this to my config:

intent:
intent_script:

and reboot.

If I try to handle this is the error in the log:

Logger: aiohttp.server
Source: helpers/intent.py:76
First occurred: 13:12:28 (1 occurrences)
Last logged: 13:12:28

Error handling request
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/helpers/intent.py”, line 72, in async_handle
result = await handler.async_handle(intent)
File “/usr/src/homeassistant/homeassistant/helpers/intent.py”, line 203, in async_handle
slots = self.async_validate_slots(intent_obj.slots)
File “/usr/src/homeassistant/homeassistant/helpers/intent.py”, line 155, in async_validate_slots
return self._slot_schema(slots) # type: ignore
File “/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py”, line 272, in call
return self._compiled([], data)
File “/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py”, line 594, in validate_dict
return base_validate(path, iteritems(data), out)
File “/usr/local/lib/python3.8/site-packages/voluptuous/schema_builder.py”, line 432, in validate_mapping
raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: required key not provided @ data[‘name’]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py”, line 422, in _handle_request
resp = await self._request_handler(request)
File “/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py”, line 499, in _handle
resp = await handler(request)
File “/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py”, line 119, in impl
return await handler(request)
File “/usr/src/homeassistant/homeassistant/components/http/security_filter.py”, line 60, in security_filter_middleware
return await handler(request)
File “/usr/src/homeassistant/homeassistant/components/http/forwarded.py”, line 76, in forwarded_middleware
return await handler(request)
File “/usr/src/homeassistant/homeassistant/components/http/request_context.py”, line 24, in request_context_middleware
return await handler(request)
File “/usr/src/homeassistant/homeassistant/components/http/ban.py”, line 78, in ban_middleware
return await handler(request)
File “/usr/src/homeassistant/homeassistant/components/http/auth.py”, line 144, in auth_middleware
return await handler(request)
File “/usr/src/homeassistant/homeassistant/components/http/view.py”, line 135, in handle
result = await result
File “/usr/src/homeassistant/homeassistant/components/http/data_validator.py”, line 63, in wrapper
result = await method(view, request, *args, **kwargs)
File “/usr/src/homeassistant/homeassistant/components/intent/init.py”, line 68, in post
intent_result = await intent.async_handle(
File “/usr/src/homeassistant/homeassistant/helpers/intent.py”, line 76, in async_handle
raise InvalidSlotInfo(f"Received invalid slot info for {intent_type}") from err
homeassistant.helpers.intent.InvalidSlotInfo: Received invalid slot info for HassTurnOn

Any hint?

Thank you in advance!

[romkabouter]

@jokerigno

homeassistant.helpers.intent.InvalidSlotInfo: Received invalid slot info for HassTurnOn

You have an invalid slot.
The slot {name} is correct, but {stato} is probably not.

[jokerigno]

oh yes thank you!

For others that may ask how it looks like here it is:

[HassTurnOn]

room_name = (soggiorno | cucina | camera da letto | camera di adele | bagno principale | bagno secondario | corridoio | studio) {room}
light_name = (lampadario | striscia led| lampada scrivania | lampada simonasalvo | lampada fiore | piantana | razzo | jack lamp) {name}
light_cmd = (accendi ) {command}

<light_cmd> [il] [la] <light_name> [in] [ <room_name> ]

[HassTurnOff]
room_name = (soggiorno | cucina | camera da letto | camera di adele | bagno principale | bagno secondario | corridoio | studio) {room}
light_name = (lampadario | striscia led| lampada scrivania | lampada simonasalvo | lampada fiore | piantana | razzo | jack lamp) {name}
light_cmd = (spegni) {command}

<light_cmd> [il] [la] <light_name> [in] [ <room_name> ]

Now another question:

I do have several lampadario (ceiling lamp), one in every room. So If I write:

accendi (turn on) lampadario in studio (roomroom_name) nothing happens and no error in HA logs. Does intent is able to manage also area_id besides entity_id?

Thank you in advance!

[romkabouter]

@jokerigno

Does intent is able to manage also area_id besides entity_id?

No, so you cannot use this with build-in intents this way.
If you want this, you need to create your own intent_script with entity_id’s as templates.

[jokerigno]

Ok. There’s any example out there or a coobook?

I think that my question are made thousands of times

[romkabouter]

No cookbook, but this may help
Create a new slot named commands with:

accendi:on
spegni:off

Create a new slot named lights with:

lampadario
striscia led
lampada scrivania
lampada simonasalvo
lampada fiore
piantana
razzo
jack lamp

Create a new slot rooms:

soggiorno
cucina
camera da letto
camera di adele
bagno principale
bagno secondario
corridoio
studio

Your sentences:

[Lights]
($commands){command} [il] [la] ($lights){light} [in] [ ($rooms){room} ]

Put something like this in you configuration yaml (I do not know the entity id’s of your lights

intent:
intent_script:
  Lights:
    action:
      - service_template: light.turn_{{ command }}
        data_template:
          entity_id: 'light.{{ light }}_{{ room }}'
      - service: mqtt.publish
        data:
          payload_template: '{"sessionId": "{{_intent.sessionId}}",
        "text": "Ok, {{ light }} {{ command }}"}'
          topic: hermes/dialogueManager/endSession

[romkabouter]

light.turn_{{ command }} will become light.turn_on or light.turn_on
light.{{ light }}{{ room }} will become light.lampadario soggiorno, this might be tricky because I do not know your entitiy-id’s

The intent_script will also close the Rhasspy session with the mqtt.publish call

[jokerigno]

Ok clear.

Maybe I can use this:

    action:
      - service_template: light.turn_{{ command }}
        data_template:
          entity_id: 'light.{{ light }}'
          area_id: '{{ room }}'

according to this .

Another question (sorry!) ok for creating manual slots but I was wondering if those can be “automatically” created based on HA entities. Looking here it tells about slot_programs/hass/entities but it is no clear on how to create it.

Maybe pasting this file in /slot_programs/hass/entities ? It is enough? (EDIT: it doesn’t!. Even if the file is executable)

[romkabouter]

@jokerigno

Maybe I can use this:

Possitbly yes, but then you must call it with “target”

@jokerigno

I was wondering if those can be “automatically” created based on HA entities

Yes, that is where slot_programs are for
That file should not go in slot_programs/hass/entities but in slot_programs/hass. The file itself is called entities, without extension

Here is some extra info (I have helped the user with the problem):

Home Assistant & slot_programs/hass/entities: entities download issues

Hi. First of all let me thank you for this awesome project. I really love it! I set up now everything and it’s working. But I cannot get rhasspy to download the entities. Here are the steps I took to get it partially work. At first some basic infos. HA: Runs in a venv with the recent version 2021.5.1 Rhasspy: v2.5.10 runs in a docker environment on the same system downloaded the script “entitites” from https://github.com/rhasspy/rhasspy/tree/master/examples/homeassistant/profile/slot_pro…

It might be a bit too much info, but this is important:

Also note that the script outputs the friendly names and the sentences will be trained on the friendly names.

It depend on this if you can use it in your intent_script or not

[jokerigno]

Uhm there’s something wrong.

Here’s a recap of all the steps:

1. Downloaded entities in rhasspy/it/slot_programs/hass

2. Changed proprierties to allow execution (see screenshot above).

3. Reboot Rhasspy

4. Added custom commands to sentences.ini

[HassTurnOn]
lights = $hass/entities,light
switches = $hass/entities,switch
entities = <lights> | <switches>
turn on [the] (<entities>){name}

[HassTurnOff]
turn off [the] (<HassTurnOn.entities>){name}

[HassOpenCover]
covers = $hass/entities,cover
open [the] (<covers>){name}

[HassCloseCover]
close [the] (<HassOpenCover.covers>){name}

[HassToggle]
entities = <HassTurnOn.entities> | <HassOpenCover.covers>
toggle [the] (<entities>){name}

[HassLightSet]
set [the] (<HassTurnOn.lights>){name} [color] to ($hass/colors){color}
set [the] (<HassTurnOn.lights>){name} brightness to (0..100){brightness}
set [the] (<HassTurnOn.lights>){name} to (0..100){brightness} percent brightness

Saved and retrain.

Here’s a error:
CalledProcessError: Command '['/profiles/it/slot_programs/hass/entities', 'switch']' returned non-zero exit status 127.

Full log:

[ERROR:2021-07-09 15:27:10,209] rhasspyserver_hermes: Command ‘[’/profiles/it/slot_programs/hass/entities’, ‘switch’]’ returned non-zero exit status 127.
Traceback (most recent call last):
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1313, in api_train
result = await core.train()
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 365, in train
word_transform=word_transform,
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/train.py”, line 87, in sentences_to_graph
slot_visitor=word_visitor,
File “/usr/lib/rhasspy/rhasspy-nlu/rhasspynlu/slots.py”, line 87, in get_slot_replacements
slot_command, universal_newlines=True
File “/usr/lib/python3.7/subprocess.py”, line 395, in check_output
**kwargs).stdout
File “/usr/lib/python3.7/subprocess.py”, line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[’/profiles/it/slot_programs/hass/entities’, ‘switch’]’ returned non-zero exit status 127.
[DEBUG:2021-07-09 15:27:10,196] rhasspynlu.slots: Running program for slot hass/entities,switch: [’/profiles/it/slot_programs/hass/entities’, ‘switch’]
[DEBUG:2021-07-09 15:27:10,194] rhasspynlu.ini_jsgf: Loaded ini file
[DEBUG:2021-07-09 15:27:10,193] rhasspyserver_hermes: Generating intent graph
[DEBUG:2021-07-09 15:27:10,193] rhasspyserver_hermes: Loading sentences from [PosixPath(’/profiles/it/sentences.ini’)]
[INFO:2021-07-09 15:27:10,192] rhasspyserver_hermes: Starting training
[DEBUG:2021-07-09 15:27:08,679] rhasspyserver_hermes: Writing /profiles/it/sentences.ini

[romkabouter]

Do you have switches? It might be that the error occurs when you do not have them
Try only lights first

[jokerigno]

I did before posting (and I do have switches)

Sentences.ini

[HassTurnOn]
lights = $hass/entities,light
entities = <lights>
turn on [the] (<entities>){name}

[HassTurnOff]
turn off [the] (<HassTurnOn.entities>){name}

Error:
CalledProcessError: Command '['/profiles/it/slot_programs/hass/entities', 'light']' returned non-zero exit status 127.

[romkabouter]

Exit 127 is a command not found:
https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html
Maybe the file cannot be found

Maybe better to first focus on your problem with the area then.
Like I said, maybe the friendly names or not usable for you anyway?

[jokerigno]

@romkabouter

Maybe the file cannot be found

But the file is there with the proprierties shown in the screenshot.

Also @nightingale seems to have same issue?

I tried it then to add the following to the sentences, but it did not work as it complained about not existing slots (e.g) $hass/entities,light and $hass/entities,switch

Regarding the area_id how can I test it if I am not able to connect Rhasspy to HA? Adding manually all the entities in slots?

For my curiosity: can Rhasspy use something like aliases in google assistant? For example calling a light

1. Striscia LED
2. LED
3. Striscia

[romkabouter]

@jokerigno

Regarding the area_id how can I test it if I am not able to connect Rhasspy to HA? Adding manually all the entities in slots?

Start small, add a couple of them first to get things going.

[jokerigno]

OK I will add “manually” some light and see if the intent_works. I will come back with a feedback.

[romkabouter]

Are you using Home Assistant OS and Addon or something different?
Also, can you post a screen shot of the URL and token? Maybe there is an issue with that and the script needs it.

[jokerigno]

Home assistant OS in a VM. Rhasspy is another docker (not the addon).

Below is how the settings appears:

[romkabouter]

Looks ok, you say Rhasspy is in a docker.
Are you sure the path on your host: /mnt/user/appdata/rhasspy/it/slot_programs/hass is in the docker mapped to /profiles/it/slot_programs/hass?

It looks like you are missing the profiles folder, it might be:
/mnt/user/appdata/rhasspy/profiles/it/slot_programs/hass

[jokerigno]

I mapped the whole /profiles

infact if I for example rebuild the docker with en profile another folder is created. And when I manually added

colors in /mnt/user/appdata/rhasspy/it/slots/hass I found the file without issues

EDIT:

I tested the intent and this “works” (if I switch accendi to on).

on la piantana in soggiorno

ChangeLightState:

    action:

      - service_template: >- #light.turn_{{ command }}
        target:
          area_id: "{{ room }}"
          entity_id:
            - "light.{{ light }}"

I tried to change the service template with this:

  - service_template: >-
      {% if {{ command }} = "accendi" %}
        light.turn_on
      {% else %}
        light.turn_off
      {% endif %}

But it is wrong.

So the connection with HA “exist” and the area_id works too. I still need to understand (with your help) why the script is not loaded. And the figure out how to write the template to manage command correctly. I found here a topic from a guy yesterday but it is made as an automation so the logic is different:

trigger:
    platform: event
    event_type: rhasspy_ChangeLightState
  action:
    service_template: >
      {% if trigger.event.data["command"] == "accendi" %}
        light.turn_on
      {% else %}
        light.turn_off
      {% endif %}
    data_template:
      entity_id: "light.{{ trigger.event.data['name'] }}