How to specify voice-responses in sentence.ini

Help
,
1

First of all: BIG THX for this awesome project

How can i specify to overwrite the output-voice-response on a “sentence”-level?

must have: On/Off/Wav-file (for a confirmation ding)
nice2have: Text

somthing like this: sentences.ini

[HassTurnOn]
voice_response = false
Turn on ($lights | $switches){name}

or even this: sentences.ini

[HassTurnOn]
voice_response = Okay
Turn on ($lights | $switches){name} *{{<voice_response>}}*

Im new to rhasspy, maybe a better way would be to have a responses.ini with the same “Keys” as the sentences.ini.
This way one could even build responses like: responses.ini

[HassTurnOn]
Okay, turing on {name}
2

We don’t have something like this in Rhasspy. It’s the intent handler’s responsibility to end the session by responding to the intent with the right text message. The intent handler can be Home Assistant, a Node-RED flow, your own Python script, …

1 Like
3

OMG Thank you SOO much, for answering me <3

Yes I’ve read a lot of things for the HomeAssistant-Intents. I think switching from built-in intents to custom once should give me the possibility to solve this problem on the Home-Assistance’s side.

Never the less, i think it would be amazing to at least have the option to skip the response inside Rhasspy. (of course one can just turn of the whole tts)

example:
In sentences like [GetTemperature] or [GetTime] you want to hear the answer.
but in cases of [SetLightToRed] you will get annoyed at some point, hearing “Set {name} color to red”
everytime.
If it is implemented, one could add a trigger-word like “please” to activate the response.
In that way you can have your conversation-feeling if you say “please”, and have your peace if you dont say “please”

The intent handler can be Home Assistant, a Node-RED flow, your own Python script, …

Maybe I can create a small add-on for Rhasspy as python-script.

1 Like
5

Wow, this community is IMPRESSIVE !!!
Thank you for answering me.

I will test it asap after work today

1 Like
6

So you mean that instead of an answer you would like to be able to play an audio file?

1 Like
7

Yes, exactly, but on a per-sentence-level.

8

SSML (Speech Synthesis Markup Language) has a audio tag for this purpose (https://www.w3.org/TR/speech-synthesis/#S3.3.1). So it is possible to use e.g. the sentence

'<audio src="hello_world.wav">hello world</audio>'

which will try to play the audio file and as fallback when the file is not found generate the normal sentence instead.

Rhasspy does not has support for SSML yet, but I am currently working on it :wink:

3 Likes
9

Welcome, @Blade_John! If you’re using Home Assistant, check out my HA example.

If you use the built-in intents through HA, it will automatically respond with text (which Rhasspy will play through your TTS system). So something like “turn on the kitchen light” will have HA turn the light on and respond “turned on kitchen light”.

You can also write a tiny Bash script to do the response, or configure a Node-RED flow :slight_smile:

1 Like
10

I recommend using Node-Red but it depends on what you like more of course. I use Node-Red for all HA and rhasspy automations (Node-Red triggers the intended service calls to HA for each intent).

Node-Red also sends my answer to the rhasspy. I will post my example tomorrow, maybe it will be helpful for you.

1 Like
11

Postet it here: Rhasspy Light and Cover control with Node-RED and HA

1 Like
12

AMAZING answers!! I was in a lot of forums for different things in my life,
but this one is REALLY impressive!!!

Yes this one is NICE !

This was the root cause of this topic.
The HA-autogenerated-respond was a mix of english and german words, spoken with the eSpeak male voice, resulting in really funny audio-outputs.

I managed it using custom Intents: (for german language).
At the moment i can’t use the speak key in HA, to make rhasspy speak.
I’m using a rest_command service.

configuration.yaml 
intent_script:
  GetTemperature:  # Intent type
    action:
      data_template:
        payload: >-
          es sind {{ states("sensor.temperature")|regex_replace(find="\.",
          replace=",", ignorecase=False) }} grad
      service: rest_command.rhasspy_speak
  GetTime:  # Intent type
    action:
      data_template:
        payload: 'es ist {{ now().hour }} uhr {{ now().minute }}'
      service: rest_command.rhasspy_speak

OUTSTANDING! smells like a game-changer.

If anyone is intrested in the german sentences<–>intent for switches & lights in HA:

Expand

sentences.ini

[rhasspYZTurnOnSwitch]
state_on = (ein |  an | einschalten) {state}
\[(kannst | könntest) du] [bitte] [(das | die | den)] ($switches){name} <state_on> [schalten]

[rhasspYZTurnOffSwitch]
state_off = (aus | ausschalten ) {state}
\[(kannst | könntest) du] [bitte] [(das | die | den)] ($switches){name} <state_off> [schalten]

[rhasspYZTurnOnLight]
state_on = (ein |  an | einschalten) {state}
\[(kannst | könntest) du] [bitte] [(das | die | den)] ($lights){name} <state_on> [schalten]

[rhasspYZTurnOffLight]
state_off = (aus | ausschalten ) {state}
\[(kannst | könntest) du] [bitte] [(das | die | den)] ($lights){name} <state_off> [schalten]

[rhasspYZSetLightColor]
\[(kannst | könntest) du] [bitte] [das|die|den] ($lights){name} auf ($colors){color} schalten

[rhasspYZSetLightBrightness]
\[(kannst | könntest) du] [bitte] [das|die|den] ($lights){name} auf (0..100){brightness} Prozent

HA-configuration.yaml:

intent_script:
  rhasspYZTurnOnSwitch:  # Intent type
    action:
      data_template:
        entity_id: >
            switch.{{ name }}
      service_template: >
          switch.turn_on
  rhasspYZTurnOffSwitch:  # Intent type
    action:
      data_template:
        entity_id: >
            switch.{{ name }}
      service_template: >
          switch.turn_off
  rhasspYZTurnOnLight:  # Intent type
    action:
      data_template:
        entity_id: >
            light.{{ name }}
      service_template: >
          light.turn_on
  rhasspYZTurnOffLight:  # Intent type
    action:
      data_template:
        entity_id: >
            light.{{ name }}
      service_template: >
          light.turn_off
          
  rhasspYZSetLightColor:  # Intent type
    action:
      data_template:
        entity_id: >
            light.{{ name }}
        color_name: '{{ color }}'
      service_template: >
          light.turn_on
          
  rhasspYZSetLightBrightness:  # Intent type
    action:
      data_template:
        entity_id: >
            light.{{ name }}
        brightness: '{{ ( brightness * 255 / 100 ) | int }}'
      service_template: >
          light.turn_on

I also tried to merge the 4 intents into 1, but that didn’t work:

Summary 
[rhasspYZ]
state_on = (ein |  an | einschalten) {state}
state_off = (aus | ausschalten ) {state}
new_state = <state_on> | <state_off>
\[(kannst | könntest) du] [bitte] [(das | die | den)] ($lights | $switches){name} (<new_state>){state} [schalten]

in the rhasspy log i can see “entity” and “state”, but it is not accessable in HA-templating.

Boom thx for the example. I don’t know why, but im unable to like node-red yet.
Feels like an unneccessary man-in-the-middle. Please don’t get me wrong, the problem is ME, not the world. At the moment it seems, everything can be done within HA (automations, scripts, intents, events). If at some point I can find some real nice benefits, I might fall to dark-side (i mean the red-side)

Nice one, next world opened. thx :slight_smile:

Offtopic: What’s the downside of maryTTS? I tested the online-demo, and it sounds much better than the eSpeak. Maybe perfomance is an issue on a pi 3b+ ?

Thank you for the nice spirit here!!!

Gratefully
Blade

2 Likes
13

Part of your problem with this sentence is most likely that you got the slot tag behind state_on and state_off and a second time behind them in the actual sentence.

You should try this instead:

fixed sentence 
[rhasspYZ]
state_on = (ein |  an | einschalten)
state_off = (aus | ausschalten )
new_state = (<state_on> | <state_off>)
\[(kannst | könntest) du] [bitte] [(das | die | den)] ($lights | $switches){name} (<new_state>){state} [schalten]
the sentences i use for my `ChangeLightState` intent

[ChangeLightState]
light_name = $lampe {name}
light_state = ((ein|an):ein | (aus|ab):aus | heller | dunkler) {state}
[(schalte | schalt)] [(die | das | den)] <light_name> <light_state>
(((deaktiviere):aus)|((aktiviere):ein)) {state} [(die | das | den)] <light_name>
<light_name> ((ausschalten|deaktiviere|abschalten):aus | (einschalten|anschalten|aktiviere):ein {state}

1 Like
14

Nice works, THANKS

My Light intents are:

[rhasspYZSetLightColor]
\[(kannst | könntest) du] [bitte] [das|die|den] ($lights){name} auf ($colors){color} schalten

[rhasspYZSetLightBrightness]
\[(kannst | könntest) du] [bitte] [das|die|den] ($lights){name} auf (0..100){brightness} Prozent
15

I’m using automations.yaml to compose my voice responses in Home Assistant. Where I’ve run into a bit of difficulty is obtaining the slots rawValue for any given slot. Below is the example JSON returned from the Rhasspy event where I’d like to obtain the rawValue data for the slot where slotName = “light”. It should be “brass lamp”.

What I have tried is shown here, but alas, neither are working. :frowning:

{{ trigger.event.data._intent.slots["light"].rawValue }}
{{ trigger.event.data._intent.slots.light.rawValue }}

So exactly how should I specify a slot by name?

{
    "event_type": "rhasspy_ChangeLightState",
    "data": {
        "light": "switch.brass_lamp",
        "state": "off",
        "_text": "switch.brass_lamp off",
        "_raw_text": "brass lamp off",
        "_intent": {
            "input": "switch.brass_lamp off",
            "intent": {
                "intentName": "ChangeLightState",
                "confidenceScore": 1
            },
            "siteId": "Listener1",
            "id": null,
            "slots": [
                {
                    "entity": "light",
                    "value": {
                        "kind": "Unknown",
                        "value": "switch.brass_lamp"
                    },
                    "slotName": "light",
                    "rawValue": "brass lamp",
                    "confidence": 1,
                    "range": {
                        "start": 0,
                        "end": 17,
                        "rawStart": 0,
                        "rawEnd": 10
                    }
                },
                {
                    "entity": "state_on_off",
                    "value": {
                        "kind": "Unknown",
                        "value": "off"
                    },
                    "slotName": "state",
                    "rawValue": "off",
                    "confidence": 1,
                    "range": {
                        "start": 18,
                        "end": 21,
                        "rawStart": 11,
                        "rawEnd": 14
                    }
                }
            ],
            "sessionId": "Listener1-porcupine-1ed92005-9da1-433a-8d7d-671cd79390de",
            "customData": null,
            "asrTokens": [
                [
                    {
                        "value": "switch.brass_lamp",
                        "confidence": 1,
                        "rangeStart": 0,
                        "rangeEnd": 17,
                        "time": null
                    },
                    {
                        "value": "off",
                        "confidence": 1,
                        "rangeStart": 18,
                        "rangeEnd": 21,
                        "time": null
                    }
                ]
            ],
            "asrConfidence": null,
            "rawInput": "brass lamp off",
            "wakewordId": "porcupine",
            "lang": null
        }
    },
    "origin": "REMOTE",
    "time_fired": "2020-07-05T18:15:43.376664+00:00",
    "context": {
        "id": "278fc19a54f349f7948dfa12882e2913",
        "parent_id": null,
        "user_id": "26ec3060f78f4880a6a70e903924eb1f"
    }
}
16

@FredTheFrog Why not using the Snips integration directly?

It is much easier to get all these info.

17

I’ve never used Snips before! :slight_smile: The good news is, I’ve figured out that because the light is always Slots[0] I can use that notation:
{{ trigger.event.data._intent.slots[0].rawValue }}

18

@FredTheFrog maybe this will work:
{{ trigger.event.data._intent.slots[0].rawValue }}
This will only work if light is the first slot returned.
slots is an array containing objects.

seems like you already figured it out.

1 Like
19

Take a look at

It is completely compatible with Rhasspy since Rhasspy 2.5 respect the Hermes protocol.

It greatly simplify the intent handling with Home Assistant.

Hope this helps.

@synesthesiam Maybe a Rhasspy Home Assistant integration based on the Snips one can be proposed to the HA guys…

1 Like
20

@moqart and @fastjack Thank you both for your very timely and helpful replies. I’ve just placed my first 2.5.0/voltron rhasspy unit into ‘Production’ mode. I had to learn the newer configuration syntax for sentences and slots first. I’m still using HA events to respond using HA automations. Maybe I’ll learn to use Snips/hermes next. :wink:

2 Likes
21

Proposing that to the HA developers is an EXCELLENT idea. My concern is they might soon deprecate the Snips interface, since Snips went closed-source.

22

Or just use Node-Red and get rid of that awful YAML scripting :wink:

1 Like
23

@FredTheFrog
you might find this simpler.
At the top of your event data, i can see you are using ‘light’ and ‘state’ as your tag/ property in Rhasspy. These can be accessed in a more straightforward fashion:-

{% set var2 = trigger.event.data.light %}
{% set var3 = trigger.event.data.state %}

1 Like
24

We actually sort of got this already with the “intent” HTTP endpoint :slight_smile:

If you just add intent: to your HA configuration YAML and switch Rhasspy from “event” to “intent” mode for HA, Rhasspy will trigger intents directly in HA. This works with the built-in intents as well as intent_script ones.

25

Thank you, @Gfawkes. While simpler :slight_smile: it doesn’t provide the correct text for voice confirmation. :frowning:
state is simply “on” or “off” which does translate directly/correctly.
light is actually “switch.brass_lamp” which I don’t want spoken. I just want “brass lamp” which is in trigger.event.data._intent.slots[0].rawValue

{
    "event_type": "rhasspy_ChangeLightState",
    "data": {
        "light": "switch.brass_lamp",
        "state": "on",
        "_text": "switch.brass_lamp on",
        "_raw_text": "brass lamp on",
        "_intent": {
            "input": "switch.brass_lamp on",
            "intent": {
                "intentName": "ChangeLightState",
                "confidenceScore": 1
            },
            "siteId": "Listener1",
            "id": null,
            "slots": [
                {
                    "entity": "light",
                    "value": {
                        "kind": "Unknown",
                        "value": "switch.brass_lamp"
                    },
                    "slotName": "light",
                    "rawValue": "brass lamp",
                    "confidence": 1,
                    "range": {
                        "start": 0,
                        "end": 17,
                        "rawStart": 0,
                        "rawEnd": 10
                    }
                },

I guess what I’m really asking is, how to update this sentence definition to send the value of <light> in the HA event? Bonus points if it defines three distinct values: lightSwitch, lightName, lightState.

[ChangeLightState]
light=$light
state=$state_on_off
<light>{light} <state>{state}
(turn | switch | cut) [the] <light>{light} <state>{state}
26

Just enable the snips integration and configure you HA instance and Rhasspy to both subscribe to the same MQTT broker (either Rhasspy builtin broker or HA MQTT integration). Then use the intent_scripts configuration to handle everything.

Much simpler…

1 Like
27

Thank you for that, @fastjack. Unfortunately, I am one of those folks that do not use MQTT for anything, at all. And I don’t want to setup anything new using MQTT. I can currently get the rawValue from the HA event, but was wanting to configure my Rhasspy SLOTs and SENTENCEs to directly provide this value.

28

Oh I see :slightly_smiling_face: Maybe using something like slot programs with the HA API… I think I recall others on this community to have done it.

29

I have just got Rhasspy working with 2 satellites with Home Assistant (HASS) all through MQTT that’s sitting on HA server.

I am NEW at all this and not a programmer so go easy on me please. I am really nervous about posting this!

Sentences.ini

[ChangeLightState]
light_name = ((bedroom light:light.0x0017880103b97549_light | living room light:light.0x0017880103b97557_light | garage light | matts office:light.matts_office_colour | alert light:light.anita_alert_bulb_light) {name}) | <ChangeLightColor.light_name>
light_state = (on | off) {state}

In the above “bedroom light” is the word you say and “light.0x0017880103b97549_light” is entity name in HA (yes I know I could rename everything in HA but that would be a real pain at this stage)

In HA I have this automation:

alias: Test - Rhasspy on MQTT
description: ''
trigger:
  - platform: mqtt
    topic: hermes/intent/+
condition:
  - condition: template
    value_template: '{{ trigger.payload_json.intent.intentName ==  ''ChangeLightState'' }}'
action:
  - service: mqtt.publish
    data:
    topic: hermes/tts/say
    payload_template: >
         {"text": "you switched the light on, it was {{ trigger.payload_json.asrTokens[0].3.value }}" ", "siteId": "{{ trigger.payload_json.siteId }}", "lang": null, "id": "79f31e80-c4ad-4952-a8b7-4b22f2d6a666", "sessionId": "", "volume": 1.0}
  - service_template: 'light.turn_{{ trigger.payload_json.asrTokens[0].1.value }}'
    data: {}
    target:
      entity_id: '{{ trigger.payload_json.asrTokens[0].3.value }}'
mode: single

trigger: so this is the bit where it listens on MQTT (you need to point your Rhasspy server at the shared MQTT server, in case the HA built in one the topic is the standard hermes one.
condition: template = This looks for the Rhasspy sentence you just said in this case “ChangeLightState”
action: so the first service: mqtt.publish which sends text back to the satellite to tell you what you just did, this is for testing really or you can use an mqtt listener, not really sure what the id string is, but that was a random one and it seems to work fine with it staying the same :man_shrugging: same for rest of them, not tested removing them totally, I should. Anyway the siteId is the important one, that tells Rhasspy which speaker to send the speech back to.
action: so call the service based and use the rhasspy off or on on the end (I was proud of that)
then grab the name after the : in the json string from rhasspy which would be light.0x0017880103b97549_light if i said bedroom light

Thats it, as I said I have no idea if this is the right or the wrong way to go about it, but being a newbie this seemed easier than learning intents and having to restart HA all the time, with this I can do pretty much anything. I know intents maybe easier with lights, but this way I can use anything, not just switches and lights and a really odd shopping list thing in HA. Thats my next thing to try :slight_smile:

What I would really like to do is send a .wav to the satellite can anyone help me do that ELI5 me please :slight_smile:

30

If you want to respond to and close a Rhasspy dialogue the correct topic is hermes/dialogueManager/endSession.
Both with a sessionID and a text, see this documentation:

The sessionId can probably found in the payload_json, but I use events instead of intents.
My automations are triggered by the Rhasspy events (I have set to send events instead of intents in Rhasspy)

Check for my example here:

31

Hi Romkabouter,

I get your point, but I am not doing a dialogue I just want to send a one-off message to a satellite, so is using hermes/tts/say wrong?

I avoided events because they don’t seem as flexible as using MQTT, everything in my house also uses MQTT so going fully MQTT with Rhasspy seemed like a good choice. I can send MQTT direct from Rhasspy to other things that are not HA compatible for example.

Now I am a bit worried this is the wrong way to go about things? I don’t want to go down a path that will get blocked later?

Thanks for the repsonse

MattV

32

Well, if you are using Rhasspy to trigger actions, it is best practise to end the session created by Rhasspy. You actually áre doing a dialogue when you ask Rhasspy to turn a light on or off.
Subscribe to hermes/dialogueManager/# and you will see what happens.

tts/say is not exactly wrong, but the dialogueManager produces a timeout on the session after 30sec or so. If you use endSession with the correct sessionID, the session you started is closed in the proper way. The sessionID is available on trigger.payload_json.sesionID, so you can achieve closing a session via MQTT as well. You already publish the siteID, just add the sessionID, use the endSession topic and your done :slight_smile:
Also remove the “id”: “79f31e80-c4ad-4952-a8b7-4b22f2d6a666” part. The id is actually the same as the sessionID
The spoken text will always be “you turned the light on”, even if you turned it off. Is there a reason you use asrTokens and not the slots?

Events are as flexible as MQTT, because when used in an automation, you can add a range of service being called, also MQTT.
MQTT is also a fine solution, whatever makes your clock tick :slight_smile:
I am just pointing out that ending a session is good practise, you can also achieve that via MQTT

No worries, there is no wrong or right way. MQTT is very flexible and if you are comfortable with that, its fine.

33

Hi Romkabouter,

No reason apart from I can’t get slots to work, asr split the sentence into words which I can pick off and match, but every time the MQTT payload does not fill the slots up with anything. I have had quite a search but can’t find anything simple enough for me to understand :frowning:

Here is the MQTT intent payload of hermes/intent/#
`

{“input”: “shopping list add parma ham”, “intent”: {“intentName”: “ShoppingListAdd”, “confidenceScore”: 1.0}, “siteId”: “sat2”, “id”: null, “slots”: [], “sessionId”: “sat2-seventy six-31f22c68-9050-4bb1-8020-7334d62c181b”, “customData”: “seventy six”, “asrTokens”: [[{“value”: “shopping”, “confidence”: 1.0, “rangeStart”: 0, “rangeEnd”: 8, “time”: null}, {“value”: “list”, “confidence”: 1.0, “rangeStart”: 9, “rangeEnd”: 13, “time”: null}, {“value”: “add”, “confidence”: 1.0, “rangeStart”: 14, “rangeEnd”: 17, “time”: null}, {“value”: “parma”, “confidence”: 1.0, “rangeStart”: 18, “rangeEnd”: 23, “time”: null}, {“value”: “ham”, “confidence”: 1.0, “rangeStart”: 24, “rangeEnd”: 27, “time”: null}]], “asrConfidence”: 1.0, “rawInput”: “shopping list add parma ham”, “wakewordId”: “seventy six”, “lang”: null}

and the slot[] is empty. Any pointers greatefully received, or do you mean something else entirely?

I see what you mean about the dialogue and I am working on the in MQTT, I might as well try and look as though I can code :slight_smile:

Thanks for not shooting my MQTT stuff down, for some reason I find it easier to understand :man_shrugging:

MattV

34

Hi Guys,

I two questions about this.

  1. when using events its possible to use the dev tools in HA - Events and listen for rhasspy_xyz. How can I do that for intents?
  2. How can I get the sensor states from my sensors? I found the following example many times, but I dont get it yet fully. “text: the temperature is {{ states.sensor.temperature }} degrees”

Actually I have just more than one sensor. So my consideration was that I try it like with the colors. I mean with a slot just for the rooms. Like this:

slots:
hass/rooms
Dachboden
Keller
Küche
Garage
Flur unten
Wohnzimmer
Schlafzimmer
…

sentences:
[RoomTemperature]
room = $hass/room
Wie hoch ist die Temperatur im [room] ($hass/room){room}

configration.yaml
RoomTemperature:
speech:
text: Die Temperatur {{ room }} ist {{ states(“sensor.” + “lumi_weather_temperature_” + room.replace() + “.state”) }} Grad

Unfortunalely I does exactly not what I want. To be honest it’s a little tricky to me, as I am yet not that familiar with yaml and even not with all the settings which can be done in HA
So I would like to set somehow a variable in the configuration.yaml and the variable will be filed by the voice command via rhasspy.
The Sensors entity_id’s are like this:

sensor.fibaro_sensor_flur_unten_air_temperature
sensor.fibaro_sensor_kueche_air_temperature
…
sensor.lumi_lumi_weather_temperature_arbeitszimmer
sensor.lumi_lumi_weather_temperature_badezimmer
sensor.lumi_lumi_weather_temperature_kueche
…

Now when I use the speech line like the following its working.:

text: Es ist momentan {{ states.sensor.lumi_lumi_weather_temperature_wohnzimmer.state }} Grad Celsius im Wohnzimmer

But surely I dont want to set this for all of the sensors, instead have rhasspy filling the variable/placeholder

Any help is much appreciated! :slight_smile:

kr

nightingale

35

Another reason to use events, the slots are trigger.event.data.<slotname>

You can’t