How do i use text to speech

https://rhasspy.readthedocs.io/en/latest/text-to-speech/

Oh!! Welcome to Rhasspy!! :slight_smile:

I’m not sure how to do that with MQTT because I’m just learning about MQTT but here are the descriptions of the different APIs currently available: https://rhasspy.readthedocs.io/en/latest/reference/#mqtt-api

I’ve been having Home Assistant automations use rest command to post to Rhasspy via the HTTP API at /api/text-to-speech to answer questions. I’ve never been so excited to hear the weather forecast. :sunny:

You cannot publish text to the TTS over MQTT, that is because it is not in the Hermes protocol.
It would be a good addition however.

For now, you can just post your text to the TTS Api endpoint:
https://rhasspy.readthedocs.io/en/latest/reference/#http-api

In your case: http://<your_ip>:12101/api/text-to-speech

The Hermes protocol implements TTS…
https://docs.snips.ai/reference/hermes#text-to-speech-tts

The current version of Rhasspy might not currently handle these MQTT topics though.

1 Like

I should have been a bit more accurate indeed, I meant Rhasspy.

1 Like

Can you please provide an example of yaml file(s) How you did it in Home assistant.

I have the Configurator add-on installed for Hass.io and visible in the sidebar.

In the config file (configuration.yaml) I have this line with all the other ! lines:
rest_command: !include rest_commands.yaml

Using Configurator, I created a file called rest_commands.yaml and this is what is in that:

rhasspy_speak:
 url: '{{ ipnum }}/api/text-to-speech'
 method: 'POST'
 headers:
   accept: 'text/plain'
   Content-Type: 'text/plain'
 payload: '{{ payload_data }}'

Then, I make an automation. I could do it via Configurator but I like the Automations UI better.

The automation in automations.yaml looks like this:

- id: '1577906020346'
  alias: Rhasspy GetTime
  description: ''
  trigger:
  - event_data: {}
    event_type: rhasspy_GetTime32
    platform: event
  - event_data: {}
    event_type: rhasspy_GetTime33
    platform: event
  condition: []
action:
  - data_template:
      ipnum: "{% if trigger.event.event_type == \"rhasspy_GetTime32\" %}\n  http://192.168.1.32:12101\n\
        {% elif trigger.event.event_type == \"rhasspy_GetTime33\" %}\n  http://192.168.1.33:12101\n\
        {% endif %}\n"
      payload_data: "{% set response %}\n{% if now().hour > 12 %}\n  {% set the_hours\
        \ = now().hour - 12 %}\n{% else %}\n  {% set the_hours = now().hour %}\n{%\
        \ endif %}\nthe time is {{ the_hours }} {{ now().minute }}\n{% endset %} {{\
        \ response }}\n"
    service: rest_command.rhasspy_speak

In the UI the action edited as yaml looks like this:

data_template:
  ipnum: |
    {% if trigger.event.event_type == "rhasspy_GetTime32" %}
      http://192.168.1.32:12101
    {% elif trigger.event.event_type == "rhasspy_GetTime33" %}
      http://192.168.1.33:12101
    {% endif %}
  payload_data: |
    {% set response %}
    {% if now().hour > 12 %}
      {% set the_hours = now().hour - 12 %}
    {% else %}
      {% set the_hours = now().hour %}
    {% endif %}
    the time is {{ the_hours }} {{ now().minute }}
    {% endset %} {{ response }}
service: rest_command.rhasspy_speak

Triggers are events formatted with rhasspy_<INTENT_NAME_HERE>. I have two Pis running Rhasspy, so this automation returns a spoken response to the Pi where the question was asked. I have [GetTime32] and [GetTime33] in the sentences.ini on each Rhasspy Pi, respectively.

In the action, it’s important to use data_template and/or service_template, as needed, depending on where your changeable/dependent parts are.

Hope this helps!

PS - I know this isn’t elegant but it is functional. :blush:

If you meant weather, specifically, I have this in configuration.yaml as a command_line sensor.

sensor:
   platform: command_line
   name: weatherBrooklyn
   json_attributes:
     - forecasts
   command: "curl 'https://api.weather.com/v1/geocode/40.59/-73.94/forecast/daily/7day.json?apiKey=ANAPIKEY&language=en-US&units=e'"
   value_template: '{{ value_json.forecasts[0].narrative }}'

With proper spacing that you can see here: https://www.home-assistant.io/integrations/sensor.command_line/

That takes the json from that page and gets the forecast for Sheepshead Bay, Brooklyn. I have other locations set up as sensors, too.

In Rhasspy, I have the different locations that I can ask about being passed as “place” to Home Assistant for intent handling. The automation has this in the action:

data_template:
  ipnum: |
    {% if trigger.event.event_type == "rhasspy_GetWeather32" %}
      http://192.168.1.32:12101
    {% elif trigger.event.event_type == "rhasspy_GetWeather33" %}
      http://192.168.1.33:12101
    {% endif %}
  payload_data: >
    {% if trigger.event.data.place == "brooklyn" %}
      {% set place_sensor = states.sensor.weatherbrooklyn.attributes.forecasts %}
      {% set place_name = "Brooklyn" %}
    {% else %}
      {% set place_sensor = states.sensor.weatherOther.attributes.forecasts %}
      {% set place_name = "Other" %}
    {% endif %}

    {% if trigger.event.data.relative_day == "weekend" %}
      {% set cal_day = "saturday" %}
    {% else %}
      {% set cal_day = trigger.event.data.calendar_day %}
    {% endif %}

    {% if cal_day == place_sensor.0.dow.lower() or
    trigger.event.data.relative_day == "today" or
    trigger.event.data.relative_day == "tonight" or
    trigger.event.data.relative_day == "" %}
      {% set day_num = 0 %}
      {% set next_day_num = 1 %}
    {% elif cal_day == place_sensor.1.dow.lower() or
    trigger.event.data.relative_day == "tomorrow" %}
      {% set day_num = 1 %}
      {% set next_day_num = 2 %}
    {% elif cal_day == place_sensor.2.dow.lower() %}
      {% set day_num = 2 %}
      {% set next_day_num = 3 %}
    {% elif cal_day == place_sensor.3.dow.lower() %}
      {% set day_num = 3 %}
      {% set next_day_num = 4 %}
    {% elif cal_day == place_sensor.4.dow.lower() %}
      {% set day_num = 4 %}
      {% set next_day_num = 5 %}
    {% elif cal_day == place_sensor.5.dow.lower() %}
      {% set day_num = 5 %}
      {% set next_day_num = 6 %}
    {% elif cal_day == place_sensor.6.dow.lower() %}
      {% set day_num = 6 %}
      {% set next_day_num = 7 %}
    {% else %}
      {% set day_num = 0 %}
      {% set next_day_num = 1 %}
    {% endif %}

    {% set response %}
      {% if trigger.event.data.relative_day == "today" or trigger.event.data.relative_day == "" %}
        the forecast for {{ place_name }} {{ trigger.event.data.relative_day }} is {{ place_sensor[day_num].narrative }}
      {% elif trigger.event.data.relative_day == "tonight" %}
        the forecast for {{ place_name }} {{ trigger.event.data.relative_day }} is {{ place_sensor[day_num].night.narrative }}
      {% elif trigger.event.data.relative_day == "tomorrow" %}
        the forecast for {{ place_name }} {{ trigger.event.data.relative_day }}
          {% if trigger.event.data.time == "night" %}
            {{ trigger.event.data.time }} is {{ place_sensor[day_num].night.narrative }}
          {% endif %}
        is {{ place_sensor[day_num].narrative }}
      {% elif trigger.event.data.relative_day == "weekend" %}
        the forecast for {{ place_name }} on Saturday is {{ place_sensor[day_num].narrative }} followed by Sundays forecast of {{ place_sensor[next_day_num].narrative }}
      {% else %}
        the forecast for {{ place_name }} {{ cal_day }} is {{ place_sensor[day_num].narrative }}
      {% endif %}
    {% endset %} {{ response }}

service: rest_command.rhasspy_speak

Again, I know this isn’t elegant and it is fragile but it’s working for me. The important parts are the trigger events of rhasspy_YOUR_INTENT_NAME_HERE, the data_template and/or service_template, and the command to post the text to Rhasspy. It took me forever to figure out formatting and I just got it by playing around. The template playground under Developer Tools was helpful. Good luck!

1 Like

Thank you for the examples.
I am at work now, this evening I will try to implement it in my system, and we will see how it goes.

I particularly like how you differentiated between your two satellite/client Rhasspy nodes. :slight_smile:
I could be wrong, but somewhere, sometime soon, this should be available from Rhasspy by default. I think @synesthesiam has mentioned it in the past.

data_template:
ipnum: |
    {% if trigger.event.event_type == “rhasspy_GetWeather32” %}
        http://192.168.1.32:12101
    {% elif trigger.event.event_type == “rhasspy_GetWeather33” %}
        http://192.168.1.33:12101
    {% endif %}
1 Like

Hi, I’m new in using rhasspy and also home assistant and had the same question. How can i find the way back to my satellites? I decided to use only the hermes protocol for intent handling also for home assistant, because in the protocol there is the siteId as field included. That means I can identify from which satellite the intent comes.

And because I don’t want to change the code on every new satellite, I used the ip and port as siteId like “192_168_1_33_12101”. So I can parse it to create the http request to /api/text-to-speech’.
But the better way for integrating into home assistant would be to send extra data like siteId, ip or whatever is needed.

For me worked publishing to the topic “hermes/tts/say” with “{“text”: “Das ist ein Beispiel”, “lang”: “de”}”

If i have a speaker setup as an entity in Hass.io or a browser setup as a audio device how can I use the rest call outlined in your message to send audio to that particular device ?

Hi,

for me the following works to use MQTT from HA to return a message to be spoken on the satellite:

automation:
  - alias: "Start vacuuming the house"
    trigger:
      platform: event
      event_type: rhasspy_StartVacuum
    action:
      - service: mqtt.publish 
        data:
        # jinja templating           
        payload_template: '{"siteId": "{{trigger.event.data._intent.siteId}}", "text": "Starting to vacuum the  house"}'
        topic: hermes/tts/say

Note that you need to have your MQTT broker registered on HA.

I would advise to rather use the high-end topic hermes/dialogueManager/startSession with:

{
  "siteId": "<site>",
  "init": {
    "type": "notification",
    "text": "<text>"
  }
}

@fastjack did you also manage to get hermes/dialogueManager/continueSession working? I’m looking for a pattern where I can pick up another command from the user after the first intent has been recognized

As I am not using Rhasspy anymore (I made my own system), I cannot be sure.

Other users on this forum have successfully used it so the topic hermes/dialogueManager/continueSession should work.

Hi @Riccardo_Pinosio,

This post used continueSession in combination with Home Assistant: How to Enable Continuous Mode

I’m using hassio and rhasspy. Hassio answers via mqtt with a script, I created.

Version 1: (with sessionID)
When session ist started, TV will be muted. When sesseion ended, TV will be unmuted

  script_talk_to_satellite:
    alias: script_talk_to_satellite
    sequence:
    - service: mqtt.publish
      data_template:
        topic: hermes/dialogueManager/endSession
        payload_template: >
          {
            "siteId": "{{siteId}}",
            "text":" {{text}} ",
            "sessionId":"{{sessionId}}"
          }

Version 2: (without sessionID)
Just to tell something, like “good morning”

  script_talk_to_satellite_say:
    alias: script_talk_to_satellite_say
    sequence:
    - service: mqtt.publish
      data_template:
        topic: hermes/tts/say
        payload_template: >
          {
            "siteId": "{{siteId}}",
            "text":" {{text}} "
          }  

Hassio event example:
If no room is told, the sitId of the satellite should be taken
entity_ids are named like: roomname_devicename (wohnzimmer_deckenlampe)

- id: 'rhasspy_ChangeLightState'
  alias: rhasspy_ChangeLightState
  trigger:
  - platform: event
    event_type: rhasspy_ChangeLightState

  action:      
  # switch on/off/toggle light
  - service: light.turn_ ...
   ...
   ...
 
 # Answer
  - service: script.turn_on
    data_template:
        entity_id: script.script_talk_to_satellite
        variables:
          siteId: >
            {{trigger.event.data._intent.siteId}}
          sessionId: >
            {{trigger.event.data._intent.sessionId}}
          text: >
            HERE'S THE TEXT TO SAY

I hope, that helps

Hi, Im new to all the hassio topics, and still learning. But had some trouble with TTS the Rhasspy. What Im trying to do is that Hassio announces if I turned a light or open a door.
Im currently tryning to do it by API. I have tried with mqtt but when I switch the option onthe web server to external, the voice cannot be tested anymore. I wonder if its just the spanish lenguage.
If someone could please help or just point out the right direction that would be enough.