How do i use text to speech

I am running the latest version on a pi4. I have several intents and slots that i handle through MQTT. How do i publish a MQTT message to have the TTS speak some text. For example to answer a question that was asked.

Thanks for replys. But when i searched Rhasspy i found this: https://github.com/rhasspy/rhasspy-tts-cli-hermes/issues which talked about:
rhasspy-tts-cli-hermes

MQTT service for text to speech with external program using the Hermes protocol

but i could not find any documentation. Does anyone know if this is for a future rev or is it currently implemented and if so, how is it used.

https://rhasspy.readthedocs.io/en/latest/text-to-speech/

Oh!! Welcome to Rhasspy!! :slight_smile:

I’m not sure how to do that with MQTT because I’m just learning about MQTT but here are the descriptions of the different APIs currently available: https://rhasspy.readthedocs.io/en/latest/reference/#mqtt-api

I’ve been having Home Assistant automations use rest command to post to Rhasspy via the HTTP API at /api/text-to-speech to answer questions. I’ve never been so excited to hear the weather forecast. :sunny:

You cannot publish text to the TTS over MQTT, that is because it is not in the Hermes protocol.
It would be a good addition however.

For now, you can just post your text to the TTS Api endpoint:
https://rhasspy.readthedocs.io/en/latest/reference/#http-api

In your case: http://<your_ip>:12101/api/text-to-speech

The Hermes protocol implements TTS…
https://docs.snips.ai/reference/hermes#text-to-speech-tts

The current version of Rhasspy might not currently handle these MQTT topics though.

1 Like

I should have been a bit more accurate indeed, I meant Rhasspy.

1 Like

Can you please provide an example of yaml file(s) How you did it in Home assistant.

I have the Configurator add-on installed for Hass.io and visible in the sidebar.

In the config file (configuration.yaml) I have this line with all the other ! lines:
rest_command: !include rest_commands.yaml

Using Configurator, I created a file called rest_commands.yaml and this is what is in that:

rhasspy_speak:
 url: '{{ ipnum }}/api/text-to-speech'
 method: 'POST'
 headers:
   accept: 'text/plain'
   Content-Type: 'text/plain'
 payload: '{{ payload_data }}'

Then, I make an automation. I could do it via Configurator but I like the Automations UI better.

The automation in automations.yaml looks like this:

- id: '1577906020346'
  alias: Rhasspy GetTime
  description: ''
  trigger:
  - event_data: {}
    event_type: rhasspy_GetTime32
    platform: event
  - event_data: {}
    event_type: rhasspy_GetTime33
    platform: event
  condition: []
action:
  - data_template:
      ipnum: "{% if trigger.event.event_type == \"rhasspy_GetTime32\" %}\n  http://192.168.1.32:12101\n\
        {% elif trigger.event.event_type == \"rhasspy_GetTime33\" %}\n  http://192.168.1.33:12101\n\
        {% endif %}\n"
      payload_data: "{% set response %}\n{% if now().hour > 12 %}\n  {% set the_hours\
        \ = now().hour - 12 %}\n{% else %}\n  {% set the_hours = now().hour %}\n{%\
        \ endif %}\nthe time is {{ the_hours }} {{ now().minute }}\n{% endset %} {{\
        \ response }}\n"
    service: rest_command.rhasspy_speak

In the UI the action edited as yaml looks like this:

data_template:
  ipnum: |
    {% if trigger.event.event_type == "rhasspy_GetTime32" %}
      http://192.168.1.32:12101
    {% elif trigger.event.event_type == "rhasspy_GetTime33" %}
      http://192.168.1.33:12101
    {% endif %}
  payload_data: |
    {% set response %}
    {% if now().hour > 12 %}
      {% set the_hours = now().hour - 12 %}
    {% else %}
      {% set the_hours = now().hour %}
    {% endif %}
    the time is {{ the_hours }} {{ now().minute }}
    {% endset %} {{ response }}
service: rest_command.rhasspy_speak

Triggers are events formatted with rhasspy_<INTENT_NAME_HERE>. I have two Pis running Rhasspy, so this automation returns a spoken response to the Pi where the question was asked. I have [GetTime32] and [GetTime33] in the sentences.ini on each Rhasspy Pi, respectively.

In the action, it’s important to use data_template and/or service_template, as needed, depending on where your changeable/dependent parts are.

Hope this helps!

PS - I know this isn’t elegant but it is functional. :blush:

If you meant weather, specifically, I have this in configuration.yaml as a command_line sensor.

sensor:
   platform: command_line
   name: weatherBrooklyn
   json_attributes:
     - forecasts
   command: "curl 'https://api.weather.com/v1/geocode/40.59/-73.94/forecast/daily/7day.json?apiKey=ANAPIKEY&language=en-US&units=e'"
   value_template: '{{ value_json.forecasts[0].narrative }}'

With proper spacing that you can see here: https://www.home-assistant.io/integrations/sensor.command_line/

That takes the json from that page and gets the forecast for Sheepshead Bay, Brooklyn. I have other locations set up as sensors, too.

In Rhasspy, I have the different locations that I can ask about being passed as “place” to Home Assistant for intent handling. The automation has this in the action:

data_template:
  ipnum: |
    {% if trigger.event.event_type == "rhasspy_GetWeather32" %}
      http://192.168.1.32:12101
    {% elif trigger.event.event_type == "rhasspy_GetWeather33" %}
      http://192.168.1.33:12101
    {% endif %}
  payload_data: >
    {% if trigger.event.data.place == "brooklyn" %}
      {% set place_sensor = states.sensor.weatherbrooklyn.attributes.forecasts %}
      {% set place_name = "Brooklyn" %}
    {% else %}
      {% set place_sensor = states.sensor.weatherOther.attributes.forecasts %}
      {% set place_name = "Other" %}
    {% endif %}

    {% if trigger.event.data.relative_day == "weekend" %}
      {% set cal_day = "saturday" %}
    {% else %}
      {% set cal_day = trigger.event.data.calendar_day %}
    {% endif %}

    {% if cal_day == place_sensor.0.dow.lower() or
    trigger.event.data.relative_day == "today" or
    trigger.event.data.relative_day == "tonight" or
    trigger.event.data.relative_day == "" %}
      {% set day_num = 0 %}
      {% set next_day_num = 1 %}
    {% elif cal_day == place_sensor.1.dow.lower() or
    trigger.event.data.relative_day == "tomorrow" %}
      {% set day_num = 1 %}
      {% set next_day_num = 2 %}
    {% elif cal_day == place_sensor.2.dow.lower() %}
      {% set day_num = 2 %}
      {% set next_day_num = 3 %}
    {% elif cal_day == place_sensor.3.dow.lower() %}
      {% set day_num = 3 %}
      {% set next_day_num = 4 %}
    {% elif cal_day == place_sensor.4.dow.lower() %}
      {% set day_num = 4 %}
      {% set next_day_num = 5 %}
    {% elif cal_day == place_sensor.5.dow.lower() %}
      {% set day_num = 5 %}
      {% set next_day_num = 6 %}
    {% elif cal_day == place_sensor.6.dow.lower() %}
      {% set day_num = 6 %}
      {% set next_day_num = 7 %}
    {% else %}
      {% set day_num = 0 %}
      {% set next_day_num = 1 %}
    {% endif %}

    {% set response %}
      {% if trigger.event.data.relative_day == "today" or trigger.event.data.relative_day == "" %}
        the forecast for {{ place_name }} {{ trigger.event.data.relative_day }} is {{ place_sensor[day_num].narrative }}
      {% elif trigger.event.data.relative_day == "tonight" %}
        the forecast for {{ place_name }} {{ trigger.event.data.relative_day }} is {{ place_sensor[day_num].night.narrative }}
      {% elif trigger.event.data.relative_day == "tomorrow" %}
        the forecast for {{ place_name }} {{ trigger.event.data.relative_day }}
          {% if trigger.event.data.time == "night" %}
            {{ trigger.event.data.time }} is {{ place_sensor[day_num].night.narrative }}
          {% endif %}
        is {{ place_sensor[day_num].narrative }}
      {% elif trigger.event.data.relative_day == "weekend" %}
        the forecast for {{ place_name }} on Saturday is {{ place_sensor[day_num].narrative }} followed by Sundays forecast of {{ place_sensor[next_day_num].narrative }}
      {% else %}
        the forecast for {{ place_name }} {{ cal_day }} is {{ place_sensor[day_num].narrative }}
      {% endif %}
    {% endset %} {{ response }}

service: rest_command.rhasspy_speak

Again, I know this isn’t elegant and it is fragile but it’s working for me. The important parts are the trigger events of rhasspy_YOUR_INTENT_NAME_HERE, the data_template and/or service_template, and the command to post the text to Rhasspy. It took me forever to figure out formatting and I just got it by playing around. The template playground under Developer Tools was helpful. Good luck!

1 Like

Thank you for the examples.
I am at work now, this evening I will try to implement it in my system, and we will see how it goes.

I particularly like how you differentiated between your two satellite/client Rhasspy nodes. :slight_smile:
I could be wrong, but somewhere, sometime soon, this should be available from Rhasspy by default. I think @synesthesiam has mentioned it in the past.

data_template:
ipnum: |
    {% if trigger.event.event_type == “rhasspy_GetWeather32” %}
        http://192.168.1.32:12101
    {% elif trigger.event.event_type == “rhasspy_GetWeather33” %}
        http://192.168.1.33:12101
    {% endif %}
1 Like

Hi, I’m new in using rhasspy and also home assistant and had the same question. How can i find the way back to my satellites? I decided to use only the hermes protocol for intent handling also for home assistant, because in the protocol there is the siteId as field included. That means I can identify from which satellite the intent comes.

And because I don’t want to change the code on every new satellite, I used the ip and port as siteId like “192_168_1_33_12101”. So I can parse it to create the http request to /api/text-to-speech’.
But the better way for integrating into home assistant would be to send extra data like siteId, ip or whatever is needed.

For me worked publishing to the topic “hermes/tts/say” with “{“text”: “Das ist ein Beispiel”, “lang”: “de”}”

If i have a speaker setup as an entity in Hass.io or a browser setup as a audio device how can I use the rest call outlined in your message to send audio to that particular device ?

Hi,

for me the following works to use MQTT from HA to return a message to be spoken on the satellite:

automation:
  - alias: "Start vacuuming the house"
    trigger:
      platform: event
      event_type: rhasspy_StartVacuum
    action:
      - service: mqtt.publish 
        data:
        # jinja templating           
        payload_template: '{"siteId": "{{trigger.event.data._intent.siteId}}", "text": "Starting to vacuum the  house"}'
        topic: hermes/tts/say

Note that you need to have your MQTT broker registered on HA.

I would advise to rather use the high-end topic hermes/dialogueManager/startSession with:

{
  "siteId": "<site>",
  "init": {
    "type": "notification",
    "text": "<text>"
  }
}

@fastjack did you also manage to get hermes/dialogueManager/continueSession working? I’m looking for a pattern where I can pick up another command from the user after the first intent has been recognized

As I am not using Rhasspy anymore (I made my own system), I cannot be sure.

Other users on this forum have successfully used it so the topic hermes/dialogueManager/continueSession should work.

Hi @Riccardo_Pinosio,

This post used continueSession in combination with Home Assistant: How to Enable Continuous Mode