Help Getting Started

Help
1

I need some help getting Rhasspy working with Home Assistant and Node-Red.

I’ve read through the docs a few times and I have a grasp of the theory but I need to see full examples to be able to connect the theory with real world usage, it’s just the way my brain works.

I see there are a few existing sentences so I assume I should be able to speak these and have Rhasspy connect with Home Assistant, get the response (via an intent script) and return the information?

My setup is a PiZero W (Satellite), a rhasspy base setup on a VM, Home Assistant (core) on a VM, and Node Red also on a VM. I also have an external MQTT broker running.

To keep it simple I will use the existing [GetTemperature] sentence.
In HA I have in my config:

intent_script: !include intents.yaml

and in intents.yaml I have:

GetTemperature:
speech:
type:
text:
The current temperature outside is
{{ states(‘sensor.temperature’) }}
With a wind chill of
{{ states(‘sensor.wind_chill’) }}
(Sorry the block quoting isn’t honouring the yaml spacing)

So If I ask Rhasspy “whats the temperature”, or “how hot/cold) is it” Rhasspy should send out the ‘GetTemperature’ intent over MQTT, Home Assistant will respond because I have a ‘GetTemperature’ intent setup. HA’s response will be sending the data to Rhasspy who will then vocalize the data received.

Am I understanding things correctly?

2

No, the intent will do speech over your TTS configuration of Home Assistant.
If you want TTS from rhasspy you need to send the text as payload to Rhasspy’s tts api.

Find an example here:

I am using events, not intents however.

3

Ah ok, that makes sense, I would be handing things off to HA. That would be the model I ended up using with Snips as I never got the base communicating back to the satellite properly.
I’d very much like to have Rhasspy be the voice of my house :slight_smile:

I tested the model I detailed above and I don’t think the intent was handed off to HA.
I checked the logs on the base and I see the hotword being detected, then I see the spoken words being received, analized and the proper intent being determined. Then I think I see the intent being published “rhasspynlu_hermes: Publishing 626 bytes(s) to hermes/intent/GetTemperature” if that is Rhasspy publishing the intent over MQTT then I assume that HA didn’t notice it for whatever reason since there was no audio played on my satellite.

I’m going to read that link you shared

4

what are your Rhasspy settings?

5

My Base settings are:

Speech to text: Kaldi
Intent Recognition: Fsticuffs
Test to Speech: Espeak
Audio Playing: Disabled
Dialogue Management: Rhasspy
Intent Handling: Disabled

I assume the intent Handling: Disabled is the issue. But if I set it to Home Assistant does that mean all intents are sent to HA?

6

Yes, you should set intent handling to Home Assistant when you want intents to be handled by Home Assistant :wink:
Indeed, all intents are send to Home Assistant.

Also, Audio Playing should be set to something, otherwise no audio is played.

7

I have a satellite setup that I hope the audio will play on.

8

Ah ok, missed that.
We are you using Node Red for?

9

No Worries,
That I don’t know, I was just looking to recreate my Snips setup as a starting point but I think it’s turning out to be harder than I expected.

I had setup a flow in Node Red to stream Radio stations to my Chromecast audio, adjust the volume, pause, play, and stop the stream.
When Snips died I was working on a Good Morning script which would give a brief weather report, if it is a work day give the commute time, and then play the morning news from CBC. I never got that last bit working before the plug was pulled.

10

Is there an advantage to using Events instead of Intents?
The docs are not overly clear on advantages/disadvantages are for each.

11

I think there are other integrations also using intents, so that might be an advantage on some usecases.
Otherwise, not really.

I find it easier to create automations listening to events, so you can switch them on and of per event.
You can not easily switch off some intents without a configuration change (either Rhasspy or HA)
Probably depends on your usecase which is to be preferred.

If you still have that flow, you can use it with Rhasspy as well. Node-Red can listen to events / intents from Home Assistant

12

Thanks for the info. You make a strong case for Events :slight_smile:
Can you use both intents and events?

I do still have the flow, it was a hack because I could get the Snips radio skill working. I did see something similar but as part of Rhasspy for playing media.
I think I’ll just work on the Home Assistant integration at this point.

13

No, the setting is a radiobutton so you have to choose.

14

I knew that, I selected the Intents radio button :relaxed:

15

Is there a way to determine if Rhasspy is sending the intent to Home Assistant?
I don’t see anything is either the Rhasspy server log or the Home Assistant Log.
Do I still need to enable the Snips Integration?

16

Which hostname have you set for your Homeassistant server in the rhasspy settings?

My problem was, that the rhasspy Docker container was not able to resolve the dns name of my Homeassistant. I had to add the dns server to my compose file to get it to work.

17

No, you do not need the snips integration.
Can you share your log from wakeword detection to end?

18

I am using the IP address for my HA server.
Both Rhasspy and HA are running in docker.

So my Rhasspy Base server is at 192.158.1.105:12101
My HA Server is at 192.168.1.105:8123

The HASS URL on my Rhasspy Base is: http://192.168.1.105:8123
The access token is filled in, the API password is blank, and the ‘Send intents to Home Assistant’ radio button is selected.

19

I didn’t think so but I wanted to check, thank you for confirming.

Do you want the log from the satellite and the base? and just the log that is available in the GUI or from the actual log file?

20

Here are the logs from my Satallite and my Base. The Satallite time is currently set to GMT, the Base is EST.
There are no entries in HA for this time period.

Satellite Log:

[DEBUG:2020-11-06 03:28:23,031] rhasspyserver_hermes: <- NluIntent(input=‘whats the temperature’, intent=Intent(intent_name=‘GetTemperature’, confidence_score=1.0), site_id=‘Kitchen-Sat’, id=None, slots=[], session_id=‘Kitchen-Sat-porcupine-0682f203-85e7-45bb-8153-74b6529b9a64’, custom_data=None, asr_tokens=[[AsrToken(value=‘whats’, confidence=1.0, range_start=0, range_end=5, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=6, range_end=9, time=None), AsrToken(value=‘temperature’, confidence=1.0, range_start=10, range_end=21, time=None)]], asr_confidence=None, raw_input=‘whats the temperature’, wakeword_id=‘porcupine’, lang=None)

[WARNING:2020-11-06 03:28:08,727] rhasspyserver_hermes: Dialogue management is disabled. ASR will NOT be automatically enabled.

[DEBUG:2020-11-06 03:28:08,677] rhasspyserver_hermes: <- HotwordDetected(model_id=’/usr/lib/rhasspy/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/porcupine/resources/keyword_files/raspberrypi/porcupine.ppn’, model_version=’’, model_type=‘personal’, current_sensitivity=0.5, site_id=‘Kitchen-Sat’, session_id=None, send_audio_captured=None, lang=None)

Base Log:

[DEBUG:2020-11-06 14:10:27,144] rhasspyserver_hermes: -> GetVoices(id=‘c3402fba-ff10-42b7-91de-d541db073de3’, site_id=‘Rhasspy-Base’)

[DEBUG:2020-11-06 14:10:27,242] rhasspyserver_hermes: Publishing 72 bytes(s) to rhasspy/tts/getVoices

[DEBUG:2020-11-06 14:10:28,413] rhasspyprofile.download: speech_to_text.system pocketsphinx kaldi = False

[DEBUG:2020-11-06 14:10:28,413] rhasspyprofile.download: speech_to_text.system kaldi kaldi = True

[DEBUG:2020-11-06 14:10:28,414] rhasspyprofile.download: Skipping kaldi/base_dictionary.txt (/profiles/en/kaldi/base_dictionary.txt)

[DEBUG:2020-11-06 14:10:28,414] rhasspyprofile.download: Skipping kaldi/g2p.fst (/profiles/en/kaldi/g2p.fst)

[DEBUG:2020-11-06 14:10:28,414] rhasspyprofile.download: Skipping kaldi/g2p.corpus (/profiles/en/kaldi/g2p.corpus)

[DEBUG:2020-11-06 14:10:28,538] rhasspyprofile.download: Skipping kaldi/model/conf/mfcc.conf (/profiles/en/kaldi/model/conf/mfcc.conf)

[DEBUG:2020-11-06 14:10:28,539] rhasspyprofile.download: Skipping kaldi/model/conf/mfcc_hires.conf (/profiles/en/kaldi/model/conf/mfcc_hires.conf)

[DEBUG:2020-11-06 14:10:28,539] rhasspyprofile.download: Skipping kaldi/model/conf/online_cmvn.conf (/profiles/en/kaldi/model/conf/online_cmvn.conf)

[DEBUG:2020-11-06 14:10:28,597] rhasspyprofile.download: Skipping kaldi/model/extractor/final.dubm (/profiles/en/kaldi/model/extractor/final.dubm)

[DEBUG:2020-11-06 14:10:28,597] rhasspyprofile.download: Skipping kaldi/model/extractor/final.ie (/profiles/en/kaldi/model/extractor/final.ie)

[DEBUG:2020-11-06 14:10:28,597] rhasspyprofile.download: Skipping kaldi/model/extractor/final.mat (/profiles/en/kaldi/model/extractor/final.mat)

[DEBUG:2020-11-06 14:10:28,598] rhasspyprofile.download: Skipping kaldi/model/extractor/global_cmvn.stats (/profiles/en/kaldi/model/extractor/global_cmvn.stats)

[DEBUG:2020-11-06 14:10:28,598] rhasspyprofile.download: Skipping kaldi/model/extractor/online_cmvn.conf (/profiles/en/kaldi/model/extractor/online_cmvn.conf)

[DEBUG:2020-11-06 14:10:28,598] rhasspyprofile.download: Skipping kaldi/model/extractor/splice_opts (/profiles/en/kaldi/model/extractor/splice_opts)

[DEBUG:2020-11-06 14:10:28,631] rhasspyprofile.download: Skipping kaldi/model/model/cmvn_opts (/profiles/en/kaldi/model/model/cmvn_opts)

[DEBUG:2020-11-06 14:10:28,631] rhasspyprofile.download: Skipping kaldi/model/model/den.fst (/profiles/en/kaldi/model/model/den.fst)

[DEBUG:2020-11-06 14:10:28,631] rhasspyprofile.download: Skipping kaldi/model/model/final.mdl (/profiles/en/kaldi/model/model/final.mdl)

[DEBUG:2020-11-06 14:10:28,632] rhasspyprofile.download: Skipping kaldi/model/model/normalization.fst (/profiles/en/kaldi/model/model/normalization.fst)

[DEBUG:2020-11-06 14:10:28,632] rhasspyprofile.download: Skipping kaldi/model/model/tree (/profiles/en/kaldi/model/model/tree)

[DEBUG:2020-11-06 14:10:28,681] rhasspyprofile.download: Skipping kaldi/model/phones/extra_questions.txt (/profiles/en/kaldi/model/phones/extra_questions.txt)

[DEBUG:2020-11-06 14:10:28,682] rhasspyprofile.download: Skipping kaldi/model/phones/nonsilence_phones.txt (/profiles/en/kaldi/model/phones/nonsilence_phones.txt)

[DEBUG:2020-11-06 14:10:28,682] rhasspyprofile.download: Skipping kaldi/model/phones/optional_silence.txt (/profiles/en/kaldi/model/phones/optional_silence.txt)

[DEBUG:2020-11-06 14:10:28,682] rhasspyprofile.download: Skipping kaldi/model/phones/silence_phones.txt (/profiles/en/kaldi/model/phones/silence_phones.txt)

[DEBUG:2020-11-06 14:10:28,682] rhasspyprofile.download: speech_to_text.system deepspeech kaldi = False

[DEBUG:2020-11-06 14:10:28,682] rhasspyprofile.download: speech_to_text.pocketsphinx.open_transcription True False = False

[DEBUG:2020-11-06 14:10:28,682] rhasspyprofile.download: speech_to_text.kaldi.open_transcription True False = False

[DEBUG:2020-11-06 14:10:28,683] rhasspyprofile.download: speech_to_text.deepspeech.open_transcription True False = False

[DEBUG:2020-11-06 14:10:28,683] rhasspyprofile.download: speech_to_text.pocketsphinx.mix_weight >0 0 = False

[DEBUG:2020-11-06 14:10:28,683] rhasspyprofile.download: speech_to_text.kaldi.mix_weight >0 0 = False

[DEBUG:2020-11-06 14:10:28,683] rhasspyprofile.download: speech_to_text.deepspeech.mix_weight >0 0 = False

[DEBUG:2020-11-06 14:10:34,368] rhasspytts_cli_hermes: <- GetVoices(id=‘c3402fba-ff10-42b7-91de-d541db073de3’, site_id=‘Rhasspy-Base’)

[DEBUG:2020-11-06 14:10:39,021] rhasspytts_cli_hermes: espeak --voices | tail -n +2 | awk ‘{ print $2,$4 }’


c[DEBUG:2020-11-06 14:10:44,353] rhasspytts_cli_hermes: -> Voices(voices=[Voice(voice_id=‘af’, description=‘afrikaans’), Voice(voice_id=‘an’, description=‘aragonese’), Voice(voice_id=‘bg’, description=‘bulgarian’), Voice(voice_id=‘bs’, description=‘bosnian’), Voice(voice_id=‘ca’, description=‘catalan’), Voice(voice_id=‘cs’, description=‘czech’), Voice(voice_id=‘cy’, description=‘welsh’), Voice(voice_id=‘da’, description=‘danish’), Voice(voice_id=‘de’, description=‘german’), Voice(voice_id=‘el’, description=‘greek’), Voice(voice_id=‘en’, description=‘default’), Voice(voice_id=‘en-gb’, description=‘english’), Voice(voice_id=‘en-sc’, description=‘en-scottish’), Voice(voice_id=‘en-uk-north’, description=‘english-north’), Voice(voice_id=‘en-uk-rp’, description=‘english_rp’), Voice(voice_id=‘en-uk-wmids’, description=‘english_wmids’), Voice(voice_id=‘en-us’, description=‘english-us’), Voice(voice_id=‘en-wi’, description=‘en-westindies’), Voice(voice_id=‘eo’, description=‘esperanto’), Voice(voice_id=‘es’, description=‘spanish’), Voice(voice_id=‘es-la’, description=‘spanish-latin-am’), Voice(voice_id=‘et’, description=‘estonian’), Voice(voice_id=‘fa’, description=‘persian’), Voice(voice_id=‘fa-pin’, description=‘persian-pinglish’), Voice(voice_id=‘fi’, description=‘finnish’), Voice(voice_id=‘fr-be’, description=‘french-Belgium’), Voice(voice_id=‘fr-fr’, description=‘french’), Voice(voice_id=‘ga’, description=‘irish-gaeilge’), Voice(voice_id=‘grc’, description=‘greek-ancient’), Voice(voice_id=‘hi’, description=‘hindi’), Voice(voice_id=‘hr’, description=‘croatian’), Voice(voice_id=‘hu’, description=‘hungarian’), Voice(voice_id=‘hy’, description=‘armenian’), Voice(voice_id=‘hy-west’, description=‘armenian-west’), Voice(voice_id=‘id’, description=‘indonesian’), Voice(voice_id=‘is’, description=‘icelandic’), Voice(voice_id=‘it’, description=‘italian’), Voice(voice_id=‘jbo’, description=‘lojban’), Voice(voice_id=‘ka’, description=‘georgian’), Voice(voice_id=‘kn’, description=‘kannada’), Voice(voice_id=‘ku’, description=‘kurdish’), Voice(voice_id=‘la’, description=‘latin’), Voice(voice_id=‘lfn’, description=‘lingua_franca_nova’), Voice(voice_id=‘lt’, description=‘lithuanian’), Voice(voice_id=‘lv’, description=‘latvian’), Voice(voice_id=‘mk’, description=‘macedonian’), Voice(voice_id=‘ml’, description=‘malayalam’), Voice(voice_id=‘ms’, description=‘malay’), Voice(voice_id=‘ne’, description=‘nepali’), Voice(voice_id=‘nl’, description=‘dutch’), Voice(voice_id=‘no’, description=‘norwegian’), Voice(voice_id=‘pa’, description=‘punjabi’), Voice(voice_id=‘pl’, description=‘polish’), Voice(voice_id=‘pt-br’, description=‘brazil’), Voice(voice_id=‘pt-pt’, description=‘portugal’), Voice(voice_id=‘ro’, description=‘romanian’), Voice(voice_id=‘ru’, description=‘russian’), Voice(voice_id=‘sk’, description=‘slovak’), Voice(voice_id=‘sq’, description=‘albanian’), Voice(voice_id=‘sr’, description=‘serbian’), Voice(voice_id=‘sv’, description=‘swedish’), Voice(voice_id=‘sw’, description=‘swahili-test’), Voice(voice_id=‘ta’, description=‘tamil’), Voice(voice_id=‘tr’, description=‘turkish’), Voice(voice_id=‘vi’, description=‘vietnam’), Voice(voice_id=‘vi-hue’, description=‘vietnam_hue’), Voice(voice_id=‘vi-sgn’, description=‘vietnam_sgn’), Voice(voice_id=‘zh’, description=‘Mandarin’), Voice(voice_id=‘zh-yue’, description=‘cantonese’)], id=‘c3402fba-ff10-42b7-91de-d541db073de3’, site_id=‘Rhasspy-Base’)

[DEBUG:2020-11-06 14:10:44,391] rhasspytts_cli_hermes: Publishing 3370 bytes(s) to rhasspy/tts/voices

[DEBUG:2020-11-06 14:10:45,135] rhasspyserver_hermes: Handling Voices (topic=rhasspy/tts/voices, id=7c903e5e-cd82-4dd8-8e87-d6712c3dbb75)