How to build a working german profile with hass.io on a Raspberry Pi 3B+

[RaspiManu]

Hello Rhasspy-Community

I am quite new to this community and Rhasspy, as you might see in my community profile

When following a tutorial on setting up Rhasspy on a Raspberry Pi 3B+ with Matrix Voice, I could not get a german profile working. Thats why I created a specific topic on the problem, that I thought I had. While my conversation with @fastjack, @geoffrey and @Jorg_Baumann I learned a lot of things that led me to the point, where I saw my whole problem.

the problem:
I am unable to create a working german profile.

the setup:

• Raspberry Pi 3B+
• Raspbian Buster
• Matrix Voice as Microphone
• Rhasspy Version 2.4.19 in Docker Container

what would help:
Every hint on which Toolkit to choose for which category in the settings and what to do to get them working all together will help me a lot. Also I would love to hear about getting hass.io working with Rhasspy on one Raspberry Pi.

what I tried:

• installed an english profile which worked kind of good with porcupine as wake word toolkit (no option for a german profile)
• tried to use Kaldi for Speech Recognition with a workaround of @Jorg_Baumann in german profile, that said Kaldi is “not compatible with this profile” (situation described in my other topic mentioned above); sadly did not work for me

I read a lot of topics and it seems like there are some more Germans and some Europeans with similar languages on here, that could possibly help me with building a profile. This would be very nice of you

Thank you in advance and best wishes
RaspiManu

EDIT: added setup info

[ulno]

Not sure if it’s much help, but hopefully encouraging: one of my family members runs rhasspy on three pcs (his desktop linux pc with microphone, an old thinkpad as server without microphone and running node-red too, and a rhaspberry pi sattelite with a usb microphone in the living room). He uses the English porcupine as wakeword, kaldi with german recognition, openfst as intent recognizer and is pretty happy about it. He uses only virtual environments, no docker - all his computers run current versions of either xubuntu or raspbian.

Can this information push you in the right direction?

[RaspiManu]

Hi @ulno,
every help is much appreciated, gives me an impression on what other people do and pushes me in the right direction. I got some questions about this setup:

• How can he use the english “porcupine” as wake word? When I used Porcupine as wake word toolkit in the english profile, it worked quite good, but in the german profile it did not react and I pronounced the word “porcupine” just like in english. I heard it only works with English, doesn’t it?

• How was he able to select Kaldi with german recognition? Both my profiles, EN and DE show me this:

  

  I tried using it with the workaround of @Jorg_Baumann described in my other topic linked above, but could not manage to get it working.

[Jorg_Baumann]

Hi RaspiManu,
Sorry, but I only can repeat that for me that was the only way to get Kaldi working.

My Hardware: Raspi 4 / 4Gbyte RAM, ReSpeaker 4-Mic Array.
Running Rhasspy 2.4.19 in a docker container under Raspbian Buster light.

My suggestion would be, first to make a clean environment:
1.)

docker ps
(that list you all containers)

2. ) Stop all containers:

docker stop <container id>

3.) Delete all containers (if you have multiple containers, remove all of them):

docker rm <container id>

4.) List all downloaded images:

docker images

5. Delete all downloaded images:

docker rmi <image id>

If deleting throws an error try this:

docker rmi -f <image id>

6. Delete all your profiles under /home/pi/.config/rhasspy/profiles/ (en/de)

7. Now you can try again to install Rhasspy:

docker run -d -p 12101:12101 --restart unless-stopped -v “$HOME/.config/rhasspy/profiles:/profiles” --device /dev/snd:/dev/snd synesthesiam/rhasspy-server:latest --user-profiles /profiles --profile en

(I would suggest, first try with english profile, to see, if Kaldi works now.)

If that’s fine you can stop the docker container with the english profile:

docker stop < container id >

Then you can install the German profile:

docker run -d -p 12101:12101 --restart unless-stopped -v “$HOME/.config/rhasspy/profiles:/profiles” --device /dev/snd:/dev/snd synesthesiam/rhasspy-server:latest --user-profiles /profiles --profile de

Normally, of course, Kaldi should work out of the box with the German profile, but I’ve found no way to activate it with my workaround.

BTW: I’m using snowboy as wakeword.
__
If you like, you can visit the German ioBroker forum: ioBroker - rhasspy-offline-sprachsteuerung where Rhasspy is also discussed (I’m also there as joergeli)

Good luck!

Addendum:
I read somewhere, that porcupine-wakeword is not so good for German profile, because it’ designed for English?

[ulno]

Ah, this bug I had in English too. I fixed that with deleting the profile and never selecting pocketsphinx (or via upgrading to 2.4.19 - happened at the same time). Which version are you using?

[RaspiManu]

Thank you very much for that very detailed description and the other hints, @Jorg_Baumann. I will try this as soon as possible.

Sorry @ulno for not describing my setup in the topic. Just forgot that point and will add it to the initial text now. How can I never select pocketsphinx, when it is the default, isn’t it?

[ulno]

When you start with an empty profile the first time, kaldi was selectable for me. Saving settings once disabled it. Is that the same for you?

[RaspiManu]

Yes, same for me. When I tried setting up a clean english profile in a new docker container yesterday, I was able to choose Kaldi right after the first boot. So I chose it , saved and restarted, but no download popped up and Kaldi was not choosable anymore.

[ulno]

And you are running 2.4.19?

[RaspiManu]

Yes, my setup consists of a Raspberry Pi 3B+ with a Matrix Voice, Raspbian Buster and Rhasspy Version 2.4.19 in a docker container. Sorry, I added it to the initial text of the topic but did not write it in my reply

[RaspiManu]

Hi @ulno, @Jorg_Baumann and everyone else interested in this topic,

It’s been some time now and I’m still not able to use Kaldi for speech recognition. I tried everything mentioned in here (except one thing, to be discussed later) but had no luck. That’s why I set up a fresh image of Buster and tried it again with installing the english profile. Sadly I still get the message, that Kaldi is “not compatible with this profile”.

When looking at the hardware and software setup described by @Jorg_Baumann… :

My Hardware: Raspi 4 / 4Gbyte RAM, ReSpeaker 4-Mic Array.
Running Rhasspy 2.4.19 in a docker container under Raspbian Buster light.

… I see 3 differences to my setup:

• I use a Raspberry Pi 3B+.
• My microphone is a Matrix Voice (working well).
• I use a desktop version of Raspbian Buster, because one goal of my project is voice controlling things shown on the screen.

Did anyone have problems with one of my setup’s components (soft- / hardware)?

The one thing to be discussed later:

Ah, this bug I had in English too. I fixed that with deleting the profile and never selecting pocketsphinx (or via upgrading to 2.4.19 - happened at the same time).

I still don’t know how to never select pocketsphinx, because it’s the default of every fresh installed image of Rhasspy 2.4.19. How can I set up a fresh image without pocketsphinx to test this?

[Jorg_Baumann]

Hi,
IMHO Raspi 3B is Ok, because in the past I also used a 3B with Kaldi.
It’s only a little bit slower than my Raspi 4.
Matrix Voice also should be OK, because I know from another user, that Kaldi works with his Matrix Voice.
Buster Desktop perhaps slows down the speed a bit , but should not be a problem.

Meanwhile I installed Rhasspy V2.5-pre several times on a separate Raspi4 Test-System and there Kaldi works out of the box (after downloading the necessary files) with German profile. ( on the other hand, I have other problems with Rhasspy V2.5-pre, but not with Kaldi)
Perhaps you can also try to test Rhasspy V2.5-pre, to see, if Kaldi is working?

[RaspiManu]

Hi @Jorg_Baumann,
thanks for the fast response. Can I test 2.5-pre with just stopping my 2.4.19 docker container and starting a new one with 2.5-pre, or will I have to remove the complete 2.4.19 image?
If I build my system with 2.5-pre, will it be possible to just update it to 2.5 when fully released, or will I have to build it again?

[Jorg_Baumann]

Can I test 2.5-pre with just stopping my 2.4.19 docker container and starting a new one with 2.5-pre, or will I have to remove the complete 2.4.19 image?
If I build my system with 2.5-pre, will it be possible to just update it to 2.5 when fully released, or will I have to build it again?

IMHO stopping the 2.4.19-container will be OK, but for security I would save the profile-directory before installing 2.5-pre
2.5-pre -> 2.5 final: IMHO, when 2.5 final is released, I would make a new container, because at the moment we don’t know what differences will be between 2.5-pre and 2.5 final.
But I think, that’s not hard work, when you backup your profile before.

By the way, I’m no Rhasspy specialist and can only share my experiences

[RaspiManu]

Thanks @Jorg_Baumann for answering my questions. I will try it with 2.5-pre as soon as possible.

[RaspiManu]

It works!
I stopped the 2.4.19 docker container, started a new one with 2.5.0-pre and was able to choose Kaldi. It is very nice for recognizing the german language. This is the first time, I got kind of a working base system of Rhasspy. Thank you very much for your continuing help, @Jorg_Baumann.

Finally, we can move on to the second goal of this topic:
Having Home Assistant as intent handler on the same Pi.
I will start experimenting with it in a few days, so at this point, I am very interested in these questions:

• What type of installation of home assistant would fit best for my case? (need it inside Raspbian)
• Are there any good hints for a complete Home Assistant newbie before starting?

[Daenara]

Don’t use the hassos version of home assistant. That one has some issues with hogging the audio devices and at least for me the known workarounds were never permanent. I use the home assistant docker now, that one does not interfere with the audio devices.

[RaspiManu]

Thank you very much for that hint, @Daenara.
Is the docker version of home assistant updatable or will I have to overwrite it, when a new version is released?

[Daenara]

I am actually not sure, I didn’t update since I switched to docker. The concept of docker would be to overwrite it, I think. There should be a section in the hass installation guide about how to update thought.

[RaspiManu]

I got some more RasPis and was thinking about setting up one as Rhasspy and one with HassOS and let them communicate inside my local network so both have better performance and I got the updates and AddOns for Home Assistant. Could this be a good idea?

Because of the problems you had with HassOs, @Daenara,…
Could you please explain them a little bit more? Did HassOs have a problem with the devices for communicating with Rhasspy, microphone etc., or did you have problems with some Bluetooth Speakers in your home or something similar, that directly interacts with Home Assistant? Will I also have these problems, if I go for my idea described above?

[Daenara]

Basically hassos is the supervisor that automatically starts and maintains quite a few docker containers for hass. One of them is a sound container and that one started hogging the audio devices after an update around 2 months ago. So basically rhasspy running on the same pi always reported that the audio device is busy. There are workarounds to that and apparently some seem to work long term for most ppl but I tried and tried but it always undid my workaround. There is a addon for hassos that is supposed to undo the change in the audio container but even that one stopped working for me after around a week and rhasspy did not get any input.

If you use two physical devices for rhasspy and hassos then you should be fine, but on the same device it might be a bit of a problem.

Here is the thread I first talked about that issue with the sound, it also contains the link to the github issue where you can find the workarounds and the link to the addon.

[RaspiManu]

Thank you for the explanation, @Daenara
I think I will try it with two separate Pis for the first try and report about the results.

[RaspiManu]

Thanks for waiting, @Daenara.

I set up HassOS on a second Pi (just booted it and made a HA account that is logged in).
For connecting to Rhasspy, I did the following:

After that, I saved the settings, restarted Rhasspy, trained it, went to the home screen, checked the box for handling behind the recognize field and asked for the time. Rhasspy recognized the intent GetTime, but nothing else happened. It might be a complete beginner question, but this is the point, where Rhasspy should tell me the time, isn’t it?

Do I need to set some things in HA to get intents from Rhasspy working?
Are my Rhasspy setting for intent handling faulty?
I thought the access token is filled in automatically (like described), but it wasn’t. Do I have to enter something?

EDIT:
I found the point in HA settings where you can create long-lived access tokens and created one. I copied it and pasted it in the token field in Rhasspy’s HA settings. After restarting and training, nothing changed.

[Daenara]

You need to actually configure hass to do something when the intent is detected but I never got that far because I keep getting distracted by other things. Basically rhasspy sends a json to hass when you set it up correctly but hass needs to know what to do with the intent and send an answer back. I think there is an intent component for hass (I used it with snips ages ago) that you can use to set interaction up but you might just need to read the documentation on it.

[RaspiManu]

I read the HA docs about intents but I’m not sure what to do with the given information. There are some code samples but I could not find out where they have to be stored inside HA’s file system and what to do, to get them working.

Last point of the intent docs are HA’s built-in intents. I wanted to try HassShoppingListAddItem and HassShoppingListLastItems, so I set up some sentences in Rhasspy:

When trying them out (with handle check box active), Rhasspy recognizes the right intent and item (see picture below), but I get no response. I tried sending events to HA and sending intents to HA.

How are you using HA and what for, @Daenara? Are you getting responses of your voice assistant?
I think I understood something wrong in how Rhasspy and HA work together. Would be great, if I could get an explanation on that.

Perhaps @synesthesiam or @ulno can tell me how to get one of the pre-set sentences / intents in Rhasspy to work as expected. A big goal would be, to get Rhasspy to tell me the time after recognizing the GetTime intent. Is HA even needed for this? I was trying to get it working after reading the Rhasspy docs on intent handling, but had no success.

[Daenara]

As I tried telling you in my last post, my rhasspy and homeassistant don’t work together yet. I use homeassistant via the app and rhasspy just for testing so far.

For the shopping list. Did you install/activate the shopping list in homeassistant? Before you try to control it with rhasspy always make sure it works from homeassistant itself.

As for the intents. They go in configuration.yaml like every other bit of homeassistant configuration.

My homeassistant is used to control my media center as well as a few lamps and the ceiling fan,but like I said, no voice activation yet because I keep getting distracted by other projects.

[RaspiManu]

I’m sorry, I misunderstood.

I could not find out yet, how to activate the ShoppingList intents in HA. I thought they were a good choice for a test and might work without an activation, because they are built-in. At the moment, my main goal is to connect Rhasspy to voice control a simple intent in HA. Sadly, I don’t own any smart plugs or something similar yet, so I need to test it with an intent, that gives me a respond with an information like the current time or my shopping list items. Any hints and help on this are much appreciated.

[RaspiManu]

I found this tutorial on how to set up Rhasspy and connect it to HA:

Home Assistant Voice Recognition with Rhasspy

Because I got a running Rhasspy Setup, I started with the point “Hass Integration”. I created an access token and filled the Rhasspy Settings for intent handling with HA with the local Hass URL, the token and the password of my user account on my HassOS (second Pi) (see picture below).

After that, I tested if the HA API is running with Windows console and the following command:

curl -X GET -H "Authorization: Bearer MY_CENSORED_ACCESS_TOKEN" -H "Content-Type: application/json" http://homeassistant.local:8123/api/

It returned:

{"message": "API running."}

That’s why I went to the next step of the tutorial. I opened the Web UI of my HassOS, went to:
Developer Tools > Events > Listen to events
and started listening for “rhasspy_GetTime” (see picture below) as shown in the tutorial.

Now, if I ask my Rhasspy voice assistant for the time, HA should show me some output telling me it detected “rhasspy_GetTime”. Sadly, nothing happened.

Did I set up something wrong?
What could be the next step in trying to fix this?

[RaspiManu]

I found out, that my local network does not send events from Rhasspy to HA, if Hass URL is http://homeassistant.local:8123. After changing it to the full IP of my HA, I was able to “listen” for rhasspy_GetTime and and see the incoming information. I was also able to set up a small test automation, that should send an example text back to Rhasspy’s text-to-speech API. Now, when I ask Rhasspy for the time, I can see, that the test automation gets triggered in HA.

Sadly, I can’t get the part of sending text to Rhasspy to work…

Following the tutorial mentioned above, I added rest_command to my configuration.yaml in HA, so it looks like this:

# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:

# Uncomment this if you are using SSL/TLS, running in Docker container, etc.
# http:
#   base_url: example.duckdns.org:8123

# Text to speech
tts:
  - platform: google_translate

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

rest_command:
  tts:
    url: http://localhost:12101/api/text-to-speech
    method: POST
    payload: ''

After that, I was trying, to send a test message to Rhasspy (via developer tools > services):

Sadly, I had no luck. I also tried using payload: "hallo" instead of message: "hallo" and using the full IP of my Rhasspy in the configuration.yaml.

Could anyone please describe, how he or she POSTs messages to the text-to-speech API of Rhasspy?
I saw, that @romkabouter is actively using HA with Rhasspy in a video post. Would you please describe, how you do this, Paul?

[romkabouter]

Sure, my rest command looks like this:

rest_command:
  rhasspy_speak:
    url: 'http://192.168.43.54:12101/api/text-to-speech'
    method: 'POST'
    payload: '{{ payload }}' 
    content_type: text/plain

So a little different that yours, check payload and an extra content_type setting.
Also, the IP address is the IP from Rhasspy, most probably localhost is incorrect.

To call this rest_command, I have an automation listening to the event:

- id: '1581372525473'
  alias: EventLampen
  trigger:
  - event_data: {}
    event_type: rhasspy_Lights
    platform: event
  condition: []
  action:
  - data_template:
      payload: OK, {{ trigger.event.data.location }} {% if trigger.event.data.action
        == "on" %}aan{% else %}uit{%  endif %}
    service: rest_command.rhasspy_speak
  - data_template:
      entity_id: light.{{ trigger.event.data.location }}
    service_template: light.turn_{{ trigger.event.data.action }}

The service rest_command.rhasspy_speak is the one to use.

I think your test call should work as well, with

payload: hello

There is no need for quotes.
When you do not hear anything, this might also be a problem in your Rhasspy config.

[RaspiManu]

Thank you so much, @romkabouter. It worked like a charm

I was finally able to get the full circle working (with german profile):
wake word > STT > intent recognition > passing the event to HA > Handling the event in HA > passing text to Rhasspy > TTS

Because a lot of people helped me on the way to get this working, this is a summary, that I will mark as solution. Massive thanks go out to the following community members:
@fastjack, @geoffrey, @Jorg_Baumann, @Daenara, @ulno and @romkabouter

This is my working setup:

Hardware:

• Raspberry Pi 3B+ with Matrix Voice as microphone and cheap speaker connected over audio jack of the Pi (Rhasspy in Docker container on Raspbian Buster, Raspbian audio default set to analog, connected via WiFi)
• Raspberry Pi 3B (HassOS, connected via WiFi)

Rhasspy Settings:

Home Assistant Settings:

• add rest_command to configuration.yaml (for example via file editor addon in WebUI):

  rest_command:
    rhasspy_speak:
      url: 'http://ip.of.your.rhasspy:12101/api/text-to-speech'
      method: 'POST'
      payload: '{{ payload }}' 
      content_type: text/plain
  

• create automation, that listens for event rhasspy_IntentName (for example: rhasspy_GetTime) and starts service rest_command.rhasspy_speak with specific payload (for example: payload: hello)

EDIT: If you want to use a payload with templates to include data like the current time, you need to change data: in your HA automation’s action to data_template:. This is only possible by clicking on the three dots symbol and choosing “edit as YAML” while working on your automation via WebUI or by editing automations.yaml file. Here is an example for sending the current time to Rhasspy (german format):

  data_template:
    payload: 'es ist {{ now().hour }} uhr {{ now().minute }}'
  service: rest_command.rhasspy_speak

I hope this will help someone as much as it helped me.

RaspiManu