Multilingual profiles

Hi all,

Just starting to set up rhasspy now. One drawcard for me was the Rhasspy is an offline, multilingual voice assistant toolkit portion of the readme. Perhaps though, I interpreted this incorrectly.

I live in a multilingual household, and ultimately would like to have English, Swedish and Mandarin interchangeable. Seemingly, whilst rhasspy supports all three languages, I can only choose one profile from this set at a time.

Would it be possible to create a combined profile? Would this be straightforward enough to edit say the Swedish profile.json to include the other two endpoints, and merge the profile folders/additional files together in the same manner?

Or is this something that fundamentally isn’t really possible with the current architecture of rhasspy?

1 Like

Hi @Libbum, this is a great question. I actually ended up changing the wording in the docs to avoid this misconception (from “multilingual” to “support for many languages”). Looks like I forgot to update the README :confused:

This is not possible with Rhasspy in a single profile, but could be done with some tweaking and by running multiple instances of Rhasspy. At a high level, there would be a backend Rhasspy instance per language, a satellite listening for wake words (one per language), and a program to direct audio at the right Rhasspy instance (NodeRED).

Here’s the details of how I would do it:

  1. Run a seperate instance of Rhasspy for each language
    • Disable wakeword
    • Use MQTT and a unique siteId per instance
    • Use the Hermes microphone for audio input over MQTT
  2. Run a wakeword service (snowboy, porcupine) on a satellite device with a different model for each language
  3. Use the webhook feature to catch the wake-up event using a NodeRED flow
  4. Switch on the wakewordId property and stream audio over MQTT to the corresponding siteId
  5. Catch hermes/intent/<INTENT_NAME> or hermes/nlu/intentNotRecognized MQTT events in NodeRED and shut audio off
1 Like

@synesthesiam wouldn’t it be an idea to make the topic where Rhasspy listens for wake words configurable. You can then run multiple wakeword services with different models on the satellites each posting on their own MQTT topic and therefore waking the right Rhasspy instance. Could also be used if you want to use different STT engines (for example use Kaldi with an open model or google for open ended questions, and Kaldi with a closed model for home automation)

Thank you both for your input here. I’ll play around with a few methods in this case and see what I can come up with. May take some time, but will report back here once I have something cobbled together.

1 Like