New Rhasspy Mobile App - Beta

Just released the Beta version of a new Rhasspy mobile app!
I know there is already the project from @razzo04 (Rhasspy mobile app) but it didn’t fit my needs and i was not able to update the project in it’s current status. Therefore i created a new project.

Github Project

Supported devices

  • Android mobile phone and tablet (minVersion Android 6.0)

  • iOS will maybe come later, some groundwork was already done

  • Smart Watches - no plans yet

  • TV (Android/iOS) - no plans yet

  • Desktop (windows/macos/linux) - no plans yet, docker container is simpler

Main Features:

  • Local Wake Word detection with Porcupine
    This was my most important feature, it also supports custom wake words and multiple wake words.

  • Background Service
    To run the app in the background continuously, this was one of the reasons why it was not possible to update the old app - i had very big issues implementing this together with Porcupine in Flutter

  • Indication when Wake Word is detected
    It’s possible to wake up the display, show an overlay animation and play a sound indication like rhasspy.

  • Microphone Overlay and Widget
    This was requested by other users, it allows to start recording with one click from everywhere and without wake word detection

  • Local Webserver
    Pretty much acts like the default rhasspy web server. Start listening for commands, turn on/off wake word, play sounds etc.

  • Intent and Broadcast to start listening
    This was also requested by other users. It allows to start recording via Broadcast. For example when using home assistant in kiosk mode you can send a Broadcast to the app without leaving the kiosk mode.

  • MQTT
    The Mqtt protocol is also mostly supported, just ignoring some params.

Whats next

I still have some ideas that can be found on the Project Board.
There is no iOS implementation yet, but because the App uses Kotlin Multiplatform Mobile the groundwork is already done but it will still require some work. Therefore i just want to start on it when there is enough need.

What do you think?

Please test my app, it’s currently open beta in the playstore or you can simply download it from the github releases.

8 Likes

Hello,

Thanks to this application without google etc.
I am unable to set up the application configuration with rhasspy. My master has no microphone and speaker. Here is the configuration of each:

Master :

Satellite mobile :


Can you guide me?

Hi, i think you should change your “WakeWord” option in the app.

As in the Rhasspy documents (Wake Word - Rhasspy) when setting WakeWord to MQTT the app waits for a MQTT Topic hermes/hotword/<wakewordId>/detected so you should either use udp or porcupine.
In the current version udp is broken (will be fixed soon) so porcupine should be the way to go.

When you want to use MQTT for the other services (SpeechToText, TextToSpeech) etc you have to connect your Rhasspy base also with the same MQTT Broker. The reason is that the mobile app will send the audio data via MQTT Messages and the base has to receive them.

If you are going to split the processing over 2 or more devices, they ALL need to access the same MQTT broker. The Rhasspy “Internal MQTT” is a bit of a short-cut if all the modules will run on the same machine.

In other words, your “Master” needs to be set to External MQTT, pointing to the same MQTT broker as your mobile app.

1 Like

@donburch : Thank you for the precision on the configuration of the master, by going external it works perfectly.

@Nailik : Thanks for your feedback. I have the wakeword that works in MQTT on my side following the change recommended by donburch.

However, one thing persists. In host, I register my domain name which is attached to my personal ip. It works outside my home (the connection is established without worry) but not at home (I then have to put the local ip of the master). Is there a reason/trick?

Regarding the vocal feedback, I have none at all. Did I make the right configuration? The wish is that it be managed by the master and the feedback is done by voice on the phone.

About the domain issue i am not sure why this happens. Do you use a local dns? Then you could map your domain name to your local ip.

How do you trigger your vocal feedback?

No, I use my internet provider’s DNS.

I use Jeedom to my vocal feedback.

And how do you send the data to the app?

Your domain name is linked to point to your routers public IP address so that anyone on the internet can find your router. Your router’s job is (in part) to redirect any incoming IP packets received on the WAN (internet) port to the appropriate LAN private IP address - on your router the setting are probably under NAT > Port Redirection or similar

However when you are at home, you are connected to a designated private IP address range (probably 192.168.x.x) and the router’s job is to pass DNS requests to the WAN port, translating your LAN IP address to its WAN IP address as it goes. And when the reply comes back it tends to go pear-shaped :frowning:

Some routers have the ability to map a public IP address to a LAN IP address - though there is no standardisation as what it’s called.

I send the data with this configuration :

OK with :
out : mydomain.end
in : 192.168.1.X

It would be nice to add a config for the local and another for the external. If wifi connection activated, verification of the local, if ko, use the external method

I could add a “Backup” field for another Domain/Ip but i would recommend you to “fix” your local network to directly resolve your external address. An option would be pihole or check if your router supports that. Also i think you should not use MQTT from an external network without SSL enabled (how to setup ssl will be documented soon).

About playing sound through the app you could check “Audio Playing” there you can setup if the audio is played as sound/media or as notification. To play the audio data you can use:

  • Mqtt by publishing a Message on the topic hermes/audioServer/<siteId>/playBytes/<requestId>
  • Or you can play audio by enabling the server and calling /api/play-wav and adding the wav data as payload but this is probably not working when your outside your network

So you have to check if jeedom does post on the correct topic, for you that would be hermes/audioServer/mobileflorian/playBytes/<requestId>

1 Like

Adding this internal and external distinction would indeed be great. Would it also be possible to be able to put a different external and internal port? I use a Synology NAS to easily have a secure configuration so the port is different (locally I stay without ssl).
Regarding the ssl for the exterior, it is actually in the works, I will put it in place when I can use it with the local on the app.

I’ll take a look at the sound.