New Rhasspy Mobile App - Beta

In the app. I’m still figuring this out and didn’t even know mqtt was an option there. Which is better?

It’s a TCL 30 running Android 12.

After answering your question, I decided to try increasing the sensitivity. I’ll see if that helps.

Jeah when you use MQTT as speech to text service then the server will recognize the silence.
In the app it checks that the audio is below the audio level for the desired time. I could add a minimum length to a voice command, I think that would help.

I’ll investigate the MQTT option.

Tinkering with the sensitivity didn’t seem to have any effect at all.

Before reaching out, it was set to somewhere around 45.

I tried increasing it to the MAXIMUM and when that had no effect, I brought it way down to 5 and there was still no change.

https://youtu.be/qVI6MNL3ZE4

Can you test the “test Audi level” button and make a video ? That would help me to investigate the issue.

I did two test videos. In the first one, I’m trying to speak very softly. I’m trying to emulate someone speaking in the background without Rhasspy trying use those sounds too.

https://youtube.com/shorts/9NG8vP0I5Yc?feature=share

In the second video, I just spoke normally as if I wanted RHASSPY to do something – I just didn’t use the wake word.
https://youtu.be/b4oIwPMckWM

For me it looks like there might be some sort of exception happening, causing the recording to stop.
On the weekend there will be a new version with some bugfixes and also a setting to set a minimum time for the recording duration.

Hi.

I need some more help, sorry.
I followed the instruction Tutorials - Rhasspy to setup the satellite (mobile APP) with a base (Docker container in raspeberry pi ) but after several cobinations I couldn’t execute the intent and play the answer sound in mobile APP. When I used only the base station I used TTS and STT of google (STT with local command).

In the tutorial neither the satellite nor the base set the intent handle and don’t work for me. I set the intent handle in mobile APP (to Home Assistant) and I set the satellite id in the actived services of base station. The automation on Home assistant is fired, but the answer doesn’t play in mobile APP.

The current combination is this:


I triend publishing a message on hermes/dialogueManager/endSession or/and hermes/tts/say but nothing work (in actually combination I can’t load the sessionId from intent, in other’s combinations I could load but doesn’t work anyway)

I followed too the configuration of @ledz1996 but doesn’t work for me.
What are doing wrong? :sob:

Thanks

Convert the text to WAV data and use “hermes/audioServer/< siteId>/playBytes/< requestId>” or you use the http endpoint “/api/say” where you post the text or send the text to “Hermes/tts/say” on your rhasspy server with the correct satellite site id.

Edit:
The app doesn’t handle “hermes/tts/say” for the following reason:

When receiving “hermes/tts/say” in app and using mqtt for text-to-speech then the app has to call “hermes/tts/say” by itself to enable the base to do the text-to-speech operation. But this would result in a loop because the app itself would also receive the command.

Therefore you have to send “hermes/tts/say” with the siteId from the app, this will be consumed by the rhasspy base, it will do the text-to-speech operation and call “hermes/audioServer/< siteId>/playBytes/< requestId>” with the given siteId, this then results in the app playing the sound.

Ok, thanks I will wait until next version (I don’t know how compilate Kotling code).
I have commit the translation to catalan.

Thanks a lot for your work.

I updated my answer to clarify that the app won’t recognize “Hermes/tts/say” (also not in the future) and why. Hope it’s now clearer.

OK, thanks for clearing it up

I decided to support “Hermes/tts/say” with the newest Release because it simplifies app usage. It uses a specific sessionId so there won’t be a loop when also using mqtt as text-to-speech service.

Note that this endpoint expects a JSON string as input as described here Reference - Rhasspy

Nice app.
Is there a possibility to make this app for android TV?
Thanks a lot for your work.

Jeah it’s already planned.
Mostly I would need to do testing as well as thinking about how I make file selection.
Most AndroidTVs don’t have a File App therefore I think about a download dialog in order to eg use custom porcupine wakewords.
Also I would need to do some testing just to make sure everything works as expected.

Thank you for this. I had been looking for this for ages.

Is it possible to change the voices?

Do you mean for Text-To-Speech playback?

Right, is it possible to change voices for TTS playback?

Also, How to manage multiple satellites that we may be hearing the same command? So e.g. with Google speakers, only one of them responds at one point in time even though they all heard the wake word.

Also, Also, Any way of integrating with local AI for conversation? may be via Home Assistant?

For tts the selected System (http or mqtt) is used, you need to change the voice there. Another possibility is to process the command in Home assistant eg via nodered and the use some tts service and send only the audio to the app.

In order to have only one satellite react to a wakeword I think Mqtt dialog might be the way to go. You would need to listen to the wake word message and then tell only one satellite to react.

Maybe they found/will found a solution for this with rhasspy3/Wyoming protocol and I currently work on supporting it in the app.

1 Like

Hi there! I am quite new to Rhasspy and Mobile app seems to be very promising. However I can not get it to work. I am not sure where problem is but I suspect that I can not record the speech for some reason. I can activate microphone by pressing or by wake word but the listening time expires and no record is made. At least if I press Play Recording nothing happens. In Dialog section I am getting RecordingIntentState > SendAudioCaptured: false, asrtext Null, AsrTeimeoutError. I have allowed microphone recording to this app. Besides wake word works fine so the mic is accesible to the app. Installed it on other Android device - it is the same.