UDP Wakeword detection with Rhasspy mobile app

Hi, I’m trying to set up the udp wakeword detection for hours now but I can’t get it to work. I followed the rhasspy docs but it isn’t clear for me what to do.

  • I’m running on docker and opened the port 12202 (looking at portainer)
  • in the app I entered the ip of the rhasspy base : port (192.168.178.40:12202)
  • in rhasspy base under audio recording I selected pyaudio and entered ip:192.168.178.40 port:12202
  • in rhasspy base under wakeword I selected porcupine and entered the port 12202
    but it still doesn’t work.

One thing I noticed is, if I enter 192.168.178.40:12101 in the app at the first line (not under wakeword)
I get a cannot connect with rhasspy message
Could anyone help? Thanks

You don’t need to set up a recording system on the rhasspy base as the app will provide audio data. on wake word you might try to put the ip of the docker container. however it is strange that the app can not connect to rhasspy even if this does not affect the wake word. To check if the app is not working or there is some other problem you can try to set a rhasspy satellite to send udp packet to rhasspy base.

Thanks for the reply,
I tested again and for some reason the app can connect now but the wakeword still does not work. I think I’ve tried every combination of docker host ip, docker container ip and phone ip on both sattelite and base but it still does not work. Detection over mqtt works. Do I have to open a port on my phone too as the docs say:

For this case, set both microphone.<MICROPHONE_SYSTEM>.udp_audio and wake.porcupine.udp_audio to the same free port number on your satellite

I try to explain the wake word configuration more clearly.

First, we open a UDP port to the container. Add in the docker run command “-p 20000:20000/udp” (in my example I will use the port number 20000) you must open a UDP no TCP port. So we can now check portainer.
image
In the app we enable wake word over UDP in the text field we put ip of the machine where rhasspy is running and after a colon the port we have opened previously. in my case 192.168.1.10:20000.


in the rhasspy base under Wake Word in UDP Audio (Input) first put the ip address of docker container:the port we have opened previously: siteId which was set in the app. in my case 172.17.0.2:20000:telefonovirtuale

now if everything worked by pronouncing the wake word the microphone symbol of the app should turn red indicating that it is listening for a command.

1 Like

Thank you very much for that detailed explanation. Turns out that I entered everything right but restarting rhasspy from the web UI didn’t apply the changes. I had to restart the whole container in order to apply them.
Thanks for the help.

1 Like

Why there are two different ip Addresses?
And one class c and class b private IP? I dont understand how this can work. I think both IP Addresses should be equal?

Hi, i solved my Problem. I am new to docker and its Container thing, esspecialy how networking is with docker Container. I installed portainer, great Software, and then i understand were the class b ip addresses came from.
Wake Word detection is more or less good. I think i have to Play around with own models and train them. Sometimes i have to wait 4 seconds before the wake Word put on the mic red. If you speak wake Word blind, you do not know how long waiting until you can speak the command. But this is not the problem of your mobile App… Great work!

Maybe in future versions I could make it possible to play a sound when a wake word is detected.

1 Like

Hi @razzo04, rather than create a new issue I thought I’d progress with this one already in process on topic of the wakeword detection with Rhasspy mobile app. (great app by the way!). I have set everything up as defined above, but something is going very wrong…
I cannot get Rhasspy to wake via the app with the UDP wakeword setting. So, I have created a small Node Red to analyse port 20000 UDP traffic to make sure that the APP UDP message is getting through. I find that as soon as I “Start Wake Word” in the app, there are UDP messages flooding through to port 20000 on the Rhasspy Host machine. Can you think of anything that I might have done that could cause this please? I attach screen shots of the Rhasspy Wakeword entry, the App entry and the Node Red detected port 20000 traffic on the Rhasspy server. The messages starting 14:15:03 are the ones from the Rhasspy mobile App… thanks in advance for your thoughts on how to progress this one.


image

Most likely you are using docker to run rhasspy, so Docker by default applies isolation between the host network and the container network so you can’t then bind the ip of the host machine instead you have to use the docker container ip or “0.0.0.0” which indicates that we listen for any interface.

@razzo04. OK thanks. I’ll look into it further. I run Rhasspy on a RPi within HASSIO. Node red is also within HASSIO on the same RPi. Node red is therefore seeing the traffic from the Rhasspy mobile app UDP messages ok with the IP address of HASSIO / Rhasspy.

I am curious as to why the mobile APP is sending 15 messages per second by UDP. Is it expected? The wake word can’t be spoken in 1/15th of a second, and so could not logically be contained in one of these UDP messages. Its possible that I misunderstand the purpose of the UDP wake word function on the App in connection with the UDP messages, but I thought it was to listen for the potential spoken wakeword and pass it to Rhasspy.

I’ll post an update if I get any further by changing the IP address.

If rhasspy runs on hassio some configurations are different, here you can find some suggestions.

The purpose of the UDP wake word is to start listening if wake word is detected. 15 messages are sent in one second because rhasspy expects to receive frames of 2092 bytes otherwise they will simply be placed in a buffer waiting for it to fill up, even if the wake word is on multiple frames it will be detected anyway.