Please help with repeating detections by ASR and NLU

Hello community,

I’m new here, my name is Martin and I am from Austria, so pls excuse if my
English is not perfect.
I like the Rhasspy project, following it quietly since the Snips disappeared,
I think @synesthesiam and the other supporters do a really great job.

Here some basic information about my setup:

  • Which version of Rhasspy you’re using: 2.5-pre as of today
  • How you installed Rhasspy: Docker, satellite/master setup as described
  • Your hardware: satellite: Rpi 3B + seeedstudio 4-Mic Array for Raspberry Pi; master: PC
  • Your operating system: satellite: Raspbian buster lite; master: ubuntu 20.04 LTS
  • Software: mosquitto on ubuntu via snap; wake: porcupine, asr: pocketsphinx, nlu: fsticuffs, dialogue mgmt: Rhasspy

I installed and configured everything along the very good documentation which can be called directly off the rhasspy frontend. Audio in and out is working, transfer from satellite to master, everything looks fine.

Synopsis to my problem:
I say the hotword (“picovoice”) - which is recognized correctly. A message is issued on the MQTT.
The Wake-WAV is played (through audio-jack on the satellite).
The dialogue mgmt activates the ASR, which is then listening. I can see on the MQTT.
I say “Wie spät ist es?” (German for: What’s the time?)
And then the ASR detects this sentence not only once but several times (e.g. 8 times - see below), everytime it forwards the information via MQTT to the NLU, which understands and issues every occurence an intent on the MQTT - which ends up in a mess on the MQTT - no chance to process it seriously.

I have tried everything that came to my mind, e.g.:
Completely reinstalled the Rpi incl. OS and Rhasspy from scratch,
Tried eclipse-mosquitto, normal mosquitto, snap mosquitto and the internal ones.
Played around with all settings that can be played with (I think :)).

I made sure on both the satellite and the master only one instance (docker container) of Rhasspy is running (someone else on the forum had that problem, resulting in multiple messages).

For three days now I’m trying to solve this problem, read all forums I found, but could not work it out.
It seems I’m the only one that encounters that specific problem… :anguished:

What am I doing wrong?
Every help is appreciated.

Thanks
Martin

The MQTT bus trace I recorded with rhasspy-watch (thanks for that great tool @ced_cox!):
Pay attention to the timestamps, the action happens within 2-3 seconds…

[2020-05-12 23:39:37] [hotword] detected on site satez, for model /usr/lib/rhasspy-voltron/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/porcupine/resources/keyword_files/raspberrypi/picovoice.ppn
[2020-05-12 23:39:37] [hotword] was asked to toggle itself 'off' on site satez
[2020-05-12 23:39:37] [Asr] was asked to toggle itself 'off' on site satez
[2020-05-12 23:39:38] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:38] [Asr] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:38] [Dialogue] session with id satez-picovoice-e2896c60-ab8c-407c-ad14-6bd2272d6b23 was started on site satez.
[2020-05-12 23:39:38] [hotword] was asked to toggle itself 'off' on site satez
[2020-05-12 23:39:38] [Asr] was asked to listen on site satez
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 9.357407305993547s on site satez
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.14690904499730095s on site satez
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.14820824000344146s on site satez
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.15568158400128596s on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.15702073900320102s on site satez
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.16000881500076503s on site satez
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.16960250199917937s on site satez
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.15388689399696887s on site satez
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 0.1875761359988246s on site satez
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:48] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:48] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:48] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:48] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:48] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:48] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:48] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:48] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:48] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:48] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:48] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:39:48] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:48] [Nlu] Intent GetTime with confidence score 1.0 on site satez 
[2020-05-12 23:40:08] [Asr] was asked to stop listening on site satez
[2020-05-12 23:40:08] [Dialogue] session with id satez-picovoice-e2896c60-ab8c-407c-ad14-6bd2272d6b23 was ended on site satez. Reason: timeout
           with customData : 
               picovoice 

Did you restarted the docker container several times? Had the same bug. After some restarts it has just been gone. Weird.

Hello,

and you have these logs everytime, even after full reinstall ?

It’s strange, i reinstall rhasspy for test every 2 days and i never had this. But i use venv and not docker. However, many people use the docker without problem.

it’s as you have more than one ASR/NLU running…

This part is OK :

[2020-05-12 23:39:37] [hotword] detected on site satez, for model /usr/lib/rhasspy-voltron/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/porcupine/resources/keyword_files/raspberrypi/picovoice.ppn
[2020-05-12 23:39:37] [hotword] was asked to toggle itself 'off' on site satez
[2020-05-12 23:39:37] [Asr] was asked to toggle itself 'off' on site satez
[2020-05-12 23:39:38] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:38] [Asr] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:38] [Dialogue] session with id satez-picovoice-e2896c60-ab8c-407c-ad14-6bd2272d6b23 was started on site satez.
[2020-05-12 23:39:38] [hotword] was asked to toggle itself 'off' on site satez
[2020-05-12 23:39:38] [Asr] was asked to listen on site satez
[2020-05-12 23:39:47] [Asr] captured text 'wie spät ist es' in 9.357407305993547s on site satez
[2020-05-12 23:39:47] [Asr] was asked to stop listening on site satez
[2020-05-12 23:39:47] [hotword] was asked to toggle itself 'on' on site satez
[2020-05-12 23:39:47] [Nlu] was asked to parse input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Detected intent GetTime with confidence score 1.0 for input 'wie spät ist es'
[2020-05-12 23:39:47] [Nlu] Intent GetTime with confidence score 1.0 on site satez 

There is something little strange, it’s time of captured text. 9s… but, the intent is recognized so, not a blocking problem

And after, ASR and NLU have a strange behaviour.

Other thing, your intent is not handled :

[2020-05-12 23:40:08] [Dialogue] session  [...]. Reason: timeout
           with customData : 
               picovoice 

timeout means that your handling intent tool does not process the intent.

Did you try for first time, just Rhasspy alone ? (without a base/satellite configuration)

Ced