Rhasspy hang after successfull wakeword detection

Hello,

I need some help to troobleshoot the following problem please:

I’m using 3 different versions of Rhasspy on 3 devices (migration in progress)

  • A standard computer running Debian Buster (amd64) --> Rhasspy 2.5-pre (siteID: jeedom)
  • A laptop on Debian Buster (amd64) --> Rhasspy 2.5.6 (siteID: default)
  • A respeaker core v2 on Debian Buster (armv71) --> Rhasspy 2.6.5 (siteID: séjour)

All Rhasspy are running on docker.

  • What is working:

Rhasspy on my standard computer works fine as the “master”, with these roles:

  • mqtt --> internal
  • STT --> Kaldi
  • Intents recognition --> Fsticuffs
  • TTS --> MaryTTS

Rhasspy deployed as a satellite on the Laptop interacts fine with the Rhasspy master
This satellite has the following roles:

  • mqtt --> internal
  • arecord
  • aplay
  • Dialogue management --> Rhasspy
  • Intent Handling --> Remote HTTP (targeting jeedom API)
  • All others options --> Remote HTTP targeting the relevant Rhasspy API entrypoint on the Rhasspy master

This config runs fine.

  • What is not working:

The respeaker core v2 is configured as a sattelite with the exact same config than the Laptop sattelite.
Only things that changes on Rhasspy: the siteID

Description of the problem:
When the wakeword is triggered, I can hear the “hi” sound and that’s all; Rhasspy seems to “hang”.
Nothing happens, whatever I can say, I have no intent recognition and no reaction.
If I try to trigger the wakeword again, nothing happens. I then have to restart Rhasspy to come back to the behaviour described above.

What I don’t understand:

  • Entering an intent and hitting the “Recognize” button on Rhasspy webui on the respeaker sattelite works. Intent is recognized and correctly applied.
  • Hitting the “Wake-up” button on Rhasspy webui on the respeaker sattelite and saying an intent also works. Intent is recognized and correctly applied.

So I think there is really a problem related to the wekeword functionnality. But what is happening ?

Here is an extract of the rhasspy container logs on the respeaker from the wakeword detection to Rhasspy “hanging”.
We can see a lot of strange errors related to “session timeout”; but I’m not sure to understand what it really means.

[DEBUG:2020-10-14 23:24:41,954] rhasspyremote_http_hermes: Receiving audio

[DEBUG:2020-10-14 23:24:49,774] rhasspywake_snowboy_hermes: Wake word detected: jarvis (site_id=séjour)
[DEBUG:2020-10-14 23:24:49,784] rhasspywake_snowboy_hermes: -> HotwordDetected(model_id=‘jarvis’, model_version=’’, model_type=‘personal’, current_sensitivity=0.0, site_id=‘séjour’, session_id=None, send_audio_captured=None, lang=None)
[DEBUG:2020-10-14 23:24:49,785] rhasspywake_snowboy_hermes: Publishing 178 bytes(s) to hermes/hotword/jarvis/detected
[DEBUG:2020-10-14 23:24:49,797] rhasspyserver_hermes: <- HotwordDetected(model_id=‘jarvis’, model_version=’’, model_type=‘personal’, current_sensitivity=0.0, site_id=‘séjour’, session_id=None, send_audio_captured=None, lang=None)
[DEBUG:2020-10-14 23:24:49,801] rhasspydialogue_hermes: <- HotwordDetected(model_id=‘jarvis’, model_version=’’, model_type=‘personal’, current_sensitivity=0.0, site_id=‘séjour’, session_id=None, send_audio_captured=None, lang=None)
[DEBUG:2020-10-14 23:24:49,804] rhasspydialogue_hermes: Playing WAV /usr/lib/rhasspy/etc/wav/beep_hi.wav
[DEBUG:2020-10-14 23:24:49,809] rhasspydialogue_hermes: -> HotwordToggleOff(site_id=‘séjour’, reason=<HotwordToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:49,810] rhasspydialogue_hermes: Publishing 48 bytes(s) to hermes/hotword/toggleOff
[DEBUG:2020-10-14 23:24:49,815] rhasspydialogue_hermes: -> AsrToggleOff(site_id=‘séjour’, reason=<AsrToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:49,816] rhasspydialogue_hermes: Publishing 48 bytes(s) to hermes/asr/toggleOff
[DEBUG:2020-10-14 23:24:49,821] rhasspydialogue_hermes: -> AudioPlayBytes(83948 byte(s)) to hermes/audioServer/séjour/playBytes/495742f3-1f4c-4b7d-aabc-91cdeac9e2bf
[DEBUG:2020-10-14 23:24:49,825] rhasspywake_snowboy_hermes: <- HotwordToggleOff(site_id=‘séjour’, reason=<HotwordToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:49,827] rhasspywake_snowboy_hermes: Disabled
[DEBUG:2020-10-14 23:24:49,834] rhasspyremote_http_hermes: <- AsrToggleOff(site_id=‘séjour’, reason=<AsrToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:49,836] rhasspyremote_http_hermes: ASR disabled
[DEBUG:2020-10-14 23:24:49,825] rhasspydialogue_hermes: Waiting for playFinished (timeout=1.2012925170068027)
[DEBUG:2020-10-14 23:24:49,863] rhasspyspeakers_cli_hermes: <- AudioPlayBytes(83948 byte(s))
[DEBUG:2020-10-14 23:24:49,865] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘plughw:CARD=seeed8micvoicec,DEV=1’]
[DEBUG:2020-10-14 23:24:50,644] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘495742f3-1f4c-4b7d-aabc-91cdeac9e2bf’, session_id=’’)
[DEBUG:2020-10-14 23:24:50,646] rhasspyspeakers_cli_hermes: Publishing 63 bytes(s) to hermes/audioServer/séjour/playFinished
[DEBUG:2020-10-14 23:24:50,658] rhasspydialogue_hermes: <- AudioPlayFinished(id=‘495742f3-1f4c-4b7d-aabc-91cdeac9e2bf’, session_id=’’)
[DEBUG:2020-10-14 23:24:50,664] rhasspydialogue_hermes: -> HotwordToggleOn(site_id=‘séjour’, reason=<HotwordToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:50,666] rhasspydialogue_hermes: Publishing 48 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2020-10-14 23:24:50,672] rhasspydialogue_hermes: -> AsrToggleOn(site_id=‘séjour’, reason=<AsrToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:50,674] rhasspydialogue_hermes: Publishing 48 bytes(s) to hermes/asr/toggleOn
[DEBUG:2020-10-14 23:24:50,676] rhasspydialogue_hermes: Starting new session (id=séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729)
[DEBUG:2020-10-14 23:24:50,683] rhasspydialogue_hermes: -> DialogueSessionStarted(session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, site_id=‘séjour’, custom_data=‘jarvis’, lang=None)
[DEBUG:2020-10-14 23:24:50,684] rhasspydialogue_hermes: Publishing 135 bytes(s) to hermes/dialogueManager/sessionStarted
[DEBUG:2020-10-14 23:24:50,690] rhasspydialogue_hermes: -> HotwordToggleOff(site_id=‘séjour’, reason=<HotwordToggleReason.DIALOGUE_SESSION: ‘dialogueSession’>)
[DEBUG:2020-10-14 23:24:50,692] rhasspydialogue_hermes: Publishing 54 bytes(s) to hermes/hotword/toggleOff
[DEBUG:2020-10-14 23:24:50,693] rhasspydialogue_hermes: Listening for session séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729
[DEBUG:2020-10-14 23:24:50,702] rhasspydialogue_hermes: -> AsrStartListening(site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, lang=None, stop_on_silence=True, send_audio_captured=True, wakeword_id=‘jarvis’, intent_filter=None)
[DEBUG:2020-10-14 23:24:50,706] rhasspydialogue_hermes: Publishing 207 bytes(s) to hermes/asr/startListening
[DEBUG:2020-10-14 23:24:50,707] rhasspywake_snowboy_hermes: <- HotwordToggleOn(site_id=‘séjour’, reason=<HotwordToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:50,708] rhasspywake_snowboy_hermes: Enabled
[DEBUG:2020-10-14 23:24:50,706] rhasspyremote_http_hermes: <- AsrToggleOn(site_id=‘séjour’, reason=<AsrToggleReason.PLAY_AUDIO: ‘playAudio’>)
[DEBUG:2020-10-14 23:24:50,716] rhasspyremote_http_hermes: ASR enabled
[DEBUG:2020-10-14 23:24:50,732] rhasspywake_snowboy_hermes: <- HotwordToggleOff(site_id=‘séjour’, reason=<HotwordToggleReason.DIALOGUE_SESSION: ‘dialogueSession’>)
[DEBUG:2020-10-14 23:24:50,732] rhasspyremote_http_hermes: <- AsrStartListening(site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, lang=None, stop_on_silence=True, send_audio_captured=True, wakeword_id=‘jarvis’, intent_filter=None)
[DEBUG:2020-10-14 23:24:50,733] rhasspywake_snowboy_hermes: Disabled
[DEBUG:2020-10-14 23:24:50,736] rhasspyremote_http_hermes: <- AsrStartListening(site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, lang=None, stop_on_silence=True, send_audio_captured=True, wakeword_id=‘jarvis’, intent_filter=None)
[ERROR:2020-10-14 23:25:20,742] rhasspydialogue_hermes: Session timed out for site séjour: séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729
[DEBUG:2020-10-14 23:25:20,749] rhasspydialogue_hermes: -> AsrStopListening(site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’)
[DEBUG:2020-10-14 23:25:20,751] rhasspydialogue_hermes: Publishing 97 bytes(s) to hermes/asr/stopListening
[DEBUG:2020-10-14 23:25:20,764] rhasspydialogue_hermes: -> DialogueSessionEnded(termination=DialogueSessionTermination(reason=<DialogueSessionTerminationReason.TIMEOUT: ‘timeout’>), session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, site_id=‘séjour’, custom_data=‘jarvis’)
[DEBUG:2020-10-14 23:25:20,766] rhasspydialogue_hermes: Publishing 159 bytes(s) to hermes/dialogueManager/sessionEnded
[DEBUG:2020-10-14 23:25:20,778] rhasspydialogue_hermes: -> HotwordToggleOn(site_id=‘séjour’, reason=<HotwordToggleReason.DIALOGUE_SESSION: ‘dialogueSession’>)
[DEBUG:2020-10-14 23:25:20,784] rhasspydialogue_hermes: Publishing 54 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2020-10-14 23:25:20,823] rhasspywake_snowboy_hermes: <- HotwordToggleOn(site_id=‘séjour’, reason=<HotwordToggleReason.DIALOGUE_SESSION: ‘dialogueSession’>)
[DEBUG:2020-10-14 23:25:20,824] rhasspywake_snowboy_hermes: Enabled
[DEBUG:2020-10-14 23:25:20,829] rhasspyremote_http_hermes: <- AsrStopListening(site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’)
[DEBUG:2020-10-14 23:25:20,832] rhasspyremote_http_hermes: <- AsrStopListening(site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’)
[DEBUG:2020-10-14 23:25:20,834] rhasspyremote_http_hermes: Received 8684 byte(s) of WAV data
[DEBUG:2020-10-14 23:25:20,837] rhasspyremote_http_hermes: http://192.168.1.251:12101/api/speech-to-text
[DEBUG:2020-10-14 23:25:21,037] rhasspyremote_http_hermes: -> AsrTextCaptured(text=’’, likelihood=0.0, seconds=0.15199856599792838, site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, wakeword_id=None, asr_tokens=None, lang=None)
[DEBUG:2020-10-14 23:25:21,040] rhasspyremote_http_hermes: Publishing 213 bytes(s) to hermes/asr/textCaptured
[DEBUG:2020-10-14 23:25:21,043] rhasspyremote_http_hermes: -> AsrAudioCaptured(8684 byte(s)) to rhasspy/asr/séjour/séjour/audioCaptured
[DEBUG:2020-10-14 23:25:21,058] rhasspydialogue_hermes: <- AsrTextCaptured(text=’’, likelihood=0.0, seconds=0.15199856599792838, site_id=‘séjour’, session_id=‘séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729’, wakeword_id=None, asr_tokens=None, lang=None)
[WARNING:2020-10-14 23:25:21,060] rhasspydialogue_hermes: Ignoring unknown session séjour-jarvis-d5b7ce62-c137-4029-94cb-0272a1daf729

Some of you guys could help me on this ?
Thank you for your help

Hey there!

Looking at your configuration, I would suggest to use a central MQTT broker (expose the MQTT port on your master docker container and plug the satellites on it) for inter device communication instead of HTTP.

Also, only enable the dialogue management and intent handling on the master (not on the satellite). The satellite only needs audio In, wakeword and audio out. The rest should be only on the master and use MQTT on the satellite.

Also try to update to the latest Rhasspy version to get the multiple session/site fix.

Hope this helps.

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

Hi everyone,
Some news.

I did some cleaning on my install and tried to follow @fastjack advice.

Now my setup is:

  • All rhasspy on 2.5.6 with latest docker images pulled
  • the master is on a dedicated computer
  • satellite1 is on a laptop
  • satellite2 is on the respeaker core v2

There are 2 problems:

  • the master and satellite1 are working, except I had to let “Intent handling” on the satellite1. I was not able to make intent handling working from the master when the intent comes from the satellite1. If I use the recognize button from the master’s webui it works.
  • the satellite2 (respeaker) doesn’t work at all with the master when I tries to say an order to the respeaker by waking up the wake word. But it works if I use the recognize button from the satellite2 webui. It also works if I use the “Wake up” button from the satellite2 webui and say an order.

It seems rhasspy still hangs on the satellite2 whe I triggers the wake word.
Please find the logs bellow:

  • Rhasspy logs (from logs streaming api):
    [WARNING:2020-10-15 21:33:34,916] rhasspyserver_hermes: Dialogue management is disabled. ASR will NOT be automatically enabled.
    [DEBUG:2020-10-15 21:33:34,911] rhasspyserver_hermes: <- HotwordDetected(model_id=’/profiles/fr/raven/default/example-2.wav’, model_version=’’, model_type=‘personal’, current_sensitivity=0.5, site_id=‘satellite2’, session_id=None, send_audio_captured=None, lang=None)

  • Rhasspy container logs:
    [DEBUG:2020-10-15 21:33:33,597] rhasspywake_raven_hermes: Receiving audio
    [DEBUG:2020-10-15 21:33:34,885] rhasspy-wake-raven: Enter refractory for 2.0 second(s)
    [DEBUG:2020-10-15 21:33:34,895] rhasspywake_raven_hermes: -> HotwordDetected(model_id=’/profiles/fr/raven/default/example-2.wav’, model_version=’’, model_type=‘personal’, current_sensitivity=0.5, site_id=‘satellite2’, session_id=None, send_audio_captured=None, lang=None)
    [DEBUG:2020-10-15 21:33:34,895] rhasspywake_raven_hermes: Publishing 211 bytes(s) to hermes/hotword/default/detected
    [DEBUG:2020-10-15 21:33:34,911] rhasspyserver_hermes: <- HotwordDetected(model_id=’/profiles/fr/raven/default/example-2.wav’, model_version=’’, model_type=‘personal’, current_sensitivity=0.5, site_id=‘satellite2’, session_id=None, send_audio_captured=None, lang=None)
    [WARNING:2020-10-15 21:33:34,916] rhasspyserver_hermes: Dialogue management is disabled. ASR will NOT be automatically enabled.
    [DEBUG:2020-10-15 21:33:34,919] rhasspywake_raven_hermes: <- HotwordToggleOff(site_id=‘satellite2’, reason=<HotwordToggleReason.PLAY_AUDIO: ‘playAudio’>)
    [DEBUG:2020-10-15 21:33:34,920] rhasspywake_raven_hermes: Disabled
    [DEBUG:2020-10-15 21:33:34,997] rhasspyspeakers_cli_hermes: <- AudioPlayBytes(83948 byte(s))
    [DEBUG:2020-10-15 21:33:34,998] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘plughw:CARD=seeed8micvoicec,DEV=1’]
    [DEBUG:2020-10-15 21:33:35,075] rhasspyremote_http_hermes: Namespace(asr_command=None, asr_train_command=None, asr_train_url=None, asr_url=None, casing=‘ignore’, certfile=None, debug=True, handle_command=None, handle_url=‘http://192.168.1.251/core/api/jeeApi.php?plugin=jeerhasspy&apikey=LOE0QkHMi7ANVLKMvyRZRed8KFIedxPI&plugin=jeerhasspy&type=jeerhasspy’, host=‘192.168.1.253’, keyfile=None, log_format=’[%(levelname)s:%(asctime)s] %(name)s: %(message)s’, nlu_command=None, nlu_train_command=None, nlu_train_url=None, nlu_url=None, password=None, port=1883, site_id=[‘satellite2’], tls=False, tls_ca_certs=None, tls_cert_reqs=‘CERT_REQUIRED’, tls_certfile=None, tls_ciphers=None, tls_keyfile=None, tls_version=None, tts_url=None, username=None, voice_before_seconds=0.5, voice_min_seconds=1.0, voice_sensitivity=3, voice_silence_seconds=0.5, voice_skip_seconds=0.0, voice_speech_seconds=0.3, wake_channels=1, wake_command=None, wake_sample_rate=16000, wake_sample_width=2, webhook=None)
    [DEBUG:2020-10-15 21:33:35,081] asyncio: Using selector: EpollSelector
    [DEBUG:2020-10-15 21:33:35,095] rhasspyremote_http_hermes: Connecting to 192.168.1.253:1883
    [DEBUG:2020-10-15 21:33:35,114] asyncio: Using selector: EpollSelector
    [DEBUG:2020-10-15 21:33:35,118] rhasspyremote_http_hermes: Connected to MQTT broker
    [DEBUG:2020-10-15 21:33:35,120] rhasspyremote_http_hermes: Subscribed to hermes/intent/#
    [DEBUG:2020-10-15 21:33:35,121] rhasspyremote_http_hermes: Subscribed to rhasspy/handle/toggleOn
    [DEBUG:2020-10-15 21:33:35,123] rhasspyremote_http_hermes: Subscribed to rhasspy/handle/toggleOff
    [DEBUG:2020-10-15 21:33:35,768] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘5d7454a0-dcf0-4621-b47a-c37a9330395e’, session_id=’’)
    [DEBUG:2020-10-15 21:33:35,769] rhasspyspeakers_cli_hermes: Publishing 63 bytes(s) to hermes/audioServer/satellite2/playFinished
    [DEBUG:2020-10-15 21:33:35,789] rhasspywake_raven_hermes: <- HotwordToggleOn(site_id=‘satellite2’, reason=<HotwordToggleReason.PLAY_AUDIO: ‘playAudio’>)
    [DEBUG:2020-10-15 21:33:35,791] rhasspywake_raven_hermes: Enabled
    [DEBUG:2020-10-15 21:33:35,832] rhasspywake_raven_hermes: Receiving audio
    [DEBUG:2020-10-15 21:33:35,852] rhasspywake_raven_hermes: <- HotwordToggleOff(site_id=‘satellite2’, reason=<HotwordToggleReason.DIALOGUE_SESSION: ‘dialogueSession’>)
    [DEBUG:2020-10-15 21:33:35,853] rhasspywake_raven_hermes: Disabled
    [DEBUG:2020-10-15 21:34:05,823] rhasspywake_raven_hermes: <- HotwordToggleOn(site_id=‘satellite2’, reason=<HotwordToggleReason.DIALOGUE_SESSION: ‘dialogueSession’>)
    [DEBUG:2020-10-15 21:34:05,824] rhasspywake_raven_hermes: Enabled
    [DEBUG:2020-10-15 21:34:05,854] rhasspywake_raven_hermes: Receiving audio
    [DEBUG:2020-10-15 21:34:06,500] rhasspy-wake-raven: Exiting refractory period

I wonder if it could be a problem related to the docker image I’ve pulled for the respeaker or something else ?
If somebody has got an idea, wou’re welcome :slight_smile:

Thank you and have a nice evening.

I unfortunately cannot post my configs because of links posting restrictions.

But what I can say to help is that the satellite1 and satellite2 have exactly the same rhasspy config, except the siteID of course.

For an easier logs reading:

  • 192.168.1.21 --> satellite2 (respeaker)
  • 192.168.1.253 --> sattelite1 (laptop)
  • 192.168.1.251 --> master (computer)

Hope I will be abble to post some details later.

Don’t hesitate to ask some information if it can help in solving my problem :wink:

Thank you again

Best regards

The fix for the dialogue management issue is under way.

Dialogue manager fix has been pushed to Docker. Debian packages are coming soon!