Using Rhasspy and Snips at the same time?

Help
, , , ,
1

Dear all,

my aim is to use Snips and Rhasspy at the same time. The reason for this is that I had some well functioning Snips Skills (for example asking for regional train connections or adding stuff to my “Bring”-grocery-list). Another reason is that I would like to keep using the “Hey Snips”-Hotword.

But until now I wasn`t able to get it up and running - due to changing and sometimes recurring problems.

Now my question: Has anyone a similar setup and can me show the needed settings and preferences? Or can anyone give me hints?

My setup:
** Hardware **

  • Raspberry Pi 4 with Raspbian Buster (2GB RAM)
  • Seed Respeaker 4-Mic-Array (with HermesLED installed)

** Software **

  • Snips installed (not in a container)
  • Home Assistant/hass.io in a Docker Container on the same Raspberry Pi 4
  • Rhasspy 2.4.19 in a Docker Container on the same Raspberry Pi 4 (german profile)
  • Rhasspy 2.5-pre in another Container, pointing to the same profile in /.config/Rhasspy/profiles/de/ (but I am waiting for some features like Google TTS Wavenet-Cache support to switch for good)

My idea until now:

  • Listen for wakeword on start-up
  • MQTT enabled, pointing to IP Adress and Port of Snips’ MQTT broker (Publish intent over MQTT checked)
  • Intent Handling via Hass (Token and PW entered), send events to Home Assistant checked
  • Wake up on MQTT message
  • Voice detection: Use MQTT
  • Speech recognition: Pocketsphinx (or Kaldi, if it does not take too long on my Raspberry Pi 4)
  • Do intent recognition with OpenFST on this device (don’t know which ist the best option here for my setup / language is german)
  • TTS: Google Wavenet (credentials.json in the profile directory, path /tts/googlewavenet/credentials.json)
  • Audio recording: Get microphone input remotely with MQTT
  • Play sound remotely with MQTT
  • WAV sounds: changed the path file to Snips’-WAV-Files (e.g. ${RHASSPY_BASE_DIR}/usr/share/snips/assisstant/custom_dialogue/sound/start_of_input.wav), and disabled the sound feeback in snips itself (but did not hear any sound feedback since)
2

One of this problems is for example, that I say “hey Snips” then I hear an immediate error-Sound before I am able to say anything. This is the log of “sam watch”:

[15:55:58] [Hotword] detected on site default, for model hey_snips
[15:55:58] [Asr] was asked to stop listening on site default
[15:55:58] [Hotword] was asked to toggle itself ‘off’ on site default
[15:55:58] [Dialogue] session with id ‘f177d132-13c2-444f-ade6-2047d820c354’ was started on site default
[15:55:58] [AudioServer] was asked to play a wav of 41.1 kB with id ‘fe0d75c1-4b21-4202-907d-a77e9f7e1217’ on site default
[15:55:58] [Asr] captured text “” in 0.0s
[15:55:58] [AudioServer] was asked to play a wav of 61.1 kB with id ‘1b34605e-a197-40e5-b62f-d263e81a4761’ on site default
[15:55:59] [AudioServer] finished playing wav with id ‘fe0d75c1-4b21-4202-907d-a77e9f7e1217’
[15:55:59] [AudioServer] finished playing wav with id ‘1b34605e-a197-40e5-b62f-d263e81a4761’
[15:55:59] [Dialogue] session with id ‘f177d132-13c2-444f-ade6-2047d820c354’ was ended on site default. The session was ended because an error happened: Event not supported: Response { waiting: AudioFinishedPayload }, with response: AsrCapture(AsrPayload { text: “”, tokens: None, likelihood: 1.0, seconds: 0.0, speaker_hypotheses: None })

[15:55:59] [Asr] was asked to stop listening on site default
[15:55:59] [Hotword] was asked to toggle itself ‘on’ on site default

3

Can anyone help me with this, please?

4

I had a similar setup working with rhasspy 2.4, I used an external MQTT broker and the snips satellites. My Rhasppy and snips servers where on a pc without microphones, all input was from MQTT from the snips sattelites. Did you turn off Rhasspys own wakeword detection?