Rhasspy with Matrix Voice ESP32: no sound input and output

Help
1

Hi,

i have tried to install Rhasspy with a Matrix Voice Mic Array. I followed this guide: https://www.hackster.io/matrix-labs/rhasspy-voice-assistant-on-matrix-voice-and-matrix-creator-97f92e
but the result is that there is no sound output and probably no input. After pushing the test button in the settings category, ‘no sound’ is appended to the Matrix Voice:
matrix
even after saving and restarting the default device is selected not the MatrixIO.

Does anyone have an idea to solve this?

thanks in advance!
Norbert

2

Hi Norbert,
What Raspberry Pi are you using?
I am using a Raspberry Pi 3B+ with a Matrix Voice, followed the same tutorial and had no problems with getting the Matrix Voice to work.
Did you test your Matrix Voice with arecord and aplay?

3

Hi RaspiManu,

yes i could make it work with aplay.
My setup is a Raspi 3B+, the speaker are connected to speaker ports (not headphone jack). My asoud.conf begins with:

 pcm.!default {
type asym
capture.pcm "mic"
playback.pcm "speaker"
}

pcm.mic {
   type softvol
   slave {
     pcm "array"
   }
   control {
     name "MicArray Master"
        card "MATRIXIOSOUND"
   }
}

pcm.speaker {
  type plug
  slave {
    pcm "hw:2,1"
    rate 44100
  }
}

pcm.array {
  type plug
  slave {
    pcm "hw:MATRIXIOSOUND"
  }
}

After saying the wake word the logs looks like:

[ERROR:541931] main: api_websocket
Traceback (most recent call last):
File “app.py”, line 1232, in wrapper
return await func(queue, *_args, **kwargs)
File “app.py”, line 1247, in api_events_intent
message_type, text = await queue.get()
File “/usr/lib/python3.6/asyncio/queues.py”, line 167, in get
yield from getter
concurrent.futures._base.CancelledError
[INFO:541910] quart.serving: 192.168.178.20:57723 GET /api/events/intent 1.1 101 - 64570745
[INFO:541902] quart.serving: 192.168.178.20:57706 GET /api/events/intent 1.1 101 - 110712529
[DEBUG:541882] PorcupineWakeListener: started -> listening
[DEBUG:541858] DialogueManager: ready -> asleep
[INFO:541854] DialogueManager: Automatically listening for wake word
[DEBUG:541847] DialogueManager: handling -> ready
[DEBUG:541846] APlayAudioPlayer: [‘aplay’, ‘-q’, ‘/usr/share/rhasspy/etc/wav/beep_error.wav’]
[DEBUG:541841] WebSocketObserver: {“text”: “wie sp\u00e4t ist das”, “intent”: {“name”: “”, “confidence”: 0}, “entities”: [], “raw_text”: “wie sp\u00e4t ist das”, “speech_confidence”: 0.1277936440913965, “wakeId”: “porcupine”, “siteId”: “default”, “slots”: {}}
[DEBUG:541840] DialogueManager: recognizing -> handling
[DEBUG:541838] DialogueManager: {‘text’: ‘wie spät ist das’, ‘intent’: {‘name’: ‘’, ‘confidence’: 0}, ‘entities’: [], ‘raw_text’: ‘wie spät ist das’, ‘speech_confidence’: 0.1277936440913965, ‘wakeId’: ‘porcupine’, ‘siteId’: ‘default’}
[ERROR:541828] FsticuffsRecognizer: in_loaded
Traceback (most recent call last):
File “/usr/share/rhasspy/rhasspy/intent.py”, line 183, in in_loaded
assert recognitions, “No intent recognized”
AssertionError: No intent recognized
[DEBUG:541823] DialogueManager: decoding -> recognizing
[DEBUG:541822] DialogueManager: wie spät ist das (confidence=0.1277936440913965)
[DEBUG:541819] PocketsphinxDecoder: wie spät ist das
[DEBUG:541818] PocketsphinxDecoder: Transcription confidence: 0.1277936440913965
[DEBUG:541814] PocketsphinxDecoder: Decoded WAV in 1.2859702110290527 second(s)
[DEBUG:540526] APlayAudioPlayer: [‘aplay’, ‘-q’, ‘/usr/share/rhasspy/etc/wav/beep_lo.wav’]
[DEBUG:540524] PocketsphinxDecoder: rate=16000, width=2, channels=1.
[DEBUG:540522] DialogueManager: awake -> decoding
[DEBUG:540519] WebrtcvadCommandListener: listening -> loaded
[DEBUG:540516] WebrtcvadCommandListener: Voice command finished
[DEBUG:538537] WebrtcvadCommandListener: Voice command started
[INFO:523180] quart.serving: 192.168.178.20:57707 GET /api/events/wake 1.1 101 - 91990633

So for me it looks like, the microphone ‘hears’ the input, but can’t give feedback.

I’m a bit at a loss

thanks in advance
Norbert

4

The wakeword and intent seem to be recognized correctly. How do you handle your recognized intent, because you expect a response telling the time.

The tutorial also makes use of the Rhasspy 2.4.19 version and not the beta version of 2.5. Not sure if it makes a difference but this can be a handy thing to know when comparing installations.

5

Rhasspy does not give you a direct answer on your question. It will identify the intent and has to send it over to homeassistant, node-red oder jeedom for example. Then the intent will get handled and you will get an answer.

6

Good morning,

i haven’t configured the intend handling, your are right.
But messages in the log like “aplay’, ‘-q’, '/usr/share/rhasspy/etc/wav/beep_lo.wav” doesn’t make a sound output. Even executed in the docker internal bash it doesn’t generate sound output.

best,
Norbert

7

Did you try to connect a speaker to the headphone jack aswell?
What is defined as audio output in your Raspbian system on the Pi?

8

yes, even on the headphone is no output.
i have selected the speaker output on the raspian and aplay … works,
inside the docker container not.

9

At this point, I’m not sure what to try next, because I’m also quite new to the Rhasspy community. Can anyone else in here help @NSchnitzler with this?

10

Not sure if you already checked but did you take the audio devices through to docker?

11

@NSchnitzler could you or someone else fix this problem with raspberry 4 and matrix voice?

I have the same issue, especially with sound output (beeps or play record) over the matrix voice speaker ports (no jack).

With your asound.conf settings I could get the sound input running, but still no output.
The log with “okay rhasspy” after “wake up - button” says:

[DEBUG:2020-04-11 12:17:06,507] rhasspyserver_hermes: Sent 267 char(s) to websocket
[DEBUG:2020-04-11 12:17:06,498] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/GetGarageState, id=21661a75-0beb-48a2-8bbd-fcae1857ef26)
[DEBUG:2020-04-11 12:17:06,495] rhasspyserver_hermes: <- NluIntent(input=‘ist das garagentor geschlossen’, intent=Intent(intent_name=‘GetGarageState’, confidence_score=0.85), site_id=‘default’, id=‘ea404648-44bb-40c3-91ba-498f1263cffb’, slots=[], session_id=‘ea404648-44bb-40c3-91ba-498f1263cffb’, custom_data=None, asr_tokens=[[AsrToken(value=‘ist’, confidence=1.0, range_start=0, range_end=3, time=None), AsrToken(value=‘das’, confidence=1.0, range_start=4, range_end=7, time=None), AsrToken(value=‘garagentor’, confidence=1.0, range_start=8, range_end=18, time=None), AsrToken(value=‘geschlossen’, confidence=1.0, range_start=19, range_end=30, time=None)]], asr_confidence=None, raw_input=‘ist es wie’, wakeword_id=None)
[DEBUG:2020-04-11 12:17:06,352] rhasspyserver_hermes: Publishing 185 bytes(s) to hermes/nlu/query
[DEBUG:2020-04-11 12:17:06,350] rhasspyserver_hermes: -> NluQuery(input=‘ist es wie’, site_id=‘default’, id=‘ea404648-44bb-40c3-91ba-498f1263cffb’, intent_filter=None, session_id=‘ea404648-44bb-40c3-91ba-498f1263cffb’, wakeword_id=None)
[DEBUG:2020-04-11 12:17:06,345] rhasspyserver_hermes: Publishing 74 bytes(s) to hermes/asr/stopListening
[DEBUG:2020-04-11 12:17:06,344] rhasspyserver_hermes: -> AsrStopListening(site_id=‘default’, session_id=‘ea404648-44bb-40c3-91ba-498f1263cffb’)
[DEBUG:2020-04-11 12:17:06,340] rhasspyserver_hermes: Waiting for intent (session_id=ea404648-44bb-40c3-91ba-498f1263cffb)
[DEBUG:2020-04-11 12:17:06,330] rhasspyserver_hermes: Handling AsrTextCaptured (topic=hermes/asr/textCaptured, id=6e82b29e-be2c-4da4-8d51-060963fbc6dc)
[DEBUG:2020-04-11 12:17:02,616] rhasspyserver_hermes: Publishing 144 bytes(s) to hermes/asr/startListening
[DEBUG:2020-04-11 12:17:02,615] rhasspyserver_hermes: -> AsrStartListening(site_id=‘default’, session_id=‘ea404648-44bb-40c3-91ba-498f1263cffb’, stop_on_silence=True, send_audio_captured=True, wakeword_id=None)
[DEBUG:2020-04-11 12:17:02,612] rhasspyserver_hermes: Waiting for transcription (session_id=ea404648-44bb-40c3-91ba-498f1263cffb)

If I try to play a sample, the sound output works great over the speaker ports with:

wget https://goo.gl/CDF6sf -O ./audio-sample.wav
aplay -D "hw:2,1" ./audio-sample.wav

But when I choose matrixvoice from the available devices, nothing happens, except this error after hit the button “Play Recording”:

RuntimeError: Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘dmix:CARD=MATRIXIOSOUND,DEV=1’]’ returned non-zero exit status 1.

a
a1491×496 194 KB

Does anyone have an idea or a hint?

thx and greetings,
Thomas

12

i didn’t pass any audio configuration to the docker file, but the audio input seems to work. Just the output seems to make a problem.

13

Is a general output working as well with your setup @NSchnitzler, like:

Could it be a Rhasspy phenomenon or is it for sure an aplay issue?

Maybe someone else is running with matrix voice and sound output?

I like to isolate some causes.

14

I am running Rhasspy on RPi 3B+ with Raspbian Buster (Desktop Version) and Matrix Voice and I have no problem with sound output. I got a normal speaker plugged into the audio jack of the Pi and set the Pi’s sound to analog (via right click on sound icon in taskbar).

15

Yes, the jack output of the pi is working. The “challenge” is to get some sound out of the onboard speaker connections of the matrix voice.

a

16

Oh, I see. Sadly, I do not own a speaker to test this, sorry. Can anyone else test this for @thomas_cologne?

17

Is this issue still open?

18

So far I couldn’t solve this issue and have no idea anymore

19

OK. I’m not using Matrix Voice ESP32 but Matrix Voice with Raspberry Pi. I guess the ESP32 functionality is not relevant in this context. I also have speakers connected to speaker ports and they are working in principal.

What you have to do to enable them:

  1. configure Rhasspy audio settings to aplay, and use this device: hw:CARD=MATRIXIOSOUND,DEV=1

  2. start alsamixer in a terminal. F6 to select Matrixvoice. Switch output from headphone to speakers

That helped me. Pls try