Unable to use Respeaker card with Rhasspy!

vincen · June 19, 2021, 5:01pm

Hi

I jut installed with deb package rhasspy 2.5.10 on a clean raspbian on a Pi 0 W. So far Rhasspy looks to run fine but I can’t use the Respeaker audio recording interface in Rhasspy.
When I select Audio Recording (setup with PyAudio), I click Refresh then I select my respeaker audio card and Save settings, Rhasspy restarts and when I go back in Audio recordings tab it is on Default I have installed the nice little script for LED animation on the respeaker and I see well LEDs go on when I put Rhasspy in Wakeup mode but it detects nothing. I have also checked in alsamixer if all input levels on respeaker cards were ok (they are all at mid range).
When I check in JSon config file it looks like it doesn’t save the setting

“microphone”: {
“pyaudio”: {
“device”: “0”
},
“system”: “pyaudio”
},

Not sure if I missed something obvious there but I tried with a different Respeaker card just in case and it’s same

For playing it looks to record properly the respeaker but I don’t have any sound coming out and I have a timeout with that error after:

[ERROR:2021-06-19 18:02:15,615] rhasspyserver_hermes:
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1692, in api_text_to_speech
results = await asyncio.gather(*aws)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1678, in speak
say_chars_per_second=say_chars_per_second,
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 616, in speak_sentence
handle_finished(), messages, message_types, timeout_seconds=timeout_seconds
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 994, in publish_wait
result_awaitable, timeout=timeout_seconds
File “/usr/lib/rhasspy/usr/local/lib/python3.7/asyncio/tasks.py”, line 449, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2021-06-19 18:01:45,555] rhasspyserver_hermes: Publishing 141 bytes(s) to hermes/tts/say
[DEBUG:2021-06-19 18:01:45,519] rhasspyserver_hermes: → TtsSay(text=‘quel temps fait il’, site_id=‘salon’, lang=None, id=‘7ff16a8e-ea7c-4999-b0d8-9507a7317f66’, session_id=’’, volume=1.0)
[DEBUG:2021-06-19 18:01:45,458] rhasspyserver_hermes: Subscribed to hermes/error/audioServer/play
[DEBUG:2021-06-19 18:01:45,437] rhasspyserver_hermes: Subscribed to hermes/audioServer/salon/playBytes/#
[DEBUG:2021-06-19 18:01:45,417] rhasspyserver_hermes: Subscribed to hermes/error/tts
[DEBUG:2021-06-19 18:01:45,391] rhasspyserver_hermes: Subscribed to hermes/tts/sayFinished
[DEBUG:2021-06-19 18:01:45,339] rhasspyserver_hermes: TTS timeout will be 30 second(s)
[DEBUG:2021-06-19 18:01:43,124] rhasspyserver_hermes: Publishing 34 bytes(s) to rhasspy/audioServer/setVolume
[DEBUG:2021-06-19 18:01:43,063] rhasspyserver_hermes: → AudioSetVolume(volume=1.0, site_id=‘salon’)

My system is setup as Remote HTTP server for one server running as a service in Home Assistantand satellite on the PI 0 W where I have these audio problems !

Thanks for your ideas

Vincèn

DustyArmstrong · June 20, 2021, 10:48am

Hey Vincèn, I had the same issue as you when trying to use Pyaudio.

Have you tried using arecord? This worked for me.

    "microphone": {
        "arecord": {
            "device": "sysdefault:CARD=Device"
        },
        "pyaudio": {
            "device": "1"
        },
        "system": "arecord"
       },

It will also (if your issue is like mine) act strangely when trying to select the microphone, but if you select defaults then Test, it should show *working!* in the menu for the default option, where you can then select it.

I also had this timeout error when using TTS on the base server, the only way I worked out to resolve it was through using TTS on the satellite itself. You can check this or this thread for more info, I think romkabouter may have had the solution to remote TTS → audio on satellite, even if I wasn’t successful in implementing it.

vincen · June 20, 2021, 1:21pm

Yep I had tried both buth both were failing same. As I found it in all matters a little too slow on Pi 0 I have moved it to a spare Pi4 and now no more problems to setup respeaker with Rhasppy

Thanks for these links I’ll check them more in details once I have a fully working setup on a standalone Rhasspy setup. Right now wake up word functions, it understands my orders and send them at HA but no way to get it to play the wav files to confirm each step. I’m well using Rhasspy for Dialogue Management but nothing ! I have setup these file paths (after checking in ssh on pi): /usr/lib/rhasspy/etc/wav/ and no errors in logs too will continue to investigate !

romkabouter · June 20, 2021, 3:14pm

What is it you want to achieve?

vincen · June 20, 2021, 4:22pm

If I understand properly the Sounds TAb at end of Settings is supposed to play a distinct noise to identify wake-up, order recorded or error but I can’t get these sounds playing while if I do an audio output test in web interface it works and same also when I ask HA to answer with voice message at a command ! Am I wrong ?

romkabouter · June 20, 2021, 5:55pm

That is correct, what are you settings now and what is your setup (hardware etc)

vincen · June 21, 2021, 3:35pm

Standalone setup of rhasspy on a PI4 with Respeaker 2Mic Hat. I have well setup Dialogue Management to Rhasppy but these sounds never play !

All paths checked in SSH on the PI4 ! and I have no error in logs of Rhasppy indicating a problem to play these files.
I have also installed on that PI4 the Python script to get LEDs on the Respeaker to indicate the different states of the voice assistant and they work fine and the voice assistant itself out of these sounds works perfect (able to wake it up, send orders to HA, and get answer back from HA with TTS.
Bug in Rhasspy ?

romkabouter · June 21, 2021, 4:36pm

Highly unlikely.
Can you make screenshots of your other settings?
Why do you not use ${RHASSPY_PROFILE_DIR} ?

Can you post a log where we can see the wakeword detected?

vincen · June 21, 2021, 5:08pm

going to post json file, going to be easier I think no ?

{
“dialogue”: {
“system”: “rhasspy”
},
“handle”: {
“system”: “hass”
},
“home_assistant”: {
“access_token”: “eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiI1OTBhZjAxNDUxODM0OTM4ODYzNjE3OTkyxxxxxxxxxxxxxxxxx.XcZMWG7pPcmvX5a4PHLhUIlRW0_gAHRVwvq-0h3DdBs”,
“url”: “http://192.168.1.40:8123”
},
“intent”: {
“system”: “fsticuffs”
},
“microphone”: {
“pyaudio”: {
“device”: “0”
},
“system”: “pyaudio”
},
“mqtt”: {
“site_id”: “salon”
},
“sounds”: {
“aplay”: {
“device”: “sysdefault:CARD=seeed2micvoicec”
},
“error”: “/usr/lib/rhasspy/etc/wav/beep_error.wav”,
“recorded”: “/usr/lib/rhasspy/etc/wav/beep_lo.wav”,
“system”: “aplay”,
“wake”: “/usr/lib/rhasspy/etc/wav/beep_hi.wav”
},
“speech_to_text”: {
“system”: “kaldi”
},
“text_to_speech”: {
“system”: “nanotts”
},
“wake”: {
“porcupine”: {
“keyword_path”: “jarvis_raspberry-pi.ppn”
},
“raven”: {
“keywords”: {
“Jarvis”: {
“enabled”: true
}
},
“probability_threshold”: “0.4”,
“vad_sensitivity”: “2”
},
“system”: “raven”
}
}

Well these files are not stored in profile dir by default on my system ! Should I copy them somewhere in profile dir ?

I have difficulties to get logs as the popup in webinterface doesn’t really refresh well I tried to access the API logs url indicated in the popup but not working !

romkabouter · June 21, 2021, 6:38pm

Profile looks ok and since you can hear sound from the TTS the audio part is not the problem.
That leaves us with the file path, so I suggest copy them to the profiles folder of Rhasspy.
${RHASSPY_PROFILE_DIR} is including the language, so this:
${RHASSPY_PROFILE_DIR}/sounds/error.wav represents profiles/en/sounds/error.wav.
Depending on your language obviously

You can find the log on the filesystem as well, but since you are using the deb I cannot tell where.
I do not use that myself, but can you do a search.

vincen · June 21, 2021, 7:18pm

did that and not better unhappy Will do some searches about place where are logs of rhasspy to understand the problem hopefully !

hugocoolens · June 24, 2021, 6:54am

Where can I find that script Vincen?

koan · June 24, 2021, 7:02am

I believe he’s talking about:

vincen · June 24, 2021, 7:40am

Yep right it’s that one that is a fork and update of the original one used for SNIPS in past !

DivanX10 · April 6, 2023, 5:46pm

Configuring the sound card for microphone and speaker operation