Rhasspy with Adafruit speaker bonnet and Pulseaudio

tenn0 · March 11, 2021, 10:59pm

Hey,
I tried setting up rhasspy on a raspi 3b+ with an adafruit speakerbonnet and generic usb mic. When i try to access the audio devices via ALSA, i get sound output easy. But im also using the raspi for pi cam streaming via udp over raspivid raspivid -n -t 0 -w 1280 -h 720 -vf -drc "off" -sh 0 -br 55 -co 0 -sa 0 -ifx "none" -g 60 -fps 60 -b 10000000 -mm "average" -ih -pf "high" -awb "auto" -o udp://192.168.2.100:8888 wich works like a charm, but if rhasspy is running, it wont register any wakewords or tts isnt working. I also plan to use mopidy, but obv running both rhasspy and mopidy will be problematic. Thats the reason to use Pulseaudio.
Im using raspi os lite buster, since i wont need any desktop env. Ive installed the drivers for the Adafruit speaker bonnet (it appears to be a hifiberry dac, i havent checked the datasheets of the hifiberry chip and the adafruit one yet). Ive installed pulseaudio and a few other modules i dont remember exactly, rebootet and tried it with rhasspy installed via docker. This way, i get always connection refused, from the containers shell i can get anything pulse related with aplay -List, only the hardware device. So i switched to venv, but i get setup errors. Then switched to native debian package. Quickly installed the armhf version, started it with sudo rhasspy -p de. It starts up, just configured sound output to use pulse and nanotts as tts engine, tested with “hi”, only getting an error about a non-zero exit status of aplay. Btw, if used the commands arecord -f S16_LE -d 10 -r 44100 --device="hw:1,0" /tmp/test-mic.wav and aplay -q -t wav -D 'pulse' /tmp/test-mic.wav are working on the pi like a charm. In the logs i get this

Logs

[ERROR:2021-03-11 22:50:10,636] rhasspyserver_hermes: Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1700, in api_text_to_speech
results = await asyncio.gather(*aws)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1686, in speak
volume=volume,
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 614, in speak_sentence
raise AudioServerException(play_response.error)
rhasspyserver_hermes.AudioServerException: Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.
[DEBUG:2021-03-11 22:50:11,515] rhasspyserver_hermes: -> TtsSay(text=‘hi’, site_id=‘default’, lang=‘de-DE’, id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=’’, volume=1.0)
[DEBUG:2021-03-11 22:50:11,517] rhasspyserver_hermes: Publishing 130 bytes(s) to hermes/tts/say
[DEBUG:2021-03-11 22:50:11,525] rhasspytts_cli_hermes: <- TtsSay(text=‘hi’, site_id=‘default’, lang=‘de-DE’, id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=’’, volume=1.0)
[DEBUG:2021-03-11 22:50:11,529] rhasspytts_cli_hermes: [‘nanotts’, ‘-v’, ‘de-DE’, ‘-o’, ‘/tmp/tmpmjsmr7kt.wav’]
Using Lingware directory: /usr/lib/rhasspy/lib/nanotts/pico/lang
read: 2 bytes from stdin
using lang: de-DE
wrote “/tmp/tmpmjsmr7kt.wav” (15020 bytes)
[DEBUG:2021-03-11 22:50:11,610] rhasspytts_cli_hermes: Got 15020 byte(s) of WAV data
[DEBUG:2021-03-11 22:50:11,611] rhasspytts_cli_hermes: -> AudioPlayBytes(15020 byte(s)) to hermes/audioServer/default/playBytes/8cc70df8-1517-4cf5-8430-399964ca4b98
[DEBUG:2021-03-11 22:50:11,613] rhasspytts_cli_hermes: Waiting for play finished (timeout=0.718)
[DEBUG:2021-03-11 22:50:11,617] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/default/playBytes/8cc70df8-1517-4cf5-8430-399964ca4b98, id=1847f231-4701-45a3-93d2-e6a532b6d163)
[DEBUG:2021-03-11 22:50:11,617] rhasspyspeakers_cli_hermes: <- AudioPlayBytes(15020 byte(s))
[DEBUG:2021-03-11 22:50:11,618] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]
ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Access denied

aplay: main:828: audio open error: Connection refused
[ERROR:2021-03-11 22:50:11,680] rhasspyspeakers_cli_hermes: handle_play
Traceback (most recent call last):
File “/usr/lib/rhasspy/rhasspy-speakers-cli-hermes/rhasspyspeakers_cli_hermes/init.py”, line 80, in handle_play
subprocess.run(self.play_command, input=wav_bytes, check=True)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/subprocess.py”, line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.
[DEBUG:2021-03-11 22:50:11,684] rhasspyspeakers_cli_hermes: -> AudioPlayError(error=“Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.”, site_id=‘default’, context=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’)
[DEBUG:2021-03-11 22:50:11,685] rhasspyspeakers_cli_hermes: Publishing 224 bytes(s) to hermes/error/audioServer/play
[DEBUG:2021-03-11 22:50:11,689] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’)
[DEBUG:2021-03-11 22:50:11,689] rhasspyspeakers_cli_hermes: Publishing 99 bytes(s) to hermes/audioServer/default/playFinished
[DEBUG:2021-03-11 22:50:11,692] rhasspyserver_hermes: Handling AudioPlayError (topic=hermes/error/audioServer/play, id=1847f231-4701-45a3-93d2-e6a532b6d163)
[DEBUG:2021-03-11 22:50:11,696] rhasspytts_cli_hermes: <- AudioPlayFinished(id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’)
[DEBUG:2021-03-11 22:50:11,699] rhasspytts_cli_hermes: -> TtsSayFinished(site_id=‘default’, id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=’’)
[DEBUG:2021-03-11 22:50:11,700] rhasspytts_cli_hermes: Publishing 84 bytes(s) to hermes/tts/sayFinished
[DEBUG:2021-03-11 22:50:11,705] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=1847f231-4701-45a3-93d2-e6a532b6d163)
[ERROR:2021-03-11 22:50:11,712] rhasspyserver_hermes: AudioPlayError(error=“Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.”, site_id=‘default’, context=‘8cc70df8-1517-4cf5-8430-399964ca4b98’, session_id=‘8cc70df8-1517-4cf5-8430-399964ca4b98’)
[ERROR:2021-03-11 22:50:11,714] rhasspyserver_hermes: Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1700, in api_text_to_speech
results = await asyncio.gather(*aws)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1686, in speak
volume=volume,
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 614, in speak_sentence
raise AudioServerException(play_response.error)
rhasspyserver_hermes.AudioServerException: Command ‘[‘aplay’, ‘-q’, ‘-t’, ‘wav’, ‘-D’, ‘pulse’]’ returned non-zero exit status 1.

SO i guess rhasspy cant talk to pulseaudio because of the ALSA lib pulse.c:243:(pulse_connect) PulseAudio: Unable to connect: Access denied error. I cant check, if rhasspy native works when startet as a service, since i cant get it to work, because it cant connect to a mqtt broker. Even tho in the profile when startet manually ive set an external broker. So any clues what i can do?

tenn0 · March 13, 2021, 1:41am

Ok, just for testing ive installed raspberrypi os full, used native debian package and pulseaudio. Works.
But, sometimes the wakeword triggers a few times more and then tells me ten times my current temperature (one of the sentences). And still the issue with video streaming from the pi cam and rhasspy not reacting persists.
Just installed the OS, connected to wifi, activated camera and ssh, installed drivers for the adafruit speaker bonnet, updated everything, installed rhasspy per native install.
Maybe there are pulseaudio modules installed, which make this work and are missing on the minimal installation (which i prefer)? Or is my pulseaudio config a mess? After taking a nap im going to post the configs and loaded modules of both installations

tenn0 · March 14, 2021, 6:48pm

So, i dunno, what is the culprit, but ive managed to get rhasspy to work with pulseaudio. instead of starting it as sudo, im starting it as normal user pi. Even when I use the udp streaming of the pi cam, which port i changed to 9999, i can talk with rhasspy as intended.
But: Ive still not managed to get rhasspy as service working.

/etc/systemd/system/rhasspy.service

[Unit]
Description=Rhasspy
After=syslog.target network.target

[Service]
Type=simple
ExecStart=/usr/bin/rhasspy -p de
User=pi
Group=pi
UMask=0002

RestartSec=1
Restart=on-failure

StandardOutput=syslog
StandardError=syslog

SyslogIdentifier=rhasspy

[Install]
WantedBy=multi-user.target

With this service file, i can start rhasspy but still cant use any audio. When i want to refresh devices in audio playing, i just get timeouts. This is all logs i get:

Rhasspy logs

[ERROR:2021-03-14 18:47:07,112] rhasspyserver_hermes:
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 810, in api_speakers
speakers = await core.get_speakers()
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 858, in get_speakers
handle_finished(), messages, message_types
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 971, in publish_wait
result_awaitable, timeout=timeout_seconds
File “/usr/lib/rhasspy/usr/local/lib/python3.7/asyncio/tasks.py”, line 449, in wait_for
raise futures.TimeoutError()
concurrent.futures.base.TimeoutError
[ERROR:2021-03-14 18:46:59,653] rhasspyserver_hermes:
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 775, in api_microphones
microphones = await core.get_microphones()
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 825, in get_microphones
handle_finished(), messages, message_types
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 971, in publish_wait
result_awaitable, timeout=timeout_seconds
File “/usr/lib/rhasspy/usr/local/lib/python3.7/asyncio/tasks.py”, line 449, in wait_for
raise futures.TimeoutError()
concurrent.futures.base.TimeoutError
[ERROR:2021-03-14 18:46:59,640] rhasspyserver_hermes:
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 810, in api_speakers
speakers = await core.get_speakers()
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 858, in get_speakers
handle_finished(), messages, message_types
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 971, in publish_wait
result_awaitable, timeout=timeout_seconds
File “/usr/lib/rhasspy/usr/local/lib/python3.7/asyncio/tasks.py”, line 449, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2021-03-14 18:46:37,105] rhasspyserver_hermes: Publishing 98 bytes(s) to rhasspy/audioServer/getDevices
[DEBUG:2021-03-14 18:46:37,104] rhasspyserver_hermes: -> AudioGetDevices(modes=[<AudioDeviceMode.OUTPUT: ‘output’>], site_id=‘pi’, id=‘dbe19127-f2ec-4b61-a048-880181e32d28’, test=False)

Any clue whats missing/broken/whatever?

daywalker1180 · March 15, 2021, 11:02pm

Hey tenn0,

shouldn’t your rhasspy service start after the PulseAudio service…?
To be honest I have no clue either, but that came immediately to my mind, maybe worth a try…

You should have a look in the unit files for systemd for PulseAudio and place
that in the [Unit] section at the line After=…

HTH
BR from Austria
Martin

tenn0 · March 16, 2021, 3:54am

Currently, ive started these service manually, after im sure PA is running