Playing sound at the same time from Rhasspy (Docker) and the host

Thank you again. Well so far I am using Rhasspy http API for integration. Is there any other benefit using MQTT for client/server integration?

Not 100% sure, but from what I can tell, Rhasspy’s websocket API isn’t as robust as MQTT to be able to get in and hack something together.

Because of what rhasspy is, a ai voice controlled hmi it does make sense to use MQTT

http://mqtt.org/

MQTT is a machine-to-machine (M2M)/“Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. For example, it has been used in sensors communicating to a broker via satellite link, over occasional dial-up connections with healthcare providers, and in a range of home automation and small device scenarios. It is also ideal for mobile applications because of its small size, low power usage, minimised data packets, and efficient distribution of information to one or many receivers

Has some well defined core building blocks and its prob wise to continue to use MQTT due to it likely being the default machine-to-machine (M2M)/“Internet of Things” connectivity protocol of practically everything.

If https://docs.snips.ai/reference/hermes should be implemented or used as a guide to the base use of opensource MQTT I guess needs some clarrification.
As to the Hermes audio transport I am beginning to think audio might be a bad format when there are so many advantages in sending and processing Mel Filter (MFCC) Spectrograms and audio distribution via Snapcast which is really great for that purpose.
But again Hermes audio has already been defined, but passing wavs is anything but lightweight.

MQTT is prob a really good idea to follow.

Thank you both for reply. I know MQTT as I use it for Homeassistant integrations, I did even wrote few integrations for my home bred Raspberry Pi devices. My question was more like if there are any drawbacks (for example speed, resources consumption, etc of MQTT vs HTTP)

I checked on Rhasspy 2.5 (i.e. Voltron) and I saw that MQTT seems to be primary way how all component are integrated. But right now, as I use Homeassistant conversation integration for intent recognition (with Almond), I can’t use Rhasspy 2.5 (as by the docs this integration is not yet complete). So I will wait until I will be able to test it.

Than it should be no problem to write MQTT client which will subscribe for Hermes topics and could do things like snapcast client muting, or Respeaker Array LED control. :slight_smile:

1 Like

I really like the look of snapcast and how it does latency handling.
It also has codec support that can reduce bandwidth requirements substantially.
Maybe should also have an option for a lightweight codec like the ones bluetooth uses.
But it looks awesome but do you guys know if you can just stream single channels to devices or do you split them and create multiple streams?
If your setting up and playing with snapcast create a thread and let us know what and how your doing as my only criticism of snapchat is the documentation.

But I am interested in your setup, devices, speakers, AI and what you do.

I found that MQTT seemed a bit faster for me with my setup.

I use Rhasspy 2.5-pre with Home Assistant intents and it’s working great. See this other thread on how I’ve done it.

Thank you for observation.

I would preffer to keep my HA integration with Almond, which is conversation integration - which seems to be not yet finished in Rhasspy 2.5. This is actually free speach integration… You do not need to create intents at all and work supringly OK.

Thanks a lot for this thread. This helped me a lot. I managed to play sound from Rhasspy (Docker) and Squeezelite (Host) at the same time only by you !

1 Like

Hi,

maybe someone here can help me.
I have rhasspy (docker) and mopidy (host) running.
Once mopidy was on rhasspy can’t make a sound.
I have tried to take over my asound.conf in the container.
Unfortunately without success

The command “aplay /usr/share/sounds/alsa/Front_Center.wav” works on the host but not in the container

Error

ALSA lib pcm_dmix.c:1108:(snd_pcm_dmix_open) unable to open slave
aplay: main:828: audio open error: Device or resource busy

asound.conf

# use samplerate to resample as speexdsp resample is bad
defaults.pcm.rate_converter "samplerate"

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

pcm.playback {
    type plug
    slave.pcm "dmixed"
}

pcm.capture {
    type plug
    slave.pcm "array"
}

pcm.dmixed {
    type dmix
    ipc_perm 0666
    slave.pcm "hw:seeed2micvoicec"
    ipc_key 555555
}

pcm.array {
    type dsnoop
    slave {
        pcm "hw:seeed2micvoicec"
        channels 2
    }
    ipc_key 666666
}

Haaa, I was solving same issue. And with help of others I solved it. As far as I remember I needed to do several things:

  1. Create two playback devices in alsa asound.conf and dmix them to given HW card (one is for Rhasspy playback TTS and second for music playback - in my case snapcast client)
  2. Share same asound.conf between host and Rhasspy Docker container
  3. Share host IPC Rhasspy Docker container

asound.conf (with independent sof volume control for both playback channels)

pcm.mix {
    type dmix
    ipc_key 1024
    ipc_perm 0666
    slave {
        pcm "hw:0,0"
        period_time 0
        period_size 1024
        buffer_size 4096
        rate 48000
        format S24_3LE
    }
    bindings {
        0 0
        1 1
    }
}


pcm.snapcast {
    type plug
    slave.pcm "snapcast_vol"
}


pcm.snapcast_vol {
    type softvol
    slave.pcm "mix"
    control {
        name "Snapcast"
        card 0
    }
}


pcm.rhasspy {
    type plug
    slave.pcm "rhasspy_vol"
}


pcm.rhasspy_vol {
    type softvol
    slave.pcm "mix"
    control {
        name "Rhasspy"
        card 0
    }
}


pcm.snoop {
    type dsnoop
    ipc_key 2048
    ipc_perm 0666
    slave {
        pcm "hw:0,0"
        channels 1
        period_size 1024
        buffer_size 4096
        rate 48000
        format S16_LE
    }
    bindings {
        0 0
    }
}


pcm.mic {
    type plug
    slave.pcm "mic_vol"
}


pcm.mic_vol {
    type softvol
    slave.pcm "snoop"
    control {
      name "Mic"
      card 0
    }
    min_dB -10.0
    max_dB 30.0
}

docker run (with both 2. + 3. requirements)

docker run -d -p 12101:12101
           --name rhasspy \
           --restart unless-stopped \
           -v "$HOME/.config/rhasspy/profiles:/profiles" \
           -v "/etc/localtime:/etc/localtime:ro" \
           -v "/etc/asound.conf:/etc/asound.conf" \
           --device /dev/snd:/dev/snd \
           --ipc="host" \
           rhasspy/rhasspy --user-profiles /profiles  --profile en

2 Likes

Thank you.
With this asound.conf and docker command it works now.
But explain me something.
Wich parameter in your docker command makes it work?
It dosen’t work with mine.

docker run -it 
    --name rhasspy \
    --restart always \
    -v "$HOME/.config/rhasspy/satellite:/profiles"  \
    -v "/etc/asound.conf:/etc/asound.conf" \
    --network host \
    --device /dev/snd \
    rhasspy/rhasspy --profile de --user-profiles /profiles --local-mqtt-port 13183

The one missing :slight_smile:

 --ipc="host" \

Hello guys, thank you for all the hints here!

I’ve quite achieve my goal on my laptop: playing audio from Rhasspy docker and my base OS at the same time.

I’m saying quite because if I use rhasspy TTS (based on my “rhasspy” audio device) or other audio output flows from Rhasspy while using “speaker-test -D host” from a terminal on my computer (my “host” audio device ), the audio is mixed and played correctly (TTS sentence mixed with pink noise).

But if I’m using my computer normally while rhasspy is saying something through the TTS system then I got a “resource busy” (eg: while rhasspy is talking, if i’m opening a youtube video, this got stuck with loading loader icon)

I’m on Ubuntu 18.04 and this is my .asoundrc file (or asound.conf).

Maybe I’m just missing some basic step to do on my host or maybe I need to use the PulseAudio features as described here https://github.com/TheBiggerGuy/docker-pulseaudio-example ?

pcm.!default {
    type plug
    slave.pcm "host_vol"
}


pcm.host_vol {
    type softvol
    slave.pcm "mix"
    control {
        name "default"
        card 0
    }
}


pcm.rhasspy {
    type plug
    slave.pcm "rhasspy_vol"
}


pcm.rhasspy_vol {
    type softvol
    slave.pcm "mix"
    control {
        name "Rhasspy"
        card 0
    }
}

pcm.mix {
    type dmix
    ipc_key 1024
    ipc_perm 0666
    slave {
        pcm "hw:0,0"
        period_time 0
        period_size 1024
        buffer_size 4096
        rate 48000
        format S24_3LE
    }
    bindings {
        0 0
        1 1
    }
}

ctl.!default {
        type hw           
        card 0
}

Notice:

It took me too long to figure out, what was the problem and I couldn’t have rhasspy read my “virtual channels”.

So if you have rhasspy with docker-compose running and you have already added the /etc/asound.conf binded that in docker-compose.yml added also the line ipc: host then you should delete the volumes from container with:

docker-compose down -v

And then start the container.

docker-compose up -d

Just to make sure nobody else tries only to restart the rhasspy container without deleting the volumes…

The above is almost correct

So the docker run will be

docker run -d -p 12101:12101 \
    --ipc=host \
    --name rhasspy \
    --restart unless-stopped \
    -v "$HOME/.config/rhasspy/profiles:/profiles" \
    -v "/etc/asound.conf:/etc/asound.conf:ro" \
    -v "/etc/localtime:/etc/localtime:ro" \
    --device /dev/snd:/dev/snd \
    rhasspy/rhasspy \
    --user-profiles /profiles \
    --profile en

So you will have to stop the container rm to remove then docker run again
The /etc/asound.conf for the 2mic is as so

# The IPC key of dmix or dsnoop plugin must be unique
# If 555555 or 666666 is used by other processes, use another one


# use samplerate to resample as speexdsp resample is bad
defaults.pcm.rate_converter "samplerate"

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

pcm.playback {
    type plug
    slave.pcm "dmixed"
}

pcm.capture {
    type plug
    slave.pcm "array"
}

pcm.dmixed {
    type dmix
    slave.pcm "hw:seeed2micvoicec"
    ipc_key 555555
    ipc_perm 0666
}

pcm.array {
    type dsnoop
    slave {
        pcm "hw:seeed2micvoicec"
        channels 2
    }
    ipc_key 666666
    ipc_perm 0666
}

The respeaker install for some reason misses the ipc_perm 0666 and without permissions you will get an ipc error.

So to recap you need to add --ipc=“host” which uses the ipc of the host so also share a copy of /etc/asound.conf in the docker run as -v "/etc/asound.conf:/etc/asound.conf:ro" \ that way the same IPC and settings are on the host and in the container and rhasspy should be set to use default as the /etc/asound.conf sets that.

If on the cli I run speaker-test I get that horrid pink noise test and if I sudo docker exec -it rhasspy /bin/bash and run speaker-test in the container I get double the horrid pink noise test, running at the same time but mixed.

I added --ipc="host" to my docker command and added ipc_perm 0666 to pcm.array in asound.conf

I made several docker stop, rm, runs and some reboots.
Seems like now rhasspy only play sound if i had not yet used squeezelite, after start/stop squeezelite rhasspy dont play sound anymore.

Before changing rhasspy played sound after stopping the player
I remember the above behavior from the past, dont know how/when it changed to be usable after player stop.

I hear nothing from speaker-test, probably because it uses the other pi soundcard rather the respeaker one.

speaker-test -Ddmix:seeed2micvoicec -c2
gives a device busy error, after restart there is an error about 48000 Hz not supported.

docker run -d -p 12101:12101 -p 12183:12183 
  --ipc="host" --name rhasspy --restart unless-stopped 
  -v "$HOME/.config/rhasspy/profiles:/profiles" 
  -v "/etc/localtime:/etc/localtime:ro" 
  -v "/etc/asound.conf:/etc/asound.conf" 
  --device /dev/snd:/dev/snd rhasspy/rhasspy 
  --user-profiles /profiles --profile de
# The IPC key of dmix or dsnoop plugin must be unique
# If 555555 or 666666 is used by other processes, use another one


# use samplerate to resample as speexdsp resample is bad
defaults.pcm.rate_converter "samplerate"

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

pcm.playback {
    type plug
    slave.pcm "dmixed"
}

pcm.capture {
    type plug
    slave.pcm "array"
}

pcm.dmixed {
    type dmix
    ipc_perm 0666       # Andere Benutzer können ebenfalls dmix gleichzeitig nutzen
    slave.pcm "hw:seeed2micvoicec"
    ipc_key 555555.
    ipc_key_add_uid false
}

pcm.array {
    type dsnoop
    slave {
        pcm "hw:seeed2micvoicec"
        channels 2
    }
    ipc_key 666666
    ipc_perm 0666
}
$ aplay -l
card 0: Headphones [bcm2835 Headphones], device 0: bcm2835 Headphones [bcm2835 Headphones]
  Subdevices: 8/8
  ...

card 1: seeed2micvoicec [seeed-2mic-voicecard], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Did you delete the container and do the docker run again as the speaker-test not giving an output means /etc/asound.conf must not be active as if it was the default would be set.

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

I have never tried speaker-test -Ddmix:seeed2micvoicec -c2 but it would never use the IPC of -Ddmixed and on its own like that has no IPC set, so don’t think that would work anyway as a dmix and if it does it would be the same as hw: or plug: due to no IPC

At a guess somewhere in the config be it squeezelite, rhasspy web config likely there is something not using the default and has a hw: or plughw: that is likely locking others out maybe.

I haven’t got that image up and running now and also never tried squeezelite but thinking how I have it, it will prob work.
I might get round to setting it up with squeezelite later or tomorrow

I deleted the container several times.
I dont know how to do testing with a non-default device, i found the speaker-test command in my shell history.
How should i do the test?

Pi sound card was not deactivated, maybe i need them later (if i find a small speaker not needing extra power).

Btw, i have squeezelite on the host, not in a docker.
I think ifollowed http://www.gerrelt.nl/RaspberryPi/wordpress/tutorial-installing-squeezelite-player-on-raspbian/#Starting_Squeezelite_on_startup
SL_SOUNDCARD is commented out in config and there seems no changed value, most comments. Squeezlite uses the seeed card.

$ squeezelite -l
Output devices:
  null                           - Discard all samples (playback) or generate zero samples (capture)
  jack                           - JACK Audio Connection Kit
  pulse                          - PulseAudio Sound Server
  playback                      
  capture                       
  dmixed                        
  array                         
  default                       
  output                        
  sysdefault:CARD=Headphones     - bcm2835 Headphones, bcm2835 Headphones - Default Audio Device
  dmix:CARD=Headphones,DEV=0     - bcm2835 Headphones, bcm2835 Headphones - Direct sample mixing device
  dsnoop:CARD=Headphones,DEV=0   - bcm2835 Headphones, bcm2835 Headphones - Direct sample snooping device
  hw:CARD=Headphones,DEV=0       - bcm2835 Headphones, bcm2835 Headphones - Direct hardware device without any conversions
  plughw:CARD=Headphones,DEV=0   - bcm2835 Headphones, bcm2835 Headphones - Hardware device with all software conversions
  usbstream:CARD=Headphones      - bcm2835 Headphones - USB Stream Output
  sysdefault:CARD=seeed2micvoicec - seeed-2mic-voicecard, bcm2835-i2s-wm8960-hifi wm8960-hifi-0 - Default Audio Device
  dmix:CARD=seeed2micvoicec,DEV=0 - seeed-2mic-voicecard, bcm2835-i2s-wm8960-hifi wm8960-hifi-0 - Direct sample mixing device
  dsnoop:CARD=seeed2micvoicec,DEV=0 - seeed-2mic-voicecard, bcm2835-i2s-wm8960-hifi wm8960-hifi-0 - Direct sample snooping device
  hw:CARD=seeed2micvoicec,DEV=0  - seeed-2mic-voicecard, bcm2835-i2s-wm8960-hifi wm8960-hifi-0 - Direct hardware device without any conversions
  plughw:CARD=seeed2micvoicec,DEV=0 - seeed-2mic-voicecard, bcm2835-i2s-wm8960-hifi wm8960-hifi-0 - Hardware device with all software conversions
  usbstream:CARD=seeed2micvoicec - seeed-2mic-voicecard - USB Stream Output

Dunno as never use squeezelite apt-get install squeezelite really but just setting it up now which I have but didn’t read any setup lols, I never read the instructions!

I just installed squeezelite and then went off at a tangent as it was failing but that was because asound2-plugins was not installed but presumed everything else 1st.
I haven’t installed rhasspy yet as think my OC Pi02W was dying as also to make things worse kept crashing with kernel messages so set to default speed but it seemed to be the network and confused

So with LMS on my PC I install squeezelite and do nothing more
Play some music in the web control and it plays
Play speaker-test or aplay file_example_WAV_2MG.wav Sample .wav download | File Examples Download

Rhasspy just installed

docker exec -it rhasspy /bin/bash

speaker-test and again its all mixed with squeezelite playing joy division merilly

your mum is fat speaks away via TTS or anything you can think of, whilst still playing joydivision

No problem so maybe that tutorial as I just installed.

/etc/asound.conf

 # The IPC key of dmix or dsnoop plugin must be unique
# If 555555 or 666666 is used by other processes, use another one


# use samplerate to resample as speexdsp resample is bad
defaults.pcm.rate_converter "samplerate"

pcm.!default {
    type asym
    playback.pcm "playback"
    capture.pcm "capture"
}

pcm.playback {
    type plug
    slave.pcm "dmixed"
}

pcm.capture {
    type plug
    slave.pcm "array"
}

pcm.dmixed {
    type dmix
    slave.pcm "hw:Device"
    ipc_key 555555
    ipc_perm 0666
}

pcm.array {
    type dsnoop
    slave {
        pcm "hw:Device"
        channels 2
    }
    ipc_key 666666
    ipc_perm 0666
}

Its a plugable usb device but has spaces in its naming so you just get the last word from /proc/asound/cards

I am thinking my pref for snapcast is going to stay snapcast, I am presuming groups and stuff can be added and setup, but all seems an after thought.
LMS is a media server whilst snapcast is network virtual audio cable and doesn’t have a media player that is from the go organised into rooms/zones with corresponding inputs for near anything and you use whatever media player or in/out you wish, whilst LMS is very concentric to the LMS media server and isn’t as simple for multiroom design purity that snapcast has.
Pretty sure you can add plugins to do the same with LMS but snapcast is setup from the get go and likely offers more in/out methods and has audiophile quality network latency compensation.

I think i need to do the test, but dont know how.
Can you help me with this? I posted the squeezebox devices so you can see my hardware.

Beside LMS i dont know other music systems. I tried volumio which does not work out of the box and then found LMS, which worked fine + i found a used squeezebox radio. I am in no way audiophile, but i like to have the same music in 2 rooms.