Stream music or radio

Help
1

Hi @all,

I’m pretty fresh with rhasspy. Yet I’ve got everything running and accomplished to stream a single wav sound. Is there a way to stream music? Like using an url from a radio station or from spotify.

I’ve thought about creating a spotify skill using their API to search for songs.

Thanks for any hints

EDIT: I’ve seen the answer of synesthesiam: Github Issue concerning stream continuously

I don’t get yet some basics:

  1. Do I need to convert the http stream to wav to output it using rhasspy?
  2. Will rhasspy still listen for his wakeword so I can stop playing music by saying “stop music” for example?
  3. Is he reffering to the audio input section in his answer? I guessed the audio output section is fitting for this use case. Am I wrong?

Maybe the easiest way is to use something like this: mopidy
Has anyone experiences?

2

Hi,
If there is noise (music) rhasspy will listen to the microphone poorly.
I use gesture recognition with SparkFun APDS9960, esp8266 (arduino) and mqtt, Placed on the table, simple and cheap
If I approach: I cut the sound I can speech :smile:
up: I increase the sound
down: I decrease the sound
left: wakeword
For music (radio, deezer …), I create a thread, with function stop, start …
Everything is possible.

3

Rhasspy cannot stream audio OUT from an url, it only streams audio OUT which is generated by the system.
That is, generated speech or the feedback sounds.

From the answer:
1: Rhasspy does not output any INPUT audio.
2: See (1)
3: Yes, for both questions :wink:

I use mopidy, work wells but no interaction with Rhasspy in my case (yet)
In my case, it is combined with a snapcast server and several clients for multiroom audio :slight_smile:

1 Like
4

Ok thanks so far guys. As soon as kaldi is working I’ll start experimenting.

5

I’ve got everything set up now. So I can start a radio stream using rhasspy.

@romkabouter how are you solving the issue, that mopidy can’t use the same audio output as rhasspy? Atm if I start my radio mopidy just tells me in his logs that:

ERROR 2020-04-12 16:44:30,229 [16810:MainThread] mopidy.audio.gst GStreamer error: Could not open audio device for playback. Device is being used by another application.

I know that rhasspy is blocking this but actually I would like to play the radio stream over the same speakers as rhasspy is talking to me.

Any hints on how to solve this?

6

I did not, but I think snapcast can be the answer.

7

Would you mind explaining what shall be change in case of using snapcast?

Btw do you mean this: Snapcast?

8

Yes, I meant snapcast :slight_smile:

Install a server somewhere (the same device as you have installed mopidy), Mopidy can pipe the output to the server.
You can use all sorts of devices as clients (multiroom audio).

Then, you can configure Rhasspy to output to MQTT. The part missing is a connection for listening to the MQTT stream and outputting it to to snapserver.
I do not know if something like that is already available.

1 Like
9

Thanks. I’ll look into it the next days.

10

I will try to and do this as well :slight_smile:
I have got Mopidy and snapserver running on a VM on proxmox, but I have not tried to use that same snapserver with Rhasspy

11

I’ve got rhasspy to set a german radio station (swr3) and start playing it over mopidy. What’s your approach? Playing local mp3 files?

12

I only play spotify at the moment

13

@romkabouter how is your progress?

I’ve opened a forums post at the mopidy community: https://discourse.mopidy.com/t/gstreamer-error/2481

Is the piping from mopidy to snapserver working in your case?

I’ve got another question to you. Is the setup as follows?

Like you said in a post above --> Snapserver and mopidy

I have a pi4 which is running rhasspy atm as well as mopidy and a snapserver. It’s not clear for me whether I need a snapclient as well or not? I think I don’t need one, because acutally the better approach would be to connect to the websocket api of the snapserver and get the stream from there and pipe it into the mqtt stream topic of rhasspy. What do you think?

Yesterday I’ve ran an snapclient on the console and the stream worked well. Except for the errors with stopping and actually correctly starting the stream using iris in mopidy.

14

I am partime schoolteacher now, so free time is sparse.

Piping works fine, I have mopidy and snapserver on the same machine.
You only need snapclients on the devices you want to play audio.

piping the snapserver into the MQTT stream might be a good idea, but I dont think the MQTT is a continues stream of bytes, but rather a stream of a lot of small wave files.
I will see if I can free up some time to try this as well :slight_smile:

15

Would you mind showing me your config of mopidy?
How do you pipe it atm and which mopidy version and snapcast are you using?

16

Ok, lets consider another setup (the one I am trying to do)

my server:
- mopidy player (from sources like youtube, spotiy etc)  -> /tmp/snapfifo -> snapcast server -> stream -> snapcast clients

my Rpi client with Rhaaspy:   
- rhasspy            ->
                         dmixed alsa device -> sound card
- snapcast client    ->
17

This is the config file at /etc/mopidy/mopidy.conf. I run a VM with debian stretch.
I have installed mopidy and snapcast

[core]
cache_dir = /var/cache/mopidy
config_dir = /etc/mopidy
data_dir = /var/lib/mopidy

[logging]
config_file = /etc/mopidy/logging.conf
debug_file = /var/log/mopidy/mopidy-debug.log

[file]
enabled = false

[local]
enabled = false
media_dir = /var/lib/mopidy/media

[m3u]
enabled = false
playlists_dir = /var/lib/mopidy/playlists

[http]
hostname = ::

[mpd]
enabled = true
hostname = ::

[spotify]
client_id = id
client_secret = secret
username = user
password = pass

[audio]
output = audioresample ! audio/x-raw,rate=48000,channels=2,format=S16LE ! audioconvert ! wavenc ! filesink location=/tmp/snapfifo

[lastfm]
enabled = false

The [audio] part is the one that pipe the output form mopidy to the /tmp/snapfifo, which is the snapserver.
I do not know if Mopidy can output to multiple channels, but if you want to have output on the machine Mopidy is running, you can use snapclient -h I think.
I do not use that right now.

My clients are a pi zero with a BT speaker and a pi3 with a speaker. The BT speaker is actually a ceiling lamp so I can play audio over that.

So what I need to do is find a way to make Rhasspy output to the pipe on the mopidy/snapserver device.
On thing I can think of is to use this little gem:
https://github.com/koenvervloesem/hermes-audio-server

I was developed to subscribe to a MQTT topic and output the audio.
It is probably not complete, but the idea is to create a piece of code subscribed to your Rhasspy MQTT out topic and pipe that stream to you snapserver.
This is 1 idea however, so there might be already some code which does it :slight_smile:

The idea of @litinoveweedle will work, but implies Rhasspy running on a different device.
One also being able to output audio. This is a problem if your Rhasspy is on a NUC or some other server. I currently have Rhasspy running as addon in Home Assistant

18

Thanks for your fast reply.

I see. Basically with hermes-audio-server you would run wakeword as well as actually all communication with rhasspy over this?

Advantage would be that rhasspy isn’t blocking the speaker in- and output all the time isn’t it?

19

Well, you might only have to run the hermes-audio-player to grab the MQTT stream from Rhasspy and pipe it to snapserver.
I do not think that software is currently capable, right @koan?

20

Well, I only created hermes-audio-server because Rhasspy was a big monolith then and I wanted a light piece of software to only process audio on a satellite. To be honest, I consider hermes-audio-server deprecated now that Rhasspy is modularized.

I don’t have any recent experience with snapcast or mopidy, so I’m not sure how you would use hermes-audio-server with it.

21

There is always going to be some need for an audio-service but to be honest I am the same as @koan as have never ran snapcast or mopidy.

I am sort of curious to how this is handled in server/satelite situations as being a Rhasspy noob not sure how its handled or what future plans are.

Each satelite has a microphone and vad to stop streaming on no voice and on multiple satelites obviously makes huge savings in bandwidth.
You might have a 5:1 or 7:1 or whatever mutichannel audio is in flavour, as that also makes a wide area mic array, so that can be a lot of bandwidth to save.
So presuming there will always be an audio-service even if just input.

You might still have 2 tier recognition with an authoritative server or a satelite model.
So even though its been modularized doesn’t the need reapear because it can now be partitioned into tiers?

My head is spinning a bit because with media its likely to be streaming channels with at least a single stream channel accompanied by a room echo channel + mic channel.
If you have a satelite mode then audio duck/cork initially is processed locally and continues duck/cork until server authorisation or releases to normal.
Thats a local audio process isn’t it and Rasspy process? As thought I would ask as not sure of what snapcast or mopidity do or even have any function or awareness for the need of a room echo channel, or input.

I have had a brief look over snapcast which looks awesome and the default audio backend whilst apps like mopidity are just audio clients to push streams onto snapcast.
Can you stream channels to and from snapcast or does each client pull the whole stream and then can be assigned a channel?

22

Thanks, it looks like this can do the same:

When creating a program which can be used as PLAY_COMMAND and puts the output to the /tmp/pipefifo
This also might be possible with sox as play_command, the rhasspy-speakers-cli-hermes runs the play_command in a subprocess. You could use aplay for example.

23

Thanks for the link.

I’ve opened an issue for the make docker command which isn’t working.
Anyway I’ve installed it using make debian and it’s working so far.

So far so good. To be honest I haven’t completely understood how this plugin is working. I thought it will publish everything coming in from the play-command at the right topic. As a result I will hear the sound over the speaker which I configured inside rhasspy.

I have no clue whether it’s good what running following command is telling me or not. But there’s no sound coming from my speakers.

(.venv) pi@raspberrypi:~ $ rhasspy-speakers-cli-hermes --play-command "play /home/pi/Documents/testSong.mp3" --debug
[DEBUG:2020-04-22 14:15:20,587] rhasspyspeakers_cli_hermes: Namespace(debug=True, host='localhost', list_command=None, log_format='[%(levelname)s:%(asctime)s] %(name)s: %(message)s', password=None, play_command='play /home/pi/Documents/testSong.mp3', port=1883, site_id=None, username=None)
[DEBUG:2020-04-22 14:15:20,589] asyncio: Using selector: EpollSelector
[DEBUG:2020-04-22 14:15:20,590] rhasspyspeakers_cli_hermes: Connecting to localhost:1883
[DEBUG:2020-04-22 14:15:20,600] asyncio: Using selector: EpollSelector
[DEBUG:2020-04-22 14:15:20,601] rhasspyspeakers_cli_hermes: Connected to MQTT broker
[DEBUG:2020-04-22 14:15:20,602] rhasspyspeakers_cli_hermes: Subscribed to hermes/audioServer/+/playBytes/#
[DEBUG:2020-04-22 14:15:20,603] rhasspyspeakers_cli_hermes: Subscribed to hermes/audioServer/toggleOff
[DEBUG:2020-04-22 14:15:20,603] rhasspyspeakers_cli_hermes: Subscribed to rhasspy/audioServer/getDevices
[DEBUG:2020-04-22 14:15:20,604] rhasspyspeakers_cli_hermes: Subscribed to hermes/audioServer/toggleOn

Afterwards I thought that maybe adding a siteId would solve it with no success.

(.venv) pi@raspberrypi:~ $ rhasspy-speakers-cli-hermes --play-command "play /home/pi/Documents/testSong.mp3" --debug --siteId "default"
usage: rhasspy-speakers-cli-hermes [-h] --play-command PLAY_COMMAND
                                   [--list-command LIST_COMMAND] [--host HOST]
                                   [--port PORT] [--username USERNAME]
                                   [--password PASSWORD] [--site-id SITE_ID]
                                   [--debug] [--log-format LOG_FORMAT]
rhasspy-speakers-cli-hermes: error: unrecognized arguments: --siteId default

I’ve configured rhasspy to use the external mosquitto broker which is running on the pi:
Screenshot 2020-04-22 at 14.26.03

Any hint what I am doing wrong?

24

I do not know yet, but I think the play command is what is needed to play the INCOMING mqqt stream to a “device”

For instance play_command could be aplay.
I think you have a misconception about the play command, be the looks of it, is the the process which is taken input from stdin (which is actually your mqtt stream) and then OUTPUT is to something.

In your case, try:

rhasspy-speakers-cli-hermes --play-command “aplay” and publish to audio to the playBytes topic.

Hopefully that will take the incoming audio and play that with the raspbian command “aplay”
You can also add parameter to aplay to play on different speaker, but without it, it takes the system default
You can try on your Pi to play a wav with:

aplay test.wav

Obivously you should have test.wav on the Pi as a file.

25

Well I have no problems at all using the play command of sox to play a song over command line.

Anyway how do you mean and publish to audio? As far as I understood that’s the part I need to do myself. rhasspy-speakers-cli-hermes isn’t capable of doing this or am I wrong?

26

I meant publish your audio to the playBytes topic.
The playBytes topic also gets the audio from your TextToSpeech, but you can publish wave files as well.

27

Huge huge thanks to @romkabouter. After hours of playing around with him it’s finally working.

Now it’s possible to hear rhasspy and music from mopidy at the same time.

For everyone who wants to reach the same here is my asound.conf located in /etc/asound.conf

asound.conf:

pcm.!default {
  type plug
  slave.pcm "ossmix"
}

pcm.ossmix {
    type dmix
    ipc_perm 0666
    ipc_key 1024 # must be unique!
    slave {
        pcm "hw:1,0"      # you cannot use a "plug" device here, darn.
        period_time 0
        period_size 1024 # must be power of 2
        buffer_size 8192  # dito. It
    }
    bindings {
        0 0 # from 0 => to 0
        1 1 # from 1 => to 1
    }
}

ctl.!default {
  type plug
}

Adjust the pcm “hw:1,0” line to your needs.

make sure the timer under /dev/snd has the correct rights. In case of errors run:

chmod a+wr /dev/snd/timer
enables every user to use aplay command

Next change the audio output inside rhasspy to mqtt/hermes and start rhasspy-speakers-cli-hermes using:

rhasspy-speakers-cli-hermes --play-command aplay

EDIT: for everyone who’s coming around this post! My setup is rhasspy 2.5 in docker on a pi4 with mopidy running as service on the pi itself.

3 Likes
28

Is that located inside the (Rhasspy?) docker container, or on the host?

29

on the host so in my case on the pi

30

In the end we had the following:

  • Mopidy playing music with config:

[audio]
output = alsasink device=default

  • Installed and running rhasspy-speakers-cli-hermes:
    https://github.com/rhasspy/rhasspy-speakers-cli-hermes
    The docker does not work, so build from source and run:
    rhasspy-speakers-cli-hermes --host 192.168.43.54 --username xxxx --password xxxx --site-id default --debug --play-command aplay

The host is the MQTT broker you have set Rhasspy to connect to, site-id is the id in the settings of Rhasspy

Both were working correct, Mopidy was playing through the speakers and rhasspy-speaker was also correctly playing.
The problem was mixing those, so rhasspy could play audio while mopidy was playing music.

Without modification, this would not would without modification because both need to use the audio device.
This can be done however with dmix type device

We have changed /etc/asound.conf on the Pi like Bozor indicated.
But there still was a permission error, which prevented mopdiy and the user pi (running rhasspy-speaker) to access the dmix device

We checjed the permissions with:

ls -la /dev/snd

Probably the output will be something like:

drwxr-xr-x 4 root root 180 Apr 24 14:53 .
drwxr-xr-x 16 root root 3860 Apr 24 14:54 …
drwxr-xr-x 2 root root 60 Apr 24 14:53 by-id
drwxr-xr-x 2 root root 60 Apr 24 14:53 by-path
crw-rw---- 1 root audio 116, 32 Apr 24 14:54 controlC1
crw-rw---- 1 root audio 116, 56 Apr 24 14:54 pcmC1D0c
crw-rw---- 1 root audio 116, 48 Apr 24 14:54 pcmC1D0p
crw-rw---- 1 root 18 116, 1 Apr 24 14:54 seq
crw-rw---- 1 root 18 116, 33 Apr 24 14:54 timer

The last line is the problem, the group audio does not have access to the timer.
Also, the user pi is normally not in the group audio (mopidy user is when following installation)
So, add pi to audio group with:

usermod pi -G audio

We gave permission to all users to /dev/snd/timer with:

chmod a+wr /dev/snd/timer

But I think only giving acces to the audio group would be better.

Bozo could now play music from Mopidy through the speakers and Rhasspy would also be able to play sound, mixed together!

Currently, I have taken it a step further and my output is not a speaker, but the snapcast server

2 Likes
31

Oh man. I’ve just rebooted today since my last session and the last step of our solution seems to be not persistent.

Any idea on how to fix this?

I had to run:

usermod pi -G audio

Afterwards everything worked again.

And a new funny fact is that docker seems to loose his permissions somehow.

32

I am following in here, but it seems over my head right now. I am trying to get away from my Echos. I am not getting MyCroft what I would like it to do, so I need a voice assistant that can play music (thru spotify etc…) and also communicate with HA and or NodeRed which it seems Rhasspy can do. I am worried about music streaming. Can you guys recap everything that you have done and is working well now? Thanks!!

33

To be honest, I’ve paused working actively on my Rhasspy atm. I plan to start again pretty soon.
Last time I had a lot of false positives with my wakeword so I thought waiting a bit more will solve the issue with a better wakeword engine coming up.

Sorry that I can’t help you atm.

Though happy to hear your progress.

34

Have you had a chance to look into this more? Seems like Rhasspy isnt setup to play music really.

35

Well I haven’t had spare time yet. Pretty busy on work ATM.
Did you have any success?

36

So I got it to stream music thru pulse audio and it wakes up fine actually. But the sound cracks. Alot on wifi and a little when connect with cable. Did your method with Mopidy crack?

37

I can’t remember any cracking during music or voice output.

38

The discussion above looks confusing. Is it pretty easy to try and set this up?

Thanks!
Joshua

39

Well it will take some time and probably some tweaks. But it’s not that hard if you have some basic Linux knowledge I would say.

40

Just wanted to share - this post led me down the road to success as well.
I wanted a combined system, and previously used custom handler in Rhasspy to play music.
The downside is that while playing, Rhasspy couldn’t wake… so I wanted to try and duplicate what you did here.

I have Rhasspy and Mopidy installed both in containers.
it took some learning to setup Mopidy correctly… but once I had that I was close.
I use Node-Red for automation - so I have accomplished this all by just doing automation…
Rhasspy controls Mopidy through Node-Red automation… and the wake work stops the music.

There were a few work arounds…

  1. Mopidy apparently can’t queue up from a playlist via it’s API… at least I couldn’t get it to work.
    To work around this, I pull the tracks form the playlist and then push them to the tracklist.add.

  2. Pausing Mopidy did not allow Rhasspy to use the speaker… had to stop playback.
    To fix this, I saved the current position in the song, and then seek it when I start playing again.

So far (1 day in) I am happy with what I have achieved! I’m sure I’ll find problems - but it has been an awesome adventure to replicate what I think is a complex setup from cloud type speakers. The last thing I haven’t finished is passing the single song to Mopidy… I the song name retrieval in Rhasspy already setup from previous work… just have get it passed to Mopidy.

Here is what it looks like in node red…

image
image1458×652 72.4 KB

Cheers! and thanks for the inspiration!
DeadEnd

2 Likes
41

I have a pi 3B+ with respeaker 2 and alsa, no pulseaudio.
Rhasspy (satellite) runs as docker, squeezelite is installed on host.
Speech comes from the base rhasspy.
I want to hear rhasspy output while squeezelite is playing music.
Can i use this? Do i need rhasspy-speakers-cli-hermes?

Is Squeezelite + Rhasspy - Poor audio quality more relevant for me?
Unfortunately i dont know much about alsa and hope @rolyan_trauts, @GregoryWilson101 or someone else can help me with the right config.

For Home Assistant is an integration available to play TTS using LogitechMediaServer. But it works with the TTS integration.
I did not found a way to use rhasspy as TTS for Home Assistant other then by Hermes/MQTT.

42

Hi!
Using the setup like i menioned in my other topic you linked, I was able to get both speach and music simultaenously. I then had to use an automation to mute the audio when the rhasspy wake word was detected. I was never able to get the rhasspy docker to work with squeezelite.

Unfortuantely, due to the music playing the wake word very rarely worked. (And i mean rarely, i felt like a fool shouting “SNOW BOY” over anything but the quietest music playing).

I pivoted to using a physical button connected to a GPIO to activate rhasspy. And then eventually stoppd using rhasspy and used physical buttons for common music commands and a tablet for the Home Assistant GUI. Rhasspy is only as good as the hardware out there and the respeaker devices a tried were not replacement for a physical interface.

43

Haven’t had rhasspy set up for a while guys and my memory is rather dim but I would hazard a guess it could be to do with a sampling rate problem where many setup using hardware direct as in hw:0 which often is asking for trouble as if plughw:0 was used then there is a layer that will automatically resample and convert.

Snowboy never tried as I am not really sure why commercial cast off’s past sell buy have been adopted apart from they where there but there is a reason its no longer commercial as it was an early KW system and now not very good.
Hardware wise a mic is a mic its the underlying software that we are absent of even though you can make much better noise tolerant KWS than you have.
Pico Porcupine is prob the best almost non commercial as they do give totally free models to go at its just not open source.

The respeaker is as good a mic as any as my bugbear is it doesn’t matter at all and any more than 1 mic is totally pointless if it is not beeing processed by hardware or software dsp but there is no link up with the rhasspy system anyway.
You have an open mic with a KWS that is not very noise tolerant and to rub it in you have extra mics for no purpose at all apart from adding slight phase distortion due to the distance apart.

Your missing a whole load of software that I have tried in vain to raise many times that it is missing and lacking from the framework for such a long time and haven’t got the skills myself as it needs a modicum of C to do real time DSP as python really sucks so bad its pretty pointless trying, but obviously we only have a single good programmer who was Michael but only python based creating this initial audio processing hole.

The audio out is prob just some config somewhere but exactly what and where I am slightly out of the loop as got to the stage you guys are and thought yeah I will keep with the Google Nest Audio for now.
I am back playing with tensorflow as there is definately more dataset avail now and a KWS is not hard to accomplish but the interests of others lies elsewhere.

44

You might be interested in my approach with Jaco-MusicMaster. The setup works with docker and you can play songs from local file system or via youtube. To improve voice understanding, the skill automatically turns down the volume after wakeword detections, similar as @Gregory_Wilson’s concept.
Even though the skill is mainly intended for Jaco, you might be able to reuse some setup steps for a Rhasspy application.