Streaming Audio via Browser from Tablet/Phone/Desktop Mic

Help
1

We’ve had a couple of threads going about this topic but I wanted to share progress to date, a few caveats, and then I have some questions (because I’m now more familiar with streaming audio from a web browser than I am with Rhasspy!)

Here’s a shockingly small amount of JS that works really well. Only tested in Chrome and on a Chromebook tablet:

const webSocket = new WebSocket('ws://192.168.1.200:1880/panel/audio');
webSocket.binaryType = 'blob';

webSocket.onopen = event => {
    console.log("[open] Audio Node-RED websocket connection established");
    navigator.mediaDevices
        .getUserMedia({ audio: true, video: false })
        .then(stream => {
            const mediaRecorder = new MediaRecorder(stream, {
                mimeType: 'audio/webm;codecs=pcm',
            });
            mediaRecorder.addEventListener('dataavailable', event => {
                if (event.data.size > 0) {
                    webSocket.send(event.data);
                }
            });
            mediaRecorder.start(1000);
        });
};

My websocket is pointing to Node-RED right now as a simple way to confirm data was flowing.

Now the question is this: How do I string this up so it feeds my audio stream directly to Rhasspy? Will Rhasspy need a new way to accept this kind of “remote mic” or is it best to use one of the existing methods outlined here?

Right now, I’m not 100% sure what the bitrate or sample rate is but I do know that I’m sending chunks based not on size but on time. That last bit of code says to stream the recorded audio every 1,000 milliseconds and I can’t find a straightforward way of sending chunks any other way.

1 Like
2

You could play it to the source side of a loopback and then the other side of the loopback would be a sink.
You can set the resample rate in the alsa asound.conf …

3

I did pretty much exactly what you describe here last week. Unfortunately I can’t share my code with you at the moment but I’m sure we can solve it together. I built a frontend with Uibuilder in Node Red and streamed microphone to Node Red -> MQTT -> Rhasspy. I also streamed the Rhasspy output again via MQTT to Node Red and then to Uibuilder and was able to use Rhasspy completely via the browser. I simply used uibuilder send and sent the data out every few milliseconds. In the background I just used SocketIO.
I think the most interesting part ist:

this.recordAudioStream = RecordRTC(stream, {
type: ‘audio’,
mimeType: ‘audio/webm’,
sampleRate: 44100,
desiredSampRate: 22505,

                recorderType: StereoAudioRecorder,
                numberOfAudioChannels: 1,
                bitrate: 16000,
    
    
                timeSlice: 1,
    
                ondataavailable: function(blob) {
    
                    uibuilder.send( {
                                'topic': "micin",
                                'payload': blob  
                                })
                        

                }

I get wav data and can then feed the output directly to a an mqtt node in Node red. If you use the right topic in the mqtt node and choose mqtt as audio input in rhasspy. It should work out of the box. I think you can’t do that with your current snippet yet, because the WAV header should be missing, right?