Rhasspy 2.5 Pre-Release

[synesthesiam]

Rhasspy 2.5 is finally ready to be tested! Hope everyone is safe and healthy.

This has been a massive effort, and I’d like to thank everyone for their input and testing (especially @KiboOst!) Thanks also to @koan for helping to standardize projects and their tests, and to @maxbachmann for his Github/Docker skills and the super fast rapidfuzz library which is now what’s used for “fuzzywuzzy”

The documentation is still hosted on Github for now, but I encourage you to use the “Documentation” link instead in Rhasspy’s web interface, which opens a local copy with the navigation sidebar.

Because this is a pre-release, expect bugs and unsupported services or edge cases. I’ve tried to get as much in as possible, but things will surely break as folks bang on it. For now, please report bugs to the voltron repository. I’d like to keep 2.4 and 2.5 separate until the official [1] release.

See the Getting Started Guide to, well, get started. I’d highly recommend backing up your 2.4 profile before using it with 2.5! With that, give it a go:

docker run -d -p 12101:12101 \
    --restart unless-stopped \
    -v "$HOME/.config/rhasspy/profiles:/profiles" \
    --device /dev/snd:/dev/snd \
    rhasspy/rhasspy:2.5.0-pre \
    --user-profiles /profiles \
    --profile en

Folks doing a master/satellite setup may be interested in the rhasspy-satellite project (e.g., for Pi Zeros).

Enjoy!

[1] I’m just a dude in his pajamas, so “official” is whatever we make it.

[katsaplias]

great news!
I was waiting for 2.5 release to get my hands on, but I’m using HA core and I’m wondering if you’re planning to release an add-on?

[Jorg_Baumann]

@synesthesiam
I’ve just installed this new release on a Raspi4 within docker with English-Profile.
Installation without problems, but when I tried to activate an external MQTT-broker with the non standard port 1891, there is no more reaction to wakeword (snowboy.umdl).
Also when I press the Wake-Up-Button, nothing happens (no beeps).

Here is a log, where I’ve just pressed the Wake-Up-Button.
I am not familar with interpreting this log, but for me, it seems, that after a timeout rhasspy runs in error.

[ERROR:2020-03-21 14:39:07,370] rhasspyserver_hermes: 
Traceback (most recent call last):
  File "/usr/lib/rhasspy-voltron/.venv/lib/python3.6/site-packages/quart/app.py", line 1471, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy-voltron/.venv/lib/python3.6/site-packages/quart/app.py", line 1519, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 750, in api_listen_for_command
    handle_captured(), messages, message_types
  File "/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 832, in publish_wait
    _, result = await asyncio.wait_for(result_awaitable, timeout_seconds)
  File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2020-03-21 14:38:44,783] rhasspyserver_hermes: Publishing 21 bytes(s) to hermes/asr/toggleOn
[DEBUG:2020-03-21 14:38:44,779] rhasspyserver_hermes: -> AsrToggleOn(siteId='default')
[DEBUG:2020-03-21 14:38:44,777] rhasspyserver_hermes: Publishing 21 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2020-03-21 14:38:44,773] rhasspyserver_hermes: -> HotwordToggleOn(siteId='default')
[DEBUG:2020-03-21 14:38:37,362] rhasspyserver_hermes: Publishing 142 bytes(s) to hermes/asr/startListening
[DEBUG:2020-03-21 14:38:37,358] rhasspyserver_hermes: -> AsrStartListening(siteId='default', sessionId='c6e96d32-95cd-4925-bfca-9176a94ad39d', stopOnSilence=True, sendAudioCaptured=True, wakewordId='')
[DEBUG:2020-03-21 14:38:37,356] rhasspyserver_hermes: Subscribed to hermes/error/asr
[DEBUG:2020-03-21 14:38:37,355] rhasspyserver_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2020-03-21 14:38:37,354] rhasspyserver_hermes: Waiting for transcription (sessionId=c6e96d32-95cd-4925-bfca-9176a94ad39d)

When I switch back to the internal MQTT-broker, everything works fine.

It’s still the same problem, which I described here:
Documentation-preview-for-2-5

FYI:
I have no problems with external MQTT-broker and Port 1891 on another Raspi4, on which Rhasspy Version 2.4.19 is installed.

[j3mu5]

Fantastic, thank you very much for the early Easter present.

I can’t wait - as soon as I get the chance later or in the next few days I will try version 2.5 and report any bugs I find. I’ve already seen in Github that Precise is included. I am excited!

[jrb5665]

Is there a way to install this in a virtual environment instead of Docker?

[hawkeye217]

Great work! I’m new to Rhasspy and so I’m jumping in with this prerelease. I had started with 2.4 and messed around for about 30 minutes until I found this thread.

I have a few Pi Zeros around with PS Eye mics connected - should I install the same rhasspy/rhasspy:2.5.0-pre docker image on each of them that I installed on the master/server or does the rhasspy-sattelite project have a different image?

[Martin_Maier]

Hi together, I hope I can help a little bit.
@jrb5665
you should take a look at the voltron repository (the link is in the topic of synestesiam). I didn’t try it but I guess it wll work.
@hawkeye217
I already installed the rhasspy-satellite venv (the link is in the topic of synestesiam) on two Pi’s (Zero and 3B+) and they work nearly perfect.

greetings Martin

[jrb5665]

Thanks @Martin_Maier I already tried cloning the repository but it doesn’t have the same scripts as the 2.4 versions and running a make just threw fatal errors. At the moment I am running the docker version in a VM but I have been running the 2.4 versions under an lxc container in ProxMox and would prefer something I can configure without docker.

I have been running the rhasspy-satellite for a few days and have much of it working but on the pi zero with a respeaker 2-mic I can get wake word working with snowboy but I haven’t managed to get it to do STT. The rest appears to work pretty well typed sentences on the test page are correctly converted to intents. I suspect my issue relates to noise through the respeaker but haven’t worked out how to configure it properly.

[jrb5665]

btw. Are you running the rhasspy-satellite installation from the command-line or as a service? I have managed the former but trying to run as a service have seen a few strange errors

[tobetobe]

Congrats! I haven’t installed 2.5 yet. But what I can see from the docs it really looks great and seems t be an enormous step forward. Excellent work!

Before I start installing, I have some few and simple questions. Currently I am running a master/satellite system based on 2.4.19 that works fine.

I have followed your hint and read the setup instructions for the satellite system:

Folks doing a master/satellite setup may be interested in the rhasspy-satellite project

I saw, that the installation is based on a python virtual environment. Since currently my installations (both master and satellite) in 2.4.19 are based on docker I would prefer docker for the 2.5 satellite as well. Is this still possible, or is the python venv mandatory? To ask my question more open: Are there any advantages to do the satellite installation with python and not with docker or is it up to the user’s preference?

My second question: I heard that in 2.5 Kaldi is not supported in the German profile. Is this true (at least at the time being)?

Thank you in advance.

Best
Thomas

[hawkeye217]

I saw, that the installation is based on a python virtual environment. Since currently my installations (both master and satellite) in 2.4.19 are based on docker I would prefer docker for the 2.5 satellite as well. Is this still possible, or is the python venv mandatory? To ask my question more open: Are there any advantages to do the satellite installation with python and not with docker or is it up to the user’s preference?

Wondering the exact same thing, @tobetobe. I’d prefer Docker on my satellites as well. @synesthesiam, what do you think? Thanks for your hard work!

[Martin_Maier]

@jrb5665
my satellites are a PiZero with respealer-2 and Pi3B+ with respeaker-4. Both running with a rhasspy-satellite as service without any major problem. Here is the content of my rhasspy.service

[Unit]
Description=Rhasspy
After=syslog.target network.target
[Service]
Type=simple
User=pi
Group=pi
WorkingDirectory=/home/pi/rhasspy-satellite
ExecStart=/bin/bash -lc ‘/home/pi/rhasspy-satellite/bin/rhasspy-satellite --profile de’
RestartSec=1
Restart=on-failure
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=rhasspy
[Install]
WantedBy=multi-user.target

To get the STT working it is important that you define the satellites within the master profile … here is mine

{
“dialogue”: {
“satellite_site_ids”: “192.168.178.33,192.168.178.36”,
“system”: “rhasspy”
},
“intent”: {
“satellite_site_ids”: “192.168.178.33,192.168.178.36”,
“system”: “fsticuffs”
},
“microphone”: {
“system”: “pyaudio”
},
“mqtt”: {
“enabled”: “true”,
“host”: “192.168.178.200”
},
“sounds”: {
“system”: “aplay”
},
“speech_to_text”: {
“satellite_site_ids”: “192.168.178.33,192.168.178.36”,
“system”: “kaldi”
},
“text_to_speech”: {
“satellite_site_ids”: “192.168.178.33,192.168.178.36”,
“system”: “espeak”
},
“rhasspy”: {
“listen_on_start”: false
}
}

The rhasspy voltron venv installation should work if you ignore the ‘make venv’ in the first part of the installation description.

Martin

[Martin_Maier]

Hi @tobetobe, I have installed the 2.5.0-pre (latest) and use Kaldi as STT an it works excellent with profile de.

[geoffrey]

How do you connect your satellite to the server? When I use an external MQTT broker (used for everything else I do with Home Assistant) I see in the server debug log

rhasspy-asr-kaldi-hermes: error: unrecognized arguments: --username rhasspy-server --username my_password

My MQTT password is thus shown after the second argument --username, the first argument rhasspy-server is the username of the external MQTT broker.

[Martin_Maier]

Hi @geoffrey, my main (master) server is an Ubuntu server 18.04.4. There I’ve installed mosquitto with default port without username and password, the new Rhasspy 2.5.0-pre and Openhab. Maybe this is the reason why it works for me.

Martin

[synesthesiam]

Thanks for pointing this out. I forgot to add those arguments! Fixing it now…

[jrb5665]

Thanks @Martin_Maier , That’s pretty much how mine is set up.
Is the respeaker-2 the 2 mic or the v2?

If it is the 2 mic, did you have to do anything in particular to get it working or tune it?
I have got wakeword working with a 2-mic but I think it may be picking up noise or something.
It looks like the wakeword triggers audio to be passed to the server for STT but it doesn’t detect when I have finished talking and just runs until it times out and then doesn;t return anything.

[Martin_Maier]

@jrb5665
I use the 2 mic version. I’ve used the hermesLedControl_v2.0.6 to install the drivers and the led control and checked the installation using arecord and aplay. Maybe you can take a look at your log. Here is a log of mine working:

[DEBUG:2020-03-23 09:38:23,211] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/asr/toggleOn
[DEBUG:2020-03-23 09:38:23,203] rhasspyserver_hermes: -> AsrToggleOn(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:23,185] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2020-03-23 09:38:23,177] rhasspyserver_hermes: -> HotwordToggleOn(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:23,147] rhasspyserver_hermes: Handling AudioPlayFinished (topic=hermes/audioServer/192.168.178.36/playFinished, id=0c3ea7ee-ec1e-4b36-bff9-6a1f8e94a7cc)
[DEBUG:2020-03-23 09:38:23,141] rhasspyserver_hermes: Handling AudioPlayFinished (topic=hermes/audioServer/192.168.178.36/playFinished, id=fd7454b7-88b7-47df-af15-8128a118a616)
[DEBUG:2020-03-23 09:38:23,137] rhasspyserver_hermes: Handling AudioPlayFinished (topic=hermes/audioServer/192.168.178.36/playFinished, id=e3a1e4e7-7866-4ddd-be3a-74c90d9b080a)
[DEBUG:2020-03-23 09:38:23,131] rhasspyserver_hermes: <- AudioPlayFinished(id=‘fb08bc67-dc29-4d22-bad5-6ea9769b76de’, sessionId=’’)
[DEBUG:2020-03-23 09:38:23,120] rhasspyserver_hermes: <- AudioPlayFinished(id=‘fb08bc67-dc29-4d22-bad5-6ea9769b76de’, sessionId=’’)
[DEBUG:2020-03-23 09:38:22,032] rhasspyserver_hermes: Sent 599 char(s) to websocket
[DEBUG:2020-03-23 09:38:21,999] rhasspyserver_hermes: <- NluIntent(input=‘schalte schreibtischlicht ein’, intent=Intent(intentName=‘SchalterIntent’, confidenceScore=1.0), slots=[Slot(entity=‘device’, slotName=‘device’, confidence=1, raw_value=‘schreibtischlicht’, value=‘schreibtischlicht’, range=SlotRange(start=8, end=25, raw_start=8, raw_end=25)), Slot(entity=‘state’, slotName=‘state’, confidence=1, raw_value=‘ein’, value=‘ein’, range=SlotRange(start=26, end=29, raw_start=26, raw_end=29))], id=’’, siteId=‘192.168.178.36’, sessionId=‘192.168.178.36-default-e76736fa-83ba-464e-af7e-099c8e4a6f28’, customData=’’, asrTokens=[‘schalte’, ‘schreibtischlicht’, ‘ein’], asrConfidence=1.0, wakewordId=‘default’, rawAsrTokens=[‘schalte’, ‘schreibtischlicht’, ‘ein’])
[DEBUG:2020-03-23 09:38:21,966] rhasspyserver_hermes: <- NluIntent(input=‘schalte schreibtischlicht ein’, intent=Intent(intentName=‘SchalterIntent’, confidenceScore=1.0), slots=[Slot(entity=‘device’, slotName=‘device’, confidence=1, raw_value=‘schreibtischlicht’, value=‘schreibtischlicht’, range=SlotRange(start=8, end=25, raw_start=8, raw_end=25)), Slot(entity=‘state’, slotName=‘state’, confidence=1, raw_value=‘ein’, value=‘ein’, range=SlotRange(start=26, end=29, raw_start=26, raw_end=29))], id=’’, siteId=‘192.168.178.36’, sessionId=‘192.168.178.36-default-e76736fa-83ba-464e-af7e-099c8e4a6f28’, customData=’’, asrTokens=[‘schalte’, ‘schreibtischlicht’, ‘ein’], asrConfidence=1.0, wakewordId=‘default’, rawAsrTokens=[‘schalte’, ‘schreibtischlicht’, ‘ein’])
[DEBUG:2020-03-23 09:38:21,892] rhasspyserver_hermes: -> AudioPlayBytes(95312 byte(s))
[DEBUG:2020-03-23 09:38:21,884] rhasspyserver_hermes: Subscribed to hermes/audioServer/192.168.178.36/playFinished
[DEBUG:2020-03-23 09:38:21,873] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/asr/toggleOff
[DEBUG:2020-03-23 09:38:21,866] rhasspyserver_hermes: -> AsrToggleOff(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:21,853] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/hotword/toggleOff
[DEBUG:2020-03-23 09:38:21,839] rhasspyserver_hermes: -> HotwordToggleOff(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:21,830] rhasspyserver_hermes: Playing WAV /home/pi/rhasspy-satellite/etc/wav/beep_lo.wav
[DEBUG:2020-03-23 09:38:21,798] rhasspyserver_hermes: <- AsrTextCaptured(text=‘schalte schreibtischlicht ein’, likelihood=1, seconds=4.663509100999988, siteId=‘192.168.178.36’, sessionId=‘192.168.178.36-default-e76736fa-83ba-464e-af7e-099c8e4a6f28’, wakewordId=’’)
[DEBUG:2020-03-23 09:38:21,778] rhasspyserver_hermes: <- AsrTextCaptured(text=‘schalte schreibtischlicht ein’, likelihood=1, seconds=4.663509100999988, siteId=‘192.168.178.36’, sessionId=‘192.168.178.36-default-e76736fa-83ba-464e-af7e-099c8e4a6f28’, wakewordId=’’)
… here I am speaking …
[DEBUG:2020-03-23 09:38:17,278] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/asr/toggleOn
[DEBUG:2020-03-23 09:38:17,270] rhasspyserver_hermes: -> AsrToggleOn(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:17,259] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2020-03-23 09:38:17,253] rhasspyserver_hermes: -> HotwordToggleOn(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:17,220] rhasspyserver_hermes: Handling AudioPlayFinished (topic=hermes/audioServer/192.168.178.36/playFinished, id=3df65b13-9cad-4f99-9b60-c53b955ea535)
[DEBUG:2020-03-23 09:38:17,212] rhasspyserver_hermes: Handling AudioPlayFinished (topic=hermes/audioServer/192.168.178.36/playFinished, id=fd7454b7-88b7-47df-af15-8128a118a616)
[DEBUG:2020-03-23 09:38:17,202] rhasspyserver_hermes: Handling AudioPlayFinished (topic=hermes/audioServer/192.168.178.36/playFinished, id=e3a1e4e7-7866-4ddd-be3a-74c90d9b080a)
[DEBUG:2020-03-23 09:38:17,196] rhasspyserver_hermes: <- AudioPlayFinished(id=‘ce2ddbb1-641c-42d5-8d96-11c7330f505b’, sessionId=’’)
[DEBUG:2020-03-23 09:38:17,178] rhasspyserver_hermes: <- AudioPlayFinished(id=‘ce2ddbb1-641c-42d5-8d96-11c7330f505b’, sessionId=’’)
[DEBUG:2020-03-23 09:38:16,587] rhasspyserver_hermes: -> AudioPlayBytes(42088 byte(s))
[DEBUG:2020-03-23 09:38:16,579] rhasspyserver_hermes: Subscribed to hermes/audioServer/192.168.178.36/playFinished
[DEBUG:2020-03-23 09:38:16,561] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/asr/toggleOff
[DEBUG:2020-03-23 09:38:16,552] rhasspyserver_hermes: -> AsrToggleOff(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:16,531] rhasspyserver_hermes: Publishing 28 bytes(s) to hermes/hotword/toggleOff
[DEBUG:2020-03-23 09:38:16,525] rhasspyserver_hermes: -> HotwordToggleOff(siteId=‘192.168.178.36’)
[DEBUG:2020-03-23 09:38:16,517] rhasspyserver_hermes: Playing WAV /home/pi/rhasspy-satellite/etc/wav/beep_hi.wav
[WARNING:2020-03-23 09:38:16,508] rhasspyserver_hermes: Dialogue management is disabled. ASR will NOT be automatically enabled.
[DEBUG:2020-03-23 09:38:16,502] rhasspyserver_hermes: <- HotwordDetected(modelId=‘hey_snips’, modelVersion=‘workflow-hey_snips_subww_feedback_10seeds-2018_12_04T12_13_05_evaluated_model_0002’, modelType=‘universal’, currentSensitivity=0.5, siteId=‘192.168.178.36’, sessionId=’’, sendAudioCaptured=None)
[DEBUG:2020-03-23 09:38:16,486] rhasspyserver_hermes: <- HotwordDetected(modelId=‘hey_snips’, modelVersion=‘workflow-hey_snips_subww_feedback_10seeds-2018_12_04T12_13_05_evaluated_model_0002’, modelType=‘universal’, currentSensitivity=0.5, siteId=‘192.168.178.36’, sessionId=’’, sendAudioCaptured=None)

greetings Martin

[Adibou]

Hi everyone!

Thanks for all the great work already achieved !

My setup is as follow:

• Rhasspy Master in docker on a laptop
• 1 rhasspy Pi Zero satellite with respeaker 2 mic (rhaspy satellite venv)
  -1 rhasspy PiA+ satellite with jabra 510 (docker install)

My problem comes from the A+ satellite, where I can record sound but can’t have sound output/. On 2.4 everything worked fine, configuration is similar.

Profile.json

"microphone": {
    "arecord": {
        "device": "sysdefault:CARD=USB",
        "udp_audio_port": "12202"
    },
    "system": "arecord"
},
"sounds": {
    "aplay": {
        "device": "sysdefault:CARD=USB",
    },
    "system": "aplay"
}

asound.conf

pcm.!default {
type asym
playback.pcm {
type plug
slave.pcm “hw:1,0”
}
capture.pcm {
type plug
slave.pcm “hw:1,0”
}
}
ctl.!default {
type hw
card 1
}

Sounds work though cause I can hear sound in the terminal with aplay -D sysdefault:CARD=USB 1.wav

Looking at docker logs on the A+, I have this when trying to play a sound via TTS for instance
[DEBUG:2020-03-23 09:27:54,220] rhasspyserver_hermes: <- AudioPlayBytes(27954 byte(s))
[DEBUG:2020-03-23 09:27:54,220] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’]
[DEBUG:2020-03-23 09:27:54,225] rhasspyserver_hermes: <- AudioPlayBytes(27954 byte(s))
[DEBUG:2020-03-23 09:27:54,228] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/satellite-A/playBytes/f3d9598e-d149-4bd3-b2a9-3a7d769ecb8e, id=dd46c6b4-d783-4a63-9802-f8bcf36e4ecf)
[DEBUG:2020-03-23 09:27:55,078] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘f3d9598e-d149-4bd3-b2a9-3a7d769ecb8e’, sessionId=’’)
[DEBUG:2020-03-23 09:27:55,081] rhasspyspeakers_cli_hermes: Publishing 63 bytes(s) to hermes/audioServer/satellite-A/playFinished
[ERROR:2020-03-23 09:28:24,155] rhasspyserver_hermes:
Traceback (most recent call last):
File “/usr/lib/rhasspy-voltron/.venv/lib/python3.6/site-packages/quart/app.py”, line 1471, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy-voltron/.venv/lib/python3.6/site-packages/quart/app.py”, line 1519, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1414, in api_text_to_speech
sentence, language=(language or voice), capture_audio=True, siteId=siteId
File “/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 479, in speak_sentence
handle_finished(), messages, message_types
File “/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 832, in publish_wait
_, result = await asyncio.wait_for(result_awaitable, timeout_seconds)
File “/usr/lib/python3.6/asyncio/tasks.py”, line 362, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

Any clue ?

Thaks a lot

[Vandaag]

Hello,

I followed the instruction on the github page: https://github.com/rhasspy/rhasspy-voltron, but when I ran the last command to start rhasspy-voltron I got the following error:

vandaag@speech:~/rhasspy-voltron$ bin/rhasspy-voltron -p nl
Starting up…
/home/vandaag/rhasspy-voltron/.venv/bin/python3: No module named rhasspysupervisor
vandaag@speech:~/rhasspy-voltron$

Any suggestions?

Thanks

[synesthesiam]

Try doing the clone with git clone --recurse-submodules instead of git clone --recursive

I’m occasionally getting issues where one or more submodules fail to get pulled. Running a git submodule update and then going to any empty rhasspy-* directories and running git checkout master seems to do the trick.

Any Git guru who can explain what’s going on?

[Vandaag]

@synesthesiam, cool that got me further.

So I am trying to run it with the nl profile. At this moment I am unable to train as I get the following error:
Command ‘fstcompile --keep_isymbols --keep_osymbols --isymbols=/tmp/tmprmh5e11y/isymbols.txt --osymbols=/tmp/tmprmh5e11y/osymbols.txt /tmp/tmprmh5e11y/graph.fst.txt “/tmp/tmprmh5e11y/graph.fst.fst”’ returned non-zero exit status 127.

On the console I see the folllowing:
[DEBUG:2020-03-23 22:19:24,097] rhasspynlu.arpa_lm: compile_fst: fstcompile --keep_isymbols --keep_osymbols --isymbols=/tmp/tmprmh5e11y/isymbols.txt --osymbols=/tmp/tmprmh5e11y/osymbols.txt /tmp/tmprmh5e11y/graph.fst.txt “/tmp/tmprmh5e11y/graph.fst.fst”
/bin/sh: 1: fstcompile: not found

In my 2.4 I see fstcompile installed in:
vandaag@speech:~/rhasspy$ find . -name fstcompile -print
./.venv/bin/fstcompile
./opt/kaldi/tools/openfst/bin/fstcompile

Next to this is that I also see the following entries:
2020-03-23 22:22:18,793 INFO spawnerr: can’t find command ‘mosquitto’
2020-03-23 22:22:18,793 INFO success: intent_recognition entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-03-23 22:22:18,793 INFO success: speech_to_text entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-03-23 22:22:18,793 INFO success: dialogue entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-03-23 22:22:20,796 INFO spawnerr: can’t find command ‘mosquitto’
2020-03-23 22:22:23,800 INFO spawnerr: can’t find command ‘mosquitto’
2020-03-23 22:22:23,800 INFO gave up: mqtt entered FATAL state, too many start retries too quickly

But I am unable to identify which process is trying to execute mosquitto.

Any advise again?

[synesthesiam]

Yes, forgot to add libfst-tools and libngram-tools to the list of required packages to install. I’ve also updated the Makefile in rhasspy-voltron to automatically detect and repair failed checkouts.

Thanks!

[Vandaag]

After a reinstall the training issue is fixed and all seem to be working fine.

The only thing I still see is:
2020-03-23 22:51:47,335 INFO spawnerr: can’t find command ‘mosquitto’
2020-03-23 22:51:47,336 INFO gave up: mqtt entered FATAL state, too many start retries too quickly

Is there any way to identify what is causing this? At this moment I only have the intent recognition and the Dialogue Management active.

Thanks.

[geoffrey]

Something else I had to install on a raspberry pi zero is libatlas-base-dev or the make file would fail when installing Snowboy.

[geoffrey]

Some more observations with regards to the rhasspy-satellite that I hope are useful.

First of all I was no longer able to execute the installation because now the make command fails due to an issue with rapidfuzzy. By manually editing the requirements.txt and bumping the version to currently the latest (0.2.2), it works. The previous bump to from 0.1.0 to 0.2.1 might have caused this.

Next I have an issue to get it working using a systemd service. When run the satellite on the commandline like the documentation states:

bin/rhasspy-satellite --profile nl

Then the internal mosquitto instance is started and listening on 12183. When I place this however in a systemd service file as follows, the internal mqtt broker is not started, also resulting in debug message related to mqtt connectivity.

ExecStart=/bin/bash -lc '/home/pi/rhasspy-satellite/bin/rhasspy-satellite --profile nl'

[maxbachmann]

@geoffrey hm strange between 0.2.1 and 0.2.2 I only added the license text file to the tarball
Can you send me the error message? I suppose it failed compiling for some reason, since it is required to build the wheel e.g. on the raspberry pi (since arm is not part of the manylinux spec -> can not be uploaded to pypi)

[geoffrey]

Sure thing @maxbachmann.

  Building wheel for rapidfuzz (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/pi/rhasspy-satellite/.venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9kbwk31k/rapidfuzz/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9kbwk31k/rapidfuzz/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-b951tl28
       cwd: /tmp/pip-install-9kbwk31k/rapidfuzz/
  Complete output (32 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-armv6l-3.7
  creating build/lib.linux-armv6l-3.7/rapidfuzz
  copying python/src/rapidfuzz/__init__.py -> build/lib.linux-armv6l-3.7/rapidfuzz
  copying python/src/rapidfuzz/fuzz.py -> build/lib.linux-armv6l-3.7/rapidfuzz
  copying python/src/rapidfuzz/process.py -> build/lib.linux-armv6l-3.7/rapidfuzz
  running egg_info
  writing python/src/rapidfuzz.egg-info/PKG-INFO
  writing dependency_links to python/src/rapidfuzz.egg-info/dependency_links.txt
  writing requirements to python/src/rapidfuzz.egg-info/requires.txt
  writing top-level names to python/src/rapidfuzz.egg-info/top_level.txt
  reading manifest file 'python/src/rapidfuzz.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'python/src/rapidfuzz.egg-info/SOURCES.txt'
  running build_ext
  creating tmp
  arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/pi/rhasspy-satellite/.venv/include -I/usr/include/python3.7m -c /tmp/tmpp3epgc44.cpp -o tmp/tmpp3epgc44.o -fvisibility=hidden
  building '_rapidfuzz_cpp' extension
  creating build/temp.linux-armv6l-3.7
  creating build/temp.linux-armv6l-3.7/python
  creating build/temp.linux-armv6l-3.7/python/src
  creating build/temp.linux-armv6l-3.7/cpp
  creating build/temp.linux-armv6l-3.7/cpp/src
  arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/pi/rhasspy-satellite/.venv/include/site/python3.7 -I/home/pi/rhasspy-satellite/.venv/include/site/python3.7 -Icpp/src -I/home/pi/rhasspy-satellite/.venv/include -I/usr/include/python3.7m -c python/src/rapidfuzz.cpp -o build/temp.linux-armv6l-3.7/python/src/rapidfuzz.o -O3 -std=c++17 -DVERSION_INFO="0.2.1" -fvisibility=hidden
  python/src/rapidfuzz.cpp:1:10: fatal error: pybind11/pybind11.h: No such file or directory
   #include <pybind11/pybind11.h>
            ^~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for rapidfuzz
  Running setup.py clean for rapidfuzz

Hopefully this gives you some insights.

[maxbachmann]

Hm yes I have seen this before but could not track it down yet. Basically I use pybind11 to build the python lib from c++. However in some cases apparently it does get confused where the pybind sources are placed and therefore fails compiling. The same issue exists when compiling with travis, but e.g. not with github actions
While for most platforms I can provide prebuild wheels I can’t do this for arm since pypi does not support arm wheels. Well still confused why it worked in one version and not in the other one even though I did not change this …

[geoffrey]

It works for now, so one less to tackle for me

Thank you for having a look at it.

[synesthesiam]

I get this error all the time, but it seems inconsequential. rapidfuzz seems to work for me either way I’ve seen this happen with pip sometimes when trying to install things in parallel. I’ve had to sequence the pip install commands for some things to get it to build right.

[geoffrey]

Even though I already became quite familiar with the components of Rhasspy, this one was new for me, so I had no idea if it would break things or not.

In helping out with the beta testing I also came across the MQTT issues I described and try to tackle them one by one

At the moment I have the rhasspy-satellite running in a screen session until I figure out the systemd issue I have and it’s communicating to the server using the HTTP API.

Next up is getting the RPi Zero entirely correctly setup with the Seeed Respeaker 2.

[synesthesiam]

Thank you for all the testing help! I suspect the systemd service issue is something to do with pathing. I use supervisord to run Rhasspy’s services, and it calls mosquitto without an absolute path. I manipulate $PATH in the rhasspy-voltron script to get the other rhasspy-* scripts resolved correct, but maybe you have to manually add /usr/bin to $PATH in your service file?

[Vandaag]

It looks like that the mosquitto error originates from the mqtt service, because I got the following error:
2020-03-24 16:38:15,550 INFO gave up: mqtt entered FATAL state, too many start retries too quickly

After I changed the mqtt service to use an external server and pointed it to the localhost and the correct port, the error disappeared

[geoffrey]

Did you have to do anything with the PATH variable to get this service working?

I have an RPi zero with a respeaker 2 mic as well, but using your example for /etc/systemd/system/rhasspy.service or any other example gives me the issue that e.g. the mosquitto executable in /usr/sbin/mosquitto is not found.

[Martin_Maier]

Hi @geoffrey, I always install mosquitto by ‘sudo apt-get install mosquitto’

[Vandaag]

I have now activated Speech to text Kaldi and I am getting the following error:

bash: /home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/kaldi/egs/wsj/s5/utils/prepare_lang.sh: No such file or directory
[ERROR:2020-03-24 19:49:56,528] rhasspyasr_kaldi_hermes: train
Traceback (most recent call last):
File “/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi-hermes/rhasspyasr_kaldi_hermes/init.py”, line 477, in handle_train
missing_words_path=self.unknown_words,
File “/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/train.py”, line 85, in train
model_dir, graph_dir, dictionary, language_model, kaldi_dir=kaldi_dir
File “/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/train.py”, line 183, in train_kaldi
subprocess.check_call(prepare_lang, cwd=model_dir, env=extended_env)
File “/usr/lib/python3.7/subprocess.py”, line 347, in check_call
raise CalledProcessError(retcode, cmd)

and

[ERROR:2020-03-24 19:49:56,567] rhasspyserver_hermes: AsrError(error=“Command ‘[‘bash’, ‘/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/kaldi/egs/wsj/s5/utils/prepare_lang.sh’, ‘/home/vandaag/.config/rhasspy/profiles/nl/kaldi/model/data/local/dict’, ‘’, ‘/home/vandaag/.config/rhasspy/profiles/nl/kaldi/model/data/local/lang’, ‘/home/vandaag/.config/rhasspy/profiles/nl/kaldi/model/data/lang’]’ returned non-zero exit status 127.”, context=’’, siteId=‘default’, sessionId=‘601bafca-c9d5-4476-905a-04c490f69beb’)
[ERROR:2020-03-24 19:49:56,568] rhasspyserver_hermes: Command ‘[‘bash’, ‘/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/kaldi/egs/wsj/s5/utils/prepare_lang.sh’, ‘/home/vandaag/.config/rhasspy/profiles/nl/kaldi/model/data/local/dict’, ‘’, ‘/home/vandaag/.config/rhasspy/profiles/nl/kaldi/model/data/local/lang’, ‘/home/vandaag/.config/rhasspy/profiles/nl/kaldi/model/data/lang’]’ returned non-zero exit status 127.
Traceback (most recent call last):
File “/home/vandaag/rhasspy-voltron/.venv/lib/python3.7/site-packages/quart/app.py”, line 1471, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/home/vandaag/rhasspy-voltron/.venv/lib/python3.7/site-packages/quart/app.py”, line 1519, in dispatch_request
return await handler(**request_.view_args)
File “/home/vandaag/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1145, in api_train
result = await core.train()
File “/home/vandaag/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 387, in train
raise TrainingFailedException(reason=asr_response.error)

Again any suggestions?

[bart11064]

First of all, thanks @synesthesiam for your great work !
I’ve tried the 2.5 prerelease and have the same problem as Jorg_Baumann above in a master/satellite configuration, and trying to use precise wakeword.

When I set an external MQTT broker, i have the following error on restart :

[ERROR:2020-03-24 22:18:47,754] rhasspyserver_hermes:
Traceback (most recent call last):
  File "/usr/lib/rhasspy-voltron/.venv/lib/python3.6/site-packages/quart/app.py", line 1471, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy-voltron/.venv/lib/python3.6/site-packages/quart/app.py", line 1519, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 758, in api_listen_for_command
    handle_captured(), messages, message_types
  File "/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 842, in publish_wait
    _, result = await asyncio.wait_for(result_awaitable, timeout_seconds)
  File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
[DEBUG:2020-03-24 22:18:17,747] rhasspyserver_hermes: Publishing 140 bytes(s) to hermes/asr/startListening
[DEBUG:2020-03-24 22:18:17,735] rhasspyserver_hermes: -> AsrStartListening(siteId='Salon', sessionId='260266c6-5ecf-4e90-8185-8bd194099242', stopOnSilence=True, sendAudioCaptured=True, wakewordId='')
[DEBUG:2020-03-24 22:18:17,732] rhasspyserver_hermes: Subscribed to hermes/error/asr
[DEBUG:2020-03-24 22:18:17,728] rhasspyserver_hermes: Subscribed to hermes/asr/textCaptured
[DEBUG:2020-03-24 22:18:17,725] rhasspyserver_hermes: Waiting for transcription (sessionId=260266c6-5ecf-4e90-8185-8bd194099242)
[DEBUG:2020-03-24 22:18:06,991] rhasspyprofile.download: intent.system flair hermes = False
[DEBUG:2020-03-24 22:18:06,990] rhasspyprofile.download: speech_to_text.kaldi.open_transcription True False = False
[DEBUG:2020-03-24 22:18:06,988] rhasspyprofile.download: speech_to_text.kaldi.mix_weight >0 0 = False
[DEBUG:2020-03-24 22:18:06,987] rhasspyprofile.download: speech_to_text.pocketsphinx.mix_weight >0 0 = False
[DEBUG:2020-03-24 22:18:06,985] rhasspyprofile.download: speech_to_text.pocketsphinx.open_transcription True False = False
[DEBUG:2020-03-24 22:18:06,982] rhasspyprofile.download: speech_to_text.system kaldi hermes = False
[DEBUG:2020-03-24 22:18:06,980] rhasspyprofile.download: speech_to_text.system pocketsphinx hermes = False  

Here my profile :

"mqtt": {
        "enabled": "true",
        "host": "x.x.x.x",
        "port": "1883",
        "site_id": "xxx"
},

Any ideas ?
Thanks a lot !

[Adibou]

Hi everyone!

Any hint on my audio issue ? Sound seems work fine from the CLI I’ve tried every option for aplay with the profile.json, still no sound and no error others than the one I’ve found, could it be related to trying to install it on a Pi A+ ?

Hope you’re all well on these troubled times.

[hawkeye217]

@synesthesiam Any plans to release a rhasspy-satellite Docker image for Pi Zeros?

[synesthesiam]

Yes, I actually have one already built but not updated. I haven’t figured out how to include it in the main Docker manifest without conflicting with the armv7l image.

The Docker image is slightly different than rhasspy-satellite since it includes the ability to do training and speech to text with Pocketsphinx.

[synesthesiam]

All: as I track down and squash bugs, it’s becoming clear that I need to move Rhasspy 2.5 to Python 3.7. Does anyone see that being a problem now that buster comes with 3.7?

[ulno]

No problem on any system I run here.

[synesthesiam]

I should point out this only affects those running Rhasspy outside of Docker

[geoffrey]

No problem for me either.

[geoffrey]

That’s exactly what I did as well as mentioned on the GitHub page

Thank you for the clarification. I’ll keep digging.

[hawkeye217]

Great. I tried installing and running the rhasspy/rhasspy:2.5.0-pre image on my Pi Zero without luck. Looking forward to whatever solution you can come up with. Docker just makes life so much easier and it would be amazing to deploy Rhasspy on a few satellites around my house!

[Vandaag]

@synesthesiam, I am currently seeing the following error:
[ERROR:2020-03-26 19:22:20,756] rhasspyasr_kaldi_hermes: train
Traceback (most recent call last):
File “/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi-hermes/rhasspyasr_kaldi_hermes/init.py”, line 477, in handle_train
missing_words_path=self.unknown_words,
File “/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/train.py”, line 85, in train
model_dir, graph_dir, dictionary, language_model, kaldi_dir=kaldi_dir
File “/home/vandaag/rhasspy-voltron/rhasspy-asr-kaldi/rhasspyasr_kaldi/train.py”, line 183, in train_kaldi
subprocess.check_call(prepare_lang, cwd=model_dir, env=extended_env)
File “/usr/lib/python3.7/subprocess.py”, line 347, in check_call
raise CalledProcessError(retcode, cmd)

This is when I try to train rhasspy for speech to text with the use of kaldi using an nl profile.

I did a full reinstall with a git clone --recurse-submodules https://github.com/rhasspy/rhasspy-voltron

Any suggestions?

[maxbachmann]

@geoffrey whats the gcc version on your device? I just figured out that in travis the issue with rapidfuzz was that it used gcc-5 which did not support C++17 yet.

[geoffrey]

I’m no expert at all, but it seems the error indicates a missing file. In my Dutch profile I have the following file:

profiles/nl/kaldi_unknown_words.txt

Perhaps check if you stil have that and in case it is missing, try creating it manually.

[geoffrey]

Not sure if the following output is correct, but I executed the command gcc -v

gcc version 8.3.0 (Raspbian 8.3.0-6+rpi1)

[maxbachmann]

Hm this one has C++17 support already. I suppose I need to test a bit around on the pi aswell then

[Carsten_Graf]

Google Wavenet isnt supported in 2.5?

[rickmini]

I just installed 2.5 on my pi4 with Matrix voice. (fuzzywuzzy, kaldi, pyaudio mic, alsa playback,porcupine). Its working great and very responsive. I have what is probably a dumb question, but how do i subscribe to the mqtt on the localhost. In previous version i was able to point to an external mqtt host, and subscribe to it. This version seems to be much different. I prefer to subscribe to the same localhost that rhasspy is installed on, if possible, but not necessary. I have a multi-robot app that parses and responds to the intents on mqtt, then controls motors and servos etc. I have just randomly tried different settings, but its now time to ask for help.

[synesthesiam]

I have what is probably a dumb question, but how do i subscribe to the mqtt on the localhost

Not a dumb question – it can be a bit unintuitive if you’re using Docker. You should just need to do 2 things:

1. Set Rhasspy’s MQTT to “external” with localhost/1883
2. Add --network host to your docker run command so “localhost” in the container is your actual machine.

[rickmini]

Thanks. 2.5 looks great! What a powerful and flexible solution, a huge step forward!

[synesthesiam]

Google Wavenet isnt supported in 2.5?

Not yet. This would be a welcomed contribution

[rickmini]

tried your suggestion, did docker stop then run but got this and rhasspy didnt start:

WARNING: Published ports are discarded when using host network mode
1209f691dc12fdbaf5c735fb197feaa6d78a63d7f2deb5af4f604b544b7e81b9

ran the following:
docker run -d -p 12101:12101
–restart unless-stopped
–network host
-v “$HOME/.config/rhasspy/profiles:/profiles”
–device /dev/snd:/dev/snd
rhasspy/rhasspy:2.5.0-pre
–user-profiles /profiles
–profile en

[synesthesiam]

Looks like it did start, just in the background (-d). Do you see it running with docker ps ?

The warning about published ports just means you can leave off the -p 12101:12101 part because it’s meaningless with the Docker container attached to the host network.

[Pittermaennchen]

I just tried to re-download the rhasspy:2.5.0-pre image.

But the new container seems to crash time and again. I am not able to open port :12101 and in Portainer the container is shown as running for a few seconds, even if I started it hours ago.

My setup: Raspberry Pi 4, Rasbian Buster
Docker command:
docker run -d --name Rhasspy -p 12101:12101
–restart unless-stopped
-v “$HOME/.config/rhasspy/profiles:/profiles”
–device /dev/snd:/dev/snd
rhasspy/rhasspy:2.5.0-pre
–user-profiles /profiles
–profile de

Maybe worth mentioning:
I have another docker container with Rhasspy 2.4.19 - but stopped it before installing the Rhasspy 2.5-pre container. Both de-profiles point to the same directory.

[rickmini]

Sorry im late getting back to you. When i said it didnt run, i did so based on the web interface not being able to connect. I rebooted, then docker ps, i found 3 instances of the docker image listed. After closing all and trying again, i had strange results, not worth relating, so i went back to my external mqtt server, and everything works great. So i am satisfied to use this solution. I am spending my time now modifiying my application that formerly ran on Snips to run on Rhasspy. Ver 2.5 allows me to do this for the first time, since i make extensive use of hermes mqtt and the program flow that it provides. As of right now, I was able to compile about 95% of it. The port depends upon internal hermes systems like its dialogue flow: “hermes/dialogueManager/sessionStarted” and …ended as well as how the intents are parsed for slot values. I also use other hermes messages to determine how to make robot functions fire.
I have found a few inconsistencies that work on Snips but not Rhasspy.
I am in the process of documenting this now. When i am finished (probably before apr 4) i will provide this to you. What is the best method to do that.
Also, in previous versions i did not use Kaldi. I am using it now, and it seems much more accurate and faster? (not sure about the speed since im now on a Pi4) Is this your experience as well?

[kaykoch]

My experiences with 2.5:

Master

marryTTS: Docker on Ubuntu
mqtt: Docker on unbuntu (mosquitto)
Master: Docker on Ubuntu

–> Works with satellites

• 2.4.16, zero, env, ReSpeaker2, API
• 2.4.18, PI3, Docker, ReSpeaker2, API
• 2.5, PI3, Docker, ReSpeaker2, mqtt

Satellites:

2.5-pre, zero, env, ReSpeaker2

All satellites with UDP Audio Port :12202 and Output siteId: wohnzimmer

Satellite reacts on “Wake Up -Button”:

• Microphone works, Intents are recognized, Beep-sound OK, Answer from node-red

Satellite reacts on spoken "Wake Word:"

• Beep-Sound OK, nothing more, after 30 sec: Beep,

[DEBUG:2020-03-28 19:09:13,878] rhasspyspeakers_cli_hermes: Publishing 63 bytes(s) to hermes/audioServer/schlafzimmer/playFinished

Then mqtt messages are sent for 30 sec.

  "hermes/audioServer/schlafzimmer/audioFrame"

Then:

 hermes/asr/stopListening
 hermes/dialogueManager/sessionEnded
 hermes/asr/textCaptured (no text inside)
 hermes/hotword/toggleOff

Then beep sound and

 hermes/hotword/toggleOn
 hermes/asr/toggleOn

AFTER 30 SECONDS:

[DEBUG:2020-03-28 19:09:41,964] rhasspymicrophone_cli_hermes: <- AsrStopListening(siteId=‘schlafzimmer’, sessionId=‘schlafzimmer-default-2b6e8623-f681-4f12-bab6-fbc91b6f0d9e’)
[DEBUG:2020-03-28 19:09:41,982] rhasspyserver_hermes: <- AsrTextCaptured(text=’’, likelihood=0, seconds=0, siteId=‘schlafzimmer’, sessionId=‘schlafzimmer-default-2b6e8623-f681-4f12-bab6-fbc91b6f0d9e’, wakewordId=’’)
[DEBUG:2020-03-28 19:09:41,985] rhasspymicrophone_cli_hermes: Enable UDP output
[DEBUG:2020-03-28 19:09:42,000] rhasspyserver_hermes: Playing WAV /root/.config/rhasspy/profiles/de/wav/answer.wav
[DEBUG:2020-03-28 19:09:42,007] rhasspyserver_hermes: -> HotwordToggleOff(siteId=‘schlafzimmer’)
[DEBUG:2020-03-28 19:09:42,015] rhasspyserver_hermes: Publishing 26 bytes(s) to hermes/hotword/toggleOff
[DEBUG:2020-03-28 19:09:42,029] rhasspywake_snowboy_hermes: <- HotwordToggleOff(siteId=‘schlafzimmer’)
[DEBUG:2020-03-28 19:09:42,034] rhasspyserver_hermes: -> AsrToggleOff(siteId=‘schlafzimmer’)
[DEBUG:2020-03-28 19:09:42,039] rhasspywake_snowboy_hermes: Disabled
[DEBUG:2020-03-28 19:09:42,042] rhasspyserver_hermes: Publishing 26 bytes(s) to hermes/asr/toggleOff
[DEBUG:2020-03-28 19:09:42,058] rhasspyserver_hermes: Subscribed to hermes/audioServer/schlafzimmer/playFinished
[DEBUG:2020-03-28 19:09:42,062] rhasspyserver_hermes: -> AudioPlayBytes(48940 byte(s))
[DEBUG:2020-03-28 19:09:42,122] rhasspyspeakers_cli_hermes: <- AudioPlayBytes(48940 byte(s))
[DEBUG:2020-03-28 19:09:42,143] rhasspyspeakers_cli_hermes: [‘aplay’, ‘-q’, ‘-t’, ‘wav’]
[DEBUG:2020-03-28 19:09:42,187] rhasspyserver_hermes: <- AsrAudioCaptured(44 byte(s))
[DEBUG:2020-03-28 19:09:42,234] rhasspyserver_hermes: -> HotwordToggleOn(siteId=‘schlafzimmer’)
[DEBUG:2020-03-28 19:09:42,254] rhasspyserver_hermes: Publishing 26 bytes(s) to hermes/hotword/toggleOn
[DEBUG:2020-03-28 19:09:42,266] rhasspywake_snowboy_hermes: <- HotwordToggleOn(siteId=‘schlafzimmer’)
[DEBUG:2020-03-28 19:09:42,276] rhasspywake_snowboy_hermes: Enabled
[DEBUG:2020-03-28 19:09:42,272] rhasspyserver_hermes: -> AsrToggleOn(siteId=‘schlafzimmer’)
[DEBUG:2020-03-28 19:09:42,284] rhasspyserver_hermes: Publishing 26 bytes(s) to hermes/asr/toggleOn
[DEBUG:2020-03-28 19:09:43,497] rhasspyspeakers_cli_hermes: -> AudioPlayFinished(id=‘1a9e4f7c-9cb5-482d-bd4e-6b5216b1db3b’, sessionId=’’)
[DEBUG:2020-03-28 19:09:43,502] rhasspyspeakers_cli_hermes: Publishing 63 bytes(s) to hermes/audioServer/schlafzimmer/playFinished

2.5-pre, zero, Docker, ReSpeaker2

Satellite reacts on “Wake Up -Button”: (with Wake-Word: Disabled)

• Microphone works, Intents are recognized, Beep-sound OK, Answer from node-red

Satellite does NOT react on spoken "Wake Word:"

2020-03-28 18:36:37,498 INFO spawned: ‘wake_word’ with pid 408
2020-03-28 18:36:38,514 INFO success: wake_word entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-03-28 18:36:47,289 INFO exited: wake_word (exit status 1; not expected)
2020-03-28 18:36:48,319 INFO spawned: ‘wake_word’ with pid 417

ENDLESS

2.5-pre, PI3, docker, JABRA 410 (This machine works with 2.4.18)

Satellite reacts on on “Wake Up -Button” and spoken "Wake Word:"

• Microphone works, Intents are recognized, No Beep-sound, no Answer, no speaker
  I tried all possible Devices. No DEBUG Information. Speaker is simply not working.
  Stopping 2.5 Docker and starting 2.4.18 --> everything works

---

I hope, someone can handle that information.

[frkos]

I’ve just tried 2.5 and… I can’t use it =((
It’s super fast but it recognizes 1 of 10 phrases, while previous version works well

I understand it’s because of Kaldi, so is it possible to tune it somehow? I’m ok with 2-3 sec delay, but have good recognition ratio

[romkabouter]

I see some unexpected messages popping up when wakeword is used

Rhasspy 2.5 is running as docker on a i3 NUC.
I have Matrix Voice streamer as mic input, hotword get detected (snowboy), but the I see

• 1 message on hermes/hotword/default/detected -> correct
• 2 message on hermes/hotword/toggleOff -> expecting just 1
• followed within a second with hermes/hotword/toggleOn -> unexpected

This causes the leds to turn green and straight away to blue (toggleOff / toggleOn)
The speech recognition is working properly.

Image below is: wakeword detect, say nothing and wait for timeout
Clearly seen can be the double hotword toggles

[geoffrey]

I tracked down the issue a bit more and it seems to be a permissions problem with regards to /dev/stdout that is used by supervisor.

When I disable the following line in rhasspy-satellite/rhasspy-supervisor/rhasspysupervisor/__init__.py, then I am able to start it as a service.

print("stdout_logfile=/dev/stdout", file=out_file)

Would you be able to check which user and group in your setup own /dev/stdout? This is my output:

pi@raspberrypi:~ $ ls -al /dev/std*
lrwxrwxrwx 1 root root 15 Feb 14  2019 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Feb 14  2019 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Feb 14  2019 /dev/stdout -> /proc/self/fd/1

[Martin_Maier]

Hi @geoffrey, I’ m a little bit astonished about your problems and your way how you try to find these errors. I can show you how the complete installation should work on a Pi Zero an I did it a lot times in this way:

1. Flashing raspbian-buster-lite on a sd card using belana Etcher
2. After first boot apt update and upgrade
3. Configuring wlan, ssh, keyboard, timezone …
4. Installing samba to get file access to the pi
5. Install HermesLedControl https://github.com/project-alice-assistant/HermesLedControl which also provides the drivers for the Respeaker 2 Hat.
6. I also install snips-satellite (sudo apt install mosquitto mosquitto-clients snips-satellite snips-hotword-model-heysnipsv4 -y) because for me it is the best wake word at the moment.
7. After this install the snips-satellite venv … test it (should work) … install service …

that’s it. In this way I’ve installed many Pi Zero satellites which all work pretty well.

[kaykoch]

Update:

Zeros are working

I tried today again with newer docker images.
I installed the newest docker image from 27.3.20.
With that I could rhasspy work on all zeros with ReSpeakaer.
HOORAY

Jabra USB Problem

wrong Sound CARD NUMBER in rhasspy used

But the problem with usb speaker remains. Still no sound.
I was deactivating the onboard sound card on pi and zeros by default.

 dtparam=audio=off

This was working on all satellites but the one with jabra. All of them have ReSpeaker2.
Aplay recognized them as CARD 0. (aplay -l)
The USB-Jabra was recognized as CARD 1. Even if i deactivate onboard Card.
I activate the onboard soundcard and plugged in a speaker.
Rhasspy works. Rhasspy works always. Even, If i changed to USB Speaker
So, I think, there is a fault in the last Rhasspy image

The last rhasspy 2.5 image is ALWAYS using CARD 0. You can change the Device to what ever you want.
I solved this problem by changing the Index Number: (WORKARROUND)
!! USB is the id of the card (USB [Jabra SPEAK 410 USB]) !!

1. Create a file: /etc/modprobe.d/alsa-base.conf

2. insert:

   options snd_usb_audio index=0
   options snd_usb_audio id="USB"
   

3. reboot

After that, aplay -l gives:

 Karte 0: USB [Jabra SPEAK 410 USB], Gerät 0: USB Audio [USB Audio]

Now, also the last one is working.
HOORAY

marryTTS

wrong locale in rhassppy given

For those, who uses marryTTS:
Check the locale code with

 http://IP-ADRESS:59125/locales

It gives you the write one.
For a german voice it is de
IT IS NOT THE ONE WHICH IS GIVEN IN RHASSPY (de-DE)

Something more

I’m using two wakewords on different satellite, so I want different voices.
This works, if you use a standard one, which is controlled by master, and a special one which is controlled by this special satellite. You have to point the TTS to the master for standard and the special one directly to marryTTS.
IMPORTANT: Only the standard satellite names should be in the masters Satellite siteIds in TTS.
If you leave the special one also inside, you get the answer twice.

So this was a good Weekend.
@ synesthesiam

• Please look for the sound card problem
• change locale in marryTTS

And thanks for all your great work.

[geoffrey]

Thank you for the description of how you set it up.

It’s not that much of an issue to go through it as it e.g. the new Docker image for the satellite does work. It’s just that if I encounter these things, others might as well, so I dug a little deeper.

[frkos]

Guys, does anyone have issues with speech recognition using Kaldi?
Especially if you have PS3eye… I’m stuck with this

When I play last voice command through the interface I hear my voice clearly…

[jrb5665]

Hi @Martin_Maier,
I experience the some of issues that @geoffrey reported and have not been able to successfully run it as a service. I haven’t tried his workaround for starting the services. I haven’t been able to add much to the discussion so far and was wondering if I would have to wait for this to get more mature.

It’s interesting that you have had no problems.

I asked you a while back about your setup and even blew my setup away a couple of time to make sure I followed what you did properly. The only difference I think I have is that I am trying to use snowboy, not snips hotword, but I don’t think that explains the issues trying to run it as a service.

I haven’t tried the docker image yet

[hawkeye217]

I have a PS3eye and sometimes recognition has been sketchy (with both Kaldi and Pocketsphinx), but I think that’s mainly because I have the gain turned up - the few issues I’ve been having are with silence detection and the recording not terminating if I’m far away from the mic.

Though I have noticed that Kaldi sometimes doesn’t recognize me as well as Pocketsphinx does…

Is silence detection adjustable in 2.5 as it was in 2.4?

EDIT: Another question: should webrtcvad parameters be adjusted on the master or on the satellite? And are those parameters adjustable and functional as they were in 2.4?

[Martin_Maier]

I haven’t tried the docker image yet

Hi jrb5665, I did a fresh installation with only the rhasspy-satellite venv. The Test with ‘bin/rhasspy-satellite --profile en’ run Ok. Then I installed the service … started it … and got some errors from mqtt connection. So I started the test again … changed MQTT to external (ip = satellite-ip, port = 1883) … stopped the test … started service again … and now it runs. It seems there are still some problems with the internal mqtt but of course this is a satellite its mqtt connection will be always external … and so this should not be a problem.

[geoffrey]

The errors I receive are not restricted to only not being able to start mosquitto, that’s what I first noticed.

In the end it are all of Rhasspy’s service that are not able to start through supervisor. So even after connecting to my main external broker, the next service fails to launch.

Thank you for testing along. Together we will squash all the bugs

[jrb5665]

Mine is also a satellite and always set to external

[synesthesiam]

Just a heads up: lots of bug fixes will be coming soon, hopefully by the end of today. The move to Python 3.7 and a newer version of the quart web framework broke quite a bit more than I expected. But it has made the code ultimately simpler and easier to maintain!

I’m adding more and more unit tests, so bugs should stay squashed for good

As one example, the websocket API was completely broken outside of a web browser because the quart guy decided that the HTTP Origin header was now a requirement, despite being explicitly optional in the spec. Node-RED doesn’t set this header, so I have to monkey patch quart to fix the problem

[geoffrey]

Something else I noticed in NodeRed when connecting to the external MQTT broker is that the messages are not recognized as JSON objects but rather plain text.

I have to set the Output property of the mqtt in node to a parsed JSON object instead of the default auto-detect (string or buffer).

Your previous post made me think of it I came across this when having issues with the websockets connection and tried changing to MQTT.

Thank you for the continued effort and heads up!

[thomas_cologne]

Not a bug, but maybe a proposal for one of the next versions:

Sometimes it´s hard to hit the correct position in the upper bar. Maybe it is possible to enlarge the link not just to the red marked zone, but extense it to the blue marked?
Does it makes sence?
As I said, just a smaller remark.

[synesthesiam]

I agree. I’m always hoping someone will really want to take the web stuff off my hands…

[NathanC]

Hey! I just joined the forum-- I set up rhasspy ~ 2 weeks ago (used to use Snips).

This project is super cool, and I just tried the pre-release and it works like a charm (although I need to use the rhasspy dialog manager, hermes has some issues). Excited to leverage the new expanded hermes spec-compliance in my python intent handler.

One thing I noticed is that audio frames are getting streamed over MQTT even w/o the wake word being detected. What do you think about some (potentially default) setting to only stream the frames after the wakeword is detected? Otherwise mic data is going to be going over the queue 24/7 which seems privacy impacting.

I’m going to look into contributing to this project later! Also, my background is in Comp Sci but my current focus is AppSec-- feel free to ping me if you ever want to bounce around security questions or do any threat modeling.

side note: the new web UI for the pre-release is a lot smoother, especially the settings section! great job

[geoffrey]

You can setup a UDP port for both the wakeword and speech to text services what perhaps does what you refer to:

By default, Rhasspy will stream microphone audio over MQTT in WAV chunks. When using Rhasspy in a master/satellite setup, it may be desirable to only send audio to the MQTT broker after the satellite as woken up. For this case, set both microphone.<MICROPHONE_SYSTEM>.udp_audio_port and wake.snowboy.udp_audio_port to the same free port number on your satellite. This will cause the microphone service to stream over UDP until an asr/startListening message is received. It will go back to UDP stream when an asr/stopListening .

Wake Word

Hope this helps.

[NathanC]

Thanks @geoffrey! That worked perfectly with porcupine.

I’d still argue that constant streaming should not be the default behavior, otherwise a lot of people (likely w/o TLS or username/passwords on their broker) would be pretty easy to eavesdrop on with a user-level process on their network. What’s the right venue for that discussion-- here or an issue?

[geoffrey]

@synesthesiam is active in both channels, but I’d suggest to create a separate topic for this in this community.

[orca8119]

Hello @synesthesiam,

I’ve aready created a pull request for the issue above, but afterwards I realized that it might be better to ask if you are interested in a little bit of beautifying the UI. I’ve tinkered a little with the forms and what I can offer you is more or less a improvement of the existing which looks a little bit “bootstrappy” like this:

[kahless]

hint hint

[Andrew49]

Ignorant question alert…
How should I go about getting the most recent version(s) of 2.5 Docker?
(I’d want the one for Intel NUC, but I may also give Rhasspy 2.5 a try on a Raspberry 3b+)

[synesthesiam]

docker pull rhasspy/rhasspy:2.5.0-pre

[synesthesiam]

Just a heads up: the next Docker image will fix the NLU messages to align correctly again with Snips.

• raw_value in slots will now be rawValue
• Same thing for raw_start and raw_end
• value will be a JSON object with kind and value properties

Rhasspy traditionally hasn’t differentiated between Snips’ entity and slotName fields. In the Rhasspy sentence (some text){tag}, I’d normally set both fields to tag and the value to some text. I’m trying something slightly different with slots now, and looking for feedback.

If you have the sentence ($mySlot){myTag} now, I’m setting the Snips entity to “mySlot” and the slotName to “myTag”. This works really well with numbers, because those are generated with a slot program. So (1..10){myNum} will have rhasspy/number as the entity and myNum as the slotName. Additionally, kind is automatically set to “Number”.

Unfortunately, Rhasspy’s intent JSON format has only ever had entity, so I’d be breaking backward compatibility there. My first attempt has been to leave entity alone, and add two new fields to each entity: source and kind. In the above example, the entity, source, and kind fields will contain “myNumber”, “rhasspy/number”, and “Number” respectively. I don’t like that Snips’ entity and Rhasspy’s entity fields have different values, but that may just be the price to pay.

Thoughts?

[fastjack]

I think that entities should be placed in slots so I liked the way Snips did it.

The user defines entities (cities for instance that are not related to any specific intent or context) and place them in different slots with custom names depending on Intent context (from, to) that give meaning to the entities.

I think « entities in slots » is clearer than « sources in entities » (which is kind of confusing).

Regarding breaking changes, I think it might be acceptable for 2.5 if it improves clarity in the long term. it might be now or never

Maybe Discourse have a survey plugin to ask the whole community what they prefer?

[adrianofoschi]

Hello, congratulation from rhasspy 2.5!
I pulled latest docker rhasspy-2.5 image.
Actually home-assistant intent handling doens’t work anymore.

Configuration:

   "home_assistant": {
        "access_token": "...",
        "handle_type": "intent",
        "url": "http://192.168.0.100:8123"
    }

My sentences:

[LightOn]
accendi ($light){light}

My light slot:

(luce salone):luce_salone

On example “LightOn” intent triggering I have this error home-assistant side:

File “/usr/src/homeassistant/homeassistant/helpers/intent.py”, line 74, in async_handle
raise InvalidSlotInfo(f"Received invalid slot info for {intent_type}") from err
homeassistant.helpers.intent.InvalidSlotInfo: Received invalid slot info for LightOn

[manju-rn]

I think the value should just contain the value. Not the kind and the value. I think it was the same with snips earlier. Hence, it used to work directly with HA conversation intent components. Now I have to define separate intents just for Rhasspy. see the issue I raised in the git

Intent JSON has additional key - 'Kind' Its affecting HA (Version 2.5-pre)

[adrianofoschi]

OK so actually we need to do {{ slot.value }} on home-assistant intent configuration?

[manju-rn]

correct. Thatswhy I am kind of not in favor of this change as all my intents are defined in HA and I was reusing them until now. With this, I have to do a workaround to define similar intent with slot.value and call the same intent again. Kind of defeats the purpose.

[synesthesiam]

I should be able to fix this for HA without disrupting other Snips-compatible services.

[manju-rn]

thanks @synesthesiam will be looking forward for it.

It’s a wonderful project and a god send for the snips veterans

[adrianofoschi]

I am trying to install docker rhasspy image on Raspberry Pi First version.
But It doesn’t start with some “parallels” errors.

Is rhasspy definitivility incompatible with rpi 1?
Thanks you

[koan]

The Raspberry Pi 1 has the same ARMv6 architecture as the Raspberry Pi Zero, it probably has something to do with it, because the images with 2.5.0-pre and latest tags on Docker Hub are only built for ARMv7, ARM64 and AMD64.

Have you tried the 2.5.0-pre-arm32v6 image?

[synesthesiam]

@koan is correct. I had tried to bundle the ARMv6 image in with the others, but my Pi Zero kept downloading the ARMv7 image and failing to start it. I’ll revisit this at some point, but you can at least manually specify the image for now

[adrianofoschi]

I tried 2.5.0-pre-arm32v6 and it works perfectly. Thanks you!

[ulno]

AFAIK, porcupine and mycroft-precise don’t work on ARMv6?
Or have you seen something else?

[Vandaag]

@synesthesiam, what is the best way to update an 2.5 pre-release version to the latest github version. I did a git pull and a make, and after that all seemed to work untill I tried to access the api. I get the following error:

[ERROR:2020-04-13 22:08:56,774] rhasspyserver_hermes: ‘servers’
Traceback (most recent call last):
File “/home/vandaag/rhasspy-voltron/.venv/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/home/vandaag/rhasspy-voltron/.venv/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/home/vandaag/rhasspy-voltron/.venv/lib/python3.7/site-packages/swagger_ui/core.py”, line 202, in swagger_blueprint_config_handler
return jsonify(self.get_config(request.host))
File “/home/vandaag/rhasspy-voltron/.venv/lib/python3.7/site-packages/swagger_ui/core.py”, line 79, in get_config
for server in config[‘servers’]:
KeyError: ‘servers’

I did do a remove of the rhasspy directory in .config aswell.

Regards,

Richard

[frkos]

And are those parameters adjustable and functional as they were in 2.4?

That’s the question I’m trying to answer now… Does anyone know?
I’m using the same config, but the timeout is about 30 sec, not 4 sec

    "command": {
        "webrtcvad": {
            "min_sec": 3,
            "speech_buffers": 7,
            "throwaway_buffers": 9,
            "timeout_sec": 4,
            "vad_mode": 2
        }
    },

I really like 2.5 version… Now it can recognize my voice

[synesthesiam]

AFAIK, porcupine and mycroft-precise don’t work on ARMv6?

Porcupine should with the latest update, but Precise won’t.

[synesthesiam]

That’s the question I’m trying to answer now… Does anyone know?

Timeouts are now controlled by the dialogue session timeout setting (should be dialogue.session_timeout). I don’t have that exposed yet on the Web UI.

[synesthesiam]

Hi @Vandaag, this error would occur if the git submodules were out of date, specifically rhasspy-server-hermes. I had to add a “servers” section to the swagger.yaml file in that service,

Perhaps someone with more git experience can give better advice, but I’ve just been running git submodule foreach git pull origin master in the rhasspy-voltron root to update. So the steps would look like:

1. git submodule foreach git pull origin master
2. git pull origin master
3. make

Submodules are still a bit of a mystery to me, so hopefully these aren’t terrible practices

[CrankyCoder]

Finally getting a chance to work on rhasspy Going to jump straight in to 2.5. Will a pi 3 be sufficient at this time? My end goal once I have a better understanding of it is to remove all the audio pieces from it and hopefully run the main dockers on a vm or more powerful piece of hardware in my rack and just have satellites. but for now, i have a pi3 sitting on my desk i am about to get another microsd card for and a playstation eye camera/mic.

So would the pi3 work for 2.5 and what size card is needed?

Can’t wait to get running

[NullEntity]

Will a pi 3 be sufficient at this time?

Yes, a Pi 3 should is sufficient. I just tested Rhasspy 2.5 on a 3B+ without any issues. Thanks to the new “rapidfuzz” drop-in replacement you can now even use “fuzzywuzzy” for intent recognition - which was quite slow before version 2.5.

[maxbachmann]

Thanks to the new “rapidfuzz” drop-in replacement you can now even use “fuzzywuzzy” for intent recognition - which was quite slow before version 2.5.

I am pleased to see that the speed improvements make it more usable

[synesthesiam]

Just a quick update on what’s coming in the next Docker image. I’ve got Mozilla’s DeepSpeech 0.6 added now for English and German (still uploading). Works on the Raspberry Pi 3/4 as well as long as you don’t have aarch64. Please point me in the direction of any more v0.6 models you can find!

I’m also having to make changes to the UDP audio settings for microphone/wake. In short, udp_audio_port is now udp_audio in the profile. In long, udp_audio can contain multiple host:port:siteId triples; the plan is to allow master servers to do wake word detection for multiple satellites simultaneously over UDP.

Currently working on integrating Snips NLU, but training is not finished yet. I may need some advice on how to translate Rhasspy’s concept of slots/entities over to Snips.

[KiboOst]

Just curious but why snips nlu ?
If something from snips is to be kept alive I would take wakeword detection. Actually nlu works nice.

[maxbachmann]

Just curious but why snips nlu ?
If something from snips is to be kept alive I would take wakeword detection. Actually nlu works nice.

Well the Snips NLU is fully open source and at least I really like how its performing. The snips wakeword detection worked pretty well, but as far as i remember that is not open sourced. (Would love to be proven wrong on this xD)

Currently working on integrating Snips NLU, but training is not finished yet. I may need some advice on how to translate Rhasspy’s concept of slots/entities over to Snips.

The main problem people appear to have with it is that cross compiling their rust stuff for the pi sucks a bit.
Are the slots/entities of Rhasspy any different to the slots/entities of the snips NLU? There is basically a json structure the Snips NLU is using for the training examples, that you would have to generate and then there should be a training feature of the NLU you can pass it to that will train the model for the NLU.

[Andrew49]

Hi all,
Just pulled the latest Docker of Rhasspy 2.5.0-pre and very impressed by the general speed of response - it’s much slicker than the earlier version I was trying before. Thanks @synesthesiam and all involved in this great project!
I’ve got just one issue which I’m hoping for advice about - When I speak with an intent which involves feedback speech (from Home Assistant, such as GetTime), I get more than one (sometimes upto 10) repeats of the end of speech sound, and then the same number of repeats of the reply speech. In the log in Portainer for my Rhasspy container, I see this warning:-

[WARNING:2020-04-16 19:27:58,685] rhasspytts_cli_hermes: Did not receive playFinished before timeout

How can I diagnose/fix this?

[synesthesiam]

Just curious but why snips nlu ?

It has good language support and seems like a nice complement to fsticuffs and fuzzywuzzy. Not as strict as fsticuffs, but intelligently flexible instead of literally matching text.

Are the slots/entities of Rhasspy any different to the slots/entities of the snips NLU?

Rhasspy’s slots can have substitutions and conversions inside them, so I’ll do my best to take those into account. A Rhasspy slot value like foo:bar!upper will match foo from the ASR, but should output BAR in the intent.

[synesthesiam]

I get more than one (sometimes upto 10) repeats of the end of speech sound, and then the same number of repeats of the reply speech

Yikes, that’s really bad. I’ll test speech again from HA and see if I have the same issue. Is this with master/satellite or just a single system?

The audio output issues are much more difficult to diagnose than in 2.4, where everything was in the same Python process. The dialogue manager pauses the wakeword/ASR systems when TTS or audio is played to avoid feedback, but what to do if the playFinished message never comes (service crash, etc.)? So I guess the WAV duration and timeout then. Maybe I should add a small delay to give the audio service more time…

[Andrew49]

Answering my own question - something made me try switching from the internal MQTT broker, to External broker (which is the Mosquitto 5.1 MQTT Addon of Home Assistant, running in a different Docker container on the same machine) - and Dada! the multiple repeats stopped! (For completeness, I had also just rebooted the whole machine about 10 mins earlier).
I have the feeling that the broker built in to Rhasspy is somehow more sluggish, but no idea why.
Hope this experience may help others.

[maxbachmann]

Rhasspy’s slots can have substitutions and conversions inside them, so I’ll do my best to take those into account. A Rhasspy slot value like foo:bar!upper will match foo from the ASR, but should output BAR in the intent.

The snips nlu has the concept of slot value and synonyms. So you should be able to build this using

"entities": {
    "slot_name": {
      "data": [
        {
          "value": "BAR",
          "synonyms": [
              "foo"
            ]
        },
      ],
      "use_synonyms": true,
      "automatically_extensible": false,
      "matching_strictness": 1.0
    }
  }

https://snips-nlu.readthedocs.io/en/latest/dataset.html#json-format
For more complex converters this might not be applicable -> you might have to go over the results of the snips nlu and run the converter afterwards. I suppose thats what your doing for the other solutions aswell.

[Don_Murphy]

Hi all,

First thank you for your great Work!

I want to try Deepspech, but it’s not working. I made fresh install (Docker) on a Raspberry Pi 4 with german Profil. But i get a error ‘TrainingFailedException: CreateModel failed with error code 12288’ after Download the files for Deepspeech. How can handle with this?

Thank you for your help

[synesthesiam]

Can you check in the deepspeech directory of your profile and tell me how big the output_graph file is?

Also please let me know the output of this command on the Pi 4:

python3 -c 'import platform; print(platform.machine())'

I’m guessing the Pi 4 reports something other than arm64.

[rolyan_trauts]

python3 -c ‘import platform; print(platform.machine())’

armv7l is the usual for raspbian pi4 Pi3 is just armv7 I think which may cause you probs also zero might report something different but forgot.

[manju-rn]

Its armv71

[Don_Murphy]

Can you check in the deepspeech directory of your profile and tell me how big the output_graph file is?

it’s 188.939.505 Bytes

python3 -c 'import platform; print(platform.machine())'

armv71

Thank you for your help!

[winandus]

@synesthesiam

I ran into an issue with uploading Slots. Here’s the logging from my code:

04-22 12:08:49 rhasspy.updater.RhasspyUpdater INFO     {"Room": ["Dining", "Kitchen", "Landing", "Guest room", "Study","Backyard"]}
04-22 12:08:49 urllib3.connectionpool DEBUG    Starting new HTTP connection (1): 192.168.178.13:12101
04-22 12:08:49 urllib3.connectionpool DEBUG    http://192.168.178.13:12101 "POST /api/slots?overwrite_all=true HTTP/1.1" 500 44
04-22 12:08:49 rhasspy.updater.RhasspyUpdater ERROR    b"TypeError: 'NoneType' object is not iterable" 

The first line is the content I’m sending to the slots endpoint, the second shows the url, and the last line is a print of response.content (I’m using the requests library in python).

[CrankyCoder]

slightly off topic… i recently got the 2.5-pre image running. As i am moving forward with it and have questions ect, should we just put all the 2.5 questions on this thread? or regular posts?

I don’t want to create a bunch of posts regarding 2.5 if we aren’t there yet

[CrankyCoder]

for the mqtt option in 2.5. I don’t want to push all my audio frames to an external broker. So i was trying to connect to the internal mqtt. But I can’t connect to it. It seems the mqtt server runs on 127.0.0.1 only. Is there a way to connect directly to the mqtt? or would i be better off spinning up an mqtt docker on the same box and just configure rhasspy to use that one but others can connect at the same time?

Thank!

[synesthesiam]

it’s 188.939.505 Bytes

OK, this makes sense now. The German DeepSpeech model I found did not include a graph for Tensorflow Lite. I’m not sure how to do this, but it’s going to be necessary for running it on a Raspberry Pi.

[synesthesiam]

Make sure to use port 12183. The internal broker doesn’t use the standard MQTT port to avoid conflicting.

[synesthesiam]

Does the Rhasspy log contain any more info (a line number)? I’m not seeing yet where the problem could be.

[winandus]

@synesthesiam

Log contains this:

[ERROR:2020-04-24 21:28:49,248] rhasspyserver_hermes: 'NoneType' object is not iterable
Traceback (most recent call last):
File "/usr/lib/rhasspy-voltron/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
File "/usr/lib/rhasspy-voltron/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
File "/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1600, in api_slots
    save_slots(slots_dir, new_slot_values, overwrite_all=overwrite_all)
File "/usr/lib/rhasspy-voltron/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 428, in save_slots
    for name in new_slot_values:
TypeError: 'NoneType' object is not iterable

I’ve had a bit of a look at the code. The old code (2.4.x, app.py) does:

new_slot_values = json5.loads(await request.data)

The new code (rhasspyserver_hermes/main.py) does not do a json.loads():

new_slot_values = await request.json (line 1598)

At first I wasn’t sure if that might work or not, because the request.json() might return a ready-to-use dict, but as an example, in api_slots_by_name, json.loads is used again:

data = await request.data (line 1641)
slot_path = slots_dir / name
slot_values = set(json.loads(data))

So, could that be the problem?

[synesthesiam]

Trying to reproduce this, but failing so far. Is it possible the client is not setting Content-Type to application/json? I could see that causing request.json to return nothing, otherwise I’d expect it to be a dict.

The example from line 1641 should be able to be await request.json as well. In my head, I was accepting a list instead of a JSON object, so I manually parsed it…

[CrankyCoder]

Ok, I got it working. Initial docker setup didn’t have the port being passed in to the container. I updated it and now I am able to connect into the mqtt. This is exactly what I needed!

[winandus]

That might very well be it. I guess 2.4 was more forgiving. I will check it later today

[KiboOst]

Hi,

I see different updates on rhasspy GitHub, but docker images are still 13 days old.

Any changelog or information why no updates for docker ? Still have a few issues on voltron repo.

[Jorg_Baumann]

Hi,
how did you managed, to connect to the mqtt?
Just adding IP and Port of MQTT in setup?

[CrankyCoder]

Sorry for the delay. yeah, just need to forward the 12183 in to the container in the docker run, or through portainer or whatever you are using to manage your dockers.

[synesthesiam]

Just a heads up, the virtual environment install for Rhasspy 2.5 has changed a bit.

I’ve created an autoconf script for Rhasspy (./configure) that lets you precisely control which things get installed. This can be something low-level like:

$ ./configure RHASSPY_SPEECH_SYSTEM=kaldi RHASSPY_WAKE_SYSTEM=precise

which will only install Kaldi and Mycroft Precise. Something a bit more high level can also be done:

./configure RHASSPY_LANGUAGE=fr

This will install the “best” speech to text engine for the platform, which is Kaldi unless you’re on a Pi Zero (then pocketsphinx).

Run ./configure --help to see all the options, and please suggest changes!

---

The new Makefile.in template uses the variable shell scripts in scripts/ to download/install pre-compiled binaries from the new prebuilt-apps repo or actually build them from source (mostly untested).

Going forward, I will be porting the Hermes services to autoconf so they can be configured per-platform. Where possible, each service will have a multi-arch Docker image for Pi 0/1/2/3/4 and amd64.

[Daenara]

Just started playing with 2.5-pre and I can’t get it to recognize any intent. I am reusing my 2.4 profile that I copied over and renamed, installed via docker and then installed pip3 into the docker so I can install the requirements for my slot_programs.

This is the error:

[ERROR:2020-05-29 21:26:21,607] rhasspyserver_hermes:
Traceback (most recent call last):
File “/usr/lib/rhasspy/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/lib/python3.7/site-packages/rhasspyserver_hermes/main.py”, line 1337, in api_text_to_intent
user_entities=user_entities,
File “/usr/lib/rhasspy/lib/python3.7/site-packages/rhasspyserver_hermes/main.py”, line 2438, in text_to_intent_dict
result = await core.recognize_intent(text, intent_filter=intent_filter)
File “/usr/lib/rhasspy/lib/python3.7/site-packages/rhasspyserver_hermes/init.py”, line 453, in recognize_intent
handle_intent(), messages, message_types
File “/usr/lib/rhasspy/lib/python3.7/site-packages/rhasspyserver_hermes/init.py”, line 901, in publish_wait
result_awaitable, timeout=timeout_seconds
File “/usr/lib/python3.7/asyncio/tasks.py”, line 449, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

[romkabouter]

If have just created an add-on for the 2.5 pre-release.
This is to test with Hassio.

You have to change Google Wavenet if you have set this, because that is not yet supported.
Other than that, it should work with your existing profile, but backup to be safe

Rhasspy 2.5 pre

[Daenara]

@Daenara

Just started playing with 2.5-pre and I can’t get it to recognize any intent. I am reusing my 2.4 profile that I copied over and renamed, installed via docker and then installed pip3 into the docker so I can install the requirements for my slot_programs.

Okay, this error just vanished overnight without me changing anything at all, next thing was that my intents that worked on 2.4 with fuzzywuzzy were not recognized at all, I then switched to fsticuff and it recognized my intents correctly. The big problem I have now is that rhasspy just does not call my custom_command script at all. Since I copied over my 2.4 profile the path should still work. I also saw the paths to the wav files use brackets so I tried it with those but still nothing about my script being run in the rhasspy log or any new entry in my script log.

The script itself runs fine if I execute it in the rhasspy docker and the relevant part of the config seems like it should work.

“handle”: {
“command”: {
“program”: “${RHASSPY_PROFILE_DIR}/custom_command/custom_commands.py”
},
“forward_to_hass”: true,
“system”: “command”
},

This is the relevant part of my config.

Is there any known problem with custom command systems? I saw that the documentation did not change from 2.4 for that part and it ran with 2.4.19 and 2.4.20 (after I fixed the missing pip to install my requirements) so I assume something in rhasspy changed. I would expect some kind of log entry that actually calls the script but after the intent detection nothing happens at all.

[DEBUG:2020-06-01 00:06:12,251] rhasspyserver_hermes: Sent 383 char(s) to websocket
[DEBUG:2020-06-01 00:06:12,243] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/GetWeatherForecast, id=e89590b6-19b9-4b27-9ff0-730227084670)
[DEBUG:2020-06-01 00:06:12,243] rhasspyserver_hermes: <- NluIntent(input=‘wie wird das wetter’, intent=Intent(intent_name=‘GetWeatherForecast’, confidence_score=1.0), site_id=‘pi-hime’, id=‘0d5a3c66-5229-4a49-ab8e-b57126914e21’, slots=[], session_id=‘0d5a3c66-5229-4a49-ab8e-b57126914e21’, custom_data=None, asr_tokens=[[AsrToken(value=‘wie’, confidence=1.0, range_start=0, range_end=3, time=None), AsrToken(value=‘wird’, confidence=1.0, range_start=4, range_end=8, time=None), AsrToken(value=‘das’, confidence=1.0, range_start=9, range_end=12, time=None), AsrToken(value=‘wetter’, confidence=1.0, range_start=13, range_end=19, time=None)]], asr_confidence=None, raw_input=‘wie wird das wetter’, wakeword_id=None)
[DEBUG:2020-06-01 00:06:12,030] rhasspyserver_hermes: Publishing 194 bytes(s) to hermes/nlu/query
[DEBUG:2020-06-01 00:06:12,030] rhasspyserver_hermes: -> NluQuery(input=‘wie wird das wetter’, site_id=‘pi-hime’, id=‘0d5a3c66-5229-4a49-ab8e-b57126914e21’, intent_filter=None, session_id=‘0d5a3c66-5229-4a49-ab8e-b57126914e21’, wakeword_id=None)
[DEBUG:2020-06-01 00:06:12,027] rhasspyserver_hermes: Subscribed to hermes/error/nlu

I do see it sending stuff out via mqtt for the intent handling and I know that I can actually plug into there to catch my intent (and I might do so in the future) but for right now I am trying to get my command script to work again.

If someone knows anything about that, if it was removed or changed or if it is a known bug that I just did not read about yet, I would be happy for input in that.

Daenara

[ulno]

venv doesn’t work at all on pi anymore due to dependency issues, see https://github.com/rhasspy/rhasspy/issues/58

Fresh install on a one week old raspberry pi (buster).

Not sure if related, I also get the following when trying to open settings:

[ERROR:2020-06-14 18:11:28,988] rhasspyserver_hermes: expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/rhasspyserver_hermes/__main__.py", line 2413, in page_settings
    "settings.html", page="Settings", **get_template_args()
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/quart/templating.py", line 91, in render_template
    return await _render(template, context)
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/quart/templating.py", line 109, in _render
    rendered_template = await template.render_async(context)
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 65, in render_async
    return self.environment.handle_exception()
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 25, in concat_async
    await collect()
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 22, in collect
    async for event in async_gen:
  File "/home/pi/rhasspy/.venv/share/rhasspy/templates/settings.html", line 1, in top-level template code
    {% extends "layout.html" %}
  File "/home/pi/rhasspy/.venv/share/rhasspy/templates/layout.html", line 157, in top-level template code
    {% block body %}{% endblock %}
  File "/home/pi/rhasspy/.venv/share/rhasspy/templates/settings.html", line 882, in block "body"
    <li>Base Language Model FST: <tt>{{ profile.read_path(profile.get("speech_to_text.pocketsphinx.base_language_model_fst")) }}</tt> </li>
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/rhasspyprofile/profile.py", line 125, in read_path
    full_path = profiles_dir.joinpath(self.name, *path_parts)
  File "/usr/lib/python3.7/pathlib.py", line 898, in joinpath
    return self._make_child(args)
  File "/usr/lib/python3.7/pathlib.py", line 688, in _make_child
    drv, root, parts = self._parse_args(args)
  File "/usr/lib/python3.7/pathlib.py", line 642, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
[ERROR:2020-06-14 18:11:29,168] rhasspyserver_hermes: NotFound(404)
Traceback (most recent call last):
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/home/pi/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1863, in dispatch_request
    raise request_.routing_exception
quart.exceptions.NotFound: NotFound(404)

[longunmin]

@rickmini can you share your settings for the matrix voice? I can’t get the Matrix Voice to work with 2.5 from upgrading from 2.4.20. Is there anything that needs to be changed within Rhasspy or from an ALSA perspective?