Rhasspy 2.5.9 Released

Announcements
1

Hot on the heels of 2.5.8, Rhasspy is now at 2.5.9.

The biggest change in this version is an upgrade of Mozilla’s DeepSpeech to version 0.9. And, thanks to the Jaco Polyglot, there are now DeepSpeech profiles for English, German, French, Spanish, Italian, and Polish. Additionally, DeepSpeech supports audio streaming like Kaldi, so it will be decoding as you speak :slight_smile:

After watching @itsMattShull on Twitch, I decided to make some web UI improvements for folks that are new to Rhasspy (apparently most people on Earth). I’ve tried to make it clearer when restarts are required, and when certain buttons on the home page won’t work – e.g., “Speak” does nothing if you have no text to speech system. I also made it so you can try different voices for some of the text to speech systems directly in the Settings page instead of save/restart each time.

Noisy environments can be a problem for Rhasspy :speak_no_evil:, since silence is needed to end a voice command. To help, I’ve added settings for energy-based silence detection (found in the speech to text settings). This lets you use different methods besides just VAD for detecting silence, such as the current debiased audio energy of the signal or a ratio of it and the maximum observed energy (or a fixed value). The rhasspy-silence project implements all this, and provides a command-line tool to check energy levels.

Lastly, if you didn’t see it, I made a wake word creator for snowboy that you can run on your desktop and create .pmdl files for your Pi’s. While porcupine is better in accuracy (and now at v1.9), custom wake words still expire.

Thanks, everyone! Here’s the complete changelog:

Added

  • Add DeepSpeech v0.9 profiles for English, German, French, Spanish, Italian, and Polish (Jaco)
  • Add streaming audio support for DeepSpeech (faster transcription)
  • Settings for energy-based silence detection
  • Max seconds for voice commands
  • English voice for Larynx (kathleen)
  • Reboot/shutdown menu in web UI
  • Add text to speech testing tools in settings page
  • Make it clearer in web UI when restarts are required
  • _site_id meta slot to Home Assistant intents/events (bk90)

Fixed

  • Custom converters for fsticuffs and fuzzywuzzy (thanks @arpagor62970)
  • fuzzywuzzy NluException: not enough values to unpack
  • Download links for all profiles

Changed

  • Upgrade to Mozilla DeepSpeech v0.9
  • Upgrade porcupine wake word system to 1.9
  • Move OpenAPI page from /api/ to /openapi/
  • Improved web UI for Raven keywords
14 Likes
2

Thanks for the update. The new Raven gui looks way better than the old one, but I managed to run into problems with it already

1 Like
3

Thanks, @Daenara. I’ll try and get this fixed as soon as possible. The timing of when the directory is scanned is part of my problem here :frowning:

4

Maybe try scanning directly after rhasspy starts/restarts if raven is the selected wakeword system. Since you need a restart to activate it that should help. Then, if stuff still doesn’t show up after changes (because you deleted a wakeword while using the system) just add a note to the gui to restart rhasspy if changes don’t appear automatically.

5

I’m not sure, if I had something to do with it ;).

6

It Seems that the deb package isnt updated. I installed the 2,5.9.deb and the Webinterface Said it is 2.5.8.

Please fix it :grin:

7

@synesthesiam thanks for the update! When going through the steps and running the follow run command:

docker run -it -p 12101:12101 --name rhasspy --restart unless-stopped -v “$HOME/.config/rhasspy/profiles:/profiles” -v “/etc/localtime:/etc/localtime:ro” --network=“host” --device /dev/snd:/dev/snd rhasspy/rhasspy --user-profiles /profiles --profile en

I end up getting an error but can’t figure out what’s going on:
> WARNING: Published ports are discarded when using host network mode

Starting up...
Using virtual environment at /usr/lib/rhasspy/.venv
DEBUG:rhasspysupervisor:Namespace(debug=True, docker_compose='', local_mqtt_port=12183, mosquitto_path='mosquitto', profile='en', supervisord_conf='supervisord.conf', system_profiles=None, user_profiles=PosixPath('/profiles'))
DEBUG:rhasspysupervisor:Loading profile en (user=/profiles, system=None)
DEBUG:rhasspyprofile.profile:Loading /usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/profiles/en/profile.json
DEBUG:rhasspyprofile.profile:Loading /profiles/en/profile.json
DEBUG:rhasspysupervisor:Generating supervisord conf
DEBUG:rhasspysupervisor:Intent recognition disabled (system=dummy)
DEBUG:rhasspysupervisor:Intent handling disabled (system=dummy)
DEBUG:rhasspysupervisor:Wrote /profiles/en/supervisord.conf
Starting services...
Using virtual environment at /usr/lib/rhasspy/.venv
Using virtual environment at /usr/lib/rhasspy/.venv
2021-01-16 11:19:42,601 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2021-01-16 11:19:42,611 INFO supervisord started with pid 173
2021-01-16 11:19:43,620 INFO spawned: 'dialogue' with pid 189
2021-01-16 11:19:43,638 INFO spawned: 'speech_to_text' with pid 190
2021-01-16 11:19:43,648 INFO spawned: 'microphone' with pid 191
2021-01-16 11:19:43,660 INFO spawned: 'text_to_speech' with pid 194
2021-01-16 11:19:43,668 INFO spawned: 'wake_word' with pid 195
2021-01-16 11:19:43,697 INFO spawned: 'speakers' with pid 200
2021-01-16 11:19:44,701 INFO success: dialogue entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-16 11:19:44,702 INFO success: speech_to_text entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-16 11:19:44,703 INFO success: microphone entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-16 11:19:44,705 INFO success: text_to_speech entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-16 11:19:44,706 INFO success: wake_word entered RUNNING state, process has stayed 
up for > than 1 seconds (startsecs)
2021-01-16 11:19:44,708 INFO success: speakers entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[DEBUG:2021-01-16 11:19:45,949] rhasspydialogue_hermes: Namespace(debug=True, group_separator=None, host='localhost', log_format='[%(levelname)s:%(asctime)s] %(name)s: %(message)s', no_sound=None, password=None, port=12183, session_timeout=30.0, site_id=['default'], sound=[['wake', '/usr/lib/rhasspy/etc/wav/beep_hi.wav'], ['recorded', '/usr/lib/rhasspy/etc/wav/beep_lo.wav'], ['error', '/usr/lib/rhasspy/etc/wav/beep_error.wav']], tls=False, tls_ca_certs=None, tls_cert_reqs='CERT_REQUIRED', tls_certfile=None, tls_ciphers=None, tls_keyfile=None, tls_version=None, username=None, volume=0.35, wakeword_id=None)
[DEBUG:2021-01-16 11:19:45,953] asyncio: Using selector: EpollSelector
[DEBUG:2021-01-16 11:19:45,965] rhasspydialogue_hermes: Connecting to localhost:12183     
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/rhasspy/rhasspy-dialogue-hermes/rhasspydialogue_hermes/__main__.py", line 89, in <module>
    main()
  File "/usr/lib/rhasspy/rhasspy-dialogue-hermes/rhasspydialogue_hermes/__main__.py", line 73, in main
    hermes_cli.connect(client, args)
  File "/usr/lib/rhasspy/rhasspy-hermes/rhasspyhermes/cli.py", line 93, in connect        
    client.connect(args.host, args.port)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/paho/mqtt/client.py", line 937, in connect
    return self.reconnect()
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/paho/mqtt/client.py", line 1071, in reconnect
    sock = self._create_socket_connection()
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/paho/mqtt/client.py", line 3522, in _create_socket_connection
    return socket.create_connection(addr, source_address=source, timeout=self._keepalive) 
  File "/usr/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
[DEBUG:2021-01-16 11:19:46,254] rhasspyspeakers_cli_hermes: Namespace(debug=True, host='localhost', list_command='aplay -L', log_format='[%(levelname)s:%(asctime)s] %(name)s: %(message)s', password=None, play_command='aplay -q -t wav -D default:CARD=seeed2micvoicec', port=12183, site_id=['default'], tls=False, tls_ca_certs=None, tls_cert_reqs='CERT_REQUIRED', tls_certfile=None, tls_ciphers=None, tls_keyfile=None, tls_version=None, username=None, 
volume=0.5)
[DEBUG:2021-01-16 11:19:46,261] asyncio: Using selector: EpollSelector
[DEBUG:2021-01-16 11:19:46,275] rhasspyspeakers_cli_hermes: Connecting to localhost:12183
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/rhasspy/rhasspy-speakers-cli-hermes/rhasspyspeakers_cli_hermes/__main__.py", line 69, in <module>
    main()
  File "/usr/lib/rhasspy/rhasspy-speakers-cli-hermes/rhasspyspeakers_cli_hermes/__main__.py", line 53, in main
    hermes_cli.connect(client, args)
  File "/usr/lib/rhasspy/rhasspy-hermes/rhasspyhermes/cli.py", line 93, in connect        
    client.connect(args.host, args.port)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/paho/mqtt/client.py", line 937, in connect
    return self.reconnect()
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/paho/mqtt/client.py", line 1071, in reconnect
    sock = self._create_socket_connection()
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/paho/mqtt/client.py", line 3522, in _create_socket_connection
    return socket.create_connection(addr, source_address=source, timeout=self._keepalive) 
  File "/usr/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

At first I thought it was because the MQTT server was still running on port 12183. So I stop it and tried again but it didn’t work. Any thoughts?

8

Opps, my mistake! Fixed the username.

2 Likes
9

I get this error on nuc

Starting up...
Using virtual environment at /usr/lib/rhasspy/.venv
DEBUG:rhasspysupervisor:Namespace(debug=True, docker_compose='', local_mqtt_port=12183, mosquitto_path='mosquitto', profile='en', supervisord_conf='supervisord.conf', system_profiles=None, user_profiles=PosixPath('/profiles'))
DEBUG:rhasspysupervisor:Loading profile en (user=/profiles, system=None)
DEBUG:rhasspyprofile.profile:Loading /usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/profiles/en/profile.json
DEBUG:rhasspyprofile.profile:Loading /profiles/en/profile.json
DEBUG:rhasspysupervisor:Generating supervisord conf
DEBUG:rhasspysupervisor:Microphone disabled (system=dummy)
DEBUG:rhasspysupervisor:Speakers disabled (system=dummy)
DEBUG:rhasspysupervisor:Wake word disabled (system=dummy)
DEBUG:rhasspysupervisor:Intent handling disabled (system=dummy)
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/rhasspy/rhasspy-supervisor/rhasspysupervisor/__main__.py", line 109, in <module>
    main()
  File "/usr/lib/rhasspy/rhasspy-supervisor/rhasspysupervisor/__main__.py", line 89, in main
    mosquitto_path=args.mosquitto_path,
  File "/usr/lib/rhasspy/rhasspy-supervisor/rhasspysupervisor/__init__.py", line 183, in profile_to_conf
    mqtt_password=mqtt_password,
  File "/usr/lib/rhasspy/rhasspy-supervisor/rhasspysupervisor/__init__.py", line 2545, in print_text_to_speech
    mqtt_password,
  File "/usr/lib/rhasspy/rhasspy-supervisor/rhasspysupervisor/__init__.py", line 2094, in get_text_to_speech
    shlex.quote(" ".join(str(v) for v in picotts_command)),
UnboundLocalError: local variable 'picotts_command' referenced before assignment

and also this

TtsException: [Errno 2] No such file or directory: 'pico2wave': 'pico2wave'

10

Do we pull the Docker image for the update of the username? Sorry for the questions, first time updating.

11

OK, got it. Thank you! For now, change “picotts” in your profile to “nanotts”. It’s the same underlying program, but maintained on Github.

12

No, I just meant I edited the post with the username of the person who realized that custom converters weren’t working :slight_smile:

You’ll see this warning anytime you have --network='host' and a -p <port>:<port> (not used because it’s just attached directly to the host network).

I’m guessing you’re using an external MQTT broker on 12183? Just start it up again and everything should be fine.

13

Can you try doing a full refresh in the browser with CTRL + R? Sometimes the version “sticks” until you clear the cache.

14

Hello, I’m trying to install using git version (on aarch64/Raspberry Pi).
When i run make install, I get the following error, :

Requirement already satisfied: protobuf>=3.8.0 in /root/sources/rhasspy/.venv/lib/python3.7/site-packages (from tensorboardX->TTS==0.0.8) (3.14.0)
Building wheels for collected packages: TTS
  Building wheel for TTS (setup.py) ... done
  Created wheel for TTS: filename=TTS-0.0.8-cp37-cp37m-linux_aarch64.whl size=138016 sha256=30f57db265bc4a7b3a126dfbd4fab04fa2411441a2baf714da0e04c70f1676ff
  Stored in directory: /tmp/pip-ephem-wheel-cache-9cig2ipw/wheels/79/84/c4/0c58500327ad3f7465e601c0360ce979c8fcaa0cb4e69664e5
Successfully built TTS
Installing collected packages: TTS
Successfully installed TTS-0.0.8
Compiling core.pyx because it changed.
[1/1] Cythonizing core.pyx
/root/sources/rhasspy/.venv/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /root/sources/rhasspy/rhasspy-tts-larynx-hermes/larynx/TTS/TTS/tts/layers/glow_tts/monotonic_align/core.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
running install
running build
running build_ext
building 'TTS.tts.layers.glow_tts.monotonic_align.core' extension
creating build
creating build/temp.linux-aarch64-3.7
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/root/sources/rhasspy/.venv/lib/python3.7/site-packages/numpy/core/include -I/root/sources/rhasspy/.venv/include -I/usr/include/python3.7m -c core.c -o build/temp.linux-aarch64-3.7/core.o
In file included from /root/sources/rhasspy/.venv/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822,
                 from /root/sources/rhasspy/.venv/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /root/sources/rhasspy/.venv/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from core.c:623:
/root/sources/rhasspy/.venv/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it with " \
  ^~~~~~~
creating build/lib.linux-aarch64-3.7
creating build/lib.linux-aarch64-3.7/TTS
creating build/lib.linux-aarch64-3.7/TTS/tts
creating build/lib.linux-aarch64-3.7/TTS/tts/layers
creating build/lib.linux-aarch64-3.7/TTS/tts/layers/glow_tts
creating build/lib.linux-aarch64-3.7/TTS/tts/layers/glow_tts/monotonic_align
aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.7/core.o -o build/lib.linux-aarch64-3.7/TTS/tts/layers/glow_tts/monotonic_align/core.cpython-37m-aarch64-linux-gnu.so
running install_lib
copying build/lib.linux-aarch64-3.7/TTS/tts/layers/glow_tts/monotonic_align/core.cpython-37m-aarch64-linux-gnu.so -> /root/sources/rhasspy/.venv/lib/python3.7/site-packages/TTS/tts/layers/glow_tts/monotonic_align
running install_egg_info
Writing /root/sources/rhasspy/.venv/lib/python3.7/site-packages/monotonic_align-0.0.0.egg-info
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: flask==1.1.2 in /root/sources/rhasspy/.venv/lib/python3.7/site-packages (from -r requirements.txt (line 1)) (1.1.2)
ERROR: Could not find a version that satisfies the requirement gruut~=0.5.0
ERROR: No matching distribution found for gruut~=0.5.0
make: *** [Makefile:177: install-rhasspy] Error 1
15

Hmmm…the Makefile is supposed to download gruut; not sure why it’s failing. Here’s where you can download it: https://github.com/rhasspy/gruut/releases

Do you see anything in your download directory in the root of the git repo?

16

Yes
rhasspy# find . -iname “gruut”
./download/gruut-0.5.0.tar.gz

17

I thought it started a MQTT broker when the Docker image was run? I guess I didn’t have to do anything else last time. That’s why I’m confused why it’s not starting the MQTT broker on 12183.

18

The warning I’m good with, it’s the connection errors in the log I’m referring too.

19

Those are fine as long as it eventually starts up. The MQTT broker starts first, but isn’t ready by the time other services get started. They’re all programed to try connecting several times with a delay before giving up.

20

Alright got it. Kept restarting the container and it didn’t work. Then I rebooted and it came up fine!

1 Like
21

Did full refresh and cleared Cache (Tested Others Browser) Same Result

1 Like
22

Any idea why do I get this error when updating on nuc?

23

Thank you very much for implementing :smiley:

1 Like
24

Yeah, I made a mistake with picotts. I’ll push an update today with a fix, or you can switch to “nanotts” in your profile.

You’re welcome :slight_smile:

25

Thank you very much for the new version. I want to try DeepSpeech, but unfortunately the download (german) doesn’t work.

26

The error is in the update process.
I can’t update on nuc from 2.5.8 to 2.5.9.

27

OK, I’ve made a small update to 2.5.9 with a fix for the picotts problem.

You’re welcome :slight_smile:

I haven’t been able to re-create this problem. Can you share your log, please?

28

[ERROR:2021-01-17 21:24:15,817] rhasspyserver_hermes: (‘https://raw.githubusercontent.com/synesthesiam/synesthesiam/de_deepspeech-aashishag/7aadb0561d37ada67c9353cf943eb66f06ec6636/model/alphabet.txt’, ‘File size mismatch (got 14 byte(s), expected 339)’)
Traceback (most recent call last):
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 673, in api_download_profile
ssl_context=ssl_context,
File “/usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/download.py”, line 494, in download_files
f"File size mismatch (got {final_size} byte(s), expected {bytes_expected})",
rhasspyprofile.download.DownloadFailedException: (‘https://raw.githubusercontent.com/synesthesiam/synesthesiam/de_deepspeech-aashishag/7aadb0561d37ada67c9353cf943eb66f06ec6636/model/alphabet.txt’, ‘File size mismatch (got 14 byte(s), expected 339)’)
[ERROR:2021-01-17 21:24:15,816] rhasspyprofile.download: Download failure (https://raw.githubusercontent.com/synesthesiam/synesthesiam/de_deepspeech-aashishag/7aadb0561d37ada67c9353cf943eb66f06ec6636/model/alphabet.txt, got 14 byte(s), expected 339)
[DEBUG:2021-01-17 21:24:15,552] rhasspyprofile.download: Downloading https://raw.githubusercontent.com/synesthesiam/synesthesiam/de_deepspeech-aashishag/7aadb0561d37ada67c9353cf943eb66f06ec6636/model/alphabet.txt to /profiles/de/deepspeech/model/0.9/alphabet.txt

29

Hi @Thargor, can you make sure your download.url_base in your profile is set to "https://raw.githubusercontent.com/"

It looks like an extra synesthesiam/ is being added to the URL.

30

Yes you are right - now it is working. But I am pretty sure, that I didn’t touch anything in the profile …
Thank you very much!

31

Hey!

Thanks to the update. a lot of cool stuff in there and the new raven interface looks awesome.

I am still kinda new to rhasspy, coming from Snips. I started everything from 2.5.8 so this is my first update I have to roll out. I am using four rpi0-satellites and a base. Rolling out a new update is kinda painful (each sat takes about 20-30 minutes) so I thought if there is a simple update script I can put on every pi and trigger it somehow. Is something like this possible?

32

Seems another awesome update !

Will update and check all that. Of course my testing will be background music oriented !!

1 Like
33

Hi

Does the deb package for 2.5.9 support Ubuntu 20.10? There seems to be a dependency on llvm-7-runtime when I try to install it, but Ubuntu 20.10 only has llvm-8-runtime and above.

There was mention of creating appImages when 2.5.8 was released, are these available yet ?

Kind regards
Dom

34

I’ll see if I can bump to LLVM 8 without breaking anything. Not sure why Ubuntu dropped 7.

Not officially yet, no. What hardware are you running Rhasspy on?

35

Hi.

I’m using home assistant and have rhasspy installed as add-on. I received the notification of rhasppy 2.9 and updated my addon. But still on the webpage (on :12101) the old version 2.5.7 (for some reason I have not made the update to 2.5.8) is showing up.
I checked the docker image version and it seems to be correct
3fbb1a43/amd64-addon-rhasspy 2.5.9 ed36febeb00e 29 hours ago 1.64GB

and also the container uses the correct image:
243d60d029ae 3fbb1a43/amd64-addon-rhasspy:2.5.9 “/run.sh” 29 hours ago Up 29 hours 0.0.0.0:12101->12101/tcp, 0.0.0.0:12333->12333/udp addon_3fbb1a43_rhasspy

I already tried uninstalling the addon and installing it again, but still 2.5.7 is in the header (of course I also made a refresh without cache)

am I missing something for the update? or does anybody has a hint want I can try in addition?

Regards,
Stefan

36

I tried DeepSpeach (german) instead of Kaldi in my Server Satellite Environment (both running the official docker container), but can’t geht it to work. It doesn’t seem to react. If I switch back to Kaldi, everything work again. I will try it again …

In my installation there are two lines “Satellite siteIds:…” for DeepSpeach. Is that a bug?

37

I am having the same experience. When using German Deepspeech my Server simply gives back a TimeOut. Same behaviour as I had on 2.5.8.

38

Anyone have info on how to use current_energy_threshold, max_current_energy_ratio_threshold, and max_energy? Are these helpful in silence detection?

39

I’m hoping they will be, but I’m not really an audio guy so I’m just going off of what I’ve read. To make use of these for silence detection, you will need to get the audio statistics working in the Rhasspy web UI or get the rhasspy-silence command-line tool running.

Here’s a brief overview for everyone:

  • Audio “energy” is computed right here. This was borrowed from the speech_recognition library.
  • current_energy_threshold just means that the energy of an audio chunk is compared to some threshold and, if it’s lower, the audio chunk is considered silence.
  • max_current_energy_ratio_threshold means that a ratio is computed for every audio chunk (max / current), and the chunk is considered silence if the value is above the threshold.
    • To make it intuitive, imagine the threshold ratio is 1. This means that if the current energy is ever less than the max energy, we have silence (max/current > 1). If we halve the threshold to 0.5, it means that anything quieter than half the max will be silence.
    • You can set max_energy to a specific value or let Rhasspy dynamically set it over time.

Hope this helps some.

2 Likes
40

I have been wondering for a while if a neural network VAD would be a better option if the KWS was provided inhouse.
A considerable amount of the load is the conversion of audio to a MFCC image and a streaming model KWS just splits the images into ‘strides’ of 20msec or more.
I have been thinking a neural VAD could work on the strides and pass to KWS without any additional MFCC processing and the model would be quite lite due to only working on a stride at a time.

I am thinking a more accurate VAD could be provided with little more overhead than current could even be less as those strides would already be used by a KWS.
Haven’t looked in ages but the VAD currently has very similar FFT routines to the MFCC of some KWS and at times it isn’t great at judging what silence is.
If a VAD was trained with users voice then its likely it could be extremely accurate at detecting when the user is speaking or not which is probably a better metric than what we consider silence.

2 Likes
41

I was looking into RNNoise yesterday, and it might fit the bill here. Not only does it remove noise, it also does VAD. The only downside appears to be that the model was trained on 48Khz audio, so we’d need to convert.

42

Yeah RNNoise is prob good there are some alsa wrappers that for me seemed like voodoo in terms of implemented VAD threshold.
Again though it might be better to train a 16k model?
Also RNNoise was one of the first but a github trawl will uncover a few.

The FFT processing to create a spectrogram or MFCC does create load and like Precise a more modern streaming Neural KWS is fed MFCC in strides so that a simple low latency CNN can work on each stride feature and have a VAD response time of 20ms or whatever the streaming stride length is.
My point was to use the audio analysis data we already have that we have already worked. RNNoise would be doing the same parallel tasks of creating a spectrogram/mfcc stream for RNNoise and just adding different or more modules is an increase in load.

The VAD we have isn’t that great and because its a separate module even if much of its input processing could be shared its not because its installed as a prebuilt separate lib.

The best way would be to modify RNNoise so that it passes the MFCC/Spectrogram as an output but would be far faster to keep in memory and go straight to KWS, meaning really be part of the KWS and not bolt on load.
At the stage of VAD/RNNoise its possible to do much such as Diarisation and model switching because the load is much less as the process from wav to audio image is done once.
This is not true of adding each function as completely separate libs and processes that just branch off the audio chain.

Also if you use RNNoise then its likely you would have to retrain your models to gain the advantages of noise reduction and garner accuracy with the slightly different spectrograph it will give.

I used https://github.com/werman/noise-suppression-for-voice and wasn’t at all sure if the VAD was working as it should from the Alsa settings I had provided.
Simple cheap USB Microphone / Soundcard

Took me ages to find how to get it to work and was never sure if vad was ok but rnoise itself if implemented should

  type ladspa
  slave.pcm {
    # Convert from float to int
    type lfloat
1 Like
43

Hi @synesthesiam

Do we actually need the llvm runtime? The reason I ask is that if I force install the deb package to ignore the dependency then everything still seems to work. Admittedly I haven’t tried every configuration supported by Rhasspy, but voice capture & sentence matching works but I’m not using a wakeword yet.

I’m running Rhasspy on an x86 laptop

Kind Regards
Dom

44

Its needed by Larynx (text to speech), but it may be a good idea to just move it to a suggested dependency and make a note in the documentation.

2 Likes
45

RNNoise looks really promising!

1 Like
46

To be honest Rnnoise will force all KWS & ASR models to be retrained with the datasets of being passed through Rnnoise as it passes but a audio stream.
From having a reread it has to be 48Khz or the filters provide do not work well if at all at lower SR, so as well as resample to 16k the MFCC process needs to begin again but maybe that can be passed through (I would have to look at code again for a while).

Thing is from last time I tested it isn’t that great as RTXVoice it is not as it really falls down on relatively modest levels of noise and creates a shed led of audio artefacts.
Its like RTXVoice as run on a 20 or 30 series card its supposedly pretty mindblowing even on lesser cards with fast system with AVX-512 it still does a good job but supposedly not as good as the cards with tensor cores running as AI accelerators that they use in their Deep Learning Super Sampling (DLSS) technology.

Its very much about horsepower and if you try RNNoise and you have tried RTXvoice it also does position results correctly as there is no comparison really with that amount of horsepower and a raspberry pi.

There is no way to extract noise without highspeed silicon DSP or some pretty heavyweight hardware without artefacts that create just as much noise as your extracting and that is just plain fact.
Google & Amazon just have better and more complex ASR models that cope better with noise.

Just run the link I sent as you can set up RNNoise via alsa pretty quick.
The sample of no RNNoise
https://drive.google.com/file/d/1pIH5O_TP6YoNrp9ql2rr_t5LmnpMB9QE/view
Vs RNNoise was on a Pi4 @ 2.0Ghz
https://drive.google.com/file/d/1_Qr-XaaaxEy-nQeiS8GaTVCWCd1egde_/view

I didn’t really test much more than the above but maybe you can get it to work better with higher levels of noise.
That was just a fan heater with my usual Harry Potter recording which was fairly minimal noise and its already starting to artefact the audio.

I think you might find the SNR that RNNoise can cope with is actually lower than the SNR your models can cope with anyway.
The problem might be because the models being used are too ‘clean’ and can be increased by noise sample addition anyway.
But you will have to test.

If you have an input signal of what is noise that AEC uses to subtract from mic in then you can run on much lower hardware but how or what is noise is the problem and its needs to be strictly synced without clock drift or you need horsepower to do the adaptive syncing.
But capturing known noise and running filtering such as AEC could be possible on low end hardware.

The easiest and cheapest way to deal with noise is by positioning a mic so voice=near & noise=far so distributed wide array microphones even with networked arrays of 2x, 4x, 8x or above are a fraction of the cost of where cutting edge noise reduction is.

@synesthesiam give it a go but you can prob tell I think its likely a waste of your time.

You could employ a much better VAD though.


https://github.com/hcmlab/vadnet (blingdows but how is still worth a look)
https://github.com/nicklashansen/voice-activity-detection
https://github.com/Cocoxili/VAD
https://github.com/filippogiruzzi/voice_activity_detection
https://github.com/MarcoZaror/VAD_system

Tensorflow VAD is like tensorflow KWS its not all that hard the cutting edge is tensorflow itself and its just a matter of implementing the framework.

1 Like
47

Hi,

Big support for that request ! Especially for embedded systems like raspberry, it is very important to minimize the dependencies.

As a workaround, install with dkpg --ignore-dependencies and remove it in /var/lib/dpkg/status.

1 Like
48

Hi,

thank you very much for this update!
All works fine in a docker installation for my base (i386) and one sattelite (pi3):

  • external mqtt
  • arecord
  • raven
  • pocketsphinx
  • fuzzywuzzy
  • picotTTS
  • aplay
  • rhasspy
1 Like
49

I am trying to install rhasspy in WSL2 following the directions at your web page.

As suggested I picked Ubuntu 20.04 as linux choise and intalled it following the directions.

However, I keep getting the ERROR messege as the one below:

ERROR: torch-1.6.0-cp37-cp37m-linux_x86_64.whl is not a supported wheel on this platform.
make: *** [Makefile:173: install-rhasspy] Error 1

When thrying under Debian I get these Errors:
ERROR: pip’s dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pvporcupine 1.9.0 requires enum34, which is not installed.
rhasspy 2.5.8 requires pylint==2.4.4, but you have pylint 2.5.3 which is incompatible.

Downloading Flask_Cors-3.0.9-py2.py3-none-any.whl (14 kB)
ERROR: Could not find a version that satisfies the requirement gruut~=0.5.0
ERROR: No matching distribution found for gruut~=0.5.0
make: *** [Makefile:177: install-rhasspy] Error 1

What do I do wrong? Please Help

50

Somehow it does not download this directory. But I did downloaded it using

git clone --recursive https://github.com/rhasspy/gruut

and I also did

sudo apt-get install python-enum34

To eliminate the scource of the first error. Than
./configure --enable-in-place
make
make install

And same result again - all same errors. Why?

51

Hi @synesthesiam

I was in 2.4.19 and I decided yesterday evening to install the 2.5.9 version through docker.
I’m on PI3+Jabra 410 and I’am surprised by the very good response time and I would like thanks to the adding of a change request I proposed a lot months ago (reboot, stop button in IHM).

Integration with the excellent plugin jeeRhasspy from @KiboOst also well done

Thank’s again for the great works

3 Likes
52

You may need to download the gruut release tarball: https://github.com/rhasspy/gruut/releases or disable Larynx with ./configure ... --disable-larynx

What happened was I exceeded the size limit for upload to PyPi with the most recent version of gruut, so you can’t currently pip install it (unless you do pip install https://github.com/rhasspy/gruut/releases/download/v0.5.0/gruut-0.5.0.tar.gz)

I need to refactor gruut so the language-specific pieces can be downloaded separately.

53

You’re welcome :slight_smile:

Always glad to hear success stories :+1: