Preliminary Release of Rhasspy 2.5.10

Announcements
1

Hi everyone! This is a “preliminary” release of 2.5.10, meaning I’ve created a rhasspy/rhasspy:2.5.10 Docker tag and uploaded new Debian packages, but I haven’t made these the “latest” release yet (the docs are not updated yet either). Due to time constraints, I haven’t been able to run all my usual tests; but I wanted to get something out to everyone :slight_smile:

There are some big additions in 2.5.10, such as:

  • ASR support for Swedish (sv)
  • New Larynx that’s faster and has a ton of new voices
    • Hint: switch to “Low Quality” on a Pi 4 or below for a big speed-up
    • This version should also work with older x86_64 CPUs (no AVX)
  • Kaldi ASR now has confidence value for words and sentences
  • Dialogue manager now has a minimum ASR confidence threshold (speech_to_text.<system>.min_confidence where <system> is kaldi, deepspeech, etc.)

I’d appreciate any testing and feedback that the community can offer! Thanks :+1:

Added

  • New version of Larynx with improved performance and 35 voices (20 English, 1 German, 3 French, 2 Spanish, 3 Dutch, 2 Italian, 1 Swedish, 3 Russian)
  • Kaldi ASR model for Swedish (sv)
  • Confidence and word timings for Kaldi ASR
  • Minimum ASR confidence threshold for dialogue manager
  • Detect AVX support and warn for Larynx, DeepSpeech, and Precise in Web UI
  • Handle spaces in converter arguments with word!(converter, …)
  • rhasspy-tts-cli-hermes TTS commands may be Jinja2 templates (–use-jinja2)
  • Support for MaryTTS effects (jasonhildebrand)
  • customData added to hermes/nlu/query message
  • customData is copied by NLU services from query to intent/intentNotRecognized
  • lang property added for wake, speech_to_text, and intent profile sections
  • Wake, ASR, NLU services all set lang properties if null

Fixed

  • Remote HTTP service sets site_id of satellite for ASR/NLU endpoints
  • DeepSpeech token output (was letters, now words)
  • Multiple values in custom converters are sent as a list on stdin
  • Don’t show restart/shutdown button if “sudo” isn’t available (Docker, Hass.io)
  • Added missing espeak phonemes for some profiles
  • MaryTTS voice test in Web UI
  • Remove dialogue session from site cache on end
  • Don’t throw error about system not configured if message is intent for satellite (schnopsi)

Changed

  • /api/listen-for-command uses a proper wake workflow now (requires dialogue manager)
  • Show absolute paths for custom models (precise, snowboy, porcupine) in Web UI
9 Likes
2

good work, I wiil try it out.
My focus will be the Dutch languages

1 Like
3

Thank you, just tried it with a manual update on my HomeAssistant Addon.

First thing I noticed is that I am getting an error regarding WaveNet:

 File "/usr/lib/rhasspy/rhasspy-tts-wavenet-hermes/rhasspytts_wavenet_hermes/__init__.py", line 15, in <module>
    from google.cloud import texttospeech
ModuleNotFoundError: No module named 'google.cloud'

Everything else seems to work. I am currently playing around with the Kaldi Confidence Scores in German. Very nice to have this. Although I realized that they vary widely even with the same sentence being spoken the same way. Is it possible to also see the confidence of the single words? Currently the only thing I found that changes is the likelihood of the full utterance in the ASR/textCaptured

1 Like
4

Nice ! As soon as I get home, I’ll will test it.

1 Like
5

I have only just started experimenting with rhasspy to extend my home automation system and had just got the 2.5.9 release working well with a pair of Pi4Bs. I had noticed that Larynx gave the best sounding TTS output but that it was horribly slow (around 15 seconds to generate audio for a “The time is…” sentence, so I was really looking forward to giving this a try!

Both Pi machines are running rhasspy as docker images and I use the MQTT server which runs on my Home Assistant server for both HA (also on docker) and rhasspy.

Anyway, I updated my docker installations with 2.5.10 and am glad to report that the increase in speed for Larynx is considerable. The first sentence took 5 seconds to deliver, but subsequent delivery is almost immediate.

Nice one!

Next to test: ASR confidence values and handling.

1 Like
6

I get this as well, I switched to Larynx now because I want to test it

7

On my NUC J5005 the Larynx component works well, altough even with low quality mode it takes several seconds to create the sound file. I would be interested to find out about what is required to create another german voice. Are there some predefinded sentences to record to help in creating a new voice option?

8

Is that just the 1st sentence as above as the NuC should have far more Ooomf than a pi4?

9

Just tested it again. And yes the first sentence takes several seconds. The following sentences about 1 second. High Quality adds 0.5 to 1 second to that. But i dont really hear a difference in it anyway.

10

It prob should say hi as it initialises on boot :slight_smile:

1 Like
11

This may need some tuning. I’m using the Minimum Bayes Risk from Kaldi, but I’m not entirely sure the best way to report it.

I must have gotten interrupted implementing this. The confidences are produced during transcription, they’re just not being passed up the layers into textCaptured or the NLU intent. I’ve created a bug report here to remember: https://github.com/rhasspy/rhasspy/issues/207

Ah, thank you. I was debugging a problem that ended up being with pip and forgot to turn this back on.

Yes! Thanks to volunteers like @RaspiManu, we have a set of German phrases to read :slight_smile: Anyone who’s interested, please PM me and I’ll send you a link.

Larynx delays loading the TTS/vocoder models until it’s called the first time, so this is what you’re seeing. I might be able to add an option to preload the voice if this is an issue for people.

12

I tried to update Rhasspy on my raspberry pi 3 and it takes a very long time to install dependencies with pip.
When running /home/pi/rhasspy/.venv/bin/python -m pip install "/home/pi/rhasspy" I have this warning displayed:

INFO: pip is looking at multiple versions of importlib-metadata to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
        INFO: pip is looking at multiple versions of hyperframe to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of hpack to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of h2 to determine which version is compatible with other requirements. This could take a while.

One core of the raspberry is at 100% and it takes hours to find the right version.
It’s probably a pip problem more than a rhasspy one, but maybe it should be possible to fix the version in rhasspy requirement file.

On the venv, it’s pip 21.0.1 and python 3.7 .

13

Right now, pip is still running, that’s so slow :sob: I started the installation 3 hours ago :expressionless:

14

You need to force the pip version to <= 20.2.4 for it to work now. They completely ruined pip with the new dependency resolver. I can’t get anything to install now if it has more than one dependency.

If you’re installing from source, try exporting PIP_VERSION="pip<=20.2.4” before make install.

1 Like
15

Thanks, with pip 20.2.4 it’s far faster.
However, I have an error:

ERROR: Could not find a version that satisfies the requirement onnxruntime~=1.6.0 (from larynx~=0.3.0->rhasspy==2.5.10) (from versions: none)
16

https://github.com/synesthesiam/prebuilt-apps/releases/download/v1.0/onnxruntime-1.6.0-cp37-cp37m-linux_armv7l.whl

Microsoft only has pre-compiled wheels for 64-bit ARM for some reason.

1 Like
17

Is it easy to do a manual update with HASSIO? I’ll certainly join in testing if I can figure out how to update my HASSIO 2.5.9 version. Thanks

18

It is. I think romkabouter was giving me this tip in another thread.
Basically you just copy the hassio addon repository to your addons/local folder.
Then in the dockerfile you put

FROM rhasspy/rhasspy:2.5.10

And install :slight_smile:

1 Like
19

Thanks. I’ll give it a go.

20

Because like tensorflow and all tensor based math the 2-3x speed increase of 64bit means that 32bit is now aimed at only for microcontrollers as it really is 2-3x at least with tensorflow but presume Onnx is very similar.
The Neon SIMD is highly optimised with all NN engines and with Armv8 the 128 Neon register to float math means in real terms 2-3x perf increase which is absolutely huge so they don’t see armv7 as viable or at least worth much mention as why would you?

I did some benchmarks with the exact same just PiOS64v32 and depending on the model like vs like of 64v32 the perf increase is 2-3x with the wider databus.

1 Like
21

I’m not quite sure that confidence levels are being fully honoured. Here’s an example.
I just came home from collecting a package and my daughter asked what was in the box. I replied “It’s another small computer”. Even though my microphone is in the living room, some six odd metres away from the front door, my wake word (computer) was detected.

As the ASR system does, it guessed what the following sounds might mean and decided that the result was good enough to send the perceived intent off to Home Assistant! In the log, I can see the following result:

[DEBUG:2021-04-06 15:22:41,309] rhasspyasr_kaldi_hermes: Transcription result: Transcription(text='toggle kodi', likelihood=0, transcribe_seconds=0.3336428640031954, wav_seconds=1.024, tokens=[TranscriptionToken(token='toggle', start_time=0.0, end_time=0.0, likelihood=0.425189), TranscriptionToken(token='kodi', start_time=0.0, end_time=1.02, likelihood=0.675266)])

When I then actually said “toggle kodi” to turn my TV off again, the result was:

[DEBUG:2021-04-06 15:28:38,781] rhasspyasr_kaldi_hermes: Transcription result: Transcription(text='toggle kodi', likelihood=0.99822817, transcribe_seconds=1.3164074919986888, wav_seconds=1.856, tokens=[TranscriptionToken(token='toggle', start_time=0.0, end_time=0.81, likelihood=1.0), TranscriptionToken(token='kodi', start_time=0.81, end_time=1.86, likelihood=1.0)])

Even though the first overall likelihood was zero, it seems that the likelihood from the individual words was what actually determined the final score.

What I can tweak to ensure random nonsense does not get turned into a valid intent?

22

Did you set the LikeliHood Threshold above 0?
Its called Minimum Confidence in the Kaldi Settings. I have set it to 0.5 but have the feeling that this is too high. Have a lot of actual intents that are falsly not recognized because of it.

1 Like
23

Thanks for that. I’ve bumped the “Minimum Confidence” up to 0.6 and it seems to recognise fairly well. I have a nasty cold at the moment so my voice isn’t 100% but I just tried about five different commands and only had one unrecognised utterance.

Then again, I just said “computer”, “blah blah” and it recognised that as “toggle kodi” and turned on my TV! :smiley:

More investigation is obviously needed.

24

Nice, great work! Please update the hassio-addon to 2.5.10 too :slight_smile:

25

I would be interested i additional German voice.

26

2.5.10 does not work correctly with Swedish. It takes about 30 seconds for a command to finish.
Here’s the log after a wake word:

ERROR (online2-cli-nnet3-decode-faster-confidence[5.5]:ReadConfigFile():parse-options.cc:462) Cannot open config file: /root/.config/rhasspy/profiles/sv/model/online/conf/online.conf

[ Stack-Trace: ]
/usr/lib/rhasspy/lib/kaldi/libkaldi-base.so(kaldi::MessageLogger::LogMessage() const+0x3d5) [0x7630ebfa]
/usr/lib/rhasspy/lib/kaldi/online2-cli-nnet3-decode-faster-confidence(kaldi::MessageLogger::LogAndThrow::operator=(kaldi::MessageLogger const&)+0xf) [0x4d353c]
/usr/lib/rhasspy/lib/kaldi/libkaldi-util.so(kaldi::ParseOptions::ReadConfigFile(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)+0x151) [0x7643a272]
/usr/lib/rhasspy/lib/kaldi/libkaldi-util.so(kaldi::ParseOptions::Read(int, char const* const*)+0xa7) [0x7643a614]
/usr/lib/rhasspy/lib/kaldi/online2-cli-nnet3-decode-faster-confidence(main+0x24b) [0x4d1a04]
/lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0x75e1b718]

27

Hi @solid, can you please try deleting the sv/model directory in your profile? You will need to re-download the Swedish model and re-train. This won’t affect the rest of your profile.

The Kaldi model has changed in 2.5.10, and it may be conflicting with the old one if you have it.

28

@synesthesiam Thanks, that worked!

1 Like
29

Awesome :slight_smile:

Let me know how it works out! This was my first attempt to train a Swedish model, so there may be tweaks necessary.

30

After some testing with the Swedish model I have to say WOW! It’s working much better now! I can start timers, computers, turn on lights, change color on lights and everything now.

I have a Google translate fallback in node-red that I use to send the .wav file from /api/recording if Rhasspy does not get my intent. I don’t think I need that fallback anymore, it’s working really good now! I found that using Kaldi with Minimum Confidence: 0.4 is the best.
Thank you so much!

1 Like
31

You’re very welcome! If you get a chance too, I’d be curious what you think of the Swedish Larynx voice. The speaker has quite the deep voice.

32

The Swedish Larynx voice is not very good, it talks too slow and just sounds like a depressed voice assistant lol. I use the Google wawnet, it has much more natural voice.

33

maxresdefault
maxresdefault1280×720 151 KB

7 Likes
34

Hmm… I can’t upgrade Rhasspy to 2.5.10 on one of my devices that runs the home assistant add-on. I’ve tried to completely remove and re-install the add-on, and all folders in /share/rhasspy but it still says 2.5.9 in the left corner when I run it. /api/version also says 2.5.9 but the home assistant add-on itself says “Current version: 2.5.10”.

35

Correct, I have a PR open:

2 Likes
36

Alright. I’m using your add-on with 2.5.10 until @synesthesiam fixes it :slight_smile:

37

The pull request is merged, you might need to uninstall and install again (don’t worry, no data will be lost)

1 Like
38

hm, I should train “hey marvin” as a wake-word now.

1 Like
39

For anyone like me who’s struggeling to get the latest version installed for docker:

You have to use

docker pull rhasspy/rhasspy:2.5.10

instead of only

docker pull rhasspy/rhasspy

because 2.5.9 is still marked as “latest” so docker pull will not update until you specify a version.

hope this will help some of you :wink:

cheers Dan

40

:frowning: I have to correct myself I’m still on 2.5.9 after pulling the new version…

What I did:

pulled 2.5.10

docker pull rhasspy/rhasspy:2.5.10

stopped rhasspy

docker stop rhasspy

removed rhasspy

docker rm rhasspy

and finally run rhasspy again

 docker run -d -p 12101:12101 \
  --name rhasspy \
  --restart unless-stopped \
  -v "$HOME/.config/rhasspy/profiles:/profiles" \
  -v "/etc/localtime:/etc/localtime:ro" \
  --device /dev/snd:/dev/snd \
  rhasspy/rhasspy \
  --user-profiles /profiles \
  --profile de

but web interface still shows I’m on 2.5.9 after rebooting

I tried on 2 machines, my satellite on pi3 and server on a debian x64 nuc based system, both have the same problem…

what can I do to fix this??

thanks

Dan

41

This part in your docker command is pulling latest, which is still 2.5.9

Change it to
rhasspy/rhasspy:2.5.10

42

The fix is easy. Correct your docker run command to use the 2.5.10 image:

docker run -d -p 12101:12101 \
  --name rhasspy \
  --restart unless-stopped \
  -v "$HOME/.config/rhasspy/profiles:/profiles" \
  -v "/etc/localtime:/etc/localtime:ro" \
  --device /dev/snd:/dev/snd \
  rhasspy:2.5.10/rhasspy \
  --user-profiles /profiles \
  --profile de

If you don’t tell docker to use the specific image it will default to using the latest version.

(edit: it might well be rhasspy/rhasspy:2.5.10, I’ll check when I get home and update my post appropriately)

43

I had an unfortunate error in one of my scripts, and it was failing to push the latest tag. Should be fixed now!

44

thanx a lot guys :wink: I’ll give it a new try now

cheers

45

@synesthesiam
I forgot to mention you, sorry

46

Thank you! I knew someone had volunteered to do a German voice, but my searches weirdly turned up nothing. I’ll PM you the info and we can go from there :slight_smile: