A preview of Rhasspy 2.5.11 is now available via Docker! Just run docker pull rhasspy/rhasspy:2.5.11 to update. I’ll mark it as latest once we work out the inevitable handful of bugs I introduce in each version
Thanks to everyone who contributed, and to the Rhasspy community for keeping the party going
The full changelog is below, but here are some of the big highlights:
Vosk support for ASR! This works best in the “open transcription” mode where you can say anything. In restricted voice command mode, I’m only able to limit Vosk’s vocabulary; I can’t control the actual grammar yet.
“Unknown words” in Kaldi. Check the “Replace unknown words with …” in the settings and re-train. Incorrect words should now be replaced with <unk> and (usually) result in a “not recognized”. You can adjust the probability of unknown words too
No, it means you can have multiple satellites pushing an audioStream to Rhasspy.
Previously, when connecting a second (esp32) satellite, Rhasspy would stop working because the two audiostream were mixed.
Now each siteid/audioFrame topic has its own process.
Has anyone had any luck with this? I’m using a Pi 4 as a base station with Text FST grammar selected. Training usually completes in roughly 14-17 seconds. I’m not able to complete training with unknown words enabled. I’m seeing a variety of behaviour whilst following the log during the process.
Training often stumbles/halts quite early on without error, though I can’t identify any patterns.
If training does progress further through the procedure it throws timeouts
File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1313, in api_train
result = await core.train()
File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 462, in train
timeout_seconds=self.training_timeout_seconds,
File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__init__.py", line 995, in publish_wait
result_awaitable, timeout=timeout_seconds
File "/usr/lib/python3.7/asyncio/tasks.py", line 423, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
Regardless of 1 or 2, a ‘405 Not Allowed’ response is thrown after 60 seconds in place of a training completed in X seconds.
Rhasspy can be woken once training has “completed/failed”, but ultimately does not capture any speech and ultimately timeouts.
Curious to know if anyone else has had any success!
Hmmmm…I may need to add some more options here. For a lot of sentences, my current implementation will increase training time a lot. I need to at least make the training timeout length an option, at least.
Try decreasing the number of frequent words it uses (default is 100), and see if that helps. If not, I may need to add an option to have a “possible unknown word” inserted for every 2nd or 3rd (etc.) actual word. Right now, every word in every sentence gets a “possible unknown word” path in the intent graph.
This config says to accept audio on the server on UDP port 12345 for satellite1. Additionally, audio from satellite1 is raw PCM rather than WAV chunks ("raw_audio": true).
Once the wake word has been detected, this raw PCM audio will be automatically packaged up as WAV chunks and forwarded over MQTT to the rest of Rhasspy’s services ("forward_to_mqtt": true). So you will get ASR, NLU, dialogue, etc. for that satellite’s site ID
The trigger for MQTT forwarding is currently whether or not the wake word service has been disabled. This may or may not be a good idea, but Rhasspy’s dialogue manager automatically disables the wake word service after detection and re-enables it after ASR + feedback sounds have finished. So it seemed natural to have MQTT forwarding start when the wake word service is disabled, and then stop when it’s re-enabled. Thoughts?
A configuration option for training timeout length would be great nonetheless. I tried setting app.config['BODY_TIMEOUT'] in Quart to extend the training timeout but it didn’t seem to take/had no effect - not sure why, any ideas? I had hoped to see quite how long training would take with frequent words set as 10.
Did you happen to see any other errors in the logs before the timeout? I did use some different Kaldi features to handle unknown words, so maybe something is also wrong there.
There’s a hard-coded training timeout of 600 seconds (10 minutes) currently. If it’s still taking that long with a value of 1, I don’t think it’s going to work great for most people in the Pi.
One option is to do the second, third, etc. word thing I mentioned. Another might be to just create a separate, low probability “catch all” intent that is made up of these frequent words. That should add almost no time to the training.
is there a way to get the best of both options? i really like the exact match on my sentences but also want wildcard on the end of them like uh: define (word). I was able to get it to work with the last image but idk what to do now. in one of these patches i swear it was just perfect. It would get the sentences and any extra words I’d throw in. It would just send it in the raw text.
I actually do know how to do this now. I will need to extend the templating language with a “wildcard” symbol. Maybe just “*”?
If I use the same trick as the “unknown words”, then * would match any single word from the base dictionary. Unfortunately, multiple wildcards in a row would not be subject to the statistics of the base language model (so “the big dog” would be just as likely as “dog big the”).
multiple words i dont need. that’s just too much. u need like cloud huge ai for that. I’m just looking for one word. Sometimes i’ll ask google a couple words to throw in its search engine too. Like i would just like the ability to choose at which point open transcription starts? Idk. I mean its pretty good at recognizing my sentences with fast speed too. If we want to throw random words in for questions though it just is all over the place. If i tweak the older version just open transcription with 0.3 confidence. It will half the time get what I’m saying i think. Idk how we could do it but unless there’s huge priority. We are really close to getting there. I know bash maybe i could help u out. Would have to look at the code base. I mean I’m pretty surprised how well i got it working considering its completely offline. New voices are a huge step forward btw. good job. sound a lot better. even if it takes an extra 2-3 seconds. On a raspberry pi 4 4gb
Based on feedback, I’ve modified the “unknown words” feature for Kaldi. To reduce training time with it enabled, I now create a single “unknown sentence” path in the grammar. Speaking something outside your sentences.ini should produce something like <unk> <unk> ... up to a maximum number of words.
You can adjust the probability of the “unknown sentence” (default is 1e-5). If you end up with too many false positives for unknown words, try lowering the probability.
Cancel Word
The old version of “unknown words” didn’t work, but I realized it was perfect for a “cancel” word! So now in the Kaldi settings, you can set a special word that will immediately cancel the current voice command. Just make sure it’s not a word you’re using in your intents
So if you were to set the “cancel” word to “terminate”, you could say something like:
turn on the terminate
and Kaldi will return:
turn on the <unk>
which will cause fsticuffs to report a failed recognition. You can set the cancel word to whatever you like (I wouldn’t use a word from your intents), and change the probability.
I just pulled the new Docker Image.
Thanks for all the work.
Iam looking for the “cancel” word at Kaldi settings, but didnt found it. Are there some post tasks necessary to bring up the new features?
Btw.: Did you already add the german Voice “kerstin” to 2.5.11?
