Mycroft Precise - Installation and Use

Hello,

Now that I’ve had some time to deal with Precise, I’d like to share my experiences. Some information can also be found here. I have the feeling that I am the only one who is not satisfied with Snowboy - but there are advantages with Precise: Completely offline, the model can be trained further in case of problems, e.g. to drive out false positive recognitions. In version 2.5 Precise will be supported - thanks @synesthesiam !


Installation: Since I have installed Linux only on Raspberry Pi and they have too little computing power, I use the Windows Subsystem (WSL) to install Debian on my PC.

In Debian / WSL it is necessary to install some things. Some things may be redundant, but with the following commands I could install everything:

sudo apt-get install python-pyaudio python3-pyaudio
sudo apt-get install python3 python3-all-dev python3-pip build-essential swig git libpulse-dev
sudo apt install libasound-dev portaudio19-dev libportaudiocpp0
pip3 install pyaudio
sudo apt-get install libasound-dev portaudio19-dev libportaudio2 libportaudiocpp0
sudo apt-get install ffmpeg libav-tools
sudo pip install virtualenv
sudo add-apt-repository ppa:mc3man/trusty-media && sudo apt-get update && sudo apt-get install ffmpeg && ffmpeg -version
sudo -s
pip install pyaudio

Then I closed and reopened WSL and installed Precise as described in the instructions

git clone https://github.com/mycroftai/mycroft-precise
cd mycroft-precise
sudo ./setup.sh

After the installation it is important to know how to exchange files between Windows and the WSL. The files are located in the following path:

C:\Users\USERNAME\AppData\Local\Packages\TheDebianProject.DebianGNULinux_SOMENUMBERSANDLETTERS\LocalState\rootfs\home\USERNAME\mycroft-precise

Later, when the wake word is ready, copying from this path is no problem. But I had problems when I copied new files to the above path under Windows (Debian didn’t show the folders and files). So I created the folder structure in Debian:

cd mycroft-precise/
mkdir data/random
mkdir to_be_converted/convert_me
mkdir to_be_converted/converted
mkdir WAKEWORD_NAME/not-wake-word
mkdir WAKEWORD_NAME/test/not-wake-word
mkdir WAKEWORD_NAME/test/wake-word
mkdir WAKEWORD_NAME/wake-word

Create the following files with the contents below (in WSL) :

sudo nano to_be_converted/convert_me/convert_mp3.sh

Then in the nano editor, paste this and save using ctrl o - ENTER and exit using ctrl x

SOURCE_DIR=convert_me
DEST_DIR=converted

for i in $SOURCE_DIR/*.mp3; do echo "Converting $i..."; fn=${i##*/}; ffmpeg -i "$i" -acodec pcm_s16le -ar 16000 -ac 1 -f wav "$DEST_DIR/${fn%.*}.wav"; done

nano to_be_converted/convert_me/convert_wav.sh

Again in nano , paste this and save using ctrl o - ENTER and exit using ctrl x

SOURCE_DIR=convert_me
DEST_DIR=converted

for i in $SOURCE_DIR/*.wav; do echo "Converting $i..."; fn=${i##*/}; ffmpeg -i "$i" -acodec pcm_s16le -ar 16000 -ac 1 -f wav "$DEST_DIR/${fn%.*}.wav"; done

To convert the files you have to copy them to WSL, the easiest way to do this is to store the files in C: under Windows and copy them to WSL:

In Windows, for example, you create a path C:/to_be_converted/convert_me, fill it with the files to be converted and then copy it to Debian (replace USERNAME!).

cp -a /mnt/c/to_be_converted/. /home/USERNAME/mycroft-precise/to_be_converted/

Please copy only the files of one type here: Wakeword or files that do not contain a wakeword.

Now you can convert the files in Debian with the following commands, depending on whether you want to convert mp3 or wav:
sh to_be_converted/convert_mp3.sh
sh to_be_converted/convert_wav.sh

After successful conversion the files can be moved to the respective directory and the convert_me shall be emptied to avoid confusion and mixing:

If wakewords were converted for training

sudo mv to_be_converted/converted/ WAKEWORD_NAME/wake-word/
sudo rm to_be_converted/convert_me/*.*

If wakewords were converted for testing

sudo mv to_be_converted/converted/ WAKEWORD_NAME/test/wake-word
sudo rm to_be_converted/convert_me/*.*

If sounds that do not contain a wakeword were converted

sudo mv to_be_converted/converted/ data/random
sudo rm to_be_converted/convert_me/*.*

Whether the conversion and moving worked, can be tracked under Windows. But as I said, I had problems when I created or moved files under Windows.
When the following three folders are filled, the wakeword can be trained

WAKEWORD_NAME/wake-word/
WAKEWORD_NAME/test/wake-word
data/random


EDIT: The data in data/random is important. In my training, a fairly large amount of data was needed to reduce the number of false positives in Precise’s internal training. It took 2 hours until false positive recognitions occurred only rarely. Therefore many, diverse examples are needed:

  • Music
  • Film
  • Audio books
  • Spoken words
  • Noise (vacuum cleaners, pets, …)

At first you can search your own flies for such data. In addition, you can be a little creative and collect data according to your own sound background. If e.g. your pets (I have two parrots) or your own conversations cause false positive triggers, you can simply make recordings with your mobile phone or similar for some time, convert them and train precise against them.

Furthermore, I have links here with data that I have packed into data/random after conversion:

I am grateful for further links to sound databases!

Simply use everything that comes your way for training. The only important thing is that the wake word will most likely not appear in the data. This can only be realized by using a wake word that is untypical for your own and the English language.

You can find the parts of the files of /data/random that trigger false positive detection (against which the wake word) shall be trained in the following folder after running precise-train-incremental mentioned below:
WAKEWORD_NAME/not-wake-word/generated

However, there are so many files that it is difficult to check all of them to see if the selected wake word does not appear in these files. More practical is therefore: unusual, improbable wake word.


Now training can be done with a few simple commands:

source .venv/bin/activate

Create Wakeword

precise-train -e 100 WAKEWORD_NAME.net WAKEWORD_NAME/

Train wakeword against false positive detection (can take some time depending on PC and data set - for me with 30 GB it took 4 h using an i7)

precise-train-incremental WAKEWORD_NAME.net WAKEWORD_NAME/

Repeat training!

precise-train -e 100 WAKEWORD_NAME.net WAKEWORD_NAME/

Test performance

precise-test WAKEWORD_NAME.net WAKEWORD_NAME/

Convert to .pb

precise-convert WAKEWORD_NAME.net

Now you can copy the two files WAKEWORD_NAME.pb and WAKEWORD_NAME.pb.params from the folder mycroft-precise to Rhasspy and look forward to Rhasspy 2.5 :wink:

3 Likes

Thank you for the write up! What is your experience with it?

Personally I only looked into Snowboy based on the Rhasspy documentation that mentions that

  • Precise only has a moderate performance
  • Porcupine has an excellent performance, but requires running an optimizer every 30 days

So the one in the middle appears to be Snowboy, with a good peformance and nothing to rerun every 30 days.

How is it going? Less false positive?

Dear @geoffrey, you’re welcome!

Hopefully the instruction is understandable and without mistakes. While writing it together I have optimized a little bit, therefore there might be errors. Please let me know if something does not work.

Because of the documentation I tried Porcupine first, because there are some free wake words which can be used for an unlimited time. For this reason my Rhasspy listens to “hey pico” from the very beginning. But often the wake word was triggered unintentionally by background noise or conversations. If the detection rate is low, the wake word must be called several times - no solution for me.

Afterwards I tried Sonowboy, both with one (i.e. 3 sample files) and several attempts (I created numerous accounts, and spoke about 20 x 3 wake words into the model - that took as long as getting Precise to run). Still the same problem as with Porcupine. The cofiguration via the website is terrible, especially if you do several iterations of the training using numberous accounts.

Since Rhasspy always detects something (and does so very well when triggered intentionally), false positive reactions are undesirable. Especially if the smart home can already control some things. Therefore I have - sad as it is - deactivated the wake word detection via the api at the moment. Rhasspy currently only works as voice output based on automation, but does not listen. That is a pity!

So I tried Precise, even though - as mentioned above - it is not supported yet but will come in version 2.5. At the moment I can’t say anything about the real usage.

I only have information about the internal test - here you can test the wake word model against an own data set (30 GB wav) and get a false detection rate. This has improved after repeated training as the data set grew and now stands at 99.94%.

image

I’m waiting patiently for version 2.5 and am happy to check if this number withstands real conditions. As mentioned before: If problems occur, your work is not in vain. You can always add more files with your wake word using different tones / speeds / … or enlarge the data set with audio files that do not contain your wake word and improve quality by training again.

I give advance praise and claim: More data = better recognition.

What is interesting about this approach is that we could create hopefully really personalized wakewords. I experimented with custom Snowboy wakewords like e.g. Pollie, but that gave lots of false positives.

At the moment Snowboy is reasonably working well, although the kids think I’m shouting “cowboy” the entire day. Today our 4 year old was able to trigger Rhasspy and to turn on the kitchen cabinet lights on using the Dutch profile.

The parts I ordered for the Rhasspy satellite are arriving today, so that’s the first thing on the list to address and once that’s working, I’m considering placing one in the kids bedroom (deactivating wakeword detection when they are in bed using an automation :blush:)

If all that goes to plan, I’ll see if I can further look into Precise, hopefully finding relevant data sources. I assume you speak German, so if it works for you, then it should be feasible to do it in Dutch (Flemish) as well.

With children it is certainly quite exciting what is intentionally and unintentionally triggered. My parrots do not imitate human language - we communicate our emotions with each other through gestures and sounds (happy, interested, bored, hungry, …). Therefore it is easier for me to separate these sounds from human sounds in training (as compared to small children).

Funny: The satellite of Rhasspy is waiting in my drawer and waits, because my motto is: First a working system with a proper wake word (everything else works great) and then the satellites.

Some data on sounds will be useful for training, regardless of the language: white, brown, … noises and ambient sounds like vacuum cleaners etc. In addition, the English language is present in movies, series and music - you can use some of it here. For data in your own language you can use data from podcasts, news, TV shows. You only need a way to get mp3 or wav files.

It is important with this approach that the selected hot word is very unlikely to appear in the data set. My hotword “hey pico” will most likely not appear in the selected songs, German news broadcasts, …

I have added links to sources of data for data/random in my instructions given aboveand will extend these list of links, if I find or get mentioned further sources.

This whole topic gets now much more important as snowboy is wrapping up (as you can read here): https://github.com/Kitt-AI/snowboy and therefore should probably not even be an option for rhasspy anymore. I wonder how well mycroft.ai is doing with the current crisis.

@ulno: That sounds exciting!
I really hope that Precise’s performance is better as compared to snowboy as well as porcupine and that mycroft will survive the crisis.
I have already played with the pre-2.5 - but I can’t make Precise work yet.

wow, F!!! Snowboy works amazing on my part, no false positive !

Have you tried on 2.4.19? I think I had
“Hey Mycroft” running there for a short time - back to “porcupine” for the moment. What is going wrong for you on 2.4.19 with precise or just trouble on 2.5?

And did you see this? https://github.com/synesthesiam/rhasspy/issues/169

I had to apply that to make it work.

Hope, we can convince you to pitch in for some precise effort in the future :wink:

Oha, I thought precise only worked from 2.5 on! :upside_down_face:
Just tried it again with 2.4.19. Once I defined “system”: “precise”, rhasspy does not load the configuration anymore.

@ulno How does your configuration look like, when “Hey Mycroft” worked for you? :hugs:
EDIT: I don’t quite understand how precise could work for you. I can’t get Rhasspy 2.4.19 started with precise. The log is quite clear: ModuleNotFoundError: No module named ‘precise_runner’

@KiboOst:My condolences on the slow death of snowboy.

My config for the wakeword looks like this (precise does not work, porcupine and snowboy do work):

"wake": {
    "system": "precise",
    "precise": {
        "model": "hey-mycroft-2.pb",
        "sensitivity": "0.5",
        "trigger_level": "3",
        "chunk_size": "2048"
    },
    "porcupine": {
        "keyword_path": "porcupine/hey_pico_raspberrypi.ppn",
        "sensitivity": "0.65"
    },
    "snowboy": {
        "audio_gain": "0.90",
        "model": "snowboy/hey_pico/33_hey_pico_tone_mod_full_2x.pmdl",
        "sensitivity": "0.6"
    }
},

"rhasspy": {
    "listen_on_start": true
}

the runner error is the one you can fix here: https://github.com/synesthesiam/rhasspy/issues/169

Config looks fine. Try the runner fix. It’s pretty dirty to apply as you have to edit a mycroft precise file in your virtualenv - for me that was edit /home/pi/rhasspy/.venv/lib/python3.7/site-packages/precise_runner/runner.py:65

This was unfortunately predictable… They should open source everything. Alas it won’t happen…

Bye bye Snowboy…

The vocal assistant field is starting to become scarce (Snips, Snowboy, …).

Porcupine is only working correctly for English speakers… How long until Mycroft is bought or shuts down… at least Precise is fully open sourced and can be maintained even if Mycroft disappears.

I’d be very interested in Precise detection performance…

I think, we all should give mycroft precise some love and maybe see if we can even train as a community some more wakewords - I can offer help for training English, German, and Portuguese wakewords and also happy to push this for some more commercial applications outside the home automation environment (when I ever am allowed to return to university). Maybe, we could even push the mycroft community to re-implement some of rhasspy’s achievements back into their system- not sure if this is a political issue @synesthesiam? Sorry if I overstep here. For me rhasspy is so much easier to use than mycroft, but that might be personal - I think there should be at least mutual acknowledgement.

I am happy to let our community use my youtube tracks for noise or failure training (especially my lectures should have a lot of real background noise) - I bet there are some other youtubers around in this community or ones we know that we could ask?

2 Likes

I guess I didn’t read consciously enough - you have already written it above. Excuse me!

@ulno EDIT:

I have not found a corresponding file in my docker container from Rhasspy. Your path is:
/home/pi/rhasspy/.venv/lib/python3.7/site-packages/precise_runner/runner.py:65

I searched Rhasspy for “precise_runner” and “site-packages”:
docker exec -it Rhasspy bash

find / -name "*site-packages*"
/usr/local/lib/python2.7/site-packages –> This path is empty.

find / -name "*precise_runner*"
No result!

So I set precise as wakeword again in the profile file, restarted Rhasspy to make sure that the current container has already tried to start with Precise. Again: No success, because no search result. A pity!

The code I want to fix is as follows:
return_time = 1e10 if timeout is None else ( timeout + time.time() )

Therefore I have set out on a search with the following order:
grep -iRls "return_time" ./
No result!

I don’t seem to have the runner.py

Not sure about docker - I despise docker and try to avoid it at all costs - it’s permission system is even worse than chroot and it’s an abolute nightmare to set up on windows and macos - but that is personal preference.

When you are in the docker, is there some kind of default python environment - you could just do a pip install mycroft-precise there - after that you should have precise (and maybe also the runner.py-file)

On a side note installing rhasspy in a virtual env is actually not that hard.

And reading your description again, did you actually search inside the docker image? You might have to start bash inside the docker and then search there.

This would be much appreciated :slight_smile: I’d be happy to host the training data in the Rhasspy repos, unless you’d prefer to keep it separate. I’d also be willing to run training on my home server.

I have another project that’s been shelved for the time being, but may be relevant here. I’ve found and trained a number of MaryTTS voices, but I don’t have full coverage yet for all of Rhasspy’s supported languages. Perhaps we could launch something to help people donate voice samples for wake words or TTS voices?

No political issue, I just don’t have the extra time to reach back into the Mycroft community. Anyone is welcome to pull pieces of Rhasspy into whatever they’d like.

I haven’t looked into if there’s any way to interoperate with Mycroft. I do mention them in the documentation as the current best alternative if you’re cool with your voice being shipped over the internet to some server.

1 Like

For me with semi-laymanlike Linux knowledge (I’m failing myself up) Docker has certain advantages through the container approach.
Thanks for the idea! I installed python, and then tried to install precise.

apt install python-pip
pip install mycroft-precise

Unfortunately, that doesn’t work as staed below. Maybe I just have to wait until Rhasspy 2.5 is finished forging.

  Could not find a version that satisfies the requirement tensorflow<1.14,>=1.13 (from mycroft-precise) (from versions: )
No matching distribution found for tensorflow<1.14,>=1.13 (from mycroft-precise)

just a guess:

is that python2 or 3?
Maybe give pip3 a chance?