PicoVoice offline Voice AI engine gets free tier for up to 3 users

Perfect, this was the only thing putting me off having a custom wake word that needed registered every 30 days. I now have a free-tier account. Thanks for posting.

Also they have cobra in that and slightly puzzled why they did go all out with AEC & Beamforming also or at least wish they had but seems just VAD.

Hi

Has anyone got this to work? I created a custom wake word and reconfigured rhasspy, but after it restarts the wake word doesn’t trigger. If I use one of the supplied wake words it works as expected.

I see this in the start logs…

[DEBUG:2021-12-10 23:31:42,249] rhasspywake_porcupine_hermes: Loading porcupine for lappy
[ERROR:2021-12-10 23:31:42,251] rhasspywake_porcupine_hermes: detection_thread_proc
Traceback (most recent call last):
File “/usr/lib/rhasspy/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/init.py”, line 254, in detection_thread_proc
sensitivities=self.sensitivities,
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine/init.py”, line 68, in create
sensitivities=sensitivities)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine/porcupine.py”, line 94, in init
raise self._PICOVOICE_STATUS_TO_EXCEPTION[status] ()
ValueError

I noticed the Picovoice console is generating v2.0 ppn files, but Rhasspy is using porcupine 1.9.5 so maybe they have to match?

correct, this is not working with Rhasspy yet

Got it to work with install from scratch on Pi3a+

Install bullseye
Install pvporcupine in actual version,
Install rhasspy
Copy pv porcupine libs over the old porcupine dir
Link libffi6 to libffi7
Change some files in rhasspiprofile, rhasspisupervisor and rhasspy-wake-porcupine-hermes to supply access_key
enter access_key in profile file
copy custom wakeword file in profile/porcupine dir

do some custom additional steps (compile driver for matrix voice-bullseye, pipe audio to command - sox - matrix speaker , change porcupine process file to german version….)

enjoy

I will write a detailed howto - tomorrow

1 Like

Part 1 - Basic installation

Download latest Pi OS lite from:

Flash it with a tool like Win32DiskImager

Create a new empty file named ssh on the boot drive

Create a text file on boot drive and name ist wpa_supplicant.conf, paste the following content in.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=DE
network={
ssid=“yourSSID”
psk=“yourWPAKey”
}

Use eject hardware to unmount sd card –otherwise sometimes boot filesystem will by inconsistent

Insert sd in pi, connect mic array and power

Connect to pi with putty, log in with user pi and password raspberry

Change password with

passwd

Update system with

sudo apt-get update

sudo apt-get upgrade

Disable onboard audio with

sudo vi /boot/config.txt

comment out by adding hashtag

#dtparam=audio=on

Quit with :wq!

Do a reboot

sudo reboot

Download and install Rhasspy

wget https://github.com/rhasspy/rhasspy/releases/latest/download/rhasspy_armhf.deb

sudo apt install ./rhasspy_armhf.deb

Install python3-pip and midnight commander

sudo apt-get install python3-pip mc

remove old porcupine

sudo rm -r /usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine-1.9.5.dist-info

sudo rm -r /usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine

copy new porcupine

sudo mv /usr/local/lib/python3.9/dist-packages/pvporcupine /usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine

sudo mv /usr/local/lib/python3.9/dist-packages/pvporcupine-2.0.1.dist-info/ /usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine-2.0.1.dist-info/

Start Rhasspy and configure as needed. Leave Porcupine unconfigured

Rhasspy –profile de

Stop rhasspy (ctrl-c)

Next part - changing py Files to Run porcupine 2.0.1

intermediate step:

Build Audio-Drivers for matrix voice and bullseye

Install kernel headers and git

sudo apt-get -y install raspberrypi-kernel-headers raspberrypi-kernel git

clone matrix kernel modules

git clone https://github.com/matrix-io/matrixio-kernel-modules

make superuser and change permissions for build dir (build will fail otherwise)

sudo su
chmod 666 /lib/modules/5.10.63-v7+ --recursive

make driver

cd /home/pi/matrixio-kernel-modules/src/
make && make install
cp /home/pi/matrixio-kernel-modules/misc/matrixio.conf /etc/modules-load.d/
cp /home/pi/matrixio-kernel-modules/misc/asound.conf /etc/

reboot

add matrixio driver to boot/config.txt

sudo su
nano /boot/config.txt

add

dtoverlay=matrixio

reboot

check sound card devices with aplay -l and arecord -l

Link /usr/lib/arm-linux-gnueabihf/libffi.so.7 to /usr/lib/arm-linux-gnueabihf/libffi.so.6

sudo ln -s /usr/lib/arm-linux-gnueabihf/libffi.so.7 /usr/lib/arm-linux-gnueabihf/libffi.so.6

Start rhasspy

rhasspy --profile de

Configure Audio recording in rhasspy

Stop rhasspy

(Ctrl+c)

Create new file in /home/pi/.config/rhasspy/profile/de

touch /home/pi/.config/rhasspy/profiles/de/copy-audio.sh

open flie and paste content

nano /home/pi/.config/rhasspy/profiles/de/copy-audio.sh

#!/usr/bin/env bash
DIR="$( cd “$( dirname “$0” )” && pwd )"
output_dir="$DIR/output"
mkdir -p “$output_dir”
sox - -r 44100 -c 2 -b 16 -e signed -t wav - | aplay -D hw:0,1

mark file as executable

chmod +x /home/pi/.config/rhasspy/profiles/de/copy-audio.sh

Start rhasspy

rhasspy --profile de

Configure Audio playing in rhasspy

Set mode to Local command and enter as command:

/home/pi/.config/rhasspy/profiles/de/copy-audio.sh

Test but keep in mind default output of the matrix voice is headaphone jack

Changes for Porcupine V 2.0.1

Replace in

/usr/lib/rhasspy/rhasspy-profile/rhasspyprofile/profiles/defaults.json

“porcupine”: {
“compatible”: true,
“keyword_path”: “porcupine.ppn”,
“sensitivity”: 0.5,
“udp_site_info”: {}
},

with

“porcupine”: {
“compatible”: true,
“keyword_path”: “porcupine.ppn”,
“sensitivity”: 0.5,
“udp_site_info”: {},
“access_key”: “”
},

Replace in

/usr/lib/rhasspy/rhasspy-supervisor/rhasspysupervisor/init.py

if wake_system == "porcupine":
    keyword = profile.get("wake.porcupine.keyword_path") or "porcupine.ppn"
    if not keyword:
        _LOGGER.error("wake.porcupine.keyword_path required")
        return []

    sensitivity = profile.get("wake.porcupine.sensitivity", "0.5")

    wake_command = [
        "rhasspy-wake-porcupine-hermes",
        "--keyword",
        shlex.quote(str(keyword)),
        "--sensitivity",
        str(sensitivity),
        "--keyword-dir",
        shlex.quote(str(write_path(profile, "porcupine"))),
    ]

with

if wake_system == “porcupine”:
keyword = profile.get(“wake.porcupine.keyword_path”) or “porcupine.ppn”
if not keyword:
_LOGGER.error(“wake.porcupine.keyword_path required”)
return []

    sensitivity = profile.get("wake.porcupine.sensitivity", "0.5")
    access_key = profile.get("wake.porcupine.access_key")

    wake_command = [
        "rhasspy-wake-porcupine-hermes",
        "--keyword",
        shlex.quote(str(keyword)),
        "--sensitivity",
        str(sensitivity),
        "--access_key",
        str(access_key),
        "--keyword-dir",
        shlex.quote(str(write_path(profile, "porcupine"))),
    ]

Replace in

/usr/lib/rhasspy/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/init.py

replace

def __init__(
    self,
    client,
    model_ids: typing.List[str],
    wakeword_ids: typing.List[str],
    sensitivities: typing.List[float],
    keyword_dirs: typing.Optional[typing.List[Path]] = None,
    site_ids: typing.Optional[typing.List[str]] = None,
    sample_rate: int = 16000,
    sample_width: int = 2,
    channels: int = 1,
    udp_audio: typing.Optional[typing.List[typing.Tuple[str, int, str]]] = None,
    udp_chunk_size: int = 2048,
    udp_raw_audio: typing.Optional[typing.Iterable[str]] = None,
    udp_forward_mqtt: typing.Optional[typing.Iterable[str]] = None,
    lang: typing.Optional[str] = None,
):

with

def __init__(
    self,
    client,
    model_ids: typing.List[str],
    wakeword_ids: typing.List[str],
    sensitivities: typing.List[float],
    access_key: str = "",
    keyword_dirs: typing.Optional[typing.List[Path]] = None,
    site_ids: typing.Optional[typing.List[str]] = None,
    sample_rate: int = 16000,
    sample_width: int = 2,
    channels: int = 1,
    udp_audio: typing.Optional[typing.List[typing.Tuple[str, int, str]]] = None,
    udp_chunk_size: int = 2048,
    udp_raw_audio: typing.Optional[typing.Iterable[str]] = None,
    udp_forward_mqtt: typing.Optional[typing.Iterable[str]] = None,
    lang: typing.Optional[str] = None,
):

replace

    self.subscribe(AudioFrame, HotwordToggleOn, HotwordToggleOff, GetHotwords)

    self.wakeword_ids = wakeword_ids
    self.model_ids = model_ids
    self.sensitivities = sensitivities

    self.keyword_dirs = keyword_dirs or []

with

    self.subscribe(AudioFrame, HotwordToggleOn, HotwordToggleOff, GetHotwords)

    self.wakeword_ids = wakeword_ids
    self.model_ids = model_ids
    self.sensitivities = sensitivities
    self.access_key = access_key
    self.keyword_dirs = keyword_dirs or []

replace

def detection_thread_proc(self, site_info: SiteInfo):
    """Handle WAV audio chunks."""
    try:
        if site_info.porcupine is None:
            _LOGGER.debug("Loading porcupine for %s", site_info.site_id)
            site_info.porcupine = pvporcupine.create(
                keyword_paths=[str(kw) for kw in self.model_ids],
                sensitivities=self.sensitivities,
            )

with

def detection_thread_proc(self, site_info: SiteInfo):
    """Handle WAV audio chunks."""
    try:
        if site_info.porcupine is None:
            _LOGGER.debug("Loading porcupine for %s", site_info.site_id)
            _LOGGER.debug("access_key %s", self.access_key)
            for kw in self.model_ids:
                 _LOGGER.debug("kw %s", kw)
            site_info.porcupine = pvporcupine.create(
                keyword_paths=[str(kw) for kw in self.model_ids],
                sensitivities=self.sensitivities,
                access_key=self.access_key,
            )		

Replace in /usr/lib/rhasspy/rhasspy-wake-porcupine-hermes/rhasspywake_porcupine_hermes/main.py

def main():
“”“Main method.”""
parser = argparse.ArgumentParser(prog=“rhasspy-wake-porcupine-hermes”)
parser.add_argument(
“–keyword”,
required=True,
action=“append”,
help=“Path(s) to one or more Porcupine keyword file(s) (.ppn)”,
)
parser.add_argument(
“–keyword-dir”,
action=“append”,
default=[],
help=“Path to directory with keyword files”,
)

with

def main():
“”“Main method.”""
parser = argparse.ArgumentParser(prog=“rhasspy-wake-porcupine-hermes”)
parser.add_argument(
“–keyword”,
required=True,
action=“append”,
help=“Path(s) to one or more Porcupine keyword file(s) (.ppn)”,
)
parser.add_argument(
“–access_key”,
required=True,
action=“append”,
help=“access_key”,
)
parser.add_argument(
“–keyword-dir”,
action=“append”,
default=[],
help=“Path to directory with keyword files”,
)

replace

# Listen for messages
client = mqtt.Client()
hermes = WakeHermesMqtt(
    client,
    args.keyword,
    keyword_names,
    sensitivities,
    keyword_dirs=args.keyword_dir,
    udp_audio=udp_audio,
    udp_raw_audio=args.udp_raw_audio,
    udp_forward_mqtt=args.udp_forward_mqtt,
    site_ids=args.site_id,
    lang=args.lang,
)

with
# Listen for messages
akey = args.access_key[0]
client = mqtt.Client()
hermes = WakeHermesMqtt(
client,
args.keyword,
keyword_names,
sensitivities,
keyword_dirs=args.keyword_dir,
udp_audio=udp_audio,
udp_raw_audio=args.udp_raw_audio,
udp_forward_mqtt=args.udp_forward_mqtt,
site_ids=args.site_id,
lang=args.lang,
access_key=akey,
)

Put your custom created ppn file to /home/pi/.config/rhasspy/lang/porcupine/

Edit your config file and add
“wake”: {
“porcupine”: {
“keyword_path”: “put your ppn file name here”,
“access_key”: “put Your Access_Key here”
},
“system”: “porcupine”
}

Install german params modell:
cd /usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/pvporcupine/lib/common
sudo wget https://github.com/Picovoice/porcupine/raw/master/lib/common/porcupine_params_de.pv
sudo rm porcupine_params.pv
sudo mv ./porcupine_params_de.pv ./porcupine_params.pv

Start rhasspy and test

rhasspy –profile de

btw.: PVP2.0.1 needs internet for access key activation
Code isn’t build perfect - it’s a proof of concept.
Im trying to get the “etwas speziellen²” Matrix Voice Leds to work. If they will work i will write here.

Have fun

Andreas

“²etwas speziellen”=“fuc**ng”

2 Likes

The install Script from

doesnt work on Bullseye. Building venv will fail so we have to do it self

Install build utils

sudo apt-get install cmake g++ git libfftw3-dev libgflags-dev swig

Install Py Bindings

sudo python3 -m pip install pybind11

Clone and build WiringPi lib

git clone --recursive https://github.com/WiringPi/WiringPi-Python.git
cd WiringPi-Python/
sudo python3 setup.py install

cd WiringPi

sudo ./build

cd ~

Clone and build Matrix Creator Hal

git clone https://github.com/matrix-io/matrix-creator-hal.git
cd matrix-creator-hal
mkdir build

cd home/pi/matrix-creator-hal/cpp/driver/

replace in cmakelists.txt

add_library(matrix_creator_hal SHARED ${matrix_creator_hal_src})
set_property(TARGET matrix_creator_hal PROPERTY CXX_STANDARD 11)
target_link_libraries(matrix_creator_hal ${CMAKE_THREAD_LIBS_INIT} ${CMAKE_THREAD_LIBS_INIT})
target_link_libraries(matrix_creator_hal ${FFTW_LIBRARIES})
target_link_libraries(matrix_creator_hal ${WIRINGPI_LIB} ${WIRINGPI_DEV_LIB} ${CRYPT_LIB})

with

add_library(matrix_creator_hal SHARED ${matrix_creator_hal_src})
set_property(TARGET matrix_creator_hal PROPERTY CXX_STANDARD 11)
target_link_libraries(matrix_creator_hal ${CMAKE_THREAD_LIBS_INIT} ${CMAKE_THREAD_LIBS_INIT})
target_link_libraries(matrix_creator_hal ${FFTW_LIBRARIES})
target_link_libraries(matrix_creator_hal ${WIRINGPI_LIB} ${WIRINGPI_DEV_LIB} ${CRYPT_LIB} -lcrypt -lrt)

cd ~/matrix-creator-hal/build
cmake …

make -j4 && sudo make install

cd ~

Clone and build Matrix lite

git clone https://github.com/matrix-io/matrix-lite-py
cd matrix-lite-py
sudo python3 -m pip install ./

download and exctract development repository - build will fail - dont try…RPI.GPIO will fail for venv.
We run it without virtual environment.

copy content to /home/pi/HermesLedControl-3.0.0

install dependencies

pip3 install PyYAML
pip install paho-mqtt

create config file

mkdir /home/pi/.config/hermesLedControl
nano /home/pi/.config/hermesLedControl/configuration.yml

Paste content

engine: “rhasspy”
pathToConfig: “/home/pi/.config/rhasspy/profiles/de/profile.json”
hardware: “matrixvoice”
pattern: “alexa”
enableDoA: false

Add mqtt Port to:

/home/pi/.config/rhasspy/profiles/de/profile.json

    "enabled": "true",
    "host": "your_mqtt_host",
    "port": "your_mqtt_port",
    "site_id": "your_rhasspy_site_id"

Build startup file for hermesLedControl

cd ~/HermesLedControl-3.0.0 $

nano hermes.sh

paste the following content

#!/usr/bin/env bash
LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH
/usr/bin/python3 /home/pi/HermesLedControl-3.0.0/main.py --hermesLedControlConfig=/home/pi/.config/hermesLedControl/configuration.yml

mark as executable

chmod +x hermes.sh

Run hermes.sh for testing - led’s should lit up short

stop with ctrl+c

Build startup Scipt

sudo touch /etc/systemd/system/hermesledcontrol.service
sudo nano /etc/systemd/system/hermesledcontrol.service

paste this content

[Unit]
Description=Hermes Led Control

[Service]
Type=simple
User=pi
WorkingDirectory=/home/pi/HermesLedControl-3.0.0
ExecStart=/bin/bash -c ‘/home/pi/HermesLedControl-3.0.0/hermes.sh 2>&1 | cat’
Restart=always
RestartSec=5
User=pi

[Install]
WantedBy=multi-user.target

safe (ctrl+o)

enable the startup script

sudo systemctl enable hermesledcontrol.service

reboot

sudo reboot

1 Like