Wakeword + stt new problem

jerome83136_tux · October 17, 2020, 1:29am

Hi guys,
For me the problem is still there

Initially I was on 2.5.6
Tried with 2.5.7
I only updated the respeaker, should I try to update other rhasspy ? (the other instances are working, only the respaker suffers the issue)

Still having the issue

Here is my config if it can help

{
    "dialogue": {
        "satellite_site_ids": "master"
    },
    "handle": {
        "remote": {
            "url": "http://192.168.1.251/core/api/jeeApi.php?plugin=jeerhasspy&apikey=LOE0QkHMi7ANVLKMvyRZRed8KFIedxPI&plugin=jeerhasspy&type=jeerhasspy"
        },
        "system": "remote"
    },
    "intent": {
        "remote": {
            "url": "http://192.168.1.251:12101/api/text-to-intent"
        },
        "system": "hermes"
    },
    "microphone": {
        "arecord": {
            "device": "plughw:CARD=seeed8micvoicec,DEV=0"
        },
        "system": "arecord"
    },
    "mqtt": {
        "enabled": "true",
        "host": "192.168.1.253",
        "site_id": "satellite2"
    },
    "sounds": {
        "aplay": {
            "device": "plughw:CARD=seeed8micvoicec,DEV=1"
        },
        "system": "aplay"
    },
    "speech_to_text": {
        "remote": {
            "url": "http://192.168.1.251:12101/api/speech-to-text"
        },
        "system": "hermes"
    },
    "text_to_speech": {
        "remote": {
            "url": "http://192.168.1.251:12101/api/text-to-speech"
        },
        "system": "hermes"
    },
    "wake": {
        "system": "raven"
    }
}

SiteIDs:

respeaker --> satellite2 (192.168.1.21)
computer --> master (192.168.1.251)

If someone has an advice, I will be happy to get some recommandations.

Best regards

Platup · October 17, 2020, 8:14pm

probably a dumb question, but how do you roll back?
I was trying…

docker pull --platform linux/arm/v6 rhasspy/rhasspy:2.5.6

but that’s not working.
I also noticed some newer updates in GitHub. But “latest” says it’s up to date.

jerome83136_tux · October 18, 2020, 11:31am

Hello,

On my side, I did this:

docker pull rhasspy/rhasspy:2.5.6 (this is on my laptop, so I didn’t need to specify the platform. Adapt according you needs)
It made the rhasspy 2.5.6 image available locally (alongside with the 2.5.7 image)

docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
rhasspy/rhasspy 2.5.7 5d4c93ceb23d 2 days ago 1.64GB
rhasspy/rhasspy 2.5.6 b320c65b825a 2 weeks ago 1.5GB
rhasspy/rhasspy latest b320c65b825a 2 weeks ago 1.5GB
Then I killed and removed the Rhasspy container using the 2.5.7 image

Identify the container name (for me its “rhasspy”, we can see it uses the “rhasspy/rhasspy:2.5.7” image)

docker ps | grep rhasspy

 f8d03395b35f        rhasspy/rhasspy:2.5.7      "bash /usr/lib/rhass…"   4 minutes ago       Up 4 minutes        0.0.0.0:12101->12101/tcp                         rhasspy

Kill the container
docker kill rhasspy
Remove the container (in my case, I store rhasspy config files on my local disk with a mountpoint. Please adapt to your setup to prevent data loss )

docker rm rhasspy
Finally, recreate your container and tell him to use the rhasspy 2.5.6 image

docker run (adapt your custom options here) rhasspy/rhasspy:2.5.6

NB: here we can see we will create a container that will use the rhasspy/rhasspy:2.5.6 image

Verify your container is using the expected image version

docker ps | grep rhasspy
1e8fd931bb17 rhasspy/rhasspy:2.5.6 "bash /usr/lib/rhass…" 2 minutes ago Up 2 minutes 0.0.0.0:12101->12101/tcp

NB: We can see that the container “rhasspy” is using the image rhasspy/rhasspy:2.5.6 as expected

Hope it will help you

Good luck and have fun

Platup · October 18, 2020, 9:41pm

Thank you! That was very helpful and I’ve successfully rolled back.

dror-israel · October 19, 2020, 6:24am

Any news about the fix to this problem?

jerome83136_tux · October 19, 2020, 1:57pm

+1
I would be happy to get some news

fastjack · October 21, 2020, 7:27pm

The fix is under way. Hang in there

synesthesiam · October 21, 2020, 8:09pm

Dialogue manager fix has been pushed to Docker. Debian packages are coming soon!

Platup · October 21, 2020, 10:01pm

Thank you!
I can indeed confirm this fixed the Satellite wakeword issue. However, the Pi Zero’s are still taking a much longer time to “Save” and then “Reboot” when making a change (compared to 2.5.6).

Should the “Server” handle the Home Assistant Intents for the Satellites? I tried, but they don’t seem to be triggering unless I setup intent handling on each satellite.

And it also looks like Home Assistant still isn’t receiving the Site ID… or am I possibly doing something wrong in not adding site_id as a slot?

synesthesiam · October 22, 2020, 12:33am

I took me a moment, but I realized what I did that made this happen. It’s fixed in master, so I’ll push it into the next update. Of all things, it’s because of JSON parsing.

For some reason, I didn’t add “Satellite Site Ids” to the intent handling in the settings page! This is what’s missing. I’ll add it, but for now you can just add this directly to your profile:

{
  "handle": {
    "satellite_site_ids": "satellite1,satellite2"
  }
}

Was it supposed to? It should be available inside _intent as a siteId property (or site_id – I forget which now).

Platup · October 22, 2020, 2:20pm

I’m trying to use some old intent scrips I was using with Snips… and it occurred to me they might have had a setting in the console to “include Site ID” or something like that. So supposed to… I’m not sure.

My intent script pulls “room” from the intent or defaults to the site id, which is the room the satellite is in.

> onOffIntent:
>   speech:
>     type: plain
>     text: >
>         {% if room == null %}
>           yes sir
>         {% else %}
>           Turning {{action}} {{room}} {{device}}   
>         {% endif %}
> 
>   action:
>   - service_template: >
>         {% if action == "on" %}
>             {% if device == "lights" %}
>               homeassistant.turn_on
>             {% elif device == "tv" %}
>               media_player.turn_on
>             {% else %}
>               switch.turn_on
>             {% endif %}
>         {% elif action == "off" %}
>             {% if device == "lights" %}
>               homeassistant.turn_off
>             {% elif device == "tv" %}
>               media_player.turn_off
>             {% else %}
>               switch.turn_off
>             {% endif %}
>         {% endif %}
>     data_template:
>       entity_id: >
>         {% if room == null %}
>           {% set room = site_id %}
>         {% else %}  
>           {% set room = room | replace(" ","_") %}
>         {% endif %}
>         {% if device == "lights" %}
>             group.{{room}}_lights
>         {% elif device == "tv" %}
>             media_player.{{room}}_tv
>         {% else %}
>             switch.{{room}}_{{device}}
>         {% endif %}

The action, room and device all seem to be received properly, but the site_id is null. When I looked at the logs, the thing I noticed was action, room and device were all slots. site_id is in the MQTT, but not defined as a slot.

[DEBUG:2020-10-22 10:11:14,727] rhasspyserver_hermes: <- NluIntent(input=‘turn on the office lights’, intent=Intent(intent_name=‘onOffIntent’, confidence_score=1.0), site_id=‘the_office’, id=None, slots=[Slot(entity=‘action’, value={‘kind’: ‘Unknown’, ‘value’: ‘on’}, slot_name=‘action’, raw_value=‘on’, confidence=1.0, range=SlotRange(start=5, end=7, raw_start=5, raw_end=7)), Slot(entity=‘room’, value={‘kind’: ‘Unknown’, ‘value’: ‘the office’}, slot_name=‘room’, raw_value=‘the office’, confidence=1.0, range=SlotRange(start=8, end=18, raw_start=8, raw_end=18)), Slot(entity=‘device’, value={‘kind’: ‘Unknown’, ‘value’: ‘lights’}, slot_name=‘device’, raw_value=‘lights’, confidence=1.0, range=SlotRange(start=19, end=25, raw_start=19, raw_end=25))], session_id=‘the_office-porcupine-4d311965-dd9c-4a61-8f30-325bf00e2354’, custom_data=None, asr_tokens=[[AsrToken(value=‘turn’, confidence=1.0, range_start=0, range_end=4, time=None), AsrToken(value=‘on’, confidence=1.0, range_start=5, range_end=7, time=None), AsrToken(value=‘the’, confidence=1.0, range_start=8, range_end=11, time=None), AsrToken(value=‘office’, confidence=1.0, range_start=12, range_end=18, time=None), AsrToken(value=‘lights’, confidence=1.0, range_start=19, range_end=25, time=None)]], asr_confidence=None, raw_input=‘turn on the office lights’, wakeword_id=‘porcupine’, lang=None)

So possibly I need to change something in my Home Assistant intent script to pull it properly

synesthesiam · October 22, 2020, 3:39pm

Here’s where the slots are being created for Home Assistant intents: https://github.com/rhasspy/rhasspy-homeassistant-hermes/blob/d78c9286171b593ced42608aa70a3c703d0516ec/rhasspyhomeassistant_hermes/init.py#L146

You can see it copies the slots from the intent itself, then adds a few “meta” slots (_text, _raw_text, and _intent). The site id is inside _intent, which is an object/dictionary. I would guess in your YAML, you could reference _intent.siteId and get the value, but I could be wrong.

jerome83136_tux · October 22, 2020, 5:04pm

Hi all,

Thank you for the grat support on this problem.

I have a very strange behaviour I would like to share in case it could help troobleshooting the issue we are talking about here.

I pulled the latest docker image 2.5.7 today at ~12:30 (FR time); and even if I guess I didn’t get the latest fix that was released later, I would like to share the problem I noticed:

I pulled the 2.5.7 image on:
– satellite1 --> standard computer (debian amd64)
– satellite2 --> respeaker core v2
– master --> laptop (debian amd64)
I replicated Rhasspy configuration from satellite1 to satellite2, so that they are identical (except required customizations like main siteID declaration, etc…)

After this, satellite1 and master were running without any problem (as they were before)
satellite2 was not working (Rhasspy hangs after successfull wakeword detection, like described in the current topic)

The strange thing: I have noticed that if I just disable “Audio output” on the satellite2, it works.
In this case, I have no audio (obviously) so I have no “beep” when wakeword detection triggers.
I have no logs at all on Rhasspy webui.
But the intent is handled and the intent handling is executed correctly (remote http to jeedom in my case)

I have tested with all possible options for “Audio output”, the only ones that are working are:

Disabled
Hermes MQTT
Both are preventing any sound output on the respeaker wich sounds logic, I guess.

By changing only this setting, Rhasspy seems to behave in different ways.

I don’t know if this feedback could be usefull. But I really don’t understand what kind of logic could exists between Wakeword hanging and Audio output configuration ?

PS: I also have to precise that intent handling is working as expected on my setup between all my satellites and the master.
Only the master has Intent handling activated and configured to target the Jeedom API
(seems to work since I have added the master itself into satellites siteIDs on the intent handling tab, on the master configuration UI)

Hope it will help
Best regards

nYce · October 22, 2020, 5:32pm

I also updated my docker image after your fix and i ran into the same behaviour than before the fix.
After the first recognized intent after a restart i can wake rhasspy but it doesnt do anything more. i can just rewake it with the wakeword again, but no intents are getting pushed to mqtt or the web interface.

Did a rollback to 2.5.6.

Platup · October 22, 2020, 9:37pm

_intent.siteId worked! Thank you!

and in case anyone else needs to parse the wake word, _intent.wakewordId worked for me.

e-tobi · October 23, 2020, 6:49am

Ran in to the same/similar issue. Just wanted to “play around” with Rhasspy an installed it locally on my notebook running Debian Buster (clone, from git (HEAD), configure, make, make install).
Once started, the wakeword is detected an it also detects the sentence the first time. The second time it only detects the wakeword, but not the sentence. The sentence is detected fine, when doing a manual wake up. After restarting Rhasspy wakeword+sentence work again on time, then only the wakeword is detected.

When switching to the v2.5.6 tag it works just fine. With a little git bisect work, I could pinpoint the issue to this commit in the dialog sub module:

synesthesiam · October 23, 2020, 1:18pm

Hi @e-tobi, you’re correct that that commit where things started to go downhill. With the recent update to 2.5.7, though, things should be working correctly now. Are you still seeing problems?

e-tobi · October 23, 2020, 3:14pm

I’m working on the git version, when I clone with git clone --recursive https://github.com/rhasspy/rhasspy then the rhasspy-dialogue-hermes submodule does not point to origin/master but to dde6bdde.
If I manually update the submodule, everything works fine.
A git submodule update --remote --merge may fix this , but I’m not an expert when it comes to submodules in Git - I usually try to avoid using git submodules

synesthesiam · October 23, 2020, 3:31pm

Good catch, thank you I’ve updated the submodule and pushed the changes.

I do too, but I couldn’t think of a better way to handle the split of Rhasspy 2.5 into independent services and still keep track of them all. It occasionally bites me, but I think I’ve managed to tame the submodules by not doing anything fancy with them.

e-tobi · October 23, 2020, 5:55pm

Thanks! Works fine now “out-of-the-git”