Local python intent skills

Hi,

There a lot Rhasspy can do yet, with HA and I hope soon with Jeedom, but is it possible to handle some of our intent with a simple python script ?

For example, if intent recognized is ‘GetTime’, it would start a python script than will import rhasspy as a package, get the hour, and send back a tts command to say the answer on the right site_id ?
Sort of Snips skills :upside_down_face:

I have been looking at AppDaemon to do this. I made a very simple proof of concept and this works, but there’s still a lot of work to make this easy and maintainable for app developers, including adding intents in the app, supporting multiple languages, and so on.

Eventually we should have something like SnipsKit, which I wrote to streamline creating Snips apps. I don’t want to have a hardcoded URL for a Rhasspy installation in apps, for instance, which was a real pain in the ass with the way Snips apps worked by default.

SnipsKit is awesome but as I do not code in python, I cannot use it :slight_smile:

Running local intent skills should be pretty straightforward using the Command intent handler.
https://rhasspy.readthedocs.io/en/latest/intent-handling/

On the other hand, running installable intent skills like Snips looks like a lot of work:

  • a SDK to build the skill (in multiple language like Javascript, Python, etc.) and the binaries to execute them
  • a skill store to find them
  • a skill repository to store them (Github?)
  • a skill installation process (Oauth2? APi keys?)
  • a skill manager to control all this
  • etc…

There is the Alice project that do just that but is still based on Snips softwares. The devs are migrating away from these Snips dependencies since the Sonos acquisition but they are not there yet (that’s why I recently spoke to them about Rhasspy to provide the parts they lack and plug everything together via the Hermes protocol but it looks like they want to do their own system).

I think Home Assistant, Jeedom, Hermes protocol MQTT/Websocket and Command intent handlers should be enough and that Rhasspy should focus on the voice interaction part (that’s the hardest part for makers to get right).

The more advanced users probably want to implement their own skills system based on their favorite language/system (as I did) because the interactions end up to be quite complex (playing music via Spotify, triggering alarms across multiple satellites, controling a multiroom playback, per user restrictions, assistant personality, etc.).

1 Like

You can create a shell_command in Home Assistant

that can be a python command

Call that from an automation (create to respond to an event coming from Rhasspy)

A better option might be python scripts:

But that also has limitations.

It would need a command handler per intent then !

I don’t wish/ask for a store, repo, install etc, just a way to handle local python scripts for devs.
Once @synesthesiam add the Command intent handler, I will be able to handle all my need in Jeedom, even running python/php script on some intent. So that’s not high priority for me, just though some other users would also get back into such skills. I use a few with Snips, but can handle them with Jeedom once Rhasspy got its connection to it :wink:

No, you would have 1 command handler handling all events.

Ah, then the command start a python dispatcher that check intent name and run the right script ? Easy enough I guess.

Yes, see details here:

This already should work by the way, but you will have to edit je profile.json file because the webUI does not seem to have a radiobutton for it.

Hmm I don’t talk about externalising intent recognition.
Just once intent is recognized, if intent = myintentname call this python script

Yes, but you can do that in the command intent handler.
Either you have to program it in HA (or other software), or program it in the command intent handler.
This is not intent recognition, but intent handling. There might be some confusion about that?

What I find really missing from the current Command intent handler is the ability to return a response via stdout to be automatically spoken by the TTS system (if enabled).

2 Likes

Just added this: https://rhasspy.readthedocs.io/en/latest/intent-handling/#command (see the Speech sub-section) :slight_smile:

4 Likes

I’m building an interactive robot for my kids (and for me) to have fun with.

I’ve been using the rhasspy-client package and the HTTP API. Here is a short example of the pattern I’ve been using. Hopefully it may be useful to others:

#!/usr/bin/env python

import asyncio
import aiohttp
import websockets
import json
import datetime
from rhasspyclient import RhasspyClient

url = 'http://localhost:12101/api'


async def speak(text):
    async with aiohttp.ClientSession() as session:
        client = RhasspyClient(url, session)

        # Call sub-commmand
        await client.text_to_speech(text)


def async_speak(text):
    global loop
    print(text)
    asyncio.ensure_future(speak(text), loop=loop)


def GetTime(intent):
    # If you need, you can access a dict of slots in the intent
    # slots = intent['slots']
    now = datetime.datetime.now().time()
    hour = now.hour
    minute = now.minute
    t = hour + (minute / 60)
    if (hour > 12):
        hour -= 12
    if minute == 0:
        minute = 'oclock'
    elif minute < 10:
        minute = 'o %d' % minute
    stime = "%s %s" % (hour, minute)
    async_speak("Let me check. It is %s." % (stime))


def Christmas(intent):
    # If you need, you can access a dict of slots in the intent
    # slots = intent['slots']
    today = datetime.datetime.now()
    christmas = datetime.datetime(today.year, 12, 25)
    if (today >= christmas):
        christmas = datetime.datetime(today.year + 1, 12, 25)
    days = (christmas - today).days
    async_speak("There are %s days until Christmas, not including today." % days)


async def get_event():
    uri = "ws://localhost:12101/api/events/intent"
    global timers
    async with websockets.connect(uri) as websocket:
        intent = await websocket.recv()
        intent = json.loads(intent)
        #print(f"< {intent}")
        cmd = (intent['intent']['name'])
        f = globals().get(cmd)
        if f:
            f(intent)
        else:
            print('Got unhandled command: %s' % cmd)


loop = asyncio.get_event_loop()
while True:
    asyncio.get_event_loop().run_until_complete(get_event())

I am wondering about ways to have the sentences grouped with the intents. So that when implementing an intent you can see at a glance which slots and structure you have in the grammar. To this end, I am thinking about embedding the sentence/grammar in docstrings, then sending them to rhasspy (and running training) using the API when my robot script starts up.

5 Likes

I did try to set a python script as intent handler.

  • Can’t get the script to run, with config:
"handle": {
    "system": "command",
    "command": {
        "program": "/home/pi/.config/rhasspy_handler/intentDispatcher.py",
        "arguments": []
    },
    "forward_to_hass": false
},

Always got:

FileNotFoundError: [Errno 2] No such file or directory: ‘/home/pi/.config/rhasspy_handler/intentDispatcher.py’: ‘/home/pi/.config/rhasspy_handler/intentDispatcher.py’

  • And setting system command make me loosing system remote, so no more post to server (jeedom plugin).

My Jeedom plugin is set to handle only intentNameJeedom names (finish with jeedom), so I could have a python script handling other intents. But can’t get it starting, and can’t get the two, command AND remote.

I could handle to post myself to jeedom plugin in the intent handler, but do you have an exemple of path working ?

1 Like

Hi Jason,

i tried your script but i have the following error :

ModuleNotFoundError: No module named 'rhasspyclient'

Indeed i can not find a rhasspyclient.py file in Git :thinking:

pip3 install rhasspyclient

pip3 install rhasspyclient
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting rhasspyclient
Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://pypi.org/simple/rhasspyclient/

package name is rhasspy-client :wink:

Thx, error is gone but now i have this one :

Command '['/home/pi/intentDispatcher.py']' returned non-zero exit status 1. 

script runs (for ever) without error in the shell

Here are my logs :

[INFO:8485438] quart.serving: 192.168.1.17:61453 POST /api/text-to-intent 1.1 200 423 1105846
[ERROR:8485428] CommandIntentHandler: in_started
Traceback (most recent call last):
  File "/home/pi/rhasspy/rhasspy/intent_handler.py", line 323, in in_ready
    self.command, check=True, input=json_input, stdout=subprocess.PIPE
  File "/usr/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/pi/intentDispatcher.py']' returned non-zero exit status 1.
[ERROR:8484402] __main__: api_events_intent
Traceback (most recent call last):
  File "app.py", line 1122, in api_events_intent
    text = await q.get()
  File "/usr/lib/python3.7/asyncio/queues.py", line 159, in get
    await getter
concurrent.futures._base.CancelledError
[INFO:8484387] quart.serving: 127.0.0.1:51632 GET /api/events/intent 1.1 101 - 166374865
[DEBUG:8484383] HermesMqtt: Published intent to hermes/intent/GetTime
[DEBUG:8484356] CommandIntentHandler: ['/home/pi/intentDispatcher.py']
[DEBUG:8484345] __main__: {"intent": {"name": "GetTime", "confidence": 1.0}, "entities": [], "text": "quelle heure est-il", "raw_text": "quelle heure est-il", "recognize_seconds": 0.002108573000441538, "tokens": ["quelle", "heure", "est-il"], "raw_tokens": ["quelle", "heure", "est-il"], "speech_confidence": 1, "slots": {}, "wakeId": "", "siteId": "default", "time_sec": 0.009440422058105469}

@duch, I wrote the script intending it to be a standalone script that runs continuously, handling intents that Rhasspy detects. I took this approach because I am not running home assistant in my project. Rhasspy needs to also be running (I run it in docker).

What is your context? How you are calling this script?

that’s what i thought reading the code but i don’t know python, i’m more a node guy.

anyway, i tried to call this script from rhasspy using the command system : https://rhasspy.readthedocs.io/en/latest/intent-handling/#command

completely off topic :rofl: i’ll try to run your script as a service and see what’s going on.