Hermes skill server for intent handling

Intro

Coming from a longer discussion in rhasspy’s issue tracker, I’d like to share a prototype for a hermes-based skill server application I’ve made a month ago. Its called hss-server (Hermes Skill Server), and allows for easy intent handling implementation and management.

I’ve started development on this as some kind of replication of the snips-skill-server, since I had a working voice assistant already for the snips.ai platform, which now is dead. It follows similar design principles, and aims to provide a stupidly easy to use programming interface, CLI and configuration.

In short, hss-server is meant to be a system service/daemon which is hosting third-party intent handlers (“skills”), and connects to MQTT to receive hermes protocol messages. hss-server talks to individual skill instances via RPC, and each skill is running as separate process with its own python virtual environment. Currently, all this is done in python, however there have been discussions on allowing other technologies as well.

To develop skills for the hss-server, I’ve provided a client library hss-skill, which needs as little as 2 methods to be implemented for a working skill.

Repositories for the server and the client library are available here and here, together with detailed setup instructions and guidelines.

Example

A minimal working example of a skill cool look like:

main.py

import myskill

if __name__ == "__main__":
    skill = myskill.WeatherSkill()
    skill.run()

myskill.py

from hss_skill import hss

class WeatherSkill(hss.BaseSkill):
    def __init__(self):
        super().__init__()   # important, call super's constructor

    def get_intentlist(self):
        return ["howAreYou"]

    def handle(self, request, session_id, site_id, intent_name, slots):
        return self.done(session_id, site_id, intent_name, "Thanks, I am fine")

The installation of the server could be as easy as:

/home/s710 $> mkdir hss
/home/s710 $> cd hss
/home/s710/hss $> mkdir hss
/home/s710/hss $> python3.7 -m venv /home/s710/hss/venv

/home/s710/hss $> source venv/bin/activate
(venv) /home/s710/hss $> pip install hss_server

Features

When starting the server, it is going to load all installed skills and connect to MQTT:

(venv) /home/s710/hss $> hss-server
INFO:hss: Hermes Skill Server v1.0.0
INFO:hss: Using config dir '/home/s710/.config/hss_server'
INFO:hss: Loading skills from '/home/s710/.config/skills'
INFO:hss_server.skillserver: Loading skills ...
INFO:hss_server.collection: Initializing skills ...
INFO:hss_server.collection: Skill 'hss-skill-s710-weather' loaded
INFO:hss_server.collection: Loaded 1 skill
INFO:hss_server.skillserver: Connecting to MQTT server ...
INFO:hss_server.mqtt: Connected to 10.0.50.5:1883
INFO:hss_server.mqtt: Publishing TTS to topic 'hermes/tts/say'
INFO:hss_server.mqtt: Subscribing to topic 'hermes/intent/#' ...

A CLI tool hss-cli is provided to easily install and update skills:

(venv) pi@calypso:~/hss/venv $ hss-cli -l
--------------------------------------------------------------------------------
Skill                               Version  Commit  Date                 Config
--------------------------------------------------------------------------------
hss-s710-weather                    -        2ff87ec 05/06/2020, 07:00:18 y
(venv) pi@calypso:~/hss/venv $ hss-cli -u
Updating ALL skills. Continue? (YES|no)
Updating skill 'hss-s710-weather' ...

Skill 'hss-s710-weather' successfully updated.

1 Skills updated, 0 skills without update

Roadmap

Currently, hss-server is based on my usage of the version 2.4 of rhasspy, and thus does not yet cover the whole hermes dialog functionality. This is currently in the works and will be available during the next few days.

In addition, ideas of providing a list of skills which are available for installation (“marketplace”) and automatic injection of training sentences is under discussion.

Thats it for now, as this is only a very short intro, I’ll probably complement more info later :slight_smile:

5 Likes

Sounds like impressive work, awesome job.

Thanks :slight_smile:

As a reference, heres the link to the issue tracker: https://github.com/synesthesiam/rhasspy/issues/217

As a new user I was only allowed to have 2 links in a post :man_shrugging:

Maybe you check out @koan 's post:
https://community.rhasspy.org/t/helper-library-to-develop-rhasspy-apps-in-python
and his git repo:


This sounds very similar to me.

However since both of your projects are still in the making, maybe there could be a solution with the best of both worlds?
Just to prevent doing twice the work.

Yes we already had a discussion on github. I didnt know there was any existing implementation when I made the hss in april.

And I already offered to join resources and develop the best possible solution for intent handling :slight_smile:

But since the work on hss-server is already done, I wanted to post it here on the forums. Also see my comment on github.

1 Like

I don’t see any problem with multiple implementations. I was aware of Patrick’s hss-server and hss-skill when I started thinking about rhasspy-hermes-app, and while I was impressed by the server part (especially for installing and managing skills, although I prefer a Docker container for each skill for security reasons), I didn’t like the way the skill library works. Essentially all the logic is in hss-server.

That’s not wrong, it’s just another approach for this task. While hss-server and hss-skill are tightly intercoupled, I prefer loosely coupled systems. Essentially the API for Rhasspy apps is the Hermes protocol over MQTT, and the task of an app library is to hide implementation details for the app writer. The hss-skill library does that by communicating to hss-server, which talks Hermes to the MQTT broker. The rhasspy-hermes-app does that by acting as a wrapper around the rhasspy-hermes library to talk directly Hermes to the MQTT broker but with some bookkeeping tasks hidden.

Other implementations are possible and equally viable, such as letting AppDaemon do the MQTT communication, an approach that I also experimented with and that @daniele_athome is working on now. Actually, the hss-server architecture looks a lot like AppDaemon but with the advantage that each skill runs in their own Python virtual environment.

And now I have only talked about the Hermes protocol over MQTT. Rhasspy also has a REST API, and this can also be used to create apps :slight_smile: Which approach you choose is a matter of taste and specific requirements you have.

So we should definitely keep sharing some thoughts and resources between developers of different app libraries and other supporting tools for Rhasspy, but I think it’s impossible to come up with one solution that’s tailored to everyone’s needs.

1 Like

Agreeing with what @koan said, I want to point out that the idea of hss-server is more than just developing apps. In fact, its meant to give developers as well as users the possibility to quickly get apps/skills up and running, by either developing custom skills with few efforts, or pick existing skills and install them in a convenient way.

In addition, I am working on concepts for some kind of skill marketplace/registry, which will ease the usage and installation of skills even more.

That beeing said, @koan maybe it can be possible to hook your docker-apps too on said marketplace, and have the CLI install it in the appropriate way (either hss-cli -i ... or docker pull ...). In this way, users can choose whatever approach suits them best, and pick from a wider range of existing skills.

Yes that’s the part I really like about hss-server (and snips-skill-server in the past): it makes it very easy to install skills.

Well, to be more precise the rhasspy-hermes-app library doesn’t care about Docker: it just lets you create a Python program that can work standalone. You can run it as is, in a Python virtual environment or in a Docker container. That’s up to the user. Of course the app developer can generate a Docker container for the app, and I have plans to automatically generate a Dockerfile and Mosquitto ACL file to securely run a Rhasspy app.

I do think a marketplace should support these various ways to run an app, and also apps written in various programming languages. Users don’t care about the programming language an app is created in.

So what would be your idea for a skills marketplace?

I haven’t really been able to think about it yet, because I have been focusing on the app library part for now, and I expect that this will remain the same in the next few months. So apart from the ideas you quoted, I don’t have anything to add yet.

Okay so I have been busy the last few days integrating Node.JS functionality into hss-server (welcome node-hss-skill), and now I am working on integrating sentences, slots, training & marketplace functionality.

I am going to start with some really simple registry server, which shouldn’t cost too much effort, and can be extended if there is enough interest for it.

1 Like

@patrickjane Have you taken a look at making your projects work with Rhasspy 2.5? I just compiled a list of awesome Rhasspy projects and resources but your projects didn’t make it yet because I don’t allow projects that only work on older releases. It would be awesome to have your projects on the list :slight_smile:

I have implemented the hermes dialog protocol, so on that regard it should be compatible with rhasspy 2.5, while still beeing backwards compatible to rhasspy 2.4.

However so far I havent gotten 2.5 to run, so I cannot really test it out. My last attempts were some two weeks ago, so I might give it another shot next week maybe.

1 Like

Cool! If you (or someone else) has verified that it works on Rhasspy 2.5, feel free to open a pull request.

Just catching up - without copying over all the detail of the Github discussion, I’ve been using hss-server with 2.5 exclusively (and don’t recall any particular issues doing so, once I’d got the rest of 2.5 set up sensibly), but haven’t tried @patrickjane’s most recent changes.

1 Like

Thanks for the confirmation, @philtweir! I have added hss-server, hss-skill and node-hss-skill to the list.

Nice, thats great. BTW: still unable to test it out (see #244 on github). Also had other issues with voltron/2.5 pre-release earlier, so somehow it seems like 2.5 does not like me :slight_smile: :frowning: