Offline Voice Assistant with Rhasspy and Home Assistant Intent Recognition

Hi folks, first time poster and recently new Rhasspy / HA user here. I am trying to setup an offline voice assistant (no time like quarantine!), and The Internet says Home Assistant + Rhasspy is the way to go. I have a lot of the pieces working, but I can’t quite close the loop. I am hoping someone can help me out.

What I’m using:

  • My MacBook Pro laptop; I would prefer to delay buying dedicated hardware until I have a working proof-of-concept
  • Home Assistant Core version 0.113.3 running in Docker
  • Rhasspy 2.5.5 also running in Docker

The sticking point is intent recognition. I want to use Home Assistant for intent recognition for two reasons:

  1. I eventually want to do automation in addition to voice/text commands, which is where Home Assistant shines; and I don’t want to have to duplicate my intents in multiple systems.
  2. My two main use cases (so far) for this voice assistant are in the kitchen: timers, and shopping list. For timers, it is convenient to able to give the timer an arbitrary name. For the shopping list case, the item to be added to the list does not need to be known to the intent recognizer ahead of time. Both cases seem to be better served by HA’s dynamic slots than by Rhasspy’s static slots (as long as the STT still works; but I consider that a separate problem).

The good news is that Rhasspy has an integration with HA’s conversation system! The bad news is that it is “Not supported yet in 2.5” :frowning: And as far as I can tell, the Docker image was not created until 2.5, so I cannot go back to an older version where the integration was supported.

So my options, as far as I can see them, are:

  1. Buy hardware, install Debian and run Rhasspy 2.4, so I can use the HA conversation integration.
  2. Try to get a working Docker image for 2.4. I’m somewhat experienced with Docker and am willing to give this a try, but from what I can tell a ton has changed between 2.4 and 2.5 (including the repo), and I’m not sure if the new Docker setup would even apply to the old version. But if someone wants to give me some pointers I will take a shot at it.
  3. Make the necessary changes to get the HA conversation integration working with 2.5. Again, willing to give it a shot, but:
    a) from what I can tell these are architectural changes we’re talking about, not minor tweaks;
    b) I know nothing about MQTT or Hermes, and very little about HA and Rhasspy;
    and probably most importantly,
    c) I have written only a few lines of Python in my life :slight_smile:
  4. Wait for someone who actually knows what they are doing to update the HA conversation integrationin 2.5. Maybe use Rhasspy intent recognition and HA intent handling until then.

Any tips, tricks, or suggestions for me? Any updates on the status of the HA conversation integration? I couldn’t find anything on the project boards.

P.S. I really appreciate the UX of Rhasspy. While I am comfortable editing YAML based on docs, the config page and download prompts make it really easy to get started and have more confidence that I haven’t messed it all up. The docs are quite good too, particularly for an open source project. Thanks for the hard work!

1 Like

There is a docker image for 2.4, or at least there was, I doubt that it was deleted. Rhasspy moved to a new repository and therefore the docker image was moved also. Here is a link to the old repository, so you should be able to install it with the install guide from there.

Have you tried using Home Assistant’s Snips integration? As Rhasspy 2.5 is talking the Snips Hermes protocol, that should work, as long as you have an MQTT broker running.

@Daenara Thanks for that. The rhasspy/rhasspy images only have 2.5, but based on your link I found the synesthesiam/rhasspy-server images which might work for me. I will give that a try.

@koan I have not tried the Snips integration. I looked at Snips early based on various recommendations I saw, but once I discovered the Sonos acquisition and that it was taken private, I kind of stopped paying attention to it. I have gotten the impression since then that the integration is more generally useful than the Snips platform itself. If you think it’s a good fit for my scenario I will take another look.

Yes, I think it is! The protocol is (as good as) the same.