The Future of Rhasspy

Hi everyone,

As some of you may know, I recently changed jobs and joined Mycroft AI as a senior developer. The two reasons for this change were (1) I needed a fully remote position in order to move closer to family, and (2) my previous job was going to force me to stop working on open source. No thanks!

Since I’m not independently wealthy :face_with_monocle: , I applied to a few places and Mycroft made me a good offer :slight_smile:
Obviously, this begs the question of what is going to happen with Rhasspy, so I’m here to talk about that.

Most importantly, I am not abandoning Rhasspy. However, updates are going to come slower due to time constraints. Mycroft hired me to help get their Mark II smart speaker over the finish line, and I am going to do that. I’ve given Github access for Rhasspy to a few folks already, but if any other maintainers would like to volunteer, please do :raised_back_of_hand:

My long term goal is to get many of Rhasspy’s key features into Mycroft. As you can see, I’ve already got offline speech to text and text to speech working! Satellites and Hermes protocol support are something I’d also like to do down the road.

Ultimately, I hope our communities can join together at some point. I stayed away from Mycroft initially because of its dependence on a cloud server, but that’s going to change now with me there. Other things, like a skill store and a dedicated hardware device, I just didn’t have the time and resources to produce. So it would seem both sides here would have something to gain.

What are your thoughts?

12 Likes

Congratulations with the new job!

I stayed away from Mycroft for the same reason as you did, and I’m happy to hear that its dependence on a cloud server is going to change with you as a developer there!

What are some things we in the Rhasspy community can do to ease the path of joining both communities together?

2 Likes

Congrats on your new job!

I have taken a look into Mycroft a while ago but the main reason for now using it was the online recognition. Also, they did not support Dutch.
You joining them gives new hope to that :smiley:

Hopefully I can put some more time into solving issues for Rhasspy, but my spare time is also limited.

Keep us up to date and good luck!

2 Likes

Nice to have some news and congrats for your new job, you highly deserve it !

I never dig up into mycroft due to cloud so I don’t how it could and even would replace rhasspy. Custom wakeword, home automation intégration, custom intents and such comes to mind …

Really hope rhasspy will continue to be enhanced for a while.

3 Likes

Hi !
I would say, as far as you keep your open mindset and still beleive in Rhasspy roadmap, it’s a triple good news:

  • You find a job to fullfill your essentiel need (pay the bills !) :money_mouth_face: in relation to your passion
  • You may bring Mycroft out of the dark clouded side and keep them far away from the GAFAM :angry:
  • Rhasspy audience may rise again and then get new supporter and may be contributors :nerd_face:

Cheers and all the best in your new job (happy Mycroft people)

2 Likes

Thanks for the kind words, everyone! I’ll keep you up to date on my plans; please feel free to message me with ideas.

For me, it would be helpful to get a sense of what people find most useful about Rhasspy, and where Mycroft is lacking. I already gave my own examples, but the Rhasspy community is pretty diverse. I don’t want to leave anyone behind, but we will need to make some decisions about what to “take with us”.

Congrats on the new job!

Coming from someone who’s done a lot of integration with Rhasspy for Home Intent, I can definitely say your technical documentation has been amazing. All the ins-and-outs of hermes and setting up various configurations (satellites!). I’ve also really enjoyed that all the various components are pluggable, easily controlled via the API, and it’s all listed as to what works with various languages. I know you’ll bring a lot to the table at Mycroft and they are very lucky to have you!

I’ve also been very impressed with your work on Larynx, and hope that it will continue!

1 Like

For me, language support and offline STT is key.

3 Likes

having worked a lot on the use of the Snips application before its takeover in early 2019 by the company Sonos, and during the shopping of Christmas gifts I was able to observe that the sonos speakers currently only worked with Google home and Alexa I conclude that to eliminate the competitor Snips Amazon and Google had to deal with sonos to definitively eliminate the Snips application. Which app is closest to the offline aspect of Snips right now ??? What do you think of the Alice project compared to Rhasppy? I have worked on other projects during this time and I have to get back to them. Thank you for your advice Regards

1 Like

Rhasspy

You can find a topic about that here:

Congrats on your new job.
Long live to both Rhasspy and MyCroft.
I think both can run offline! People have used MyCroft offline in packages for Kids and places where internet access had been issues even though not official.

I will be happy to see both Rhasspy and MyCroft grow more and more… even collaborate together

1 Like

Congrats and I understand why you did this. But I see this as a big loss, similar to when Snips was acquired. I think it means the death of the Rhaspy project even if more slowly than Snips. *** I hope I’m wrong. *** The problem is when companies take over, almost always, the open source aspect goes down. Meaning the ability for programmers to use and add onto a tech platform without friction - obtaining/paying for licenses, etc. I can see this as a win for the open source, voice developers, community if two things happen… 1) Mycroft eventually gets all the Rhaspy functionality you will hopefully help give it. and 2) Mycroft embraces the OS/developers and doesn’t go the same path as Sonos.

1 Like

I’m happy to hear that you have a positive attitude toward both Mycroft and Rhasspy.

There is something of a history of voice computing projects losing steam after developers are hired by commercial projects. The Jasper project died after the developers were hired by Microsoft to work on Cortana. KSimon has been limping along since the primary developer was hired by Apple to work on Siri. So it’s not without reason that people were concerned.

I have also avoided Mycroft for years because I don’t buy their marketing around the cloud component (“we are open source, so you can totally trust us”, “using our servers helps anonymize your activities”). I’d love to feel more comfortable inviting that project into my personal space.

I’m happy to see that Mycroft does make the Selene back end available. I will try and set that up again (the last time I tried was several years ago using an unofficial and unsupported repository, and I gave up after a couple of days). One place where I do see the benefit of a centralized back-end is for organizations like schools and health care providers who need to keep PII protected while also providing a consistent and custom set of services.

Congratulations and best of luck with your new position. Thank you for the efforts you have devoted to open source voice interfaces.

1 Like

Thank you again to everyone for the warm replies!

Very happy to hear this :slight_smile: As @romkabouter mentioned, language support is very important both with (offline) STT and TTS. Every language we can get enough data for is another group of people that don’t have to rely on Google, etc.

It will! I’ve already started on a follow-on that I’m calling Mimic 3 (to fit Mycroft’s naming scheme). It’s close in architecture to Larynx, but is using a new TTS model that is almost 2x faster on the Pi 4. I’m still figuring out the best hyper-parmeters to use, but once I do I will retrain all of the voices and upgrade Larynx :+1:

I understand, and I won’t pretend like I can 100% promise what the future will hold. However, at least with Mycroft what I work on will be completely open source. Snips’ tech was amazing, but it’s effectively worthless now because of the Sonos acquisition.

I will be working to get Rhasspy’s tech into Mycroft over the coming year. The Mark II will hopefully be shipping in September (if they get enough orders), and my goal is to have offline STT and TTS for all of Rhasspy’s supported languages on board.

Definitely, and unfortunately. I feel like having some shared standards could help ease the burden of shifting between projects. For all its warts, Hermes enabled a lot of Snips users to switch over to Rhasspy without redesigning everything. I hope to finally have some time to work with @sepia-assistant and @DANBER to establish those standards.

No company is immune from acquisition, so you shouldn’t trust them! Mycroft seems very unlikely, however, to go down the same road as Snips.

You’re welcome, and I see myself as a long ways from finished :smiley:

1 Like

Congrats on the new job!!!

Seeing everything you contribute will be open source, what works for one project will most likely work for the other project, so win win.

1 Like

For me Mycroft is lacking three important things :

  1. TTS Support is very limited. For German I only have the unusable slow Mozilla Deepspeach Model or low quality options. Hearing you want to create an even better Larynx called Mimic 3 which can be used with Mycroft sounds really great.
  2. Im not using Home Assistant for controlling my Smart Home, so I need an option to process the intend in NodeRed or at least configure Get & Post HTTP API Requests. And it’s just great if you want your own automation flows. Like my alarm flow im currently building in rhasspy does not only ring an alarm at some time but also powers up my room heating and slowly increases brightness of my lights beforehand. (While writing this comment i found this : https://github.com/JarbasHiveMind/HiveMind-NodeRed . So there seems to be some kind of community skill which is able to do this)
  3. It couldn’t be used completely offline. Sounds like this is going to change too.

Im actually really looking forward to this.

4 Likes

One trick I use to improve performance is to use the engines with caching.
So what I do is have most responses as simple as possible, “like the light is on” rather than “the bedroom light is on” and for things that are more dynamic like weather, temperature, humidity, time date, etc I actually have node read send the sentence to TTS whenever it changes but only play it when there is a request, so it is already cached and the response is close to instant.

Jarbas is an excellent buddy with one stop shop for everything AI, Voices and more… :smile:

Happy Adventures with the new job.

For me Rhasspys’ ability to do custom intents is my favorite tool.
My Rhasspy installs: In a silly bot, in two solar monitoring stations (RV and house), in an offline house assistant, in a offline RV assistant, all simply give info to my custom python programs which do all the handling, and send responses back, go online and post this or that api, do gpio stuff, turn camera on and off whatever.
Rhasspy has the Power of Whatever: so that’s my favorite feature.
I did play with its home assistant integration as well, but found my custom python stuff much more usable for my unique projects.
For me to move to mycroft:
*. No need to be online, unless updating something, and no need to have anything posted to someone elses’ cloud server to do anything, sure If I need the weather I can bring LAN or Wifi up (I have an intents for that), Python does its thing like a request to a weather api, then LAN down wifi down (intents for those as well). It all works with Rhasspy.
*.I do use the mycroft custom wake word (I log into an older install of Ubuntu Studio I didn’t erase just because I setup the mycroft custom wake word tools on it awhile back.I’ve not needed to make any new ones in awhile though.

2 Likes

I deeply respect you @synesthesiam. While It saddens me that you’ll surely have less and less time for Rhasspy, I understand the need to eat and pay the bills. What devestates me far more than losing Rhasspy though, is that your skills are being gained by somewhere as deceptive as Mycroft.

Yes they tout loudly that their code is open, but this is not a privacy respecting project, nor is the company behind it. I came to Rhasspy precisely to get away from the likes of Mycroft. There are reasons that their project is specifically structured to default to using their cloud services (sound familiar?). There are reasons they demand user logins (also familiar…?). Their project has been specifically structured in a way that makes self hosting, and I quote “not easy and is unlikely to provide an equivalent user experience [as sending your personal info to their computers].”

I mean, is Mycroft different than GAFAM? Maybe, but if so, it doesn’t seem to be for lack of trying to emulate them. Let’s take a quick look, not at their glossy marketing homepage, but at their privacy policy (which covers both their website and their services). Just a quick skim through, copy and pasting the privacy relevant bits comes up with:

When you use our Services including the Mycroft Voice Assistant, your voice and audio commands are transmitted to our Servers for processing.

Because they made it, you know “not easy” to self host, like say Rhasspy currently is.

We collect information about you directly from you and from third parties, as well as automatically through your use of our Site or Services

Just to clarify that it’s not just “opt in” stuff.

When you use our Services, your audio commands are transmitted to Mycroft for processing, as part of the Services. We may also collect other metadata about your audio commands, such as the time and location.

Note that it’s not limited to information needed to fulfill the request.

we collect information about your device, including platform type and location

Once again, specifically not limited to fulfilling the service.

If you comment or post content to the Services, we may gather data about the content you post.

Just in case you were wondering if they would also use the content.

We automatically collect information about you through your use of our Site and Services, log files, IP address, app identifier, advertising ID, location info, browser type, device type, domain name, the website that led you to our Services, the website to which you go after leaving our Services, the dates and times you access our Services, and the links you click and your other activities within the Services.

That’s a list that would make Google proud, especially the advertising ID. What could that be for?

To send you news and newsletters, special offers, and promotions, or to otherwise contact you about products or information we think may interest you.

Because privacy focused companies are all about promotions and special offers.

for other research and analytical purposes

(how much vaguer and more all encompassing can you get than “other research” and “analytical purposes”?!)

To protect our own rights and interests, such as to resolve any disputes

So you control the assistant in my home and can use it’s info against me in a legal dispute?

We may share your information, including personal information

Just to be clear…

We may disclose the information we collect from you to third party vendors, service providers, contractors or agents who perform functions on our behalf.

Once again, pretty much to anyone.

If we are acquired by or merged with another company, if any part of our assets are transferred to another company, or as part of a bankruptcy proceeding, we may transfer the information we have collected from you to the other company.

As with pretty much any company… which is one of the problems with company run projects.

We may share aggregate or de-identified information about users with third parties for marketing, advertising, research or similar purposes.

Just in case you thought they were only collecting to improve the project.

We and our third party service providers use cookies and other tracking mechanisms to track information about your use of our Site or Services. We may combine this information with other personal information we collect from you (and our third party service providers may do so on our behalf).

Oh? Like who?

We use automated devices and applications, such as Google Analytics

Ah, that’s who (at least one of the who’s…).

our Site does not recognize browser “do-not-track” requests

Real “privacy focused”, right?

If you’d like to update your profile information with us, you may do so through your account. […] we may maintain a copy as part of our business records.

So you can ask us to delete it, buuuut… “business records” y’know?

Our Services are not designed for children under 13; and children under 13 are not permitted to have an account with us. If we discover that a child under 13 has provided us with personal information, we will delete such information from our systems.

Why no users under 13? Much like Audacity moving to ban users under 13 when they added tracking and Google Analytics. In many areas it’s illegal to track personal data about children for commercial purposes. If you want an easy surveillance advertising based income, you gotta get rid of users under 13.

A couple of these by themselves could be understood, but what’s the overall pattern here? Is this the pattern of a privacy centric company, recording advertising ID’s, location, IP addresses, content and using it to give to third parties and marketing?

I’m sorry if this doesn’t come across as supportive as the other comments here. Most likely this post will lead to personal attacks against me by people who feel that they’re defending you, all while encouraging you towards being controlled by “© Mycroft AI, Inc.”’. Yes, the code may still be open, but make no mistake, this is open code to use against privacy. If I didn’t care about you I’d just be silent. I’m posting this precisely because to me this looks like a good, ethical, skilled person being bought by a company that has long fed off misleading FOSS and privacy advocates. This is a clear attempt at swallowing the more private competition, ie you and Rhasspy.

I wish you well, I really do, but this is a very sad day, indeed.

Hopefully a more honest company makes you a better offer soon. Hopefully I’m wildly wrong. Reading their privacy policy though, I doubt it, so I do hope you’ll be careful to keep an escape plan handy in case you need it later. I would have happily thrown hundreds of dollars at a crowdfunding campaign for Rhasspy, but Mycroft will never get a single cent from me.

Anyway, thank you for all the good that you did for Rhasspy and the community. It was good while it lasted.

4 Likes