I also recommend publishing new releases much more often, even if it’s just a bug fix. This way users can work around a problem easier by downgrading or upgrading a component.
Integration tests would also help, but I haven’t done this yet on this scale, so that will be quite a challenge… Maybe we should look at Rhasspy as a whole as a kind of ‘distribution’ of various programs. Currently we have been developing and testing components roughly in isolation, but when they are linked together in the Docker container or the venv installation, things break. How could we do this in a more reliable way? So that we can publish Rhasspy releases with a known-working combination of subproject versions? How do Linux distributions do this?
IMHO, @synethesiam is actually doing it right. Linux and its development process had a long time to grow - they even needed to develop their own support tools (git), but everything based in concrete needs/problems.
So, unfortunately yes, things need to break first, before we can fix them, and then yes fix one by one and break more - we just need to guarantee that it is easy to fix things and get involved in the development.
This leads to another problem. I know that @synesthesiam has a day job that does allow him to partly work on rhasspy but I assume he does also other things. And we have reached a point where the project needs to scale up to more people. I think, we need to discuss how we can involve more volunteers in the actual development and testing work (and all of these will have to cater to their day jobs first too). So, how can sone of the more development and documentatiom-focused community members get more involved?
Guess, we need a new thread for that.
I will try the in place fix, when I am back home and report back.
I agree, this is a critical question to answer for the future of Rhasspy. A project with a bus factor of 1 is no good for longevity.
@maxbachmann, @koan, and other community members have stepped up to help with development and documentation. The GitHub organization has helped, and I plan to do something similar for DockerHub and PyPI.
One thing I hadn’t considered is to set up the Python package requirements to fit with semantic versioning instead of fighting against it. So instead of:
rhasspy-nlu==0.1.10
we would have:
rhasspy-nlu>=0.1,<0.2
Then each patch version (0.1.9, 0.1.10) would automatically get picked up during the next upgrade.
I don’t have a problem with this in theory, it’s just that a “release” ends up being a lot of work to get out right now. I’m hoping this will change with more automation. What you and @maxbachmann were helping set up on GitHub before the 2.5 release (using git tags and Actions) would get us most of the way there.
I think this is the Achilles’ heel of trying to automate releases right now. I use rhasspy-test to run a suite of automated tests against the latest (amd64) Docker image before pushing an update. This spins up an isolated Docker container, downloads and trains an English profile, then runs the tests. It relies on a local web server to serve up the ~15GB of profile artifacts.
I imagine going forward, I’ll still be using rhasspy-test but when it succeeds and we’re satisfied, one of will just git tag the version and everything will be built/deployed from GitHub.
And this is something you can’t force, we just have to wait until / hope that people step up to “adopt” specific parts of Rhasspy that they’re interested in That’s how I started with some of the tests, documentation and CI/CD, the app library now and documenting Rhasspy Hermes soon.
I think the documentation that @synesthesiam created is already excellent, better than 99% of the projects that I have seen. That’s already a good base to attract contributions.
Yes, we should try to build on this. Maybe we should discuss on GitHub what exactly needs to be done to make the release process as frictionless as possible.
I recently created a development page in rhasspy-hermes-app’s documentation to help first-time contributors find their way. Maybe I should rework this with some more general information and move this to the development page of Rhasspy? Then we can refer to this page for people who want to contribute.
This dev-page looks pretty good, definetely voting to get it into rhasspy’s main documentation.
How do the tests catch testing remote setups with satellites? Testing a distributed system is usually pretty hard (and might have prevented the unsolved error that started this thread).
Basically on each repo add a donate button that can give feedback of preference of importance via donation.
Doesn’t dictate spend but does give valuable metrics.
I added this information to Rhasspy’s main documentation and changed it to a contributing page to attract both developers and non-developers. Let me know if I should add other information.
That’s how I started with some of the tests, documentation and CI/CD, the app library now and documenting Rhasspy Hermes soon.