To test my assistant and avoid repeating the same utterances all day long, I’m using an in-house Node.js script based on Snips exported unit-tests.
It would be nice to also have a “tests” feature (like Snips had).
Tests can be JSON files located in a tests folder of the profile with the WAV file bytes encoded as base64 and the expected utterance, intent and slots.
A new tab in the web UI can be added to manage tests. This tab should have the ability to record an utterance, play it back and match it with an intent and slots.
Using the API, it should be easy to execute tests in batch.
This would clearly help working on intents and slots and ensure nothing is broken.
Anyway, not sure it is high on the list. Builtins slots (duration, datetime, number etc) are really needed and for me the top priority actually. Could not use any assistant in production without this, it is a no go.
Base/satellite setup comes second, even if I could run several pi4 with full rhasspy and duplicates profiles after training on one of them (not a solution but would work).
LED control/feedback also comes to mind, etc.
I think making a snips replacement with these features would make a lot of users switch to rhasspy. And more users means more feddback, more ideas, more helpers, all the best for rhasspy Cheaters ever know how to run such batch tests
No, they do. There are a number of accuracy metrics computed, including an intent_entity_accuracy that is 1.0 when every intent matches and every slot value matches, with no extra slot values.
I’ll take a look at the tests in voice2json when I have more time.
I have some expertise in web app dev so I might try to port the web UI as a separated service when the communication layer and protocol is defined more clearly and if my free time allows it.
In the meantime, I’ll try to write some ideas in a dedicated topic so we can all discuss it.