I have been playing too, longer sentences seem worse in some ways…
Intent:-
[NonSense]
what color is the lamp post at the corner of the road
Tests:-
“what color is the lamp post at the the corner of the road” = confidence 1
“what color is the lamp at the corner of the road” = confidence 1
“what color is the corner is of the road” = confidence 1
Saying “what time is the color of the road” or “what color is the” simply triggers my [GetTime] intent which is “what time is it” The confidence is around 0.9 though so that is trappable with a reasonable score rating.
That seems to be a very wide assumption range of what is only a partial match and all score perfectly???
Does it match the whole sentence or just the first few words??
A little more, it really dislikes optional stuff it seems…
[SetTimer]
minutes = (1){min} minute | (2…59){min} minutes
seconds = (1){sec} second | (2…59){sec} seconds
set [a] timer for (1){min} and (a half){sec:30!int} minutes
Saying “set a timer for one and a half minutes” works but the score is very low at around 0.53
I just upgraded to 2.5.10 in the hopes of taking advantage of the conference values to reduce false positives. This thread however is making me think that the results are mixed. Are less false positives being reported with silence? It seems that incorrect sentences or gibberish might still be pretty likely to generate false positives.
Here’s the code where this is happening. MBR is computed for the whole sentence. The word confidences are computed from the “one best” result.
There do seem to be other notions of confidence in Kaldi. Perhaps I should be using a different calculation than MBR. From my research, it seemed better than the other sentence-level confidence measure, which was just the likelihoold difference between the first and second transcriptions.
I include an “<unk>” (unknown) word during training, but I don’t think it can ever be chosen by Kaldi in Text FST mode. I may need to experiment with putting this into the sentences, so garbage words can become “<unk>” rather than some random word.
Hi,
Just upgraded my production setup to 2.5.10, have set min confidence to 0.9 for asr and have strange result. Can’t get any intent recognized. min confidence to 0 works nice.
On mqtt explorer, for “allume la lumière” I have this:
“value”: “lumière”,
“confidence”: 0.673712,
other words are above 0.95, entire sentence correspond exactly to sentences.ini
Why such low confidence ??
EDIT: even some common words like musique get 0.5 confidence … same for “la”
the minimum confidence that you put to 0.4 refers to the “likelihood” value of the whole sentence.
That means, that your recognized sentence of “éteins la lumière” did not pass your minimum confidence of 0.4. Therefore, was discarded.
This has most probably to do with the other possible sentences.
Check for use of optional words.
In my experience, I was able to solve one particularly stubborn sentence by breaking it up into more seperate pieces.