Looks interesting but they have compared to RNNoise which they seem to do often as there are other better alternatives avail (RNNoise is pretty bad and a very early model) same with Porcupine where they compare against the likes of Snowboy.
Its a shame DeepFilterNet is single threaded only, but DTLN is a big improvement on RNNoise, so claims are not exactly true.
There is also another thing to check as for the latest and greatest in ASR it is often trained without the filter technology and in the case of Whisper can increase WER (To dicate a filter to KWS & ASR and train in a dataset can have huge increases in accuracy)
The same challenge with Porcupine as you mentioned. If you can get an SDK from Cerence, Nuance, Sensory, or Soundhound and do a benchmark, youâd do me a big favour
Both Opensource both vastly superior to RNNoise, you are a representative of a commcercial company so maybe you invest in those closed commercial SDKâs or even compare against freely available opensource that isnât so ancient?
âyou are a representative of a commcercial company so maybe you invest in those closed commercial SDKâs
Their license agreements (EULAs) donât allow publishing any benchmarks
The idea of an open-source benchmark is to be reproducible.
How is investing (or not) relevant to an open-source benchmark when people cannot reproduce? Nobody can (and should) give away othersâ products. Itâs their call to make their SDK/API accessible or not.
compare against freely available opensource that isnât so ancient?
I didnât say âRNNoise is the state-of-the-artâ. Itâs not. I agree with you. I said âRNNoise is still widely used" Enterprises know and use RNNoise and Krisp. Itâs the market. ÂŻ_(ă)_/ÂŻ
Anyway, I wanted to clarify that (1) Weâre not cherrypicking. (2) Benchmarks are open to contributions (3) We support reproducible benchmarks.
If you want benches then you provide them, I know instantly from the samples there are better and I just posted x2 opensource ones and you could get the SDK.
IMO you do cherrypick and its why you have the examples you do, many companies are the same with sales blurb that is often optimistic as that is the nature of sales blurb.
There are much better noise filters and likely a better bench would be the SNR vs load as its looks quite light whilst better filters likely provide much more load.
No-one uses RNNoise apart from some that try PulseEffects and realise how poor/antiquated it is now, enterprises use products like Krisp or RTXVoice.
RTX has hardware requirements, not cross-platform and you had probably known that. All I can do is explain the rationale and I did. Everybody is free to do their own analysis, and also what they want to believe. There is no value in repeating myself.
Thanks for sharing Koala and have a great weekend.