I was looking at benchmarks and thought wow either that is fast, but don’t remember Whisper.cpp being that slow.
So had to give whisper.cpp a refresh
system_info: n_threads = 4 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | VSX = 0 |
main: processing 'samples/gb1.wav' (3179927 samples, 198.7 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:09.000] My fellow Americans, this day has brought terrible news and great sadness to our country.
[00:00:09.000 --> 00:00:18.000] At 9 o'clock this morning, mission control in Houston lost contact with our space shuttle Columbia.
[00:00:18.000 --> 00:00:24.000] A short time later, debris was seen falling from the skies above Texas.
[00:00:24.000 --> 00:00:29.000] The Columbia's lost. There are no survivors.
[00:00:29.000 --> 00:00:37.000] On board was a crew of seven, Colonel Rick Husband, Lieutenant Colonel Michael Anderson,
[00:00:37.000 --> 00:00:46.000] Commander Laurel Clark, Captain David Brown, Commander William McCool, Dr. Kultna Shavla,
[00:00:46.000 --> 00:00:52.000] and Ilan Ramon, a colonel in the Israeli Air Force.
[00:00:52.000 --> 00:00:58.000] These men and women assumed great risk in the service to all humanity.
[00:00:58.000 --> 00:01:02.000] in an age when spaceflight has come to seem almost routine.
[00:01:02.000 --> 00:01:06.000] It is easy to overlook the dangers of travel by rocket
[00:01:06.000 --> 00:01:11.000] and the difficulties of navigating the fierce outer atmosphere of the Earth.
[00:01:11.000 --> 00:01:17.000] These astronauts knew the dangers, and they faced them willingly,
[00:01:17.000 --> 00:01:21.000] knowing they had a high and noble purpose in life.
[00:01:21.000 --> 00:01:26.000] Because of their courage and daring and idealism,
[00:01:26.000 --> 00:01:39.000] we will miss them all the more. All Americans today are thinking as well of the families of these men and women who have been given this sudden shock and grief.
[00:01:39.000 --> 00:01:51.000] You're not alone. Our entire nation grieves with you. And those you love will always have the respect and gratitude of this country.
[00:01:51.000 --> 00:01:55.720] The cause in which they died will continue.
[00:01:55.720 --> 00:02:04.120] Mankind is led into the darkness beyond our world by the inspiration of discovery and
[00:02:04.120 --> 00:02:07.000] the longing to understand.
[00:02:07.000 --> 00:02:11.160] Our journey into space will go on.
[00:02:11.160 --> 00:02:16.480] In the skies today, we saw destruction and tragedy.
[00:02:16.480 --> 00:02:22.040] farther than we can see, there is comfort and hope.
[00:02:22.040 --> 00:02:29.280] In the words of the prophet Isaiah, "Lift your eyes and look to the heavens."
[00:02:29.280 --> 00:02:31.640] Who created all these?
[00:02:31.640 --> 00:02:39.260] He who brings out the story hosts one by one and calls them each by name.
[00:02:39.260 --> 00:02:46.400] Because of His great power and mighty strength, not one of them is missing.
[00:02:46.400 --> 00:02:53.580] The same Creator who names the stars also knows the names of the seven souls we mourn
[00:02:53.580 --> 00:02:55.580] today.
[00:02:55.580 --> 00:03:03.140] The crew of the shuttle Columbia did not return safely to Earth, yet we can pray that all
[00:03:03.140 --> 00:03:05.820] are safely home.
[00:03:05.820 --> 00:03:12.640] May God bless the grieving families and may God continue to bless America.
[00:03:12.640 --> 00:03:22.640] [BLANK_AUDIO]
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: load time = 655.10 ms
whisper_print_timings: mel time = 1611.87 ms
whisper_print_timings: sample time = 436.05 ms / 520 runs ( 0.84 ms per run)
whisper_print_timings: encode time = 82173.55 ms / 8 runs (10271.69 ms per run)
whisper_print_timings: decode time = 34584.14 ms / 520 runs ( 66.51 ms per run)
whisper_print_timings: total time = 119509.48 ms
real 1m59.635s
user 7m49.414s
sys 0m2.604s
‘faster_whisper’
orangepi@orangepi5:~/faster-whisper$ time OMP_NUM_THREADS=4 python3 my_script.py
Detected language 'en' with probability 0.989548
[0.00s -> 9.00s] My fellow Americans, this day has brought terrible news and great sadness to our country.
[9.00s -> 18.00s] At 9 o'clock this morning, mission control in Houston lost contact with our space shuttle Columbia.
[18.00s -> 24.00s] A short time later, debris was seen falling from the skies above Texas.
[24.00s -> 29.00s] The Columbia's lost. There are no survivors.
[29.00s -> 37.00s] On board was a crew of seven, Colonel Rick Husband, Lieutenant Colonel Michael Anderson,
[37.00s -> 44.00s] Commander Laurel Clark, Captain David Brown, Commander William McCool,
[44.00s -> 52.00s] Dr. Kultna Shavla, and Ilan Ramon, a colonel in the Israeli Air Force.
[52.00s -> 58.00s] These men and women assumed great risk in the service to all humanity.
[58.00s -> 63.00s] In an age when space flight has come to seem almost routine,
[63.00s -> 67.00s] it is easy to overlook the dangers of travel by rocket
[67.00s -> 72.00s] and the difficulties of navigating the fierce outer atmosphere of the Earth.
[72.00s -> 78.00s] These astronauts knew the dangers, and they faced them willingly,
[78.00s -> 83.00s] knowing they had a high and noble purpose in life.
[83.00s -> 90.00s] Because of their courage and daring and idealism, we will miss them all the more.
[90.00s -> 96.00s] All Americans today are thinking as well of the families of these men and women
[96.00s -> 100.00s] who have been given this sudden shock and grief.
[100.00s -> 105.00s] You're not alone. Our entire nation grieves with you,
[105.00s -> 112.00s] and those you love will always have the respect and gratitude of this country.
[112.00s -> 116.00s] The cause in which they died will continue.
[116.00s -> 121.00s] Mankind is led into the darkness beyond our world
[121.00s -> 127.00s] by the inspiration of discovery and the longing to understand.
[127.00s -> 131.00s] Our journey into space will go on.
[131.00s -> 136.00s] In the skies today, we saw destruction and tragedy.
[136.00s -> 142.00s] As farther than we can see, there is comfort and hope.
[142.00s -> 145.00s] In the words of the prophet Isaiah,
[145.00s -> 151.00s] lift your eyes and look to the heavens who created all these,
[151.00s -> 155.00s] he who brings out the story hosts one by one
[155.00s -> 159.00s] and calls them each by name.
[159.00s -> 163.00s] Because of his great power and mighty strength,
[163.00s -> 166.00s] not one of them is missing.
[166.00s -> 169.00s] The same Creator who names the stars
[169.00s -> 175.00s] also knows the names of the seven souls we mourn today.
[175.00s -> 180.00s] The crew of the shuttle Columbia did not return safely to Earth,
[180.00s -> 185.00s] yet we can pray that all are safely home.
[185.00s -> 189.00s] May God bless the grieving families
[189.00s -> 194.00s] and may God continue to bless America.
real 1m22.424s
user 6m13.880s
sys 0m17.584s
Maybe its the chosen CPU Xeon(R) Gold 6226R as surprised but the rk3588s is faster than a Xeon Gold 6226R on 4 cores…
Faster_Whisper is faster than Whisper.cpp but not > x6 faster, as great the 8bit conversion seems to run great but confused to the benchmark figures posted on Github? Anyone else the same.
The tiny model runs approx same ratio as the small which I 1st choose as that was the supposed benchmarks.
whisper.cpp_tiny
real 0m18.860s
user 1m6.639s
sys 0m0.945s
faster_whisper_8bit_tiny
real 0m16.380s
user 1m10.504s
sys 0m2.735s
That is purely curiousity at the benchmarks posted on GitHub - guillaumekln/faster-whisper: Faster Whisper transcription with CTranslate2
Suprised you started with Whisper though as still on a Pi even the tiny model isn’t the fastest, but will have to see what benches you get with Tiny.
Do you have anything in the pipeline based on custom LM say with n-grams incorporating Assist entity construction?