@synesthesiam
I like the announcement!
Is there already a date when the mimic3 repository will be online in the Github? I would like to test the Debian packages.
Greetings, Jens
@synesthesiam
I like the announcement!
Is there already a date when the mimic3 repository will be online in the Github? I would like to test the Debian packages.
Greetings, Jens
Wow, how could Portuguese (Brazilian) not be on the list?
Iβll have to check this myself. The Mimic 3 server should also work with Rhasspyβs βremote TTSβ option, but I need to double check I havenβt broken anything with that either!
Hopefully next month, but I sent you a link with the beta packages
It was, but people told me that the voice I trained wasnβt understandable. I used this dataset: https://github.com/Edresson/TTS-Portuguese-Corpus
Do you know of any other TTS Portuguese datasets?
sorry no. Iβm clueless about lang models, data, etc.
Nice!
"text_to_speech": {
"command": {
"say_arguments": " --ssml --voice 'de_DE/m-ailabs_low#rebecca_braunert_plunkett' ",
"say_program": "mimic3"
},
"satellite_site_ids": "default",
"system": "command"
},
Das ist ein Test in deutsch <voice name="en_US/vctk_low#p236">and this is an test in english.</voice>
β¦ and Rhasspy speaks two languages in one sentence - cool.
It runs a bit slow on my old machine without GPU. With enough power and cache it will definitely get better.
Greetings, Jens
I didnβt, but I donβt see any useful data there
Awesome! The way to speed this up is to run mimic-server
as a service (check the source code for a systemd unit example), and then use mimic3 --remote ...
so it will use the web server instead.
Calling it up via the web interface wasnβt faster either. Now I have to pimp my base a bit firstβ¦
Btw, I think Mycroft should link to some demoβs in their Mimic 3 blog post announcement.
If people could hear presumably how good the TTS sounds theyβd be more likely to sign up and get involved. My 2 cents.
Will this be a drop in replacement?
Hello,
unfortunately i cannot sent PM as a new user and therefore cant test RTFβs for different architectures. Can someone give hints about RTFs, maybe for ARM?
Thanks
Hi @The1And0, on 64-bit ARM you can get an RTF of around 0.5. 32-bit ARM is slower, around 1.2 or 1.3. If youβre on a 64-bit x86/64 machine though, it can be 10x faster than ARM
Try it out for yourself: https://github.com/mycroftAI/mimic3
Oh, thereβs a Docker image now. (Although apparently without harvard-glow_tts
yet?)
Is it compatible with the βRemote HTTPβ TTS option of Rhasspy?
It is! Just set this as the URL: http://localhost:59125/api/tts
You can change the voice like this: http://localhost:59125/api/tts?voice=en_US/vctk_low#p236
I think Iβm missing something with the docker image.
After running the image I can access the web server but I cannot synthesize voices.
My guess is I have to still manually download the voices. But how do you do it using the docker image?
I tried locating the mimic3-download command in the image but no luck.
ERROR:mimic3_http.synthesis:Error during inference
Traceback (most recent call last):
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 125, in do_synthesis_proc
result = do_synthesis(item, mimic3)
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 81, in do_synthesis
raise e
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 61, in do_synthesis
mimic3.speak_text(params.text, text_language=params.text_language)
File β/home/mimic3/app/mimic3_tts/tts.pyβ, line 368, in speak_text
voice = self._get_or_load_voice(self.voice)
File β/home/mimic3/app/mimic3_tts/tts.pyβ, line 579, in _get_or_load_voice
voice = Mimic3Voice.load_from_directory(
File β/home/mimic3/app/mimic3_tts/voice.pyβ, line 283, in load_from_directory
onnx_model = Mimic3Voice._load_model(
File β/home/mimic3/app/mimic3_tts/voice.pyβ, line 403, in _load_model
onnx_model = onnxruntime.InferenceSession(
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.pyβ, line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.pyβ, line 370, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
RuntimeError: /onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char*, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:
Connect to the docker image
sudo docker exec βit nginx-test /bin/bash
as if the container name was nginx-test but change name to the container name
Its like ssh as you connect to the docker image and logon as by default your will be root
then do as you would do on a host wget and you may need to install wget as you would do βapt-get installβ
Docker to connect its always sudo docker exec βit <container-name> /bin/bash
Thatβs what I meant by running the mimic3-download command βfrom the imageβ
Shouldβve specified I was running it through docker exec /bin/bash
@tipofthesowrd You may need to run this before running the Docker image:
mkdir -p "${HOME}/.local/share/mycroft/mimic3"
chmod a+rwx "${HOME}/.local/share/mycroft/mimic3"
The Docker image runs as an unprivileged user for security, so it may not have permission to download voices.
I can see it downloading the voices and afterwards throwing the following errors & logs. When running docker exec -it /bin/bash I can enter the folder and cat the phonemes.txt file for example. So read permissions seems to be OK
ALIASES: 100%|βββββββββββββββββββββββββββββββ| 25.0/25.0 [00:00<00:00, 45.0kB/s]
LICENSE: 100%|βββββββββββββββββββββββββββββ| 6.88k/6.88k [00:00<00:00, 16.6MB/s]
README.md: 100%|βββββββββββββββββββββββββββ| 11.2k/11.2k [00:00<00:00, 6.89MB/s]
README.md.in: 100%|βββββββββββββββββββββββββββββ| 186/186 [00:00<00:00, 367kB/s]
SOURCE: 100%|βββββββββββββββββββββββββββββββββ| 52.0/52.0 [00:00<00:00, 332kB/s]
VERSION: 100%|βββββββββββββββββββββββββββββββ| 6.00/6.00 [00:00<00:00, 30.7kB/s]
config.json: 100%|βββββββββββββββββββββββββ| 3.57k/3.57k [00:00<00:00, 39.0kB/s]
generator.onnx: 100%|ββββββββββββββββββββββ| 59.9M/59.9M [00:27<00:00, 2.26MB/s]
phoneme_map.txt: 100%|βββββββββββββββββββββββ| 21.0/21.0 [00:00<00:00, 34.4kB/s]
phonemes.txt: 100%|βββββββββββββββββββββββββββββ| 329/329 [00:00<00:00, 296kB/s]
ERROR:mimic3_http.synthesis:Error during inference
Traceback (most recent call last):
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 125, in do_synthesis_proc
result = do_synthesis(item, mimic3)
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 81, in do_synthesis
raise e
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 61, in do_synthesis
mimic3.speak_text(params.text, text_language=params.text_language)
File β/home/mimic3/app/mimic3_tts/tts.pyβ, line 368, in speak_text
voice = self._get_or_load_voice(self.voice)
File β/home/mimic3/app/mimic3_tts/tts.pyβ, line 579, in _get_or_load_voice
voice = Mimic3Voice.load_from_directory(
File β/home/mimic3/app/mimic3_tts/voice.pyβ, line 283, in load_from_directory
onnx_model = Mimic3Voice._load_model(
File β/home/mimic3/app/mimic3_tts/voice.pyβ, line 403, in _load_model
onnx_model = onnxruntime.InferenceSession(
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.pyβ, line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.pyβ, line 370, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
RuntimeError: /onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char*, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:ERROR:mimic3_http.app:/onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char*, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:
Traceback (most recent call last):
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/quart/app.pyβ, line 1512, in full_dispatch_request
result = await self.dispatch_request(request_context)
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/quart/app.pyβ, line 1557, in dispatch_request
return await self.ensure_async(handler)(*request_.view_args)
File β/home/mimic3/app/mimic3_http/app.pyβ, line 225, in app_tts
wav_bytes = await text_to_wav(
File β/home/mimic3/app/mimic3_http/app.pyβ, line 102, in text_to_wav
wav_bytes = await future
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 125, in do_synthesis_proc
result = do_synthesis(item, mimic3)
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 81, in do_synthesis
raise e
File β/home/mimic3/app/mimic3_http/synthesis.pyβ, line 61, in do_synthesis
mimic3.speak_text(params.text, text_language=params.text_language)
File β/home/mimic3/app/mimic3_tts/tts.pyβ, line 368, in speak_text
voice = self._get_or_load_voice(self.voice)
File β/home/mimic3/app/mimic3_tts/tts.pyβ, line 579, in _get_or_load_voice
voice = Mimic3Voice.load_from_directory(
File β/home/mimic3/app/mimic3_tts/voice.pyβ, line 283, in load_from_directory
onnx_model = Mimic3Voice._load_model(
File β/home/mimic3/app/mimic3_tts/voice.pyβ, line 403, in _load_model
onnx_model = onnxruntime.InferenceSession(
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.pyβ, line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File β/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.pyβ, line 370, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
RuntimeError: /onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:
Are you running this on a shared (cloud) machine, by chance?