Mimic 3 TTS Preview

Announcements
41

I can see it downloading the voices and afterwards throwing the following errors & logs. When running docker exec -it /bin/bash I can enter the folder and cat the phonemes.txt file for example. So read permissions seems to be OK

ALIASES: 100%|███████████████████████████████| 25.0/25.0 [00:00<00:00, 45.0kB/s]
LICENSE: 100%|█████████████████████████████| 6.88k/6.88k [00:00<00:00, 16.6MB/s]
README.md: 100%|███████████████████████████| 11.2k/11.2k [00:00<00:00, 6.89MB/s]
README.md.in: 100%|█████████████████████████████| 186/186 [00:00<00:00, 367kB/s]
SOURCE: 100%|█████████████████████████████████| 52.0/52.0 [00:00<00:00, 332kB/s]
VERSION: 100%|███████████████████████████████| 6.00/6.00 [00:00<00:00, 30.7kB/s]
config.json: 100%|█████████████████████████| 3.57k/3.57k [00:00<00:00, 39.0kB/s]
generator.onnx: 100%|██████████████████████| 59.9M/59.9M [00:27<00:00, 2.26MB/s]
phoneme_map.txt: 100%|███████████████████████| 21.0/21.0 [00:00<00:00, 34.4kB/s]
phonemes.txt: 100%|█████████████████████████████| 329/329 [00:00<00:00, 296kB/s]
ERROR:mimic3_http.synthesis:Error during inference
Traceback (most recent call last):
File “/home/mimic3/app/mimic3_http/synthesis.py”, line 125, in do_synthesis_proc
result = do_synthesis(item, mimic3)
File “/home/mimic3/app/mimic3_http/synthesis.py”, line 81, in do_synthesis
raise e
File “/home/mimic3/app/mimic3_http/synthesis.py”, line 61, in do_synthesis
mimic3.speak_text(params.text, text_language=params.text_language)
File “/home/mimic3/app/mimic3_tts/tts.py”, line 368, in speak_text
voice = self._get_or_load_voice(self.voice)
File “/home/mimic3/app/mimic3_tts/tts.py”, line 579, in _get_or_load_voice
voice = Mimic3Voice.load_from_directory(
File “/home/mimic3/app/mimic3_tts/voice.py”, line 283, in load_from_directory
onnx_model = Mimic3Voice._load_model(
File “/home/mimic3/app/mimic3_tts/voice.py”, line 403, in _load_model
onnx_model = onnxruntime.InferenceSession(
File “/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py”, line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File “/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py”, line 370, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
RuntimeError: /onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char*, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:

ERROR:mimic3_http.app:/onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char*, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:
Traceback (most recent call last):
File “/home/mimic3/app/.venv/lib/python3.9/site-packages/quart/app.py”, line 1512, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/home/mimic3/app/.venv/lib/python3.9/site-packages/quart/app.py”, line 1557, in dispatch_request
return await self.ensure_async(handler)(*request_.view_args)
File “/home/mimic3/app/mimic3_http/app.py”, line 225, in app_tts
wav_bytes = await text_to_wav(
File “/home/mimic3/app/mimic3_http/app.py”, line 102, in text_to_wav
wav_bytes = await future
File “/home/mimic3/app/mimic3_http/synthesis.py”, line 125, in do_synthesis_proc
result = do_synthesis(item, mimic3)
File “/home/mimic3/app/mimic3_http/synthesis.py”, line 81, in do_synthesis
raise e
File “/home/mimic3/app/mimic3_http/synthesis.py”, line 61, in do_synthesis
mimic3.speak_text(params.text, text_language=params.text_language)
File “/home/mimic3/app/mimic3_tts/tts.py”, line 368, in speak_text
voice = self._get_or_load_voice(self.voice)
File “/home/mimic3/app/mimic3_tts/tts.py”, line 579, in _get_or_load_voice
voice = Mimic3Voice.load_from_directory(
File “/home/mimic3/app/mimic3_tts/voice.py”, line 283, in load_from_directory
onnx_model = Mimic3Voice._load_model(
File “/home/mimic3/app/mimic3_tts/voice.py”, line 403, in _load_model
onnx_model = onnxruntime.InferenceSession(
File “/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py”, line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File “/home/mimic3/app/.venv/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py”, line 370, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
RuntimeError: /onnxruntime_src/onnxruntime/core/platform/posix/env.cc:183 onnxruntime::{anonymous}::PosixThread::PosixThread(const char, int, unsigned int ()(int, Eigen::ThreadPoolInterface), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed, error code: 0 error msg:

42

Are you running this on a shared (cloud) machine, by chance?

43

Wow, how could Portuguese (Brazilian) not be on the list?

Wow, how could you be any more obtuse?

44

Greetings sir,
Is there any chance to access the training source code of mimic-III model?

45

I’ve uploaded the code here: GitHub - MycroftAI/mimic3-train: Mimic 3 Training Code

I can’t guarantee that it’s in a working state, unfortunately. It was being refactored when Mycroft went under, and I never got a chance to finish it.

46

Greetings sir,
I have another request to make, if it’s OK.

I intended to know the way to add extra words and phonemes to the dictionary files of mimic3. Can you please guide me with that?

47

Mimic 3 uses gruut for phonemization. You can see the structure of the English data here, for example: gruut/gruut-lang-en/gruut_lang_en at master · rhasspy/gruut · GitHub

Adding words involves modifying the lexicon.db sqlite3 database. There is a word_phonemes table that you can add words/phonemes to. Each language is installed as a separate Python package, such as gruut-lang-en. You can create a new package with your updated lexicon.db or modify the one in your virtual environment.