OpenVoice/demo_part1.ipynb

Voice Style Control Demo

In [ ]:

import os
import torch
import se_extractor
from api import BaseSpeakerTTS, ToneColorConverter

Initialization

In [ ]:

ckpt_base = 'checkpoints/base_speaker'
ckpt_converter = 'checkpoints/converter'
device = 'cuda:0'
output_dir = 'outputs'

base_speaker_tts = BaseSpeakerTTS(f'{ckpt_base}/config.json', device=device)
base_speaker_tts.load_ckpt(f'{ckpt_base}/checkpoint.pth')

tone_color_converter = ToneColorConverter(f'{ckpt_converter}/config.json', device=device)
tone_color_converter.load_ckpt(f'{ckpt_converter}/checkpoint.pth')

os.makedirs(output_dir, exist_ok=True)

Obtain Tone Color Embedding

The source_se is the tone color embedding of the base speaker. It is an average for multiple sentences with multiple emotions of the base speaker. We directly provide the result here but the readers feel free to extract source_se by themselves.

In [ ]:

source_se = torch.load(f'{ckpt_base}/source_se.pth').to(device)

The reference_speaker.mp3 below points to the short audio clip of the reference whose voice we want to clone. We provide an example here. If you use your own reference speakers, please make sure each speaker has a unique filename. The se_extractor will save the targeted_se using the filename of the audio and will not automatically overwrite.

In [ ]:

reference_speaker = 'resources/example_reference.mp3'
target_se, audio_name = se_extractor.get_se(reference_speaker, tone_color_converter)

Inference

In [ ]:

save_path = f'{output_dir}/output_friendly.wav'

# Run the base speaker tts
text = "This audio is generated by open voice."
src_path = f'{output_dir}/tmp.wav'
base_speaker_tts.tts(text, src_path, speaker='friendly', language='English', speed=1.0)

# Run the tone color converter
encode_message = "@MyShell"
tone_color_converter.convert(
    audio_src_path=src_path, 
    src_se=source_se, 
    tgt_se=target_se, 
    output_path=save_path,
    message=encode_message)

Try with different styles and speed. The style can be controlled by the speaker parameter in the base_speaker_tts.tts method. Available choices: friendly, cheerful, excited, sad, angry, terrified, shouting, whispering. The speed can be controlled by the speed parameter. Let's try whispering with speed 0.9.

In [ ]:

save_path = f'{output_dir}/output_whispering.wav'

# Run the base speaker tts
text = "This audio is generated by open voice with a half-performance model."
src_path = f'{output_dir}/tmp.wav'
base_speaker_tts.tts(text, src_path, speaker='whispering', language='English', speed=0.9)

# Run the tone color converter
encode_message = "@MyShell"
tone_color_converter.convert(
    audio_src_path=src_path, 
    src_se=source_se, 
    tgt_se=target_se, 
    output_path=save_path,
    message=encode_message)

Tech for good. For people who will deploy OpenVoice for public usage: We offer you the option to add watermark to avoid potential misuse. Please see the ToneColorConverter class. MyShell reserves the ability to detect whether an audio is generated by OpenVoice, no matter whether the watermark is added or not.