mirror of
https://github.com/rhasspy/piper-sample-generator.git
synced 2026-08-27 18:15:58 -04:00
Compare commits
20 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
2971426a55 | ||
|
|
275077a1a3 | ||
|
|
1a8c49bd29 | ||
|
|
c9d824c0e2 | ||
|
|
ded9350eaf | ||
|
|
66ec23fa49 | ||
|
|
c0d2dd9ea4 | ||
|
|
942002d0ee | ||
|
|
395aef7dbc | ||
|
|
df5799c141 | ||
|
|
af7ac7aae2 | ||
|
|
5e67370ab3 | ||
|
|
4d7e4b390c | ||
|
|
4057c1a620 | ||
|
|
9c1019c932 | ||
|
|
213d4d561a | ||
|
|
172d7b5cae | ||
|
|
315e555f49 | ||
|
|
2dbff77c61 | ||
|
|
77d8c0d4b3 |
25 changed files with 2713 additions and 471 deletions
19
CHANGELOG.md
Normal file
19
CHANGELOG.md
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Changelog
|
||||
|
||||
## 3.2.0
|
||||
|
||||
- Refactor as `piper_sample_generator` package
|
||||
|
||||
## 3.1.0
|
||||
|
||||
- Support MPS acceleration on Apple Silicon
|
||||
- Add `--phoneme-input` flag
|
||||
|
||||
## 3.0.0
|
||||
|
||||
- Move phonemization to piper 1.3.0 (piper-phonemize is deprecated)
|
||||
- Move to PyTorch 2
|
||||
- Add support for using Piper voices (`.onnx`) directly
|
||||
- Allow multiple `--model` for Piper voices (`.onnx`)
|
||||
- Remove silence trimming
|
||||
- Remove `min-phoneme-count`
|
||||
78
README.md
78
README.md
|
|
@ -1,35 +1,51 @@
|
|||
# Piper Sample Generator
|
||||
|
||||
Generates samples using [Piper](https://github.com/rhasspy/piper/) for training a wake word system like [openWakeWord](https://github.com/dscripka/openWakeWord).
|
||||
Generate spoken audio samples using [Piper][piper] for training a wake word system like [openWakeWord][] or [microWakeWord][].
|
||||
|
||||
Supports normal [Piper voices][piper voices] or a special [generator][] that can mix speaker embeddings (English only).
|
||||
|
||||
## Install
|
||||
|
||||
Create a virtual environment and install the requirements:
|
||||
|
||||
``` sh
|
||||
git clone https://github.com/rhasspy/piper-sample-generator.git
|
||||
cd piper-sample-generator/
|
||||
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
python3 -m pip install --upgrade pip
|
||||
python3 -m pip install -r requirements.txt
|
||||
pip install piper-sample-generator
|
||||
```
|
||||
|
||||
Download the LibriTTS-R generator (exported from [checkpoint](https://huggingface.co/datasets/rhasspy/piper-checkpoints/tree/main/en/en_US/libritts_r/medium)):
|
||||
## Piper Voices
|
||||
|
||||
Download one or more [Piper voices][piper voices] (both the `.onnx` and `.onnx.json` files for each voice). [Audio samples][piper samples] are available.
|
||||
|
||||
As an example, we'll download the U.S. English "lessac" voice in medium quality:
|
||||
|
||||
``` sh
|
||||
mkdir -p voices
|
||||
wget -O voices/en_US-lessac-medium.onnx 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx?download=true'
|
||||
wget -O voices/en_US-lessac-medium.onnx.json 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json?download=true'
|
||||
```
|
||||
|
||||
Generate a small set of samples with the CLI:
|
||||
|
||||
``` sh
|
||||
python3 -m piper_sample_generator 'okay piper.' --model voices/en_US-lessac-medium.onnx --max-samples 10 --output-dir okay_piper/
|
||||
```
|
||||
|
||||
Check the `okay_piper/` directory for 10 WAV files (named `0.wav` to `9.wav`).
|
||||
|
||||
You can add multiple `--model <voice>` arguments to cycle between different voices when generating samples.
|
||||
|
||||
See `--help` for more options, including `--length-scales` (speaking speeds).
|
||||
|
||||
## Generator
|
||||
|
||||
Download the LibriTTS-R generator (exported from [checkpoint][]):
|
||||
|
||||
``` sh
|
||||
wget -O models/en-us-libritts-high.pt 'https://github.com/rhasspy/piper-sample-generator/releases/download/v2.0.0/en_US-libritts_r-medium.pt'
|
||||
```
|
||||
|
||||
|
||||
## Run
|
||||
|
||||
Generate a small set of samples with the CLI:
|
||||
|
||||
``` sh
|
||||
python3 generate_samples.py 'okay, piper.' --max-samples 10 --output-dir okay_piper/
|
||||
python3 -m piper_sample_generator 'okay piper.' --model models/en-us-libritts-high.pt --max-samples 10 --output-dir okay_piper/
|
||||
```
|
||||
|
||||
Check the `okay_piper/` directory for 10 WAV files (named `0.wav` to `9.wav`).
|
||||
|
|
@ -37,38 +53,38 @@ Check the `okay_piper/` directory for 10 WAV files (named `0.wav` to `9.wav`).
|
|||
Generation can be much faster and more efficient if you have a GPU available and PyTorch is configured to use it. In this case, increase the batch size:
|
||||
|
||||
``` sh
|
||||
python3 generate_samples.py 'okay, piper.' --max-samples 100 --batch-size 10 --output-dir okay_piper/
|
||||
python3 -m piper_sample_generator 'okay piper.' --model models/en-us-libritts-high.pt --max-samples 100 --batch-size 10 --output-dir okay_piper/
|
||||
```
|
||||
|
||||
On an NVidia 2080 Ti with 11GB, a batch size of 100 was possible (generating approximately 100 samples per second).
|
||||
|
||||
Setting `--max-speakers` to a value less than 904 (the number of speakers LibriTTS) is recommended. Because very few samples of later speakers were in the original dataset, using them can cause audio artifacts.
|
||||
|
||||
See `--help` for more options, including adjust the `--length-scales` (speaking speeds) and `--slerp-weights` (speaker blending) which are cycled per batch.
|
||||
|
||||
Alternatively, you can import the generate function into another Python script:
|
||||
|
||||
```python
|
||||
from generate_samples import generate_samples # make sure to add this to your Python path as needed
|
||||
|
||||
generate_samples(text = ["okay, piper"], max_samples = 100, output_dir = output_dir, batch_size=10)
|
||||
```
|
||||
|
||||
There are some additional arguments available when importing the function directly, see the docstring of `generate_sample` for more information.
|
||||
See `--help` for more options, including the `--length-scales` (speaking speeds) and `--slerp-weights` (speaker blending) which are cycled per batch.
|
||||
|
||||
### Augmentation
|
||||
|
||||
Once you have samples generating, you can augment them using [audiomentation](https://iver56.github.io/audiomentations/):
|
||||
Once you have samples generated, you can augment them using [audiomentation](https://iver56.github.io/audiomentations/):
|
||||
|
||||
``` sh
|
||||
python3 augment.py --sample-rate 16000 okay_piper/ okay_piper_augmented/
|
||||
python3 -m piper_sample_generator.augment --sample-rate 22050 okay_piper/ okay_piper_augmented/
|
||||
```
|
||||
|
||||
This will do several things to each sample:
|
||||
|
||||
1. Randomly decrease the volume
|
||||
* The original samples are normalized, so different volume levels are needed
|
||||
2. Randomly [apply an impulse response](https://iver56.github.io/audiomentations/waveform_transforms/apply_impulse_response/) using the files in `impulses/`
|
||||
2. Randomly apply an [impulse response][] using the files in `piper_sample_generator/impulses/`
|
||||
* Change the acoustics of the sample to sound like the speaker was in a room with echo or using a poor quality microphone
|
||||
3. Resample to 16Khz for training (e.g., [openWakeWord](https://github.com/dscripka/openWakeWord))
|
||||
3. Resample to 16Khz for training (e.g., [openWakeWord][])
|
||||
|
||||
|
||||
<!-- Links -->
|
||||
[piper]: https://github.com/OHF-Voice/piper1-gpl/
|
||||
[openWakeWord]: https://github.com/dscripka/openWakeWord
|
||||
[microWakeWord]: https://github.com/kahrendt/microWakeWord/
|
||||
[piper voices]: https://huggingface.co/rhasspy/piper-voices
|
||||
[generator]: https://github.com/rhasspy/piper-sample-generator/releases/download/v2.0.0/en_US-libritts_r-medium.pt
|
||||
[piper samples]: https://rhasspy.github.io/piper-samples/
|
||||
[checkpoint]: https://huggingface.co/datasets/rhasspy/piper-checkpoints/tree/main/en/en_US/libritts_r/medium
|
||||
[impulse response]: https://iver56.github.io/audiomentations/waveform_transforms/apply_impulse_response/
|
||||
|
|
|
|||
|
|
@ -1,407 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import gc
|
||||
import itertools as it
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import wave
|
||||
from pathlib import Path
|
||||
from typing import List, Union
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
import torchaudio
|
||||
import webrtcvad
|
||||
from piper_phonemize import phonemize_espeak
|
||||
|
||||
from piper_train.vits import commons
|
||||
|
||||
_DIR = Path(__file__).parent
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
|
||||
|
||||
# Main generation function
|
||||
def generate_samples(
|
||||
text: Union[List, str],
|
||||
output_dir: str,
|
||||
max_samples: int = None,
|
||||
file_names: List[str] = [],
|
||||
model: str = os.path.join(
|
||||
Path(__file__).parent, "models", "en_US-libritts_r-medium.pt"
|
||||
),
|
||||
batch_size: int = 1,
|
||||
slerp_weights: List[float] = [0.5],
|
||||
length_scales: List[float] = [0.75, 1, 1.25],
|
||||
noise_scales: List[float] = [0.667],
|
||||
noise_scale_ws: List[float] = [0.8],
|
||||
max_speakers: float = None,
|
||||
verbose: bool = False,
|
||||
auto_reduce_batch_size: bool = False,
|
||||
**kwargs,
|
||||
) -> None:
|
||||
"""
|
||||
Generate synthetic speech clips, saving the clips to the specified output directory.
|
||||
|
||||
Args:
|
||||
text (List[str]): The text to convert into speech. Can be either a
|
||||
a list of strings, or a path to a file with text on each line.
|
||||
output_dir (str): The location to save the generated clips.
|
||||
max_samples (int): The maximum number of samples to generate.
|
||||
file_names (List[str]): The names to use when saving the files. Must be the same length
|
||||
as the `text` argument, if a list.
|
||||
model (str): The path to the STT model to use for generation.
|
||||
batch_size (int): The batch size to use when generated the clips
|
||||
slerp_weights (List[float]): The weights to use when mixing speakers via SLERP.
|
||||
length_scales (List[float]): Controls the average duration/speed of the generated speech.
|
||||
noise_scales (List[float]): A parameter for overall variability of the generated speech.
|
||||
noise_scale_ws (List[float]): A parameter for the stochastic duration of words/phonemes.
|
||||
max_speakers (int): The maximum speaker number to use, if the model is multi-speaker.
|
||||
verbose (bool): Enable or disable more detailed logging messages (default: False).
|
||||
auto_reduce_batch_size (bool): Automatically and temporarily reduce the batch size
|
||||
if CUDA OOM errors are detected, and try to resume generation.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
|
||||
if max_samples is None:
|
||||
max_samples = len(text)
|
||||
|
||||
_LOGGER.debug("Loading %s", model)
|
||||
model_path = Path(model)
|
||||
model = torch.load(model_path)
|
||||
model.eval()
|
||||
_LOGGER.info("Successfully loaded the model")
|
||||
|
||||
if torch.cuda.is_available():
|
||||
model.cuda()
|
||||
_LOGGER.debug("CUDA available, using GPU")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
config_path = f"{model_path}.json"
|
||||
with open(config_path, "r", encoding="utf-8") as config_file:
|
||||
config = json.load(config_file)
|
||||
|
||||
voice = config["espeak"]["voice"]
|
||||
sample_rate = config["audio"]["sample_rate"]
|
||||
num_speakers = config["num_speakers"]
|
||||
if max_speakers is not None:
|
||||
num_speakers = min(num_speakers, max_speakers)
|
||||
|
||||
max_len = None
|
||||
|
||||
sample_idx = 0
|
||||
is_done = False
|
||||
settings_iter = it.cycle(
|
||||
it.product(
|
||||
slerp_weights,
|
||||
length_scales,
|
||||
noise_scales,
|
||||
noise_scale_ws,
|
||||
)
|
||||
)
|
||||
|
||||
# Define resampler to get to 16khz (https://pytorch.org/audio/stable/tutorials/audio_resampling_tutorial.html#kaiser-best)
|
||||
sample_rate = 22050
|
||||
resample_rate = 16000
|
||||
resampler = torchaudio.transforms.Resample(
|
||||
sample_rate,
|
||||
resample_rate,
|
||||
lowpass_filter_width=64,
|
||||
rolloff=0.9475937167399596,
|
||||
resampling_method="kaiser_window",
|
||||
beta=14.769656459379492,
|
||||
)
|
||||
|
||||
speakers_iter = it.cycle(it.product(range(num_speakers), range(num_speakers)))
|
||||
speakers_batch = list(it.islice(speakers_iter, 0, batch_size))
|
||||
if isinstance(text, str) and os.path.exists(text):
|
||||
texts = it.cycle(
|
||||
[
|
||||
i.strip()
|
||||
for i in open(text, "r", encoding="utf-8").readlines()
|
||||
if len(i.strip()) > 0
|
||||
]
|
||||
)
|
||||
elif isinstance(text, list):
|
||||
texts = it.cycle(text)
|
||||
else:
|
||||
texts = it.cycle([text])
|
||||
|
||||
if file_names:
|
||||
file_names = it.cycle(file_names)
|
||||
|
||||
batch_idx = 0
|
||||
while speakers_batch:
|
||||
if is_done:
|
||||
break
|
||||
|
||||
batch_size = len(speakers_batch)
|
||||
slerp_weight, length_scale, noise_scale, noise_scale_w = next(settings_iter)
|
||||
|
||||
with torch.no_grad():
|
||||
speaker_1 = torch.LongTensor([s[0] for s in speakers_batch])
|
||||
speaker_2 = torch.LongTensor([s[1] for s in speakers_batch])
|
||||
|
||||
phoneme_ids = [
|
||||
get_phonemes(voice, config, next(texts), verbose)
|
||||
for i in range(batch_size)
|
||||
]
|
||||
|
||||
def right_pad_lists(lists):
|
||||
max_length = max(len(lst) for lst in lists)
|
||||
padded_lists = []
|
||||
for lst in lists:
|
||||
padded_l = lst + [1] * (
|
||||
max_length - len(lst)
|
||||
) # phoneme 1 (corresponding to '^' character seems to work best)
|
||||
padded_lists.append(padded_l)
|
||||
return padded_lists
|
||||
|
||||
phoneme_ids = right_pad_lists(phoneme_ids)
|
||||
|
||||
if auto_reduce_batch_size:
|
||||
oom_error = True
|
||||
counter = 1
|
||||
while oom_error is True:
|
||||
try:
|
||||
audio = generate_audio(
|
||||
model,
|
||||
speaker_1[0 : batch_size // counter],
|
||||
speaker_2[0 : batch_size // counter],
|
||||
phoneme_ids[0 : batch_size // counter],
|
||||
slerp_weight,
|
||||
noise_scale,
|
||||
noise_scale_w,
|
||||
length_scale,
|
||||
max_len,
|
||||
)
|
||||
oom_error = False
|
||||
except torch.cuda.OutOfMemoryError:
|
||||
torch.cuda.empty_cache()
|
||||
gc.collect()
|
||||
counter += 1 # reduce batch size to avoid OOM errors
|
||||
else:
|
||||
audio = generate_audio(
|
||||
model,
|
||||
speaker_1,
|
||||
speaker_2,
|
||||
phoneme_ids,
|
||||
slerp_weight,
|
||||
noise_scale,
|
||||
noise_scale_w,
|
||||
length_scale,
|
||||
max_len,
|
||||
)
|
||||
|
||||
# Resample audio
|
||||
audio = resampler(audio.cpu()).numpy()
|
||||
|
||||
audio_int16 = audio_float_to_int16(audio)
|
||||
for audio_idx in range(audio_int16.shape[0]):
|
||||
# Use webrtcvad to trip silence from the clips
|
||||
audio_data = remove_silence(audio_int16[audio_idx].flatten())[None,]
|
||||
|
||||
if isinstance(file_names, it.cycle):
|
||||
wav_path = output_dir / next(file_names)
|
||||
else:
|
||||
wav_path = output_dir / f"{sample_idx}.wav"
|
||||
with wave.open(str(wav_path), "wb") as wav_file:
|
||||
wav_file.setframerate(resample_rate)
|
||||
wav_file.setsampwidth(2)
|
||||
wav_file.setnchannels(1)
|
||||
wav_file.writeframes(audio_data)
|
||||
|
||||
sample_idx += 1
|
||||
if sample_idx >= max_samples:
|
||||
is_done = True
|
||||
break
|
||||
|
||||
# print(f"Batch {batch_idx +1}/{max_samples//batch_size} complete", " "*200, end='\r')
|
||||
|
||||
# Next batch
|
||||
_LOGGER.debug(f"Batch {batch_idx +1}/{max_samples//batch_size} complete")
|
||||
speakers_batch = list(it.islice(speakers_iter, 0, batch_size))
|
||||
batch_idx += 1
|
||||
|
||||
_LOGGER.info("Done")
|
||||
|
||||
|
||||
def remove_silence(x, frame_duration=0.030, sample_rate=16000, min_start=2000):
|
||||
"""Uses webrtc voice activity detection to remove silence from the clips"""
|
||||
vad = webrtcvad.Vad(0)
|
||||
if x.dtype == np.float32 or x.dtype == np.float64:
|
||||
x = (x * 32767).astype(np.int16)
|
||||
x_new = x[0:min_start].tolist()
|
||||
step_size = int(sample_rate * frame_duration)
|
||||
for i in range(min_start, x.shape[0] - step_size, step_size):
|
||||
vad_res = vad.is_speech(x[i : i + step_size].tobytes(), sample_rate)
|
||||
if vad_res:
|
||||
x_new.extend(x[i : i + step_size].tolist())
|
||||
return np.array(x_new).astype(np.int16)
|
||||
|
||||
|
||||
def generate_audio(
|
||||
model,
|
||||
speaker_1,
|
||||
speaker_2,
|
||||
phoneme_ids,
|
||||
slerp_weight,
|
||||
noise_scale,
|
||||
noise_scale_w,
|
||||
length_scale,
|
||||
max_len,
|
||||
):
|
||||
x = torch.LongTensor(phoneme_ids)
|
||||
x_lengths = torch.LongTensor([len(i) for i in phoneme_ids])
|
||||
|
||||
if torch.cuda.is_available():
|
||||
speaker_1 = speaker_1.cuda()
|
||||
speaker_2 = speaker_2.cuda()
|
||||
x = x.cuda()
|
||||
x_lengths = x_lengths.cuda()
|
||||
|
||||
x, m_p_orig, logs_p_orig, x_mask = model.enc_p(x, x_lengths)
|
||||
emb0 = model.emb_g(speaker_1)
|
||||
emb1 = model.emb_g(speaker_2)
|
||||
g = slerp(emb0, emb1, slerp_weight).unsqueeze(-1) # [b, h, 1]
|
||||
|
||||
if model.use_sdp:
|
||||
logw = model.dp(x, x_mask, g=g, reverse=True, noise_scale=noise_scale_w)
|
||||
else:
|
||||
logw = model.dp(x, x_mask, g=g)
|
||||
w = torch.exp(logw) * x_mask * length_scale
|
||||
w_ceil = torch.ceil(w)
|
||||
y_lengths = torch.clamp_min(torch.sum(w_ceil, [1, 2]), 1).long()
|
||||
y_mask = torch.unsqueeze(
|
||||
commons.sequence_mask(y_lengths, y_lengths.max()), 1
|
||||
).type_as(x_mask)
|
||||
attn_mask = torch.unsqueeze(x_mask, 2) * torch.unsqueeze(y_mask, -1)
|
||||
attn = commons.generate_path(w_ceil, attn_mask)
|
||||
|
||||
m_p = torch.matmul(attn.squeeze(1), m_p_orig.transpose(1, 2)).transpose(
|
||||
1, 2
|
||||
) # [b, t', t], [b, t, d] -> [b, d, t']
|
||||
logs_p = torch.matmul(attn.squeeze(1), logs_p_orig.transpose(1, 2)).transpose(
|
||||
1, 2
|
||||
) # [b, t', t], [b, t, d] -> [b, d, t']
|
||||
|
||||
z_p = m_p + torch.randn_like(m_p) * torch.exp(logs_p) * noise_scale
|
||||
z = model.flow(z_p, y_mask, g=g, reverse=True)
|
||||
o = model.dec((z * y_mask)[:, :, :max_len], g=g)
|
||||
|
||||
audio = o
|
||||
return audio
|
||||
|
||||
|
||||
def get_phonemes(voice, config, text, verbose):
|
||||
# Combine all sentences
|
||||
phonemes = [
|
||||
p
|
||||
for sentence_phonemes in phonemize_espeak(text, voice)
|
||||
for p in sentence_phonemes
|
||||
]
|
||||
if verbose is True:
|
||||
_LOGGER.debug("Phonemes: %s", phonemes)
|
||||
|
||||
id_map = config["phoneme_id_map"]
|
||||
phoneme_ids = list(id_map["^"])
|
||||
for phoneme in phonemes:
|
||||
p_ids = id_map.get(phoneme)
|
||||
if p_ids is not None:
|
||||
phoneme_ids.extend(p_ids)
|
||||
phoneme_ids.extend(id_map["_"])
|
||||
|
||||
phoneme_ids.extend(id_map["$"])
|
||||
return phoneme_ids
|
||||
|
||||
|
||||
def slerp(v1, v2, t, DOT_THR=0.9995, zdim=-1):
|
||||
"""SLERP for pytorch tensors interpolating `v1` to `v2` with scale of `t`.
|
||||
|
||||
`DOT_THR` determines when the vectors are too close to parallel.
|
||||
If they are too close, then a regular linear interpolation is used.
|
||||
|
||||
`zdim` is the feature dimension over which to compute norms and find angles.
|
||||
For example: if a sequence of 5 vectors is input with shape [5, 768]
|
||||
Then `zdim = 1` or `zdim = -1` computes SLERP along the feature dim of 768.
|
||||
|
||||
Theory Reference:
|
||||
https://splines.readthedocs.io/en/latest/rotation/slerp.html
|
||||
PyTorch reference:
|
||||
https://discuss.pytorch.org/t/help-regarding-slerp-function-for-generative-model-sampling/32475/3
|
||||
Numpy reference:
|
||||
https://gist.github.com/dvschultz/3af50c40df002da3b751efab1daddf2c
|
||||
"""
|
||||
|
||||
# take the dot product between normalized vectors
|
||||
v1_norm = v1 / torch.norm(v1, dim=zdim, keepdim=True)
|
||||
v2_norm = v2 / torch.norm(v2, dim=zdim, keepdim=True)
|
||||
dot = (v1_norm * v2_norm).sum(zdim)
|
||||
|
||||
# if the vectors are too close, return a simple linear interpolation
|
||||
if (torch.abs(dot) > DOT_THR).any():
|
||||
res = (1 - t) * v1 + t * v2
|
||||
|
||||
# else apply SLERP
|
||||
else:
|
||||
# compute the angle terms we need
|
||||
theta = torch.acos(dot)
|
||||
theta_t = theta * t
|
||||
sin_theta = torch.sin(theta)
|
||||
sin_theta_t = torch.sin(theta_t)
|
||||
|
||||
# compute the sine scaling terms for the vectors
|
||||
s1 = torch.sin(theta - theta_t) / sin_theta
|
||||
s2 = sin_theta_t / sin_theta
|
||||
|
||||
# interpolate the vectors
|
||||
res = (s1.unsqueeze(zdim) * v1) + (s2.unsqueeze(zdim) * v2)
|
||||
|
||||
return res
|
||||
|
||||
|
||||
def audio_float_to_int16(
|
||||
audio: np.ndarray, max_wav_value: float = 32767.0
|
||||
) -> np.ndarray:
|
||||
"""Normalize audio and convert to int16 range"""
|
||||
audio_norm = audio * (max_wav_value / max(0.01, np.max(np.abs(audio))))
|
||||
audio_norm = np.clip(audio_norm, -max_wav_value, max_wav_value)
|
||||
audio_norm = audio_norm.astype("int16")
|
||||
return audio_norm
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Get command line arguments
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("text")
|
||||
parser.add_argument("--max-samples", required=True, type=int)
|
||||
parser.add_argument(
|
||||
"--model", default=_DIR / "models" / "en_US-libritts_r-medium.pt"
|
||||
)
|
||||
parser.add_argument("--batch-size", type=int, default=1)
|
||||
parser.add_argument("--slerp-weights", nargs="+", type=float, default=[0.5])
|
||||
parser.add_argument(
|
||||
"--length-scales", nargs="+", type=float, default=[1.0, 0.75, 1.25, 1.4]
|
||||
)
|
||||
parser.add_argument(
|
||||
"--noise-scales",
|
||||
nargs="+",
|
||||
type=float,
|
||||
default=[0.667, 0.75, 0.85, 0.9, 1.0, 1.4],
|
||||
)
|
||||
parser.add_argument("--noise-scale-ws", nargs="+", type=float, default=[0.8])
|
||||
parser.add_argument("--output-dir", default="output")
|
||||
parser.add_argument(
|
||||
"--max-speakers",
|
||||
type=int,
|
||||
help="Maximum number of speakers to use (default: all)",
|
||||
)
|
||||
args = parser.parse_args().__dict__
|
||||
|
||||
# Generate speech
|
||||
generate_samples(**args)
|
||||
740
models/de_DE-mls-medium.pt.json
Executable file
740
models/de_DE-mls-medium.pt.json
Executable file
|
|
@ -0,0 +1,740 @@
|
|||
{
|
||||
"dataset": "mls",
|
||||
"audio": {
|
||||
"sample_rate": 22050,
|
||||
"quality": "medium"
|
||||
},
|
||||
"espeak": {
|
||||
"voice": "de"
|
||||
},
|
||||
"language": {
|
||||
"code": "de_DE"
|
||||
},
|
||||
"inference": {
|
||||
"noise_scale": 0.333,
|
||||
"length_scale": 1,
|
||||
"noise_w": 0.333
|
||||
},
|
||||
"phoneme_type": "espeak",
|
||||
"phoneme_map": {},
|
||||
"phoneme_id_map": {
|
||||
" ": [
|
||||
3
|
||||
],
|
||||
"!": [
|
||||
4
|
||||
],
|
||||
"\"": [
|
||||
150
|
||||
],
|
||||
"#": [
|
||||
149
|
||||
],
|
||||
"$": [
|
||||
2
|
||||
],
|
||||
"'": [
|
||||
5
|
||||
],
|
||||
"(": [
|
||||
6
|
||||
],
|
||||
")": [
|
||||
7
|
||||
],
|
||||
",": [
|
||||
8
|
||||
],
|
||||
"-": [
|
||||
9
|
||||
],
|
||||
".": [
|
||||
10
|
||||
],
|
||||
"0": [
|
||||
130
|
||||
],
|
||||
"1": [
|
||||
131
|
||||
],
|
||||
"2": [
|
||||
132
|
||||
],
|
||||
"3": [
|
||||
133
|
||||
],
|
||||
"4": [
|
||||
134
|
||||
],
|
||||
"5": [
|
||||
135
|
||||
],
|
||||
"6": [
|
||||
136
|
||||
],
|
||||
"7": [
|
||||
137
|
||||
],
|
||||
"8": [
|
||||
138
|
||||
],
|
||||
"9": [
|
||||
139
|
||||
],
|
||||
":": [
|
||||
11
|
||||
],
|
||||
";": [
|
||||
12
|
||||
],
|
||||
"?": [
|
||||
13
|
||||
],
|
||||
"X": [
|
||||
156
|
||||
],
|
||||
"^": [
|
||||
1
|
||||
],
|
||||
"_": [
|
||||
0
|
||||
],
|
||||
"a": [
|
||||
14
|
||||
],
|
||||
"b": [
|
||||
15
|
||||
],
|
||||
"c": [
|
||||
16
|
||||
],
|
||||
"d": [
|
||||
17
|
||||
],
|
||||
"e": [
|
||||
18
|
||||
],
|
||||
"f": [
|
||||
19
|
||||
],
|
||||
"g": [
|
||||
154
|
||||
],
|
||||
"h": [
|
||||
20
|
||||
],
|
||||
"i": [
|
||||
21
|
||||
],
|
||||
"j": [
|
||||
22
|
||||
],
|
||||
"k": [
|
||||
23
|
||||
],
|
||||
"l": [
|
||||
24
|
||||
],
|
||||
"m": [
|
||||
25
|
||||
],
|
||||
"n": [
|
||||
26
|
||||
],
|
||||
"o": [
|
||||
27
|
||||
],
|
||||
"p": [
|
||||
28
|
||||
],
|
||||
"q": [
|
||||
29
|
||||
],
|
||||
"r": [
|
||||
30
|
||||
],
|
||||
"s": [
|
||||
31
|
||||
],
|
||||
"t": [
|
||||
32
|
||||
],
|
||||
"u": [
|
||||
33
|
||||
],
|
||||
"v": [
|
||||
34
|
||||
],
|
||||
"w": [
|
||||
35
|
||||
],
|
||||
"x": [
|
||||
36
|
||||
],
|
||||
"y": [
|
||||
37
|
||||
],
|
||||
"z": [
|
||||
38
|
||||
],
|
||||
"æ": [
|
||||
39
|
||||
],
|
||||
"ç": [
|
||||
40
|
||||
],
|
||||
"ð": [
|
||||
41
|
||||
],
|
||||
"ø": [
|
||||
42
|
||||
],
|
||||
"ħ": [
|
||||
43
|
||||
],
|
||||
"ŋ": [
|
||||
44
|
||||
],
|
||||
"œ": [
|
||||
45
|
||||
],
|
||||
"ǀ": [
|
||||
46
|
||||
],
|
||||
"ǁ": [
|
||||
47
|
||||
],
|
||||
"ǂ": [
|
||||
48
|
||||
],
|
||||
"ǃ": [
|
||||
49
|
||||
],
|
||||
"ɐ": [
|
||||
50
|
||||
],
|
||||
"ɑ": [
|
||||
51
|
||||
],
|
||||
"ɒ": [
|
||||
52
|
||||
],
|
||||
"ɓ": [
|
||||
53
|
||||
],
|
||||
"ɔ": [
|
||||
54
|
||||
],
|
||||
"ɕ": [
|
||||
55
|
||||
],
|
||||
"ɖ": [
|
||||
56
|
||||
],
|
||||
"ɗ": [
|
||||
57
|
||||
],
|
||||
"ɘ": [
|
||||
58
|
||||
],
|
||||
"ə": [
|
||||
59
|
||||
],
|
||||
"ɚ": [
|
||||
60
|
||||
],
|
||||
"ɛ": [
|
||||
61
|
||||
],
|
||||
"ɜ": [
|
||||
62
|
||||
],
|
||||
"ɞ": [
|
||||
63
|
||||
],
|
||||
"ɟ": [
|
||||
64
|
||||
],
|
||||
"ɠ": [
|
||||
65
|
||||
],
|
||||
"ɡ": [
|
||||
66
|
||||
],
|
||||
"ɢ": [
|
||||
67
|
||||
],
|
||||
"ɣ": [
|
||||
68
|
||||
],
|
||||
"ɤ": [
|
||||
69
|
||||
],
|
||||
"ɥ": [
|
||||
70
|
||||
],
|
||||
"ɦ": [
|
||||
71
|
||||
],
|
||||
"ɧ": [
|
||||
72
|
||||
],
|
||||
"ɨ": [
|
||||
73
|
||||
],
|
||||
"ɪ": [
|
||||
74
|
||||
],
|
||||
"ɫ": [
|
||||
75
|
||||
],
|
||||
"ɬ": [
|
||||
76
|
||||
],
|
||||
"ɭ": [
|
||||
77
|
||||
],
|
||||
"ɮ": [
|
||||
78
|
||||
],
|
||||
"ɯ": [
|
||||
79
|
||||
],
|
||||
"ɰ": [
|
||||
80
|
||||
],
|
||||
"ɱ": [
|
||||
81
|
||||
],
|
||||
"ɲ": [
|
||||
82
|
||||
],
|
||||
"ɳ": [
|
||||
83
|
||||
],
|
||||
"ɴ": [
|
||||
84
|
||||
],
|
||||
"ɵ": [
|
||||
85
|
||||
],
|
||||
"ɶ": [
|
||||
86
|
||||
],
|
||||
"ɸ": [
|
||||
87
|
||||
],
|
||||
"ɹ": [
|
||||
88
|
||||
],
|
||||
"ɺ": [
|
||||
89
|
||||
],
|
||||
"ɻ": [
|
||||
90
|
||||
],
|
||||
"ɽ": [
|
||||
91
|
||||
],
|
||||
"ɾ": [
|
||||
92
|
||||
],
|
||||
"ʀ": [
|
||||
93
|
||||
],
|
||||
"ʁ": [
|
||||
94
|
||||
],
|
||||
"ʂ": [
|
||||
95
|
||||
],
|
||||
"ʃ": [
|
||||
96
|
||||
],
|
||||
"ʄ": [
|
||||
97
|
||||
],
|
||||
"ʈ": [
|
||||
98
|
||||
],
|
||||
"ʉ": [
|
||||
99
|
||||
],
|
||||
"ʊ": [
|
||||
100
|
||||
],
|
||||
"ʋ": [
|
||||
101
|
||||
],
|
||||
"ʌ": [
|
||||
102
|
||||
],
|
||||
"ʍ": [
|
||||
103
|
||||
],
|
||||
"ʎ": [
|
||||
104
|
||||
],
|
||||
"ʏ": [
|
||||
105
|
||||
],
|
||||
"ʐ": [
|
||||
106
|
||||
],
|
||||
"ʑ": [
|
||||
107
|
||||
],
|
||||
"ʒ": [
|
||||
108
|
||||
],
|
||||
"ʔ": [
|
||||
109
|
||||
],
|
||||
"ʕ": [
|
||||
110
|
||||
],
|
||||
"ʘ": [
|
||||
111
|
||||
],
|
||||
"ʙ": [
|
||||
112
|
||||
],
|
||||
"ʛ": [
|
||||
113
|
||||
],
|
||||
"ʜ": [
|
||||
114
|
||||
],
|
||||
"ʝ": [
|
||||
115
|
||||
],
|
||||
"ʟ": [
|
||||
116
|
||||
],
|
||||
"ʡ": [
|
||||
117
|
||||
],
|
||||
"ʢ": [
|
||||
118
|
||||
],
|
||||
"ʦ": [
|
||||
155
|
||||
],
|
||||
"ʰ": [
|
||||
145
|
||||
],
|
||||
"ʲ": [
|
||||
119
|
||||
],
|
||||
"ˈ": [
|
||||
120
|
||||
],
|
||||
"ˌ": [
|
||||
121
|
||||
],
|
||||
"ː": [
|
||||
122
|
||||
],
|
||||
"ˑ": [
|
||||
123
|
||||
],
|
||||
"˞": [
|
||||
124
|
||||
],
|
||||
"ˤ": [
|
||||
146
|
||||
],
|
||||
"̃": [
|
||||
141
|
||||
],
|
||||
"̊": [
|
||||
158
|
||||
],
|
||||
"̝": [
|
||||
157
|
||||
],
|
||||
"̧": [
|
||||
140
|
||||
],
|
||||
"̩": [
|
||||
144
|
||||
],
|
||||
"̪": [
|
||||
142
|
||||
],
|
||||
"̯": [
|
||||
143
|
||||
],
|
||||
"̺": [
|
||||
152
|
||||
],
|
||||
"̻": [
|
||||
153
|
||||
],
|
||||
"β": [
|
||||
125
|
||||
],
|
||||
"ε": [
|
||||
147
|
||||
],
|
||||
"θ": [
|
||||
126
|
||||
],
|
||||
"χ": [
|
||||
127
|
||||
],
|
||||
"ᵻ": [
|
||||
128
|
||||
],
|
||||
"↑": [
|
||||
151
|
||||
],
|
||||
"↓": [
|
||||
148
|
||||
],
|
||||
"ⱱ": [
|
||||
129
|
||||
]
|
||||
},
|
||||
"num_symbols": 256,
|
||||
"num_speakers": 236,
|
||||
"speaker_id_map": {
|
||||
"2422": 0,
|
||||
"4536": 1,
|
||||
"2037": 2,
|
||||
"9565": 3,
|
||||
"10148": 4,
|
||||
"6507": 5,
|
||||
"5055": 6,
|
||||
"3503": 7,
|
||||
"252": 8,
|
||||
"9132": 9,
|
||||
"3990": 10,
|
||||
"5753": 11,
|
||||
"5424": 12,
|
||||
"2602": 13,
|
||||
"4174": 14,
|
||||
"3885": 15,
|
||||
"12415": 16,
|
||||
"8470": 17,
|
||||
"11927": 18,
|
||||
"9639": 19,
|
||||
"3494": 20,
|
||||
"2946": 21,
|
||||
"5283": 22,
|
||||
"4533": 23,
|
||||
"2497": 24,
|
||||
"12275": 25,
|
||||
"1649": 26,
|
||||
"146": 27,
|
||||
"8337": 28,
|
||||
"4542": 29,
|
||||
"589": 30,
|
||||
"1998": 31,
|
||||
"3797": 32,
|
||||
"5244": 33,
|
||||
"7328": 34,
|
||||
"7998": 35,
|
||||
"10179": 36,
|
||||
"9610": 37,
|
||||
"20": 38,
|
||||
"253": 39,
|
||||
"12899": 40,
|
||||
"7194": 41,
|
||||
"3759": 42,
|
||||
"2677": 43,
|
||||
"6719": 44,
|
||||
"1897": 45,
|
||||
"11990": 46,
|
||||
"6880": 47,
|
||||
"19": 48,
|
||||
"9515": 49,
|
||||
"327": 50,
|
||||
"3244": 51,
|
||||
"5324": 52,
|
||||
"2234": 53,
|
||||
"3124": 54,
|
||||
"2043": 55,
|
||||
"143": 56,
|
||||
"8139": 57,
|
||||
"9646": 58,
|
||||
"8659": 59,
|
||||
"9538": 60,
|
||||
"989": 61,
|
||||
"5405": 62,
|
||||
"10087": 63,
|
||||
"8294": 64,
|
||||
"4396": 65,
|
||||
"1474": 66,
|
||||
"139": 67,
|
||||
"136": 68,
|
||||
"10791": 69,
|
||||
"7242": 70,
|
||||
"3631": 71,
|
||||
"9908": 72,
|
||||
"7906": 73,
|
||||
"1171": 74,
|
||||
"7479": 75,
|
||||
"5632": 76,
|
||||
"3731": 77,
|
||||
"4650": 78,
|
||||
"135": 79,
|
||||
"145": 80,
|
||||
"137": 81,
|
||||
"1757": 82,
|
||||
"91": 83,
|
||||
"9514": 84,
|
||||
"13494": 85,
|
||||
"1946": 86,
|
||||
"3277": 87,
|
||||
"5595": 88,
|
||||
"278": 89,
|
||||
"7120": 90,
|
||||
"7406": 91,
|
||||
"11695": 92,
|
||||
"1593": 93,
|
||||
"3862": 94,
|
||||
"138": 95,
|
||||
"141": 96,
|
||||
"9948": 97,
|
||||
"1163": 98,
|
||||
"1054": 99,
|
||||
"1844": 100,
|
||||
"4911": 101,
|
||||
"7261": 102,
|
||||
"8223": 103,
|
||||
"7624": 104,
|
||||
"144": 105,
|
||||
"13871": 106,
|
||||
"2974": 107,
|
||||
"5934": 108,
|
||||
"7002": 109,
|
||||
"8769": 110,
|
||||
"3363": 111,
|
||||
"3040": 112,
|
||||
"6067": 113,
|
||||
"9494": 114,
|
||||
"8743": 115,
|
||||
"13255": 116,
|
||||
"1660": 117,
|
||||
"3588": 118,
|
||||
"4748": 119,
|
||||
"8450": 120,
|
||||
"5295": 121,
|
||||
"4705": 122,
|
||||
"8125": 123,
|
||||
"7272": 124,
|
||||
"7320": 125,
|
||||
"1874": 126,
|
||||
"1262": 127,
|
||||
"10870": 128,
|
||||
"12379": 129,
|
||||
"4463": 130,
|
||||
"10349": 131,
|
||||
"2252": 132,
|
||||
"8325": 133,
|
||||
"3052": 134,
|
||||
"4001": 135,
|
||||
"7456": 136,
|
||||
"140": 137,
|
||||
"4739": 138,
|
||||
"11299": 139,
|
||||
"4730": 140,
|
||||
"6659": 141,
|
||||
"2034": 142,
|
||||
"2732": 143,
|
||||
"2158": 144,
|
||||
"3698": 145,
|
||||
"5675": 146,
|
||||
"6315": 147,
|
||||
"10904": 148,
|
||||
"7202": 149,
|
||||
"11480": 150,
|
||||
"10625": 151,
|
||||
"11546": 152,
|
||||
"4576": 153,
|
||||
"4512": 154,
|
||||
"8634": 155,
|
||||
"13626": 156,
|
||||
"1613": 157,
|
||||
"287": 158,
|
||||
"9207": 159,
|
||||
"6982": 160,
|
||||
"1724": 161,
|
||||
"10191": 162,
|
||||
"1091": 163,
|
||||
"2909": 164,
|
||||
"1965": 165,
|
||||
"2506": 166,
|
||||
"4414": 167,
|
||||
"5764": 168,
|
||||
"12776": 169,
|
||||
"1033": 170,
|
||||
"13726": 171,
|
||||
"2314": 172,
|
||||
"6826": 173,
|
||||
"9706": 174,
|
||||
"8427": 175,
|
||||
"9168": 176,
|
||||
"9287": 177,
|
||||
"6905": 178,
|
||||
"4153": 179,
|
||||
"3330": 180,
|
||||
"2859": 181,
|
||||
"5406": 182,
|
||||
"2840": 183,
|
||||
"1920": 184,
|
||||
"9241": 185,
|
||||
"10163": 186,
|
||||
"8305": 187,
|
||||
"12461": 188,
|
||||
"3276": 189,
|
||||
"11413": 190,
|
||||
"10536": 191,
|
||||
"10614": 192,
|
||||
"7579": 193,
|
||||
"8675": 194,
|
||||
"7483": 195,
|
||||
"7270": 196,
|
||||
"8704": 197,
|
||||
"4468": 198,
|
||||
"6611": 199,
|
||||
"11497": 200,
|
||||
"11772": 201,
|
||||
"2792": 202,
|
||||
"11481": 203,
|
||||
"10162": 204,
|
||||
"10819": 205,
|
||||
"8732": 206,
|
||||
"11328": 207,
|
||||
"11920": 208,
|
||||
"6646": 209,
|
||||
"7486": 210,
|
||||
"11870": 211,
|
||||
"12417": 212,
|
||||
"10364": 213,
|
||||
"6117": 214,
|
||||
"6448": 215,
|
||||
"10433": 216,
|
||||
"7515": 217,
|
||||
"5823": 218,
|
||||
"8567": 219,
|
||||
"10947": 220,
|
||||
"11869": 221,
|
||||
"12335": 222,
|
||||
"12500": 223,
|
||||
"13755": 224,
|
||||
"7006": 225,
|
||||
"3685": 226,
|
||||
"5819": 227,
|
||||
"9353": 228,
|
||||
"11355": 229,
|
||||
"12174": 230,
|
||||
"7150": 231,
|
||||
"6952": 232,
|
||||
"11987": 233,
|
||||
"3995": 234,
|
||||
"7449": 235
|
||||
},
|
||||
"piper_version": "1.0.0"
|
||||
}
|
||||
629
models/fr_FR-mls-medium.pt.json
Executable file
629
models/fr_FR-mls-medium.pt.json
Executable file
|
|
@ -0,0 +1,629 @@
|
|||
{
|
||||
"dataset": "mls",
|
||||
"audio": {
|
||||
"sample_rate": 22050,
|
||||
"quality": "medium"
|
||||
},
|
||||
"espeak": {
|
||||
"voice": "fr"
|
||||
},
|
||||
"language": {
|
||||
"code": "fr_FR"
|
||||
},
|
||||
"inference": {
|
||||
"noise_scale": 0.333,
|
||||
"length_scale": 1,
|
||||
"noise_w": 0.333
|
||||
},
|
||||
"phoneme_type": "espeak",
|
||||
"phoneme_map": {},
|
||||
"phoneme_id_map": {
|
||||
" ": [
|
||||
3
|
||||
],
|
||||
"!": [
|
||||
4
|
||||
],
|
||||
"\"": [
|
||||
150
|
||||
],
|
||||
"#": [
|
||||
149
|
||||
],
|
||||
"$": [
|
||||
2
|
||||
],
|
||||
"'": [
|
||||
5
|
||||
],
|
||||
"(": [
|
||||
6
|
||||
],
|
||||
")": [
|
||||
7
|
||||
],
|
||||
",": [
|
||||
8
|
||||
],
|
||||
"-": [
|
||||
9
|
||||
],
|
||||
".": [
|
||||
10
|
||||
],
|
||||
"0": [
|
||||
130
|
||||
],
|
||||
"1": [
|
||||
131
|
||||
],
|
||||
"2": [
|
||||
132
|
||||
],
|
||||
"3": [
|
||||
133
|
||||
],
|
||||
"4": [
|
||||
134
|
||||
],
|
||||
"5": [
|
||||
135
|
||||
],
|
||||
"6": [
|
||||
136
|
||||
],
|
||||
"7": [
|
||||
137
|
||||
],
|
||||
"8": [
|
||||
138
|
||||
],
|
||||
"9": [
|
||||
139
|
||||
],
|
||||
":": [
|
||||
11
|
||||
],
|
||||
";": [
|
||||
12
|
||||
],
|
||||
"?": [
|
||||
13
|
||||
],
|
||||
"X": [
|
||||
156
|
||||
],
|
||||
"^": [
|
||||
1
|
||||
],
|
||||
"_": [
|
||||
0
|
||||
],
|
||||
"a": [
|
||||
14
|
||||
],
|
||||
"b": [
|
||||
15
|
||||
],
|
||||
"c": [
|
||||
16
|
||||
],
|
||||
"d": [
|
||||
17
|
||||
],
|
||||
"e": [
|
||||
18
|
||||
],
|
||||
"f": [
|
||||
19
|
||||
],
|
||||
"g": [
|
||||
154
|
||||
],
|
||||
"h": [
|
||||
20
|
||||
],
|
||||
"i": [
|
||||
21
|
||||
],
|
||||
"j": [
|
||||
22
|
||||
],
|
||||
"k": [
|
||||
23
|
||||
],
|
||||
"l": [
|
||||
24
|
||||
],
|
||||
"m": [
|
||||
25
|
||||
],
|
||||
"n": [
|
||||
26
|
||||
],
|
||||
"o": [
|
||||
27
|
||||
],
|
||||
"p": [
|
||||
28
|
||||
],
|
||||
"q": [
|
||||
29
|
||||
],
|
||||
"r": [
|
||||
30
|
||||
],
|
||||
"s": [
|
||||
31
|
||||
],
|
||||
"t": [
|
||||
32
|
||||
],
|
||||
"u": [
|
||||
33
|
||||
],
|
||||
"v": [
|
||||
34
|
||||
],
|
||||
"w": [
|
||||
35
|
||||
],
|
||||
"x": [
|
||||
36
|
||||
],
|
||||
"y": [
|
||||
37
|
||||
],
|
||||
"z": [
|
||||
38
|
||||
],
|
||||
"æ": [
|
||||
39
|
||||
],
|
||||
"ç": [
|
||||
40
|
||||
],
|
||||
"ð": [
|
||||
41
|
||||
],
|
||||
"ø": [
|
||||
42
|
||||
],
|
||||
"ħ": [
|
||||
43
|
||||
],
|
||||
"ŋ": [
|
||||
44
|
||||
],
|
||||
"œ": [
|
||||
45
|
||||
],
|
||||
"ǀ": [
|
||||
46
|
||||
],
|
||||
"ǁ": [
|
||||
47
|
||||
],
|
||||
"ǂ": [
|
||||
48
|
||||
],
|
||||
"ǃ": [
|
||||
49
|
||||
],
|
||||
"ɐ": [
|
||||
50
|
||||
],
|
||||
"ɑ": [
|
||||
51
|
||||
],
|
||||
"ɒ": [
|
||||
52
|
||||
],
|
||||
"ɓ": [
|
||||
53
|
||||
],
|
||||
"ɔ": [
|
||||
54
|
||||
],
|
||||
"ɕ": [
|
||||
55
|
||||
],
|
||||
"ɖ": [
|
||||
56
|
||||
],
|
||||
"ɗ": [
|
||||
57
|
||||
],
|
||||
"ɘ": [
|
||||
58
|
||||
],
|
||||
"ə": [
|
||||
59
|
||||
],
|
||||
"ɚ": [
|
||||
60
|
||||
],
|
||||
"ɛ": [
|
||||
61
|
||||
],
|
||||
"ɜ": [
|
||||
62
|
||||
],
|
||||
"ɞ": [
|
||||
63
|
||||
],
|
||||
"ɟ": [
|
||||
64
|
||||
],
|
||||
"ɠ": [
|
||||
65
|
||||
],
|
||||
"ɡ": [
|
||||
66
|
||||
],
|
||||
"ɢ": [
|
||||
67
|
||||
],
|
||||
"ɣ": [
|
||||
68
|
||||
],
|
||||
"ɤ": [
|
||||
69
|
||||
],
|
||||
"ɥ": [
|
||||
70
|
||||
],
|
||||
"ɦ": [
|
||||
71
|
||||
],
|
||||
"ɧ": [
|
||||
72
|
||||
],
|
||||
"ɨ": [
|
||||
73
|
||||
],
|
||||
"ɪ": [
|
||||
74
|
||||
],
|
||||
"ɫ": [
|
||||
75
|
||||
],
|
||||
"ɬ": [
|
||||
76
|
||||
],
|
||||
"ɭ": [
|
||||
77
|
||||
],
|
||||
"ɮ": [
|
||||
78
|
||||
],
|
||||
"ɯ": [
|
||||
79
|
||||
],
|
||||
"ɰ": [
|
||||
80
|
||||
],
|
||||
"ɱ": [
|
||||
81
|
||||
],
|
||||
"ɲ": [
|
||||
82
|
||||
],
|
||||
"ɳ": [
|
||||
83
|
||||
],
|
||||
"ɴ": [
|
||||
84
|
||||
],
|
||||
"ɵ": [
|
||||
85
|
||||
],
|
||||
"ɶ": [
|
||||
86
|
||||
],
|
||||
"ɸ": [
|
||||
87
|
||||
],
|
||||
"ɹ": [
|
||||
88
|
||||
],
|
||||
"ɺ": [
|
||||
89
|
||||
],
|
||||
"ɻ": [
|
||||
90
|
||||
],
|
||||
"ɽ": [
|
||||
91
|
||||
],
|
||||
"ɾ": [
|
||||
92
|
||||
],
|
||||
"ʀ": [
|
||||
93
|
||||
],
|
||||
"ʁ": [
|
||||
94
|
||||
],
|
||||
"ʂ": [
|
||||
95
|
||||
],
|
||||
"ʃ": [
|
||||
96
|
||||
],
|
||||
"ʄ": [
|
||||
97
|
||||
],
|
||||
"ʈ": [
|
||||
98
|
||||
],
|
||||
"ʉ": [
|
||||
99
|
||||
],
|
||||
"ʊ": [
|
||||
100
|
||||
],
|
||||
"ʋ": [
|
||||
101
|
||||
],
|
||||
"ʌ": [
|
||||
102
|
||||
],
|
||||
"ʍ": [
|
||||
103
|
||||
],
|
||||
"ʎ": [
|
||||
104
|
||||
],
|
||||
"ʏ": [
|
||||
105
|
||||
],
|
||||
"ʐ": [
|
||||
106
|
||||
],
|
||||
"ʑ": [
|
||||
107
|
||||
],
|
||||
"ʒ": [
|
||||
108
|
||||
],
|
||||
"ʔ": [
|
||||
109
|
||||
],
|
||||
"ʕ": [
|
||||
110
|
||||
],
|
||||
"ʘ": [
|
||||
111
|
||||
],
|
||||
"ʙ": [
|
||||
112
|
||||
],
|
||||
"ʛ": [
|
||||
113
|
||||
],
|
||||
"ʜ": [
|
||||
114
|
||||
],
|
||||
"ʝ": [
|
||||
115
|
||||
],
|
||||
"ʟ": [
|
||||
116
|
||||
],
|
||||
"ʡ": [
|
||||
117
|
||||
],
|
||||
"ʢ": [
|
||||
118
|
||||
],
|
||||
"ʦ": [
|
||||
155
|
||||
],
|
||||
"ʰ": [
|
||||
145
|
||||
],
|
||||
"ʲ": [
|
||||
119
|
||||
],
|
||||
"ˈ": [
|
||||
120
|
||||
],
|
||||
"ˌ": [
|
||||
121
|
||||
],
|
||||
"ː": [
|
||||
122
|
||||
],
|
||||
"ˑ": [
|
||||
123
|
||||
],
|
||||
"˞": [
|
||||
124
|
||||
],
|
||||
"ˤ": [
|
||||
146
|
||||
],
|
||||
"̃": [
|
||||
141
|
||||
],
|
||||
"̊": [
|
||||
158
|
||||
],
|
||||
"̝": [
|
||||
157
|
||||
],
|
||||
"̧": [
|
||||
140
|
||||
],
|
||||
"̩": [
|
||||
144
|
||||
],
|
||||
"̪": [
|
||||
142
|
||||
],
|
||||
"̯": [
|
||||
143
|
||||
],
|
||||
"̺": [
|
||||
152
|
||||
],
|
||||
"̻": [
|
||||
153
|
||||
],
|
||||
"β": [
|
||||
125
|
||||
],
|
||||
"ε": [
|
||||
147
|
||||
],
|
||||
"θ": [
|
||||
126
|
||||
],
|
||||
"χ": [
|
||||
127
|
||||
],
|
||||
"ᵻ": [
|
||||
128
|
||||
],
|
||||
"↑": [
|
||||
151
|
||||
],
|
||||
"↓": [
|
||||
148
|
||||
],
|
||||
"ⱱ": [
|
||||
129
|
||||
]
|
||||
},
|
||||
"num_symbols": 256,
|
||||
"num_speakers": 125,
|
||||
"speaker_id_map": {
|
||||
"1840": 0,
|
||||
"3698": 1,
|
||||
"123": 2,
|
||||
"1474": 3,
|
||||
"12709": 4,
|
||||
"7423": 5,
|
||||
"9242": 6,
|
||||
"8778": 7,
|
||||
"3060": 8,
|
||||
"4512": 9,
|
||||
"6249": 10,
|
||||
"12541": 11,
|
||||
"13634": 12,
|
||||
"10065": 13,
|
||||
"6128": 14,
|
||||
"5232": 15,
|
||||
"5764": 16,
|
||||
"12713": 17,
|
||||
"12823": 18,
|
||||
"6070": 19,
|
||||
"12501": 20,
|
||||
"9121": 21,
|
||||
"1649": 22,
|
||||
"2776": 23,
|
||||
"11772": 24,
|
||||
"5612": 25,
|
||||
"11822": 26,
|
||||
"1590": 27,
|
||||
"5525": 28,
|
||||
"10827": 29,
|
||||
"1243": 30,
|
||||
"13142": 31,
|
||||
"62": 32,
|
||||
"13177": 33,
|
||||
"10620": 34,
|
||||
"8102": 35,
|
||||
"8582": 36,
|
||||
"11875": 37,
|
||||
"7239": 38,
|
||||
"9854": 39,
|
||||
"7377": 40,
|
||||
"10082": 41,
|
||||
"12512": 42,
|
||||
"1329": 43,
|
||||
"2506": 44,
|
||||
"6856": 45,
|
||||
"10058": 46,
|
||||
"103": 47,
|
||||
"14": 48,
|
||||
"6381": 49,
|
||||
"1664": 50,
|
||||
"11954": 51,
|
||||
"66": 52,
|
||||
"1127": 53,
|
||||
"3270": 54,
|
||||
"13611": 55,
|
||||
"13658": 56,
|
||||
"12968": 57,
|
||||
"1989": 58,
|
||||
"12981": 59,
|
||||
"7193": 60,
|
||||
"6348": 61,
|
||||
"7679": 62,
|
||||
"2284": 63,
|
||||
"3182": 64,
|
||||
"3503": 65,
|
||||
"2033": 66,
|
||||
"2771": 67,
|
||||
"7614": 68,
|
||||
"125": 69,
|
||||
"3204": 70,
|
||||
"5595": 71,
|
||||
"5553": 72,
|
||||
"694": 73,
|
||||
"1624": 74,
|
||||
"1887": 75,
|
||||
"2926": 76,
|
||||
"7150": 77,
|
||||
"3190": 78,
|
||||
"3344": 79,
|
||||
"4699": 80,
|
||||
"1798": 81,
|
||||
"1745": 82,
|
||||
"5077": 83,
|
||||
"753": 84,
|
||||
"52": 85,
|
||||
"4174": 86,
|
||||
"4018": 87,
|
||||
"12899": 88,
|
||||
"1844": 89,
|
||||
"4396": 90,
|
||||
"1817": 91,
|
||||
"2155": 92,
|
||||
"2946": 93,
|
||||
"4336": 94,
|
||||
"4609": 95,
|
||||
"1977": 96,
|
||||
"10957": 97,
|
||||
"204": 98,
|
||||
"4650": 99,
|
||||
"5295": 100,
|
||||
"5968": 101,
|
||||
"4744": 102,
|
||||
"2825": 103,
|
||||
"9804": 104,
|
||||
"707": 105,
|
||||
"30": 106,
|
||||
"115": 107,
|
||||
"5840": 108,
|
||||
"2587": 109,
|
||||
"2607": 110,
|
||||
"2544": 111,
|
||||
"28": 112,
|
||||
"27": 113,
|
||||
"177": 114,
|
||||
"112": 115,
|
||||
"94": 116,
|
||||
"2596": 117,
|
||||
"3595": 118,
|
||||
"7032": 119,
|
||||
"7848": 120,
|
||||
"11247": 121,
|
||||
"7439": 122,
|
||||
"2904": 123,
|
||||
"6362": 124
|
||||
},
|
||||
"piper_version": "1.0.0"
|
||||
}
|
||||
554
models/nl_NL-mls-medium.pt.json
Executable file
554
models/nl_NL-mls-medium.pt.json
Executable file
|
|
@ -0,0 +1,554 @@
|
|||
{
|
||||
"audio": {
|
||||
"sample_rate": 22050
|
||||
},
|
||||
"espeak": {
|
||||
"voice": "nl"
|
||||
},
|
||||
"language": {
|
||||
"code": "nl_NL"
|
||||
},
|
||||
"inference": {
|
||||
"noise_scale": 0.333,
|
||||
"length_scale": 1,
|
||||
"noise_w": 0.333
|
||||
},
|
||||
"phoneme_type": "espeak",
|
||||
"phoneme_map": {},
|
||||
"phoneme_id_map": {
|
||||
" ": [
|
||||
3
|
||||
],
|
||||
"!": [
|
||||
4
|
||||
],
|
||||
"\"": [
|
||||
150
|
||||
],
|
||||
"#": [
|
||||
149
|
||||
],
|
||||
"$": [
|
||||
2
|
||||
],
|
||||
"'": [
|
||||
5
|
||||
],
|
||||
"(": [
|
||||
6
|
||||
],
|
||||
")": [
|
||||
7
|
||||
],
|
||||
",": [
|
||||
8
|
||||
],
|
||||
"-": [
|
||||
9
|
||||
],
|
||||
".": [
|
||||
10
|
||||
],
|
||||
"0": [
|
||||
130
|
||||
],
|
||||
"1": [
|
||||
131
|
||||
],
|
||||
"2": [
|
||||
132
|
||||
],
|
||||
"3": [
|
||||
133
|
||||
],
|
||||
"4": [
|
||||
134
|
||||
],
|
||||
"5": [
|
||||
135
|
||||
],
|
||||
"6": [
|
||||
136
|
||||
],
|
||||
"7": [
|
||||
137
|
||||
],
|
||||
"8": [
|
||||
138
|
||||
],
|
||||
"9": [
|
||||
139
|
||||
],
|
||||
":": [
|
||||
11
|
||||
],
|
||||
";": [
|
||||
12
|
||||
],
|
||||
"?": [
|
||||
13
|
||||
],
|
||||
"X": [
|
||||
156
|
||||
],
|
||||
"^": [
|
||||
1
|
||||
],
|
||||
"_": [
|
||||
0
|
||||
],
|
||||
"a": [
|
||||
14
|
||||
],
|
||||
"b": [
|
||||
15
|
||||
],
|
||||
"c": [
|
||||
16
|
||||
],
|
||||
"d": [
|
||||
17
|
||||
],
|
||||
"e": [
|
||||
18
|
||||
],
|
||||
"f": [
|
||||
19
|
||||
],
|
||||
"g": [
|
||||
154
|
||||
],
|
||||
"h": [
|
||||
20
|
||||
],
|
||||
"i": [
|
||||
21
|
||||
],
|
||||
"j": [
|
||||
22
|
||||
],
|
||||
"k": [
|
||||
23
|
||||
],
|
||||
"l": [
|
||||
24
|
||||
],
|
||||
"m": [
|
||||
25
|
||||
],
|
||||
"n": [
|
||||
26
|
||||
],
|
||||
"o": [
|
||||
27
|
||||
],
|
||||
"p": [
|
||||
28
|
||||
],
|
||||
"q": [
|
||||
29
|
||||
],
|
||||
"r": [
|
||||
30
|
||||
],
|
||||
"s": [
|
||||
31
|
||||
],
|
||||
"t": [
|
||||
32
|
||||
],
|
||||
"u": [
|
||||
33
|
||||
],
|
||||
"v": [
|
||||
34
|
||||
],
|
||||
"w": [
|
||||
35
|
||||
],
|
||||
"x": [
|
||||
36
|
||||
],
|
||||
"y": [
|
||||
37
|
||||
],
|
||||
"z": [
|
||||
38
|
||||
],
|
||||
"æ": [
|
||||
39
|
||||
],
|
||||
"ç": [
|
||||
40
|
||||
],
|
||||
"ð": [
|
||||
41
|
||||
],
|
||||
"ø": [
|
||||
42
|
||||
],
|
||||
"ħ": [
|
||||
43
|
||||
],
|
||||
"ŋ": [
|
||||
44
|
||||
],
|
||||
"œ": [
|
||||
45
|
||||
],
|
||||
"ǀ": [
|
||||
46
|
||||
],
|
||||
"ǁ": [
|
||||
47
|
||||
],
|
||||
"ǂ": [
|
||||
48
|
||||
],
|
||||
"ǃ": [
|
||||
49
|
||||
],
|
||||
"ɐ": [
|
||||
50
|
||||
],
|
||||
"ɑ": [
|
||||
51
|
||||
],
|
||||
"ɒ": [
|
||||
52
|
||||
],
|
||||
"ɓ": [
|
||||
53
|
||||
],
|
||||
"ɔ": [
|
||||
54
|
||||
],
|
||||
"ɕ": [
|
||||
55
|
||||
],
|
||||
"ɖ": [
|
||||
56
|
||||
],
|
||||
"ɗ": [
|
||||
57
|
||||
],
|
||||
"ɘ": [
|
||||
58
|
||||
],
|
||||
"ə": [
|
||||
59
|
||||
],
|
||||
"ɚ": [
|
||||
60
|
||||
],
|
||||
"ɛ": [
|
||||
61
|
||||
],
|
||||
"ɜ": [
|
||||
62
|
||||
],
|
||||
"ɞ": [
|
||||
63
|
||||
],
|
||||
"ɟ": [
|
||||
64
|
||||
],
|
||||
"ɠ": [
|
||||
65
|
||||
],
|
||||
"ɡ": [
|
||||
66
|
||||
],
|
||||
"ɢ": [
|
||||
67
|
||||
],
|
||||
"ɣ": [
|
||||
68
|
||||
],
|
||||
"ɤ": [
|
||||
69
|
||||
],
|
||||
"ɥ": [
|
||||
70
|
||||
],
|
||||
"ɦ": [
|
||||
71
|
||||
],
|
||||
"ɧ": [
|
||||
72
|
||||
],
|
||||
"ɨ": [
|
||||
73
|
||||
],
|
||||
"ɪ": [
|
||||
74
|
||||
],
|
||||
"ɫ": [
|
||||
75
|
||||
],
|
||||
"ɬ": [
|
||||
76
|
||||
],
|
||||
"ɭ": [
|
||||
77
|
||||
],
|
||||
"ɮ": [
|
||||
78
|
||||
],
|
||||
"ɯ": [
|
||||
79
|
||||
],
|
||||
"ɰ": [
|
||||
80
|
||||
],
|
||||
"ɱ": [
|
||||
81
|
||||
],
|
||||
"ɲ": [
|
||||
82
|
||||
],
|
||||
"ɳ": [
|
||||
83
|
||||
],
|
||||
"ɴ": [
|
||||
84
|
||||
],
|
||||
"ɵ": [
|
||||
85
|
||||
],
|
||||
"ɶ": [
|
||||
86
|
||||
],
|
||||
"ɸ": [
|
||||
87
|
||||
],
|
||||
"ɹ": [
|
||||
88
|
||||
],
|
||||
"ɺ": [
|
||||
89
|
||||
],
|
||||
"ɻ": [
|
||||
90
|
||||
],
|
||||
"ɽ": [
|
||||
91
|
||||
],
|
||||
"ɾ": [
|
||||
92
|
||||
],
|
||||
"ʀ": [
|
||||
93
|
||||
],
|
||||
"ʁ": [
|
||||
94
|
||||
],
|
||||
"ʂ": [
|
||||
95
|
||||
],
|
||||
"ʃ": [
|
||||
96
|
||||
],
|
||||
"ʄ": [
|
||||
97
|
||||
],
|
||||
"ʈ": [
|
||||
98
|
||||
],
|
||||
"ʉ": [
|
||||
99
|
||||
],
|
||||
"ʊ": [
|
||||
100
|
||||
],
|
||||
"ʋ": [
|
||||
101
|
||||
],
|
||||
"ʌ": [
|
||||
102
|
||||
],
|
||||
"ʍ": [
|
||||
103
|
||||
],
|
||||
"ʎ": [
|
||||
104
|
||||
],
|
||||
"ʏ": [
|
||||
105
|
||||
],
|
||||
"ʐ": [
|
||||
106
|
||||
],
|
||||
"ʑ": [
|
||||
107
|
||||
],
|
||||
"ʒ": [
|
||||
108
|
||||
],
|
||||
"ʔ": [
|
||||
109
|
||||
],
|
||||
"ʕ": [
|
||||
110
|
||||
],
|
||||
"ʘ": [
|
||||
111
|
||||
],
|
||||
"ʙ": [
|
||||
112
|
||||
],
|
||||
"ʛ": [
|
||||
113
|
||||
],
|
||||
"ʜ": [
|
||||
114
|
||||
],
|
||||
"ʝ": [
|
||||
115
|
||||
],
|
||||
"ʟ": [
|
||||
116
|
||||
],
|
||||
"ʡ": [
|
||||
117
|
||||
],
|
||||
"ʢ": [
|
||||
118
|
||||
],
|
||||
"ʦ": [
|
||||
155
|
||||
],
|
||||
"ʰ": [
|
||||
145
|
||||
],
|
||||
"ʲ": [
|
||||
119
|
||||
],
|
||||
"ˈ": [
|
||||
120
|
||||
],
|
||||
"ˌ": [
|
||||
121
|
||||
],
|
||||
"ː": [
|
||||
122
|
||||
],
|
||||
"ˑ": [
|
||||
123
|
||||
],
|
||||
"˞": [
|
||||
124
|
||||
],
|
||||
"ˤ": [
|
||||
146
|
||||
],
|
||||
"̃": [
|
||||
141
|
||||
],
|
||||
"̊": [
|
||||
158
|
||||
],
|
||||
"̝": [
|
||||
157
|
||||
],
|
||||
"̧": [
|
||||
140
|
||||
],
|
||||
"̩": [
|
||||
144
|
||||
],
|
||||
"̪": [
|
||||
142
|
||||
],
|
||||
"̯": [
|
||||
143
|
||||
],
|
||||
"̺": [
|
||||
152
|
||||
],
|
||||
"̻": [
|
||||
153
|
||||
],
|
||||
"β": [
|
||||
125
|
||||
],
|
||||
"ε": [
|
||||
147
|
||||
],
|
||||
"θ": [
|
||||
126
|
||||
],
|
||||
"χ": [
|
||||
127
|
||||
],
|
||||
"ᵻ": [
|
||||
128
|
||||
],
|
||||
"↑": [
|
||||
151
|
||||
],
|
||||
"↓": [
|
||||
148
|
||||
],
|
||||
"ⱱ": [
|
||||
129
|
||||
]
|
||||
},
|
||||
"num_symbols": 256,
|
||||
"num_speakers": 52,
|
||||
"speaker_id_map": {
|
||||
"2450": 0,
|
||||
"1724": 1,
|
||||
"1666": 2,
|
||||
"5809": 3,
|
||||
"496": 4,
|
||||
"2506": 5,
|
||||
"7432": 6,
|
||||
"3619": 7,
|
||||
"4429": 8,
|
||||
"3798": 9,
|
||||
"12500": 10,
|
||||
"10587": 11,
|
||||
"2951": 12,
|
||||
"1775": 13,
|
||||
"9861": 14,
|
||||
"880": 15,
|
||||
"3034": 16,
|
||||
"2825": 17,
|
||||
"5438": 18,
|
||||
"3245": 19,
|
||||
"4396": 20,
|
||||
"11290": 21,
|
||||
"11936": 22,
|
||||
"6916": 23,
|
||||
"10294": 24,
|
||||
"10079": 25,
|
||||
"7588": 26,
|
||||
"7579": 27,
|
||||
"123": 28,
|
||||
"3024": 29,
|
||||
"960": 30,
|
||||
"10984": 31,
|
||||
"2792": 32,
|
||||
"7723": 33,
|
||||
"4174": 34,
|
||||
"2981": 35,
|
||||
"5764": 36,
|
||||
"6513": 37,
|
||||
"7884": 38,
|
||||
"6697": 39,
|
||||
"12749": 40,
|
||||
"11157": 41,
|
||||
"2239": 42,
|
||||
"10879": 43,
|
||||
"1085": 44,
|
||||
"8480": 45,
|
||||
"8331": 46,
|
||||
"6282": 47,
|
||||
"10632": 48,
|
||||
"2602": 49,
|
||||
"5367": 50,
|
||||
"11472": 51
|
||||
},
|
||||
"piper_version": "1.0.0"
|
||||
}
|
||||
1
piper_sample_generator/__init__.py
Normal file
1
piper_sample_generator/__init__.py
Normal file
|
|
@ -0,0 +1 @@
|
|||
"""Piper sample generator."""
|
||||
628
piper_sample_generator/__main__.py
Executable file
628
piper_sample_generator/__main__.py
Executable file
|
|
@ -0,0 +1,628 @@
|
|||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import gc
|
||||
import itertools as it
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import unicodedata
|
||||
import wave
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple, Union, cast
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
from piper import PiperVoice, SynthesisConfig
|
||||
from piper.phonemize_espeak import EspeakPhonemizer
|
||||
|
||||
try:
|
||||
from piper_train.vits import commons
|
||||
except ImportError:
|
||||
from piper_train.vits import commons
|
||||
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
|
||||
|
||||
# Main generation function
|
||||
def generate_samples(
|
||||
text: Union[List[str], str],
|
||||
output_dir: Union[str, Path],
|
||||
model: Union[str, Path],
|
||||
max_samples: Optional[int] = None,
|
||||
file_names: Optional[Iterable[str]] = None,
|
||||
batch_size: int = 1,
|
||||
slerp_weights: Tuple[float, ...] = (0.5,),
|
||||
length_scales: Tuple[float, ...] = (0.75, 1, 1.25),
|
||||
noise_scales: Tuple[float, ...] = (0.667,),
|
||||
noise_scale_ws: Tuple[float, ...] = (0.8,),
|
||||
max_speakers: Optional[int] = None,
|
||||
verbose: bool = False,
|
||||
phoneme_input: bool = False,
|
||||
**kwargs,
|
||||
) -> None:
|
||||
"""
|
||||
Generate synthetic speech clips, saving the clips to the specified output directory.
|
||||
|
||||
Args:
|
||||
text (List[str]): The text to convert into speech. Can be either a
|
||||
a list of strings, or a path to a file with text on each line.
|
||||
output_dir (str): The location to save the generated clips.
|
||||
model (str): The path to the TTS generator model (.pt).
|
||||
max_samples (int): The maximum number of samples to generate.
|
||||
file_names (List[str]): The names to use when saving the files. Must be the same length
|
||||
as the `text` argument, if a list.
|
||||
batch_size (int): The batch size to use when generated the clips
|
||||
slerp_weights (List[float]): The weights to use when mixing speakers via SLERP.
|
||||
length_scales (List[float]): Controls the average duration/speed of the generated speech.
|
||||
noise_scales (List[float]): A parameter for overall variability of the generated speech.
|
||||
noise_scale_ws (List[float]): A parameter for the stochastic duration of words/phonemes.
|
||||
max_speakers (int): The maximum speaker number to use, if the model is multi-speaker.
|
||||
verbose (bool): Enable or disable more detailed logging messages (default: False).
|
||||
phoneme_input (bool): Set to indicate given input text is phoneme input.
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
|
||||
if max_samples is None:
|
||||
max_samples = len(text)
|
||||
|
||||
_LOGGER.debug("Loading %s", model)
|
||||
model_path = Path(model)
|
||||
|
||||
torch_model = torch.load(model_path, weights_only=False)
|
||||
torch_model.eval()
|
||||
_LOGGER.info("Successfully loaded the model")
|
||||
|
||||
if torch.cuda.is_available():
|
||||
torch_model.cuda()
|
||||
_LOGGER.debug("CUDA available, using GPU")
|
||||
elif torch.backends.mps.is_available():
|
||||
mps_device = torch.device("mps")
|
||||
torch_model.to(mps_device)
|
||||
_LOGGER.debug("MPS available, using GPU")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
config_path = f"{model_path}.json"
|
||||
with open(config_path, "r", encoding="utf-8") as config_file:
|
||||
config = json.load(config_file)
|
||||
|
||||
voice = config["espeak"]["voice"]
|
||||
sample_rate = config["audio"]["sample_rate"]
|
||||
num_speakers = config["num_speakers"]
|
||||
if max_speakers is not None:
|
||||
num_speakers = min(num_speakers, max_speakers)
|
||||
|
||||
max_len = None
|
||||
|
||||
sample_idx = 0
|
||||
is_done = False
|
||||
settings_iter = it.cycle(
|
||||
it.product(
|
||||
slerp_weights,
|
||||
length_scales,
|
||||
noise_scales,
|
||||
noise_scale_ws,
|
||||
)
|
||||
)
|
||||
|
||||
speakers_iter = it.cycle(it.product(range(num_speakers), range(num_speakers)))
|
||||
speakers_batch = list(it.islice(speakers_iter, 0, batch_size))
|
||||
if isinstance(text, str) and os.path.isfile(text):
|
||||
texts = it.cycle(
|
||||
[
|
||||
i.strip()
|
||||
for i in open(text, "r", encoding="utf-8").readlines()
|
||||
if len(i.strip()) > 0
|
||||
]
|
||||
)
|
||||
elif isinstance(text, list):
|
||||
texts = it.cycle(text)
|
||||
else:
|
||||
texts = it.cycle([text])
|
||||
|
||||
if file_names:
|
||||
file_names = it.cycle(file_names)
|
||||
|
||||
batch_idx = 0
|
||||
while speakers_batch:
|
||||
if is_done:
|
||||
break
|
||||
|
||||
batch_size = len(speakers_batch)
|
||||
slerp_weight, length_scale, noise_scale, noise_scale_w = next(settings_iter)
|
||||
|
||||
with torch.no_grad():
|
||||
speaker_1 = torch.LongTensor([s[0] for s in speakers_batch])
|
||||
speaker_2 = torch.LongTensor([s[1] for s in speakers_batch])
|
||||
|
||||
phoneme_ids_by_batch = []
|
||||
for i in range(batch_size):
|
||||
phoneme_ids = get_phonemes(
|
||||
voice, config, next(texts), verbose, phoneme_input
|
||||
)
|
||||
phoneme_ids_by_batch.append(phoneme_ids)
|
||||
|
||||
def right_pad_lists(lists):
|
||||
max_length = max(len(lst) for lst in lists)
|
||||
padded_lists = []
|
||||
for lst in lists:
|
||||
padded_l = lst + [1] * (
|
||||
max_length - len(lst)
|
||||
) # phoneme 1 (corresponding to '^' character seems to work best)
|
||||
padded_lists.append(padded_l)
|
||||
return padded_lists
|
||||
|
||||
phoneme_ids_by_batch = right_pad_lists(phoneme_ids_by_batch)
|
||||
audio, phoneme_samples = generate_audio(
|
||||
torch_model,
|
||||
speaker_1,
|
||||
speaker_2,
|
||||
phoneme_ids_by_batch,
|
||||
slerp_weight,
|
||||
noise_scale,
|
||||
noise_scale_w,
|
||||
length_scale,
|
||||
max_len,
|
||||
)
|
||||
|
||||
# Trim audio to actual length based on phoneme samples
|
||||
for i in range(audio.shape[0]):
|
||||
# Fill time after last speech with silence (zeros)
|
||||
# It will be removed in the next stage with np.trim_zeros
|
||||
last_sample_idx = int(phoneme_samples[i].flatten().sum().item())
|
||||
audio[i, 0, last_sample_idx + 1 :] = 0
|
||||
|
||||
audio_numpy = audio.cpu().numpy()
|
||||
|
||||
if torch.backends.mps.is_available():
|
||||
# There seems to be a memory leak if we don't empty the cache
|
||||
# after each batch with mps
|
||||
torch.mps.empty_cache()
|
||||
gc.collect()
|
||||
|
||||
audio_int16 = audio_float_to_int16(audio_numpy)
|
||||
for audio_idx in range(audio_int16.shape[0]):
|
||||
audio_data = np.trim_zeros(audio_int16[audio_idx].flatten())
|
||||
|
||||
if isinstance(file_names, it.cycle):
|
||||
wav_path = output_dir / next(file_names)
|
||||
else:
|
||||
wav_path = output_dir / f"{sample_idx}.wav"
|
||||
|
||||
wav_file: wave.Wave_write = wave.open(str(wav_path), "wb")
|
||||
with wav_file:
|
||||
wav_file.setframerate(sample_rate)
|
||||
wav_file.setsampwidth(2)
|
||||
wav_file.setnchannels(1)
|
||||
wav_file.writeframes(audio_data)
|
||||
|
||||
sample_idx += 1
|
||||
if sample_idx >= max_samples:
|
||||
is_done = True
|
||||
break
|
||||
|
||||
# print(f"Batch {batch_idx +1}/{max_samples//batch_size} complete", " "*200, end='\r')
|
||||
|
||||
# Next batch
|
||||
_LOGGER.debug("Batch %s/%s complete", batch_idx + 1, max_samples // batch_size)
|
||||
speakers_batch = list(it.islice(speakers_iter, 0, batch_size))
|
||||
batch_idx += 1
|
||||
|
||||
_LOGGER.info("Done")
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def generate_samples_onnx(
|
||||
text: Union[List[str], str],
|
||||
output_dir: Union[str, Path],
|
||||
model: Union[str, Path, List[Union[str, Path]]],
|
||||
max_samples: Optional[int] = None,
|
||||
file_names: Optional[Iterable[str]] = None,
|
||||
length_scales: Tuple[float, ...] = (0.75, 1, 1.25),
|
||||
noise_scales: Tuple[float, ...] = (0.667,),
|
||||
noise_scale_ws: Tuple[float, ...] = (0.8,),
|
||||
max_speakers: Optional[int] = None,
|
||||
phoneme_input: bool = False,
|
||||
**kwargs,
|
||||
) -> None:
|
||||
"""
|
||||
Generate synthetic speech clips, saving the clips to the specified output directory.
|
||||
|
||||
Args:
|
||||
text (List[str]): The text to convert into speech. Can be either a
|
||||
a list of strings, or a path to a file with text on each line.
|
||||
output_dir (str): The location to save the generated clips.
|
||||
model (str): The path to the Piper TTS model (.onnx).
|
||||
max_samples (int): The maximum number of samples to generate.
|
||||
file_names (List[str]): The names to use when saving the files. Must be the same length
|
||||
as the `text` argument, if a list.
|
||||
length_scales (List[float]): Controls the average duration/speed of the generated speech.
|
||||
noise_scales (List[float]): A parameter for overall variability of the generated speech.
|
||||
noise_scale_ws (List[float]): A parameter for the stochastic duration of words/phonemes.
|
||||
max_speakers (int): The maximum speaker number to use, if the model is multi-speaker.
|
||||
phoneme_input (bool): Set to indicate given input text is phoneme input.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
|
||||
if max_samples is None:
|
||||
max_samples = len(text)
|
||||
|
||||
if not isinstance(model, list):
|
||||
model = [model]
|
||||
|
||||
_LOGGER.debug("Loading %s", model)
|
||||
voices = [PiperVoice.load(m, use_cuda=torch.cuda.is_available()) for m in model]
|
||||
_LOGGER.info("Successfully loaded model(s)")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
sample_idx = 0
|
||||
settings_iter = it.cycle(
|
||||
it.product(
|
||||
voices,
|
||||
length_scales,
|
||||
noise_scales,
|
||||
noise_scale_ws,
|
||||
)
|
||||
)
|
||||
|
||||
if isinstance(text, str) and os.path.exists(text):
|
||||
texts = it.cycle(
|
||||
[
|
||||
i.strip()
|
||||
for i in open(text, "r", encoding="utf-8").readlines()
|
||||
if len(i.strip()) > 0
|
||||
]
|
||||
)
|
||||
elif isinstance(text, list):
|
||||
texts = it.cycle(text)
|
||||
else:
|
||||
texts = it.cycle([text])
|
||||
|
||||
if file_names:
|
||||
file_names = it.cycle(file_names)
|
||||
|
||||
for voice, length_scale, noise_scale, noise_w_scale in settings_iter:
|
||||
num_speakers = voice.config.num_speakers
|
||||
if max_speakers is not None:
|
||||
num_speakers = min(num_speakers, max_speakers)
|
||||
|
||||
for speaker_id in range(num_speakers):
|
||||
if isinstance(file_names, it.cycle):
|
||||
wav_path = output_dir / next(file_names)
|
||||
else:
|
||||
wav_path = output_dir / f"{sample_idx}.wav"
|
||||
|
||||
text_input = next(texts)
|
||||
|
||||
if phoneme_input:
|
||||
# For ONNX models with phoneme input, build phoneme IDs manually
|
||||
phonemes = list(unicodedata.normalize("NFD", text_input))
|
||||
|
||||
# Build phoneme IDs similar to get_phonemes function
|
||||
id_map = voice.config.phoneme_id_map
|
||||
|
||||
# Beginning of utterance
|
||||
phoneme_ids = list(id_map.get("^", [1])) # Default to [1] if not found
|
||||
phoneme_ids.extend(id_map.get("_", [0])) # Default to [0] if not found
|
||||
|
||||
# Add phonemes
|
||||
for phoneme in phonemes:
|
||||
p_ids = id_map.get(phoneme)
|
||||
if p_ids is not None:
|
||||
phoneme_ids.extend(p_ids)
|
||||
phoneme_ids.extend(id_map.get("_", [0]))
|
||||
else:
|
||||
_LOGGER.warning(
|
||||
"Phoneme '%s' not found in model's phoneme map", phoneme
|
||||
)
|
||||
|
||||
# End of utterance
|
||||
phoneme_ids.extend(id_map.get("$", [2])) # Default to [2] if not found
|
||||
|
||||
# Generate audio from phoneme IDs
|
||||
syn_config = SynthesisConfig(
|
||||
speaker_id=speaker_id,
|
||||
length_scale=length_scale,
|
||||
noise_scale=noise_scale,
|
||||
noise_w_scale=noise_w_scale,
|
||||
)
|
||||
audio = voice.phoneme_ids_to_audio(phoneme_ids, syn_config)
|
||||
|
||||
# Convert to int16 and write to WAV
|
||||
audio_int16 = audio_float_to_int16(audio[np.newaxis, :])
|
||||
wav_file: wave.Wave_write = wave.open(str(wav_path), "wb")
|
||||
with wav_file:
|
||||
wav_file.setframerate(voice.config.sample_rate)
|
||||
wav_file.setsampwidth(2)
|
||||
wav_file.setnchannels(1)
|
||||
wav_file.writeframes(audio_int16.flatten())
|
||||
else:
|
||||
with wave.open(str(wav_path), "wb") as wav_file:
|
||||
voice.synthesize_wav(
|
||||
text_input,
|
||||
wav_file=wav_file,
|
||||
syn_config=SynthesisConfig(
|
||||
speaker_id=speaker_id,
|
||||
length_scale=length_scale,
|
||||
noise_scale=noise_scale,
|
||||
noise_w_scale=noise_w_scale,
|
||||
),
|
||||
)
|
||||
|
||||
sample_idx += 1
|
||||
if sample_idx >= max_samples:
|
||||
return
|
||||
|
||||
_LOGGER.info("Done")
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def generate_audio(
|
||||
model,
|
||||
speaker_1,
|
||||
speaker_2,
|
||||
phoneme_ids,
|
||||
slerp_weight,
|
||||
noise_scale,
|
||||
noise_scale_w,
|
||||
length_scale,
|
||||
max_len,
|
||||
) -> Tuple[torch.FloatTensor, torch.FloatTensor]:
|
||||
x = torch.LongTensor(phoneme_ids)
|
||||
x_lengths = torch.LongTensor([len(i) for i in phoneme_ids])
|
||||
|
||||
if torch.cuda.is_available():
|
||||
speaker_1 = speaker_1.cuda()
|
||||
speaker_2 = speaker_2.cuda()
|
||||
x = cast(torch.LongTensor, x.cuda())
|
||||
x_lengths = cast(torch.LongTensor, x_lengths.cuda())
|
||||
elif torch.backends.mps.is_available():
|
||||
mps_device = torch.device("mps")
|
||||
speaker_1 = speaker_1.to(mps_device)
|
||||
speaker_2 = speaker_2.to(mps_device)
|
||||
x = cast(torch.LongTensor, x.to(mps_device))
|
||||
x_lengths = cast(torch.LongTensor, x_lengths.to(mps_device))
|
||||
|
||||
x, m_p_orig, logs_p_orig, x_mask = model.enc_p(x, x_lengths)
|
||||
emb0 = model.emb_g(speaker_1)
|
||||
emb1 = model.emb_g(speaker_2)
|
||||
g = slerp(emb0, emb1, slerp_weight).unsqueeze(-1) # [b, h, 1]
|
||||
|
||||
if model.use_sdp:
|
||||
logw = model.dp(x, x_mask, g=g, reverse=True, noise_scale=noise_scale_w)
|
||||
else:
|
||||
logw = model.dp(x, x_mask, g=g)
|
||||
w = torch.exp(logw) * x_mask * length_scale
|
||||
w_ceil = torch.ceil(w)
|
||||
y_lengths = torch.clamp_min(torch.sum(w_ceil, [1, 2]), 1).long()
|
||||
y_mask = torch.unsqueeze(
|
||||
commons.sequence_mask(y_lengths, int(y_lengths.max().item())), 1
|
||||
).type_as(x_mask)
|
||||
attn_mask = torch.unsqueeze(x_mask, 2) * torch.unsqueeze(y_mask, -1)
|
||||
attn = commons.generate_path(w_ceil, attn_mask)
|
||||
|
||||
m_p = torch.matmul(attn.squeeze(1), m_p_orig.transpose(1, 2)).transpose(
|
||||
1, 2
|
||||
) # [b, t', t], [b, t, d] -> [b, d, t']
|
||||
logs_p = torch.matmul(attn.squeeze(1), logs_p_orig.transpose(1, 2)).transpose(
|
||||
1, 2
|
||||
) # [b, t', t], [b, t, d] -> [b, d, t']
|
||||
|
||||
z_p = m_p + torch.randn_like(m_p) * torch.exp(logs_p) * noise_scale
|
||||
z = model.flow(z_p, y_mask, g=g, reverse=True)
|
||||
o = model.dec((z * y_mask)[:, :, :max_len], g=g)
|
||||
|
||||
audio = cast(torch.FloatTensor, o)
|
||||
phoneme_samples = cast(torch.FloatTensor, w_ceil * 256) # hop length
|
||||
|
||||
return audio, phoneme_samples
|
||||
|
||||
|
||||
_PHONEMIZER = EspeakPhonemizer()
|
||||
|
||||
|
||||
def get_phonemes(
|
||||
voice: str,
|
||||
config: Dict[str, Any],
|
||||
text: str,
|
||||
verbose: bool = False,
|
||||
phoneme_input: bool = False,
|
||||
) -> List[int]:
|
||||
# Combine all sentences
|
||||
if phoneme_input:
|
||||
phonemes = list(unicodedata.normalize("NFD", text))
|
||||
else:
|
||||
phonemes = [
|
||||
p
|
||||
for sentence_phonemes in _PHONEMIZER.phonemize(voice, text)
|
||||
for p in sentence_phonemes
|
||||
]
|
||||
if verbose is True:
|
||||
_LOGGER.debug("Phonemes: %s", phonemes)
|
||||
|
||||
id_map = config["phoneme_id_map"]
|
||||
|
||||
# Beginning of utterance
|
||||
phoneme_ids = list(id_map["^"])
|
||||
phoneme_ids.extend(id_map["_"])
|
||||
|
||||
# Phoneme ids for just the text
|
||||
text_phoneme_ids = []
|
||||
|
||||
for phoneme in phonemes:
|
||||
p_ids = id_map.get(phoneme)
|
||||
if p_ids is not None:
|
||||
phoneme_ids.extend(p_ids)
|
||||
text_phoneme_ids.extend(p_ids)
|
||||
phoneme_ids.extend(id_map["_"])
|
||||
text_phoneme_ids.extend(id_map["_"])
|
||||
|
||||
# End of utterance
|
||||
phoneme_ids.extend(id_map["$"])
|
||||
|
||||
return phoneme_ids
|
||||
|
||||
|
||||
def slerp(v1, v2, t: float, DOT_THR: float = 0.9995, zdim: int = -1):
|
||||
"""SLERP for pytorch tensors interpolating `v1` to `v2` with scale of `t`.
|
||||
|
||||
`DOT_THR` determines when the vectors are too close to parallel.
|
||||
If they are too close, then a regular linear interpolation is used.
|
||||
|
||||
`zdim` is the feature dimension over which to compute norms and find angles.
|
||||
For example: if a sequence of 5 vectors is input with shape [5, 768]
|
||||
Then `zdim = 1` or `zdim = -1` computes SLERP along the feature dim of 768.
|
||||
|
||||
Theory Reference:
|
||||
https://splines.readthedocs.io/en/latest/rotation/slerp.html
|
||||
PyTorch reference:
|
||||
https://discuss.pytorch.org/t/help-regarding-slerp-function-for-generative-model-sampling/32475/3
|
||||
Numpy reference:
|
||||
https://gist.github.com/dvschultz/3af50c40df002da3b751efab1daddf2c
|
||||
"""
|
||||
|
||||
# take the dot product between normalized vectors
|
||||
v1_norm = v1 / torch.norm(v1, dim=zdim, keepdim=True)
|
||||
v2_norm = v2 / torch.norm(v2, dim=zdim, keepdim=True)
|
||||
dot = (v1_norm * v2_norm).sum(zdim)
|
||||
|
||||
# if the vectors are too close, return a simple linear interpolation
|
||||
if (torch.abs(dot) > DOT_THR).any():
|
||||
res = (1 - t) * v1 + t * v2
|
||||
|
||||
# else apply SLERP
|
||||
else:
|
||||
# compute the angle terms we need
|
||||
theta = torch.acos(dot)
|
||||
theta_t = theta * t
|
||||
sin_theta = torch.sin(theta)
|
||||
sin_theta_t = torch.sin(theta_t)
|
||||
|
||||
# compute the sine scaling terms for the vectors
|
||||
s1 = torch.sin(theta - theta_t) / sin_theta
|
||||
s2 = sin_theta_t / sin_theta
|
||||
|
||||
# interpolate the vectors
|
||||
res = (s1.unsqueeze(zdim) * v1) + (s2.unsqueeze(zdim) * v2)
|
||||
|
||||
return res
|
||||
|
||||
|
||||
def audio_float_to_int16(
|
||||
audio: np.ndarray, max_wav_value: float = 32767.0
|
||||
) -> np.ndarray:
|
||||
"""Normalize audio and convert to int16 range"""
|
||||
audio_norm = audio * (max_wav_value / max(0.01, np.max(np.abs(audio))))
|
||||
audio_norm = np.clip(audio_norm, -max_wav_value, max_wav_value)
|
||||
audio_norm = audio_norm.astype("int16")
|
||||
return audio_norm
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def main() -> int:
|
||||
"""Main entry point."""
|
||||
|
||||
# Get command line arguments
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("text")
|
||||
parser.add_argument(
|
||||
"--max-samples",
|
||||
required=True,
|
||||
type=int,
|
||||
help="Maximum number of samples to generate",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--model",
|
||||
required=True,
|
||||
action="append",
|
||||
help="Path to PyTorch generator (.pt) or Piper voice model (.onnx)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--batch-size", type=int, default=1, help="CUDA batch size (generator only)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slerp-weights",
|
||||
nargs="+",
|
||||
type=float,
|
||||
default=[0.5],
|
||||
help="Speaker blending weights (generator only)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--length-scales",
|
||||
nargs="+",
|
||||
type=float,
|
||||
default=[1.0, 0.75, 1.25, 1.4],
|
||||
help="Audio length scales (< 1 is faster, > 1 is slower)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--noise-scales",
|
||||
nargs="+",
|
||||
type=float,
|
||||
default=[0.667, 0.75, 0.85, 0.9, 1.0, 1.4],
|
||||
help="Noise amounts added to audio (most voices use 0.667)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--noise-scale-ws",
|
||||
nargs="+",
|
||||
type=float,
|
||||
default=[0.8],
|
||||
help="Phoneme width variation (most voices use 0.8)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
default="output",
|
||||
help="Directory to output WAV files (default: ./output)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-speakers",
|
||||
type=int,
|
||||
help="Maximum number of speakers to use (default: no limit)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--phoneme-input", action="store_true", help="Treat input text as phoneme input"
|
||||
)
|
||||
parser.add_argument("--verbose", action="store_true")
|
||||
args = parser.parse_args().__dict__
|
||||
|
||||
# Generate speech
|
||||
model_paths = [Path(m) for m in args["model"]]
|
||||
assert model_paths
|
||||
|
||||
if any(mp for mp in model_paths[1:] if mp.suffix != model_paths[0].suffix):
|
||||
_LOGGER.error("All models must have the same suffix (.pt or .onnx)")
|
||||
return 1
|
||||
|
||||
if model_paths[0].suffix == ".onnx":
|
||||
# Use Piper voice (.onnx)
|
||||
generate_samples_onnx(**args)
|
||||
elif model_paths[0].suffix == ".pt":
|
||||
# Use PyTorch generator (.pt)
|
||||
if len(model_paths) > 1:
|
||||
_LOGGER.error("Only one generator (.pt) is supported")
|
||||
return 1
|
||||
|
||||
args["model"] = args["model"][0]
|
||||
generate_samples(**args)
|
||||
else:
|
||||
_LOGGER.error("Models must have .pt or .onnx suffix")
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -1,12 +1,11 @@
|
|||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import audioop
|
||||
import sys
|
||||
import wave
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from audiomentations import Compose, ApplyImpulseResponse, Gain
|
||||
from audiomentations import ApplyImpulseResponse, Compose, Gain
|
||||
|
||||
_DIR = Path(__file__).parent
|
||||
|
||||
|
|
@ -15,14 +14,14 @@ def main() -> None:
|
|||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("input_dir")
|
||||
parser.add_argument("output_dir")
|
||||
parser.add_argument("--sample-rate", type=int)
|
||||
parser.add_argument("--sample-rate", type=int, required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
impulses = list((_DIR / "impulses").glob("*.wav"))
|
||||
|
||||
augment = Compose(
|
||||
transforms=[
|
||||
Gain(min_gain_in_db=-12, max_gain_in_db=0),
|
||||
Gain(min_gain_db=-12, max_gain_db=0),
|
||||
ApplyImpulseResponse(impulses),
|
||||
]
|
||||
)
|
||||
|
|
@ -35,9 +34,10 @@ def main() -> None:
|
|||
output_wav = output_dir / (input_wav.relative_to(input_dir))
|
||||
output_wav.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with wave.open(str(input_wav), "rb") as input_wav_file, wave.open(
|
||||
str(output_wav), "wb"
|
||||
) as output_wav_file:
|
||||
with (
|
||||
wave.open(str(input_wav), "rb") as input_wav_file,
|
||||
wave.open(str(output_wav), "wb") as output_wav_file,
|
||||
):
|
||||
assert input_wav_file.getsampwidth() == 2
|
||||
assert input_wav_file.getnchannels() == 1
|
||||
|
||||
5
pylintrc
5
pylintrc
|
|
@ -1,3 +1,6 @@
|
|||
[MASTER]
|
||||
ignored-modules=torch
|
||||
|
||||
[MESSAGES CONTROL]
|
||||
disable=
|
||||
format,
|
||||
|
|
@ -31,7 +34,7 @@ disable=
|
|||
missing-class-docstring,
|
||||
missing-function-docstring,
|
||||
import-error,
|
||||
consider-using-with
|
||||
relative-beyond-top-level
|
||||
|
||||
[FORMAT]
|
||||
expected-line-ending-format=LF
|
||||
|
|
|
|||
45
pyproject.toml
Normal file
45
pyproject.toml
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
[build-system]
|
||||
requires = ["setuptools>=62.3"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "piper-sample-generator"
|
||||
version = "3.2.0"
|
||||
license = {text = "MIT"}
|
||||
description = "Generate TTS audio samples for training wake word systems"
|
||||
readme = "README.md"
|
||||
authors = [
|
||||
{name = "The Home Assistant Authors", email = "hello@home-assistant.io"}
|
||||
]
|
||||
keywords = ["piper", "sample", "tts", "wakeword"]
|
||||
requires-python = ">=3.9.0"
|
||||
dependencies = [
|
||||
"audiomentations==0.33.0",
|
||||
"piper-tts==1.3.0",
|
||||
"numpy>=2,<3",
|
||||
"torch>=2,<3",
|
||||
"torchaudio",
|
||||
"webrtcvad",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"black==22.12.0",
|
||||
"flake8==6.0.0",
|
||||
"isort==5.11.3",
|
||||
"mypy==0.991",
|
||||
"pylint==2.15.9",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
"Source Code" = "http://github.com/rhasspy/piper-sample-generator"
|
||||
|
||||
[tool.setuptools]
|
||||
platforms = ["any"]
|
||||
zip-safe = true
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
include = ["piper_sample_generator*"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
piper_sample_generator = ["impulses/*.wav"]
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
audiomentations==0.33.0
|
||||
piper-phonemize==1.1.0
|
||||
numpy<2
|
||||
torch
|
||||
torchaudio
|
||||
webrtcvad
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
black==22.12.0
|
||||
flake8==6.0.0
|
||||
isort==5.11.3
|
||||
mypy==0.991
|
||||
pylint==2.15.9
|
||||
|
|
@ -6,8 +6,13 @@ from pathlib import Path
|
|||
_DIR = Path(__file__).parent
|
||||
_PROGRAM_DIR = _DIR.parent
|
||||
_VENV_DIR = _PROGRAM_DIR / ".venv"
|
||||
_SCRIPT = _PROGRAM_DIR / "generate_samples.py"
|
||||
_MODULE_DIR = _PROGRAM_DIR / "piper_sample_generator"
|
||||
|
||||
context = venv.EnvBuilder().ensure_directories(_VENV_DIR)
|
||||
subprocess.check_call([context.env_exe, "-m", "black", str(_SCRIPT)])
|
||||
subprocess.check_call([context.env_exe, "-m", "isort", str(_SCRIPT)])
|
||||
if _VENV_DIR.exists():
|
||||
context = venv.EnvBuilder().ensure_directories(_VENV_DIR)
|
||||
python_exe = context.env_exe
|
||||
else:
|
||||
python_exe = "python3"
|
||||
|
||||
subprocess.check_call([python_exe, "-m", "black", str(_MODULE_DIR)])
|
||||
subprocess.check_call([python_exe, "-m", "isort", str(_MODULE_DIR)])
|
||||
|
|
|
|||
19
script/lint
19
script/lint
|
|
@ -6,11 +6,16 @@ from pathlib import Path
|
|||
_DIR = Path(__file__).parent
|
||||
_PROGRAM_DIR = _DIR.parent
|
||||
_VENV_DIR = _PROGRAM_DIR / ".venv"
|
||||
_SCRIPT = _PROGRAM_DIR / "generate_samples.py"
|
||||
_MODULE_DIR = _PROGRAM_DIR / "piper_sample_generator"
|
||||
|
||||
context = venv.EnvBuilder().ensure_directories(_VENV_DIR)
|
||||
subprocess.check_call([context.env_exe, "-m", "black", str(_SCRIPT), "--check"])
|
||||
subprocess.check_call([context.env_exe, "-m", "isort", str(_SCRIPT), "--check"])
|
||||
subprocess.check_call([context.env_exe, "-m", "flake8", str(_SCRIPT)])
|
||||
subprocess.check_call([context.env_exe, "-m", "pylint", str(_SCRIPT)])
|
||||
subprocess.check_call([context.env_exe, "-m", "mypy", str(_SCRIPT)])
|
||||
if _VENV_DIR.exists():
|
||||
context = venv.EnvBuilder().ensure_directories(_VENV_DIR)
|
||||
python_exe = context.env_exe
|
||||
else:
|
||||
python_exe = "python3"
|
||||
|
||||
subprocess.check_call([python_exe, "-m", "black", str(_MODULE_DIR), "--check"])
|
||||
subprocess.check_call([python_exe, "-m", "isort", str(_MODULE_DIR), "--check"])
|
||||
subprocess.check_call([python_exe, "-m", "flake8", str(_MODULE_DIR)])
|
||||
subprocess.check_call([python_exe, "-m", "pylint", str(_MODULE_DIR)])
|
||||
subprocess.check_call([python_exe, "-m", "mypy", str(_MODULE_DIR)])
|
||||
|
|
|
|||
|
|
@ -8,5 +8,10 @@ _DIR = Path(__file__).parent
|
|||
_PROGRAM_DIR = _DIR.parent
|
||||
_VENV_DIR = _PROGRAM_DIR / ".venv"
|
||||
|
||||
context = venv.EnvBuilder().ensure_directories(_VENV_DIR)
|
||||
subprocess.check_call([context.env_exe, "generate_samples.py"] + sys.argv[1:])
|
||||
if _VENV_DIR.exists():
|
||||
context = venv.EnvBuilder().ensure_directories(_VENV_DIR)
|
||||
python_exe = context.env_exe
|
||||
else:
|
||||
python_exe = "python3"
|
||||
|
||||
subprocess.check_call([python_exe, "-m", "piper_sample_generator"] + sys.argv[1:])
|
||||
|
|
|
|||
12
script/setup
12
script/setup
|
|
@ -1,4 +1,5 @@
|
|||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import subprocess
|
||||
import venv
|
||||
from pathlib import Path
|
||||
|
|
@ -7,6 +8,9 @@ _DIR = Path(__file__).parent
|
|||
_PROGRAM_DIR = _DIR.parent
|
||||
_VENV_DIR = _PROGRAM_DIR / ".venv"
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--dev", action="store_true", help="Install dev requirements")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Create virtual environment
|
||||
builder = venv.EnvBuilder(with_pip=True)
|
||||
|
|
@ -19,4 +23,10 @@ subprocess.check_call(pip + ["install", "--upgrade", "pip"])
|
|||
subprocess.check_call(pip + ["install", "--upgrade", "setuptools", "wheel"])
|
||||
|
||||
# Install requirements
|
||||
subprocess.check_call(pip + ["install", "-r", str(_PROGRAM_DIR / "requirements.txt")])
|
||||
subprocess.check_call(pip + ["install", "-e", str(_PROGRAM_DIR)])
|
||||
|
||||
if args.dev:
|
||||
# Install dev requirements
|
||||
subprocess.check_call(
|
||||
pip + ["install", "-e", f"{_PROGRAM_DIR}[dev]"]
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue