"This notebook demonstrates how to train custom openWakeWord models using pre-defined datasets and an automated process for dataset generation and training. While not guaranteed to always produce the best performing model, the methods shown in this notebook often produce baseline models with releatively strong performance.\n",
"\n",
"Manual data preparation and model training (e.g., see the [training models](training_models.ipynb) notebook) remains an option for when full control over the model development process is needed.\n",
"\n",
"At a high level, the automatic training process takes advantages of several techniques to try and produce a good model, including:\n",
"\n",
"- Early-stopping and checkpoint averaging (similar to [stochastic weight averaging](https://arxiv.org/abs/1803.05407)) to search for the best models found during training, according to the validation data\n",
"- Variable learning rates with cosine decay and multiple cycles\n",
"- Adaptive batch construction to focus on only high-loss examples when the model begins to converge, combined with gradient accumulation to ensure that batch sizes are still large enough for stable training\n",
"- Cycical weight schedules for negative examples to help the model reduce false-positive rates\n",
"\n",
"See the contents of the `train.py` file for more details."
]
},
{
"cell_type": "markdown",
"id": "e08d031b",
"metadata": {
"id": "e08d031b"
},
"source": [
"# Environment Setup"
]
},
{
"cell_type": "markdown",
"id": "aee78c37",
"metadata": {
"id": "aee78c37"
},
"source": [
"To begin, we'll need to install the requirements for training custom models. In particular, a relatively recent version of Pytorch and custom fork of the [piper-sample-generator](https://github.com/dscripka/piper-sample-generator) library for generating synthetic examples for the custom model.\n",
"\n",
"**Important Note!** Currently, automated model training is only supported on linux systems due to the requirements of the text to speech library used for synthetic sample generation (Piper). It may be possible to use Piper on Windows/Mac systems, but that has not (yet) been tested."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4b1227eb",
"metadata": {
"id": "4b1227eb"
},
"outputs": [],
"source": [
"## Environment setup\n",
"\n",
"# install piper-sample-generator (currently only supports linux systems)\n",
"When training new openWakeWord models using the automated procedure, four specific types of data are required:\n",
"\n",
"1) Synthetic examples of the target word/phrase generated with text-to-speech models\n",
"\n",
"2) Synthetic examples of adversarial words/phrases generated with text-to-speech models\n",
"\n",
"3) Room impulse reponses and noise/background audio data to augment the synthetic examples and make them more realistic\n",
"\n",
"4) Generic \"negative\" audio data that is very unlikely to contain examples of the target word/phrase in the context where the model should detect it. This data can be the original audio data, or precomputed openWakeWord features ready for model training.\n",
"\n",
"5) Validation data to use for early-stopping when training the model.\n",
"\n",
"For the purposes of this notebook, all five of these sources will either be generated manually or can be obtained from HuggingFace thanks to their excellent `datasets` library and extremely generous hosting policy. Also note that while only a portion of some datasets are downloaded, for the best possible performance it is recommended to download the entire dataset and keep a local copy for future training runs."
"For automated model training openWakeWord uses a specially designed training script and a [YAML](https://yaml.org/) configuration file that defines all of the information required for training a new wake word/phrase detection model.\n",
"\n",
"It is strongly recommended that you review [the example config file](../examples/custom_model.yml), as each value is fully documented there. For the purposes of this notebook, we'll read in the YAML file to modify certain configuration parameters before saving a new YAML file for training our example model. Specifically:\n",
"\n",
"- We'll train a detection model for the phrase \"hey sebastian\"\n",
"- We'll only generate 5,000 positive and negative examples (to save on time for this example)\n",
"- We'll only generate 1,000 validation positive and negative examples for early stopping (again to save time)\n",
"- The model will only be trained for 10,000 steps (larger datasets will benefit from longer training)\n",
"- We'll reduce the target metrics to account for the small dataset size and limited training.\n",
"\n",
"On the topic of target metrics, there are *not* specific guidelines about what these metrics should be in practice, and you will need to conduct testing in your target deployment environment to establish good thresholds. However, from very limited testing the default values in the config file (accuracy >= 0.7, recall >= 0.5, false-positive rate <= 0.2 per hour) seem to produce models with reasonable performance.\n"
"With the data downloaded and training configuration set, we can now start training the model. We'll do this in parts to better illustrate the sequence, but you can also execute every step at once for a fully automated process."
"After the model finishes training, the auto training script will automatically convert it to ONNX and tflite versions, saving them as `my_custom_model/<model_name>.onnx/tflite` in the present working directory, where `<model_name>` is defined in the YAML training config file. Either version can be used as normal with `openwakeword`. I recommend testing them with the [`detect_from_microphone.py`](https://github.com/dscripka/openWakeWord/blob/main/examples/detect_from_microphone.py) example script to see how the model performs!"
