{ "cells": [ { "cell_type": "markdown", "id": "825fe381", "metadata": {}, "source": [ "# Introduction\n", "\n", "This notebook demonstrates the process of training a new openWakeWord model, using synthetic speech generated with open-source TTS models, and negative data representing music, noise, and speech. While the process here is complete, only small samples of datasets are utilized so that a new model can be trained on CPUs. In practice, much larger volumes of data (both positive and negitive examples) is needed to produce robust models. See the [documentation](https://github.com/dscripka/openWakeWord/tree/main/docs/models) for the pre-trained openWakeWord models for more information about how these models were trained." ] }, { "cell_type": "markdown", "id": "bd8e4597", "metadata": {}, "source": [ "To start, we'll need to install the requirements needed to train new openWakeWord models." ] }, { "cell_type": "code", "execution_count": null, "id": "1ba07a8f", "metadata": {}, "outputs": [], "source": [ "# Install requirements (it's recommended that you do this in a new virtual environment)\n", "\n", "# !pip install openwakeword\n", "# !pip install speechbrain\n", "# !pip install datasets\n", "# !pip install scipy matplotlib" ] }, { "cell_type": "code", "execution_count": 1, "id": "c914b0c9", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:26:26.308309Z", "start_time": "2023-02-18T03:26:24.785801Z" } }, "outputs": [], "source": [ "# Imports\n", "\n", "import os\n", "import collections\n", "import numpy as np\n", "from numpy.lib.format import open_memmap\n", "from pathlib import Path\n", "from tqdm import tqdm\n", "import openwakeword\n", "import openwakeword.data\n", "import openwakeword.utils\n", "import openwakeword.metrics\n", "\n", "import scipy\n", "import datasets\n", "import matplotlib.pyplot as plt\n", "import torch\n", "from torch import nn\n", "import IPython.display as ipd" ] }, { "cell_type": "markdown", "id": "b40a7f25", "metadata": {}, "source": [ "# Data Preparation" ] }, { "cell_type": "markdown", "id": "aee94c6e", "metadata": {}, "source": [ "## Download Data" ] }, { "cell_type": "markdown", "id": "00ea736a", "metadata": {}, "source": [ "Next we'll load the data used for training. For the purposes of this demonstration, we'll use a small set of positive and negative.\n", "\n", "For the positive data, there are ~3400 synthetic examples of the phrase \"turn on the office lights\" that were produced with the text-to-speech models documented in a [separate repo](https://github.com/dscripka/synthetic_speech_dataset_generation).\n", "\n", "These positive examples can be downloaded [here](https://f002.backblazeb2.com/file/openwakeword-resources/data/turn_on_the_office_lights.tar.gz).\n", "\n", "For negative data, we'll use small, already prepared samples of the [fma-large dataset](https://github.com/mdeff/fma) for music, the [FSD50k dataset](https://zenodo.org/record/4060432#.Y-hA2BzMJhE) for noise, and the [Common Voice 11](https://huggingface.co/datasets/mozilla-foundation/common_voice_11_0) dataset for speech.\n", "\n", "The fma-large sample can be downloaded [here](https://f002.backblazeb2.com/file/openwakeword-resources/data/fma_sample.zip), and then extracted into the working director.\n", "\n", "The FSD50k sample can be downloaded [here](https://f002.backblazeb2.com/file/openwakeword-resources/data/fsd50k_sample.zip), and then extracted into the working directory.\n", "\n", "And we'll use the HuggingFace Datasets library to get a portion of the test split of the Common Voice 11 (CV11) corpus.\n", "\n", "Note the data provided here is intended for non-commerical applications only; you will need to verify the license status of this (and other) data if you intend to use it for commerical purposes." ] }, { "cell_type": "code", "execution_count": 381, "id": "a31f760c", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T16:08:32.045657Z", "start_time": "2023-02-12T16:07:24.104475Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Reading metadata...: 16354it [00:00, 26183.62it/s]\n", "100%|██████████| 5000/5000 [00:44<00:00, 112.28it/s]\n" ] } ], "source": [ "# Download CV11 test split from HuggingFace, and convert the audio into 16 khz, 16-bit wav files\n", "\n", "cv_11 = datasets.load_dataset(\"mozilla-foundation/common_voice_11_0\", \"en\", split=\"test\", streaming=True)\n", "cv_11 = cv_11.cast_column(\"audio\", datasets.Audio(sampling_rate=16000, mono=True)) # convert to 16-khz\n", "cv_11 = iter(cv_11)\n", "\n", "# Convert and save clips (only first 5000)\n", "limit = 5000\n", "for i in tqdm(range(limit)):\n", " example = next(cv_11)\n", " output = os.path.join(\"cv11_test_clips\", example[\"path\"][0:-4] + \".wav\")\n", " os.makedirs(os.path.dirname(output), exist_ok=True)\n", "\n", " wav_data = (example[\"audio\"][\"array\"]*32767).astype(np.int16) # convert to 16-bit PCM format\n", " scipy.io.wavfile.write(output, 16000, wav_data)\n" ] }, { "cell_type": "markdown", "id": "0a12ab3d", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T02:04:47.611837Z", "start_time": "2023-02-12T02:04:47.606080Z" } }, "source": [ "## Compute Audio Embeddings" ] }, { "cell_type": "markdown", "id": "bfea6a2b", "metadata": {}, "source": [ "Once all the data is downloaded, we can now get the audio embeddings for the positive and negative clips. As this part of the openWakeWord model is frozen (i.e., not updated during training), it makes sense to pre-compute these features so that they only need to be prepared once." ] }, { "cell_type": "code", "execution_count": 2, "id": "473349ce", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:26:45.282504Z", "start_time": "2023-02-18T03:26:45.093446Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/dscripka/anaconda3/envs/torch_gpu/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n", " warnings.warn(\n" ] } ], "source": [ "# Create audio pre-processing object to get openWakeWord audio embeddings\n", "\n", "F = openwakeword.utils.AudioFeatures()" ] }, { "cell_type": "markdown", "id": "9e757355", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T02:14:32.160470Z", "start_time": "2023-02-12T02:14:32.154438Z" } }, "source": [ "### Negative Clips" ] }, { "cell_type": "code", "execution_count": 3, "id": "ab401215", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:27:23.911209Z", "start_time": "2023-02-18T03:26:47.968057Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "200it [00:00, 2779.44it/s]\n", "100%|██████████| 200/200 [00:01<00:00, 141.58it/s]\n", "1000it [00:00, 2806.48it/s]\n", "100%|██████████| 1000/1000 [00:05<00:00, 177.36it/s]\n", "5000it [00:01, 2555.99it/s]\n", "100%|██████████| 5000/5000 [00:26<00:00, 188.73it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "6096 negative clips after filtering, representing ~12.0 hours\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "# Get negative example paths, filtering out clips that are too long or too short\n", "\n", "negative_clips, negative_durations = openwakeword.data.filter_audio_paths(\n", " [\n", " \"fma_sample\",\n", " \"fsd50k_sample\",\n", " \"cv11_test_clips\"\n", " ],\n", " min_length_secs = 1.0, # minimum clip length in seconds\n", " max_length_secs = 60*30, # maximum clip length in seconds\n", " duration_method = \"header\" # use the file header to calculate duration\n", ")\n", "\n", "print(f\"{len(negative_clips)} negative clips after filtering, representing ~{sum(negative_durations)//3600} hours\")" ] }, { "cell_type": "code", "execution_count": 4, "id": "221d8662", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:28:06.568812Z", "start_time": "2023-02-18T03:28:06.524651Z" } }, "outputs": [], "source": [ "# Use HuggingFace datasets to load files from disk by batches\n", "\n", "audio_dataset = datasets.Dataset.from_dict({\"audio\": negative_clips})\n", "audio_dataset = audio_dataset.cast_column(\"audio\", datasets.Audio(sampling_rate=16000))" ] }, { "cell_type": "code", "execution_count": 6, "id": "37ec1163", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:30:38.082245Z", "start_time": "2023-02-18T03:29:08.355371Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ " 98%|█████████▊| 94/96 [01:26<00:01, 1.09it/s]\n", "15it [00:03, 4.61it/s] \n" ] } ], "source": [ "# Get audio embeddings (features) for negative clips and save to .npy file\n", "# Process files by batch and save to Numpy memory mapped file so that\n", "# an array larger than the available system memory can be created\n", "\n", "batch_size = 64 # number of files to load, compute features, and write to mmap at a time\n", "clip_size = 3 # the desired window size (in seconds) for the trained openWakeWord model\n", "N_total = int(sum(negative_durations)//clip_size) # maximum number of rows in mmap file\n", "n_feature_cols = F.get_embedding_shape(clip_size)\n", "\n", "output_file = \"negative_features.npy\"\n", "output_array_shape = (N_total, n_feature_cols[0], n_feature_cols[1])\n", "fp = open_memmap(output_file, mode='w+', dtype=np.float32, shape=output_array_shape)\n", "\n", "row_counter = 0\n", "for i in tqdm(np.arange(0, audio_dataset.num_rows, batch_size)):\n", " # Load data in batches and shape into rectangular array\n", " wav_data = [(j[\"array\"]*32767).astype(np.int16) for j in audio_dataset[i:i+batch_size][\"audio\"]]\n", " wav_data = openwakeword.data.stack_clips(wav_data, clip_size=16000*clip_size).astype(np.int16)\n", " \n", " # Compute features (increase ncpu argument for faster processing)\n", " features = F.embed_clips(x=wav_data, batch_size=1024, ncpu=8)\n", " \n", " # Save computed features to mmap array file (stopping once the desired size is reached)\n", " if row_counter + features.shape[0] > N_total:\n", " fp[row_counter:min(row_counter+features.shape[0], N_total), :, :] = features[0:N_total - row_counter, :, :]\n", " fp.flush()\n", " break\n", " else:\n", " fp[row_counter:row_counter+features.shape[0], :, :] = features\n", " row_counter += features.shape[0]\n", " fp.flush()\n", " \n", "# Trip empty rows from the mmapped array\n", "openwakeword.data.trim_mmap(output_file)" ] }, { "cell_type": "markdown", "id": "c60aa86d", "metadata": {}, "source": [ "Now we have all of the negative features prepared, and saved to fixed durations clips in a Numpy array. For this data, the array is small at ~160 MB, but in-practice the memory mapping allows the array to be very large (e.g., 100s of GBs)." ] }, { "cell_type": "markdown", "id": "1b49b9c6", "metadata": {}, "source": [ "### Positive Clips" ] }, { "cell_type": "markdown", "id": "6f3f9ed0", "metadata": {}, "source": [ "First, [download](https://f002.backblazeb2.com/file/openwakeword-resources/data/turn_on_the_office_lights.tar.gz) and extract the positive clips into the working directory.\n", "\n", "Then the positive clips will be prepared in two way:\n", "\n", "1) Mixing the synthetic positive clips with negative data at random SNRs to simulate noise data\n", "\n", "2) Aligning the positive clips with background data such that the end of the input window aligns with the end of the positive clip. This way the model will learn to predict the presence of the wakeword/phrase immediately after it is spoken.\n", "\n", "In practice, there are other possible ways to augment the positive data (e.g., creating reverberation with room impulse response files, mixing with synthetic noise, etc.) but in practice we have observed that mixing with realistic background data provides the best results. Again, see the [documentation](https://github.com/dscripka/openWakeWord/tree/main/docs/models) for the pre-trained openWakeWord models for more information about the types of data augmentation used.\n", "\n", "After this prepartion, the positive clips will be converted into the openWakeWord features in the same way as the negative files." ] }, { "cell_type": "code", "execution_count": 7, "id": "fe1964fb", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:31:01.912793Z", "start_time": "2023-02-18T03:30:43.623741Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "3388it [00:01, 2771.26it/s]\n", "100%|██████████| 3388/3388 [00:17<00:00, 198.61it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "3203 positive clips after filtering\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "# Get positive example paths, filtering out clips that are too long or too short\n", "\n", "positive_clips, durations = openwakeword.data.filter_audio_paths(\n", " [\n", " \"turn_on_the_office_lights\"\n", " ],\n", " min_length_secs = 1.0, # minimum clip length in seconds\n", " max_length_secs = 2.0, # maximum clip length in seconds\n", " duration_method = \"header\" # use the file header to calculate duration\n", ")\n", "\n", "print(f\"{len(positive_clips)} positive clips after filtering\")" ] }, { "cell_type": "code", "execution_count": 8, "id": "5d9dc47b", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:31:05.710564Z", "start_time": "2023-02-18T03:31:05.699618Z" } }, "outputs": [], "source": [ "# Define starting point for each positive clip based on its length, so that each one ends \n", "# between 0-200 ms from the end of the total window size chosen for the model.\n", "# This results in the model being most confident in the prediction right after the\n", "# end of the wakeword in the audio stream, reducing latency in operation.\n", "\n", "# Get start and end positions for the positive audio in the full window\n", "sr = 16000\n", "total_length_seconds = 3 # must be the some window length as that used for the negative examples\n", "total_length = int(sr*total_length_seconds)\n", "\n", "jitters = (np.random.uniform(0, 0.2, len(positive_clips))*sr).astype(np.int32)\n", "starts = [total_length - (int(np.ceil(i*sr))+j) for i,j in zip(durations, jitters)]\n", "ends = [int(i*sr) + j for i, j in zip(durations, starts)]\n", "\n", "# Create generator to mix the positive audio with background audio\n", "batch_size = 8\n", "mixing_generator = openwakeword.data.mix_clips_batch(\n", " foreground_clips = positive_clips,\n", " background_clips = negative_clips,\n", " combined_size = total_length,\n", " batch_size = batch_size,\n", " snr_low = 5,\n", " snr_high = 15,\n", " start_index = starts,\n", " volume_augmentation=True, # randomly scale the volume of the audio after mixing\n", ")\n" ] }, { "cell_type": "code", "execution_count": null, "id": "70898754", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T03:35:32.377177Z", "start_time": "2023-02-12T03:35:32.349576Z" } }, "outputs": [], "source": [ "# (Optionally) listen to mixed clips to confirm that the mixing appears correct\n", "\n", "mixed_clips, labels, background_clips = next(mixing_generator)\n", "ipd.display(ipd.Audio(mixed_clips[0], rate=16000, normalize=True, autoplay=False))" ] }, { "cell_type": "code", "execution_count": 10, "id": "621c2ee6", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:33:35.853508Z", "start_time": "2023-02-18T03:31:44.655774Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 400/400 [01:50<00:00, 3.62it/s]\n", "4it [00:00, 5.66it/s] \n" ] } ], "source": [ "# Iterate through the mixing generator, computing audio features for positive examples and saving them\n", "\n", "N_total = len(positive_clips) # maximum number of rows in mmap file\n", "n_feature_cols = F.get_embedding_shape(total_length_seconds)\n", "\n", "output_file = \"turn_on_the_office_lights_features.npy\"\n", "output_array_shape = (N_total, n_feature_cols[0], n_feature_cols[1])\n", "\n", "fp = open_memmap(output_file, mode='w+', dtype=np.float32, shape=output_array_shape)\n", "\n", "row_counter = 0\n", "for batch in tqdm(mixing_generator, total=N_total//batch_size):\n", " batch, lbls, background = batch[0], batch[1], batch[2]\n", " \n", " # Compute audio features\n", " features = F.embed_clips(batch, batch_size=256)\n", "\n", " # Save computed features\n", " fp[row_counter:row_counter+features.shape[0], :, :] = features\n", " row_counter += features.shape[0]\n", " fp.flush()\n", " \n", " if row_counter >= N_total:\n", " break\n", "\n", "# Trip empty rows from the mmapped array\n", "openwakeword.data.trim_mmap(output_file)\n" ] }, { "cell_type": "markdown", "id": "77a31736", "metadata": {}, "source": [ "Alright! At this point the positive and negative features have been pre-computed and saved to disk, and now a model can be trained that takes these features and predicts whether the wakeword/phrase is present." ] }, { "cell_type": "markdown", "id": "cab316e2", "metadata": {}, "source": [ "# Training the Model" ] }, { "cell_type": "markdown", "id": "a514773d", "metadata": {}, "source": [ "At this point, you are free to use any type of model that you like, but in practice we've observed that a simple full-connected neural network can often perform quite well. For this example notebook, we'll create and train this network in Pytorch, but any framework that can export a model to the [ONNX](https://onnx.ai/) format will also work." ] }, { "cell_type": "markdown", "id": "f0dab37e", "metadata": {}, "source": [ "## Loading Data" ] }, { "cell_type": "code", "execution_count": 11, "id": "c7ed4de5", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:33:52.884948Z", "start_time": "2023-02-18T03:33:51.191681Z" } }, "outputs": [], "source": [ "# Load the data prepared in previous steps (it's small enough to load entirely in memory)\n", "\n", "negative_features = np.load(\"negative_features.npy\")\n", "positive_features = np.load(\"turn_on_the_office_lights_features.npy\")\n", "\n", "X = np.vstack((negative_features, positive_features))\n", "y = np.array([0]*len(negative_features) + [1]*len(positive_features)).astype(np.float32)[...,None]\n", "\n", "# Make Pytorch dataloader\n", "batch_size = 512\n", "training_data = torch.utils.data.DataLoader(\n", " torch.utils.data.TensorDataset(torch.from_numpy(X), torch.from_numpy(y)),\n", " batch_size = batch_size,\n", " shuffle = True\n", ")\n" ] }, { "cell_type": "markdown", "id": "1d1ba9e4", "metadata": {}, "source": [ "## Define Model" ] }, { "cell_type": "code", "execution_count": 12, "id": "d7c71798", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:33:54.913238Z", "start_time": "2023-02-18T03:33:54.896447Z" } }, "outputs": [], "source": [ "# Define fully-connected network in PyTorch\n", "\n", "layer_dim = 32\n", "fcn = nn.Sequential(\n", " nn.Flatten(),\n", " nn.Linear(X.shape[1]*X.shape[2], layer_dim), # since the input is flattened, it's timesteps*feature columns\n", " nn.LayerNorm(layer_dim),\n", " nn.ReLU(),\n", " nn.Linear(layer_dim, layer_dim),\n", " nn.LayerNorm(layer_dim),\n", " nn.ReLU(),\n", " nn.Linear(layer_dim, 1),\n", " nn.Sigmoid(),\n", " )\n", "\n", "loss_function = torch.nn.functional.binary_cross_entropy\n", "optimizer = torch.optim.Adam(fcn.parameters(), lr=0.001)\n" ] }, { "cell_type": "markdown", "id": "6bb834c1", "metadata": {}, "source": [ "## Train Model" ] }, { "cell_type": "code", "execution_count": 13, "id": "5bc28f8b", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:33:59.286835Z", "start_time": "2023-02-18T03:33:57.795926Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 10/10 [00:01<00:00, 6.74it/s]\n" ] } ], "source": [ "# Define training loop, metrics, and logging\n", "\n", "n_epochs = 10\n", "history = collections.defaultdict(list)\n", "for i in tqdm(range(n_epochs), total=n_epochs):\n", " for batch in training_data:\n", " # Get data for batch\n", " x, y = batch[0], batch[1]\n", " \n", " # Get weights for classes, and assign 10x higher weight to negative class\n", " # to help the model learn to not have too many false-positives\n", " # As you have more data (both positive and negative), this is less important\n", " weights = torch.ones(y.shape[0])\n", " weights[y.flatten() == 1] = 0.1\n", " \n", " # Zero gradients\n", " optimizer.zero_grad()\n", " \n", " # Run forward pass\n", " predictions = fcn(x)\n", " \n", " # Update model parameters\n", " loss = loss_function(predictions, y, weights[..., None])\n", " loss.backward()\n", " optimizer.step()\n", " \n", " # Log metrics\n", " history['loss'].append(float(loss.detach().numpy()))\n", " \n", " tp = sum(predictions.flatten()[y.flatten() == 1] >= 0.5)\n", " fn = sum(predictions.flatten()[y.flatten() == 1] < 0.5)\n", " history['recall'].append(float(tp/(tp+fn).detach().numpy()))\n" ] }, { "cell_type": "code", "execution_count": 14, "id": "4231dd84", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:34:01.172348Z", "start_time": "2023-02-18T03:34:01.043030Z" } }, "outputs": [ { "data": { "text/plain": [ "(0.0, 1.0)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGiCAYAAAA1LsZRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABjbUlEQVR4nO3dd3hUZdoG8HtaJr2QHgghQOhNQjEgRVAUFXVFRd1PYBVX7IAVu667qLuWtYAFsayusCq6rIssqPSiSIdQAoQkQEJI75nMzPn+eOdMSWbSSOZMcu7fdeXKzJkzM++Zgcw9z1uORpIkCUREREQK0SrdACIiIlI3hhEiIiJSFMMIERERKYphhIiIiBTFMEJERESKYhghIiIiRTGMEBERkaIYRoiIiEhRDCNERESkKIYRIiIiUlSLw8imTZswbdo0JCQkQKPR4LvvvmvyPhs3bkRqair8/f3Rs2dPvPfee61pKxEREXVCLQ4jlZWVGDp0KN55551m7Z+ZmYmrrroK48aNw549e/Dkk0/iwQcfxDfffNPixhIREVHno7mQE+VpNBp8++23uP766z3u8/jjj2PVqlU4fPiwfdvcuXOxb98+bN++vbVPTURERJ2Evr2fYPv27ZgyZYrLtiuuuAIfffQR6urqYDAYGtyntrYWtbW19utWqxVFRUWIjIyERqNp7yYTERFRG5AkCeXl5UhISIBW67kzpt3DSF5eHmJjY122xcbGwmw2o6CgAPHx8Q3us2jRIrzwwgvt3TQiIiLygpycHHTr1s3j7e0eRgA0qGbIPUOeqhwLFy7EggUL7NdLS0vRvXt35OTkIDQ0tP0aSkTeI0nAy4ni8uV/Akb8Qdn2NNcipz+oC08r1w5nFeeBty9yXB81Fxg5B3h3RMN9b/4M6DWp4faPrwLy9juuT/8I6HOFuLz6UWDfl677B8cCD+xy3ZaxDvja9j72uhy4+WNx2fm9ltvwr5n1GqAFYBUXL30KsFqAjS+77qLRA5JZXJ7xBdBzAvDRFCA/HbjiZSB5HPDeWAA6YOa3wGfXNjxO+9P5AVaT59tl8r/NkxuBFb8X27r0Bu7e0HDfH18Edn7gui12EHDuoOfHj+wD3PUT8Gov0R6/EMBU7rqPIQiYtQpYOhkwhgIL0oG/9QXqKptuf33DZwFj5wNvD3Ns6zcN+N2Slj9WM5SVlSExMREhISGN7tfuYSQuLg55eXku2/Lz86HX6xEZGen2PkajEUajscH20NBQhhGizqK6BDDavpB0iQba6v926WlgwyJg9FwgbnDj+5qqAL/Alj2+vx6QLOJyU22WJODEz0DsQKD0DHByPTB2HqDTAxYzoNECcum6skB80Oj9gMpCYOeHwLDbgOIsoKYE6D/Nc3ursh2vJQBoqwBNhes22cHPgIGXA35Brtvrzov9E0cDOb8AuVuAodcAp7YAJ79v+Fh15wF/nevj1J517Je7GTBqAWOw7dic7n/iPw0fT6MVrxcA1JwWr0+D9lsA2LZVZQGmfKD0sHhPRs4AgiKBrv2AgqPAoX+4P37HATgeqzE6k3ifz+90PF7lCQC2ILD5NRHi+lwB7P/Q9Tnjhoh/h/++1/Pjj74VCAsDohOAkiwAbt63hL5A17627eW2190KaFsxbGHoNCChJ5A4UIQ4ABg4ue3+/3nQ1BCLdg8jaWlp+M9//uOybe3atRgxYoTb8SJEpBKVBe3zuOueBQ5+A+z5HHi+1PN+p7YAn14LTHoaGLfA8371BUUBFefE5eoSICDc877Z24HPbwB6TQaqi4Gzu4GY/sDJDcDefwIh8cA9W8U352VTgb5XiqrBmseBA18Buz4BaisAUwVw3TvAqgeBK/4MXHyP6/NUnne9Xl0MlOe6b9OJn4DX+wP3/wYEx4htFjNQmS8uD58pwkjGOiA4zlGd0OiAiB5A0QnbA0nA+aNA1+GOxy487rhsqQWytooP6ZJs1zYc/Lphu+SABwDnj7ledyc/HbDYKhs9J4ggAgB9pogwUv85YgYC+Ycaf0x3am3/hk5usG3QAJCA1/sBWgNgrRObc34Rv7UGICJJvBb9rrFVl2z3CUsESnOA8CTx3p8/Agy9VdwvJM4WRmzCkxzXo/oA/qGAX7D4t3B2L2Cuafmx6P2BHpeIyz3GOcJIj/Etf6w21uKpvRUVFdi7dy/27t0LQEzd3bt3L7KzxT+2hQsXYuZMR/lt7ty5yMrKwoIFC3D48GEsW7YMH330ER555JG2OQIi6piqnMKIudbzfi1VkOG43NhkwcxN4gPv5PqWPb5kdVwuPOF5PwA4Z/vwyzvg+KDe/Rnw6wfiQ6UwA8jeAXwzR3x4p/9b7JNlm2lYnmsr2UvAL++L9h77X8PnaRBGSoCysw33S7AFh5pSEcac7y9ZReAYcD0ADVB2BjjyvWOf/tcAt/0LuHGZ+CADgHzHLEm3r0fpadff9SVdIrol6is4ChQcc38f2blDwLE14nK/qx3b+17tut+YB4BpfwdmfgcEdGn8MWX3bAMumS8u15QC+UeA3H3iuhweABFEEi8GrlgE9LxUBIgxDwDT3gKG/R8w+o8ivE5+Brjo/4Dr3hWvcepsYOa/gXkHgNAE8VghcY7H1RqA7hc7rkel2PaxjbHM2ua4TaMDdA17ElwEdAGCokXQlCtrPSeI36FdgchezXtd2lGLKyO//fYbLr30Uvt1eWzHrFmz8MknnyA3N9ceTAAgOTkZq1evxvz58/Huu+8iISEBb731FqZPn94GzSeiDsv5A9TSjL775grt6hj7UHgCiOrtfr/iLNffzSFJ4sNJVngc6JbqeX+5IiBXHQBRLXG2+1Og6KTrc4TEAWX1PsDlY3L3IS2/loGRQFWhrTLi2j2OwCjgj+tFdWX3p+LxBt0gbiu3BZeQONGt0iVZtEke6zDtLWDQdHFbVG8g51fg1GbxzXr/V0D6d8A1bzqCYNwQ8fhyu0pzxO+UKSIE1VWJ61NfER+0L8W4trW6uOEx1pe713G5z5WOy4mjgdBujtev2yhggG3syD1bRTXiH79z7P/HjaJi8bcUEfa0eiCqr+ODv6YM2PKGuNzvGlGZCusqutR6TgDih4rb0up1xfQY67g87mHH5afPiefQaACDv2N7sFMYCYkT1RCZfDk03hZgbWEkeTzwuw9E8FxqGwc04HrR9qjewH9tz9v7MmD6h67t6zMVmPQMkDhKtEVhLQ4jEydORGNLk3zyyScNtk2YMAG7d+9u6VO1iCRJMJvNsFiaKO1Rs+l0Ouj1ek6npvbh3E3TmpKzJ85hIXu75zAiB4WyM6KbQtfEn8NfPxRlcufg5NwtUVcDbHwF6DtV/IEHHB/C7toXGCWqQwfrLQBZW+76Lbm+sjPiA9LfqY9f/tCP6iOO2V03TZdk8Tt+iPidd8Bxmxxc5OeN7u8akAZcK4KILKa/+J1/GNhuWwCzKBOosD1O0hgRRipsIazE9jpE9xMDU0/8JK5H9gb0Rkf3Q306v6aDatxgIMxpULFWK0LWtrdsz9nXcVtoAmAIcLqzBogZIMbphHcHijPFb50e8A8Tu5w/4ghZ4xYAgV1E115r6TwMT3B+z4NjPYSRruK3XDmL6CECitxWQAyYnfCoeJ3XLBSvn/NjybRaYLzv9FB4ZTZNezOZTMjNzUVVVZXSTel0AgMDER8fDz8/P6WbQp1BXbX4Vp00pl4YacPKiEsVYgcw/Hb3+8lhxGoWH9zhiQ33qSoS3TjBccBqN3+4MzcBpvmi9P3Ti8COd4FtbwPPFrg+hzvDb3d843Zp/3nxIdyYggzXikyFHEZSPIeRCFsYibN9k891mjkj7ytXA2L6AUf/Ky4HxQABEa6PFTNA/HYONPJ4jKBoETIAx9gaOZSFdwd6ThRhJDLFURkI6OI+jHRPE5UUySLGO5hrAGhEV4gcNup3ywCuYUQ+bpl/uJidUlcpxszoba91l54ijHTpKa4bbWHv/BHxO3Yw0LWRKtiFkl97wLUyotE52iTvY64WvyN6iN9+gUBYd6A02xG+tTpR4Tl3QLyfPq7DhxGr1YrMzEzodDokJCTAz8+P3+TbgCRJMJlMOH/+PDIzM5GSktLogjVEzbL2aWDnUlG2NjlNS2zLyohz90/WFvf7mE2OrglADBR0F0bWLAT2L3f/GBotkLMDWH6bGI+w6xOx3VrnqLSUuKmMyBIuEh9uZ3aJGRdHfxDtqCxwdGN4UnDUNYzYKyO2KoC5Gig+JS7rA8R1uTISO1C0vTJfVERC4oCyemEkur/jsZ0rC/W3OQc/Wa9JjoGxcrvkUBaWCPSeDMxeLbo6ZIER4oMUAAbdKLoxijOBsQ+JcHN2DxA3SIypCYkHpvxJVKByfgFG3d2wDQkXianJ/mGOsCHTaESFpDDDMV4DEOMmTvzk+OB3rjYA7v99tKUQp/W4QuLEa5x2v2ij3jYmxLm9gCOMAMC4+cDh70WXjOyKPwMZa4GUK9qt2W2lw4cRk8kEq9WKxMREBAa2cIoeNSogIAAGgwFZWVkwmUzw9/dv+k5Ejdm5VPze/JoYgyCztNEAVnOtazdN8Smg4HjDrpqy064DUZ0rGFarY7qtpyAS0UOMkfjH9aJykv2L65oPhRliH3cf1rKwROD690QlY9htwOnfRBipKnANau6cP+p6XX6eLj3FN2nJ4uhmSbtPzCzpe5W47hcoqhIFR0V1JCSuYTeN8zdpdyV+/zDXcRlRfYDxj4nnTxgmql+A6KaRJKfKiO0D3Xk8BeA6sDQpTayRIus9WTzGhkUA/u3okkkaI348GXyj59vCutrCiFMgGnmXGPg78i7bMdab6hpcb1xLW6tfGdFoRJjwtA8gqjWyEXeIH2c9JzgGqvq4TvNVl9/a2wdfV2o3Lemmae4ptORv4lqDGNwHiG+G9dXvPpGvn90DvJIEbH9XXA+Kdv88/mGiu8FgW2Njx7uut+cd8DyDRBaWCET3AVJniXEEQVGOY6irbvy+9Qexyq9lcEzDqcZjHwQe2idCgsw+bsQ2Q0TuppG/eUemiFADuK+MAI5xI4AYsDrkJlGt0epcKyM5v4puI72/6zd5Z4FOYcTdjBeNRlQ7ANfjaK1QW6BxHmsS3UcM8oy2ha/6lZFg15XE25zz4wd7GDPkXBkZ+DtHWzsBftIQUfspPQ28PwHYa1u90/kPfFGm47KnbhqrFfj4auDDSU1/QAOOAZNB0Y4ZFhn/E2M/fnrRMRCxfhiRZ9RkbgJqy8T0WUkSg0nd8Q+zlfvjHfdzlre/8fEi+gBH+JC5hBGnbhqdUcySARzdJ85hRJJcj9s/3HGbIdAx9sFZnC2M5O4Xr7E8EFf+QDT4O8KGp4XjnMOIPPVUJocRU4Vj7Mag6Q0XWpM5B5BAN2EEEO/n3K3AlJfc394Sg28U414G/s7zPvVft/aujAREOKbo1q+AyMKTHJcnPdO+7fEyhhEiaj8nN4opmPv+Ka5bzI7bSp0+rOvPmLBagbyDotsia4tYLGz1I2KqYmNre8gVgqAoMY0UAE5tBXZ9LLqG5AGjcviQw5EcHMptAy4rz4spsp5Cknw/+UNDnooqD6bMO+B4zGinLg85VIR1azidUq7CVBa6dtMkXOQYx5CUJn5XFTpury13dHMFRbkONo3p737apr0ysl90RZVkiQ9f5wXMfvcecO3bYhCpO/IgVqBhGPELFoELcKxVMuJO948DNF0ZAcRxxA1yjJ+4EL0uBe7d7rqWR33GeuuftHdlRKMBYgeI8TyeKh5BkcDvvwZm/9cn1gZpSwwjCpk4cSLmzZundDOI2pf8Dd9UJVYS9XQujfof+r8sEecYWfOEY9uez8WYk1+d1ksw14qVQuUPb3nsRHCMmNHhHyYGlJ6xLS1gn2pqCwpJttUoi07YKgy2MFJxTkyhBcRskscygXFOs2nqhxFZP9u4jNz9jtUzk8aIvv24wUBX27linLsHZIFuuml+9z5wyxfifCKxg4Eht4jtNWWOrit5vIchSFQenMOIp9kfcmWk+BTwX9txjX/E9b5xg8UiWZ4mBLhURup9eGo0QLBTF1fsYNegU58c0upfVpJW51odae8wAgC3fQXM3SJmHXmScrljFdVOhGGEiNqP/KFaV+34oHen/piR9FXit7zCpjPntTu2vAF8cSPwte1bt727IkZ8IMpdFnIwkAe3yuM5+lwhujLKc8VAUrmNVUWO6klYV/HN3Xn2h/y49dcDSblClNqri4CDK8W22IHA3RuBP25yhBB3MzPslRGnbprEUaLaMfx24J4tojIAiAGqcgCTZ/HI1Y7mhJHALmLMCiACYliimNHTEtF9AWOY+MCO7N3w9iCnbo1+Vze+sFZzummU4BJG2rmbBhABLnZg+z+PD2IYIaL2I1c86iodQUFmCHR8o3aeTWOqFNNdnQ2aLpbbBlzXo9j1qfh97Adg61vATy+I6/K3cnkwZ7GtElJTIn7Ls0Ci+zlWId31qdOqpZJjxVN5xoVzFUSujDgPKDSGiuftNlJct1dGxopv2VqtWDwsPAkYcB0akM+t4jy111BvhqAhUEx7lY8l40fgt4/EdXkBK+fuhYRGqhFydQQARt7Z8u4PQwAwZx1w57p6C4nZOIeiPlMaf6xA2776APePpRTnABXkhTCiYp0ujEiShCqTWZGfxlambUxxcTFmzpyJiIgIBAYGYurUqcjIcJxfIysrC9OmTUNERASCgoIwcOBArF692n7f3//+94iOjkZAQABSUlLw8ccft8lrSdQsuz4VS4xbrQ1vkz9UnSsj3dOA+3eJ06BPtHXDOJ+bJucXx8nHZL0miymqgKhayOSxFACwzmlAn1xlkCsY8snOakrFypTyeVvCugLDZ4vL6d+J6Z6ys3vEb3sYcaqC2LtpnLbJlQbnEnpAhGPtD0DMwJm333UtiPptdp52XD+MaDSOb+v/mQd8MV0Evu5p4jUCXCtQ7ioWMudulItmet6vMdF9PS+oVeQ0tif+osYfR37tnKtPvsB50HRLz+5MLdLh1xmpr7rOggHPujmRlBekv3gFAv1a/pLOnj0bGRkZWLVqFUJDQ/H444/jqquuQnp6OgwGA+677z6YTCZs2rQJQUFBSE9PR3CwWJr5mWeeQXp6On744QdERUXh+PHjqK5uxqwDorayYZHo5hg5x9FVIKuzVUZMVY7KSHCMY90PefaAcxjJ3NzwOaKcpppWFYlAYbWIs6O6W9hM7nOvP821ukS0w2oWAwWD40TYcD5DqkwOI2FuKiN+wQ23Oa+hsdG2rXuaY82SpshjRpzXSakfRgCx/kV1kWMGT5+pwLQ3Hd/inacjN/bcI+4E9nwhzqkS1A7jNPpMFVOeYwc3/RpE9xXjYyJTGt/P29ryBI7UqE4XRjoaOYRs3boVY8aIBXy++OILJCYm4rvvvsNNN92E7OxsTJ8+HYMHiyl2PXs6vg1mZ2fjoosuwogRYmBcjx49vH4MpHLy2AV3y3nbKyNVjvOWOA8E1NsW0nMewCqfil1eshsQ3/DlD+mqQuC9ceIx5XPAyOKHirOl9pkqrtdfK8JS6/jGHpLgOB9N/NCGYUSesSJXRpw/5OXKjXMYkb/ddxvpOKdKY4ty1Vd/qq/Oz/35cuRjkru2Lp7rWqGZ+ISYYVN/Aaz64ocAT+WJLqT2cOlCsa7IkJubt//QW9qnHRfCzC923tLpwkiAQYf0F5VZ+jbA0PL/1IcPH4Zer8fo0aPt2yIjI9G3b18cPixOz/3ggw/innvuwdq1a3HZZZdh+vTpGDJEfAO95557MH36dOzevRtTpkzB9ddfbw81RF5hdqp+eLpNsjiWRnceCCgv1e08tVcOAcnjxADWwEgxqFEeK2Gudj0PirOLbgdG3eW47rzmhuxcuvjt3CUQNxg4vMrt4dnDiPOHtlyxcA4BcmXEECDGhBz5r2PV0+bQG8WAULlLyV1VBGi4/kX92SchccCNHzXvOZs6OeCFMIYAo//Yfo/vDVZz0/tQm+h0Y0Y0Gg0C/fSK/LTmnDiexplIkmR/vDlz5uDkyZO4/fbbceDAAYwYMQJvv/02AGDq1KnIysrCvHnzcPbsWUyePBmPPOI7Z2KkTs5qcQQJd9N2nRfvks+V4rYy4lQOl+/T72oRQJJty1kbQxyBRFZ/2fT6lZL63TSAOHEY4LoUuKcZDBqt6xTWa94ABt4gTtMOiADhvHaI7PolwCPHWr4WhHN3iacFwupXewKj3O9HF26wrarTc6KizVCDThdGOpoBAwbAbDbjl19+sW8rLCzEsWPH0L+/449gYmIi5s6di5UrV+Lhhx/Ghx861lqIjo7G7Nmz8fnnn+PNN9/EBx984NVjIBVz7l5xt0JqndPtxbYVV53DiHx2WucwIldYuqaKZcyvsy21rtE0rALIs2MA0a0TUy9UuK2M2KoqzuEhdlDD/QAx48Q50Iy4A7jpY9eTr8UPE6Elfphjm87QcNGs5nCu9HiaVdIgjPjQVNjO5uq/iYXfbuSkgPbW6bppOpqUlBRcd911uOuuu/D+++8jJCQETzzxBLp27YrrrhPT/+bNm4epU6eiT58+KC4uxs8//2wPKs8++yxSU1MxcOBA1NbW4vvvv3cJMUTtyiVEuKuMOAUU+bwxzoFCnk7qPLVXvo8hsOHiYAFdXGeLyCufAsDouxt2O7itjMjdNE6P7WmRqeYsLjXjc3FsEUlN79sU5yqHp24a5zBiDG2bFUnJPf8wsfAbtTtWRnzAxx9/jNTUVFxzzTVIS0uDJElYvXo1DAYDAMBiseC+++5D//79ceWVV6Jv375YvHgxAMDPzw8LFy7EkCFDMH78eOh0Oixf7uFMo0RtzTlsuKuMuBsA6PxhKn+Q1tUAh74TK6N6WmMDaFgZkfe95Uvgsucaf676bXLupvHUxdpjnPvtzvwC2yaIAK6DWD110ziPGfGV1UqJLhArIwrZsGGD/XJERAQ+++wzj/vK40Pcefrpp/H000+3ZdOIms+lm8YWDKqKxMnmwpPcBxTngCBP7a2rBL6aZfvwt42jctdNERjRcBvguTrg72F/oGHV5aq/AT//SZyAbLVt3FVj5y5pD83qpmEYoc6HYYSIWs85bJgqxfln3hompuFGprh2o8icv9nLA1hlzifBa05lxNPjyNx108i6JLteH3WXWCtFkoCsbWJ12Mbu3x6CWthNU386MFEHxTBCRK1nrjfWo+ysYz0Q59VMZTqjOD29zHkgKOAYV+JpjQ1PZ3T1FEbcDWAFxBgRd104Go34uUmhAYvOlRF205CKMIwQUctVFwOb/uba1VFX6TqV1536AUBXr3tFXkzMUxeFpw9fg6cw4iZwAGJVUF/kfHzNmU3DMEKdBMMIETXfqa2ielFTAmx/x/VbuqnK/RgRZ/71FuzSGQBoYB8nIvPUReFpGqunyohOL5Zur786rK+eGdVlzIindUacXkN201AnwTBCRM3z04vA5tfE5ZFzxO/aMsftddUtr4xoNGLwqfNAWKDllZHGprf6h4swEhIvzqED+HAYcZ5N04wxI6yMUCfBqb1E1LSik44gAgBFmQ33qatsRmXETbdJ/a4awHNVoHua+KnPU2UEcAxCde5SiuvA3TRG5zDCygh1DgwjRNS0ygLX6/IZeJ3VVTcdRuqfVwVwX9VobFrrHWuAi+9t+jHs9wl3/Q2IE7j5Ip0BCLBNR25ONw0rI9RJsJuGiJpWf8xFpZswYqpqeTcN0LIwYr9PvUqIvpH94wYDWVuAMfeLk911G9l+Z6ptC4FRYoCwp24aObBUFwOh8e73IepgGEaIqGn1l3qXp+A6a1Y3TTMrI56mtcqcB7hqdI2fffaKvwBjHwRCEzrGCc9C4sS0aE8zgQBg+lKgPK/hwm1EHRS7aVTk+eefx7Bhw+zXZ8+ejeuvv16x9pCPO7MLOLpGXDbVq3hI1ob7Ow9gbc6CXTKdX8NtTVVGXNYqaWS8CABotSKIdBSTngHGPAD0vszzPr0vAy76P++1iaidsTJCRO6tuF0sYvbwkYbdNO44T+0NiRODXutztwiZpa7htpZ003S2E8V1Hy1+iFSElREfYTKZlG4CkauKfACS6JJxd0be+pwXPQvxMJbB3QBWd107nior9tudwkpTlREi8nmdL4xIkvjDqcSPJDXdPpuJEyfi/vvvx4IFCxAVFYXLL78c6enpuOqqqxAcHIzY2FjcfvvtKChwzGKwWq145ZVX0Lt3bxiNRnTv3h1//vOf7bc//vjj6NOnDwIDA9GzZ08888wzqKtz862TqClWi2M11OYMTAVE101NibgcEud+H3fdNO4eu6kw4hxAPK2+SkQdRufrpqmrAv6iUP/wk2ebHnjn5NNPP8U999yDrVu3oqioCBMmTMBdd92F119/HdXV1Xj88cdx88034+effwYALFy4EB9++CHeeOMNXHLJJcjNzcWRI0fsjxcSEoJPPvkECQkJOHDgAO666y6EhITgsccea/NDpU7O5Wy8lc3rpgGAykLx21NlxN0AVlZGiFSv84WRDqR379549dVXAQDPPvsshg8fjr/85S/225ctW4bExEQcO3YM8fHx+Pvf/4533nkHs2bNAgD06tULl1xyiX3/p59+2n65R48eePjhh7FixQqGEWq5OqcwYqpydNP4hQCmcs/3q7JV8lpSGTG7CyNNDWB1DiOdbMwIkQp1vjBiCBQVCqWeuwVGjBhhv7xr1y6sX78ewcHBDfY7ceIESkpKUFtbi8mTJ3t8vK+//hpvvvkmjh8/joqKCpjNZoSGuvkmStQU54BQV+WYTRMcAxQ1FkZslRHngaoarWP2TWPTVZ15WmNDpmdlhKgz6XxhRKNpUVeJkoKCHO20Wq2YNm0aXnnllQb7xcfH4+RJNzMTnOzYsQO33HILXnjhBVxxxRUICwvD8uXL8dprrzV6PyK3zLWOyyanbprgWKDohOf7yd00zpULjc4RRtwNYHWnyW6aTjybhkiFOl8Y6aCGDx+Ob775Bj169IBe3/BtSUlJQUBAAH766SfMmTOnwe1bt25FUlISnnrqKfu2rKysdm0zdWJ19Ssjtm6a4Gj3+8tqS8Vv5zBhCABq68R6In4NK39uNTm1N8D9ZSLqkDrfbJoO6r777kNRURFuvfVW/Prrrzh58iTWrl2LO+64AxaLBf7+/nj88cfx2GOP4bPPPsOJEyewY8cOfPTRRwDE+JPs7GwsX74cJ06cwFtvvYVvv/1W4aOiDstcb8yIPOMlOLZ593cOE/7hwKSngav+KhYgq2/yc+J37CCn+7MyQqQmDCM+IiEhAVu3boXFYsEVV1yBQYMG4aGHHkJYWBi0tj/gzzzzDB5++GE8++yz6N+/P2bMmIH8fHGOkOuuuw7z58/H/fffj2HDhmHbtm145plnlDwk6sgazKaRKyMxzbu/c5iwmIDxjwKps93ve8l8YMFhYNzD7u/f1ONzzAhRh6eRpBYsjqGQsrIyhIWFobS0tMGAzJqaGmRmZiI5ORn+/vyj1Nb4+qpUxo/AF9PF5VF3Axn/A4pPAde+Dax6oOH+hkDX9ULu3gy8P05cjuoD3L+zGc+5DvjiRnF5zs9At1TP+9aUAS8nisvDZwHXvtX04xOR1zX2+e2MlREiashlNk2l02waD9009U9lbwgEbl0hul6mL23eczoPPG/R1F6GZKKOjgNYiaghl9k0VU130wRGAqU5juuGAKDvleKnuZwHtzYVRnQG2ywdC1dgJeoEWBkhooacZ9OYKkV1BACCPISRLsmu15sKE+4YncJIc6bny8/ByghRh8cwQkQNOQ9grXKcHwkB4WKKbn2RKa7XW7gAIICWVUYARwjhbBqiDq/TdNN0gHG4HRJfV5VyDiMV520XNGJND79goLpIXIft30dUH6c7a1oXEAIjRajRGQBDcyojtsDDyghRh9fhw4jBYAAAVFVVISCAix+1taoqMXBRfp1JJZzPTVNpCyOGQLFOiBxGgqIct0X1duxvCBQrIbeUVgfcu912uRlFW3msCMMIUYfX4cOITqdDeHi4fb2NwMBAaFrzh5BcSJKEqqoq5OfnIzw8HDqdTukmkTc5V0bkmTXyOA75d3CsLYxogC69HPvLS7+3hq4FoVfPMELUWXT4MAIAcXHiDKFyIKG2Ex4ebn99SUWcw4isQRiJAc5BhAF/p/UD3J2Ftz3Yu2k4ZoSoo+sUYUSj0SA+Ph4xMTGoq6tTujmdhsFgYEVErercBAp5gKkcRqL6ApmbgYgk77XL2ZCbxSyfpLHKPD8RtZlOEUZkOp2OH55EbcF5nRGZn60SYQwRv8MTgXu2inPPAI51P7xl5J3ih4g6PE7tJaKG3HW1yBWR/tcCEclAz0uB6L5AiG1V1vqrsBIRNVOnqowQURtxWxmxhZGhM8RPfcExQCXHbRFRy7EyQkQNuRsz4um8NLLr3hWDSic/2z5tIqJOi5URImrI3WyauCGN3ydhGPBEdsum5xIRgZURInLHbRgZ3PT9GESIqBUYRoiooTo3YSRmgPfbQUSqwDBCRA25m01j4EqnRNQ+GEaIqKH6s2m0HF5GRO2HYYSIGqo/m6bvVcq0g4hUgV93iKgheQDrDUuBzI3A5S8q2x4i6tQYRojIlSQ5wkjyOGDITcq2h4g6PXbTEJErSx0gWcVlPQetElH7YxghIlfOa4wwjBCRFzCMEJErlzBiVK4dRKQarQojixcvRnJyMvz9/ZGamorNmzc3uv8XX3yBoUOHIjAwEPHx8fjDH/6AwsLCVjWYiNqZPJNG7w9oNMq2hYhUocVhZMWKFZg3bx6eeuop7NmzB+PGjcPUqVORnZ3tdv8tW7Zg5syZuPPOO3Ho0CF89dVX2LlzJ+bMmXPBjSeidiCvMcIuGiLykhaHkddffx133nkn5syZg/79++PNN99EYmIilixZ4nb/HTt2oEePHnjwwQeRnJyMSy65BHfffTd+++23C248EbUDs1NlhIjIC1oURkwmE3bt2oUpU6a4bJ8yZQq2bdvm9j5jxozB6dOnsXr1akiShHPnzuHrr7/G1Vdf7fF5amtrUVZW5vJDRF4id9MYApRtBxGpRovCSEFBASwWC2JjY122x8bGIi8vz+19xowZgy+++AIzZsyAn58f4uLiEB4ejrffftvj8yxatAhhYWH2n8TExJY0k4guRF2V+O0XpGw7iEg1WjWAVVNvUJskSQ22ydLT0/Hggw/i2Wefxa5du7BmzRpkZmZi7ty5Hh9/4cKFKC0ttf/k5OS0pplE1BqsjBCRl7VoBdaoqCjodLoGVZD8/PwG1RLZokWLMHbsWDz66KMAgCFDhiAoKAjjxo3DSy+9hPj4+Ab3MRqNMBo5pZBIEQwjRORlLaqM+Pn5ITU1FevWrXPZvm7dOowZM8btfaqqqqDVuj6NTqcDICoqRORj5G4aQ6Cy7SAi1WhxN82CBQuwdOlSLFu2DIcPH8b8+fORnZ1t73ZZuHAhZs6cad9/2rRpWLlyJZYsWYKTJ09i69atePDBBzFq1CgkJCS03ZEQUdswMYwQkXe1+ER5M2bMQGFhIV588UXk5uZi0KBBWL16NZKSkgAAubm5LmuOzJ49G+Xl5XjnnXfw8MMPIzw8HJMmTcIrr7zSdkdBRG2HlREi8jKN1AH6SsrKyhAWFobS0lKEhoYq3Ryizu3nPwObXgVG3gVc/TelW0NEHVhzP795bhoicmWvjHAAKxF5B8MIEbniOiNE5GUMI0TkilN7icjLGEaIyBUHsBKRlzGMEJErVkaIyMsYRojIFdcZISIvYxghIlfspiEiL2MYISJX7KYhIi9jGCEiV6yMEJGXMYwQkSv7OiMMI0TkHQwjROSK3TRE5GUMI0TkIEnspiEir2MYISIHiwmQrOIyKyNE5CUMI0TkYKp0XGZlhIi8hGGEiBzk8SJaA6AzKNsWIlINhhEicrAPXmVVhIi8h2GEiBzsg1c5XoSIvIdhhIgcGEaISAEMI0TkYF/wLEjZdhCRqjCMEJEDFzwjIgUwjBCRA8MIESmAYYSIHLj6KhEpgGGEiBxMDCNE5H0MI0TkwNk0RKQAhhEicjDXiN96f2XbQUSqwjBCRA5yGGFlhIi8iGGEiBzqWBkhIu9jGCEiB7Ntai/DCBF5EcMIETnIlREDwwgReQ/DCBE5cAArESmAYYSIHDiAlYgUwDBCRA51HDNCRN7HMEJEDuymISIFMIwQkQMHsBKRAhhGiMjBXhnhmBEi8h6GESJyMLMyQkTexzBCRA4cwEpECmAYISIHDmAlIgUwjBCRA9cZISIFMIwQkWAxA1azuMzKCBF5EcMIEQnySfIAVkaIyKsYRohIkNcYAQCdUbl2EJHqMIwQkSBXRnRGQMs/DUTkPfyLQ0SCuVb85hojRORlDCNEJNjXGOF4ESLyLoYRIhK4+ioRKYRhhIgErr5KRAphGCEigauvEpFCGEaISODqq0SkEIYRIhLqWBkhImUwjBCRYOaYESJSBsMIEQl1nE1DRMpgGCEiwT6AlWNGiMi7GEaISOA6I0SkEIYRIhK4AisRKYRhhIgEezcNz9hLRN7FMEJEglwZ4TojRORlDCNEJMhn7eXUXiLyMoYRIhLMrIwQkTIYRohIqOOYESJSBsMIEQlcZ4SIFMIwQkSCxSR+szJCRF7GMEJEAqf2EpFCWhVGFi9ejOTkZPj7+yM1NRWbN29udP/a2lo89dRTSEpKgtFoRK9evbBs2bJWNZiI2ol9Ng3DCBF5l76ld1ixYgXmzZuHxYsXY+zYsXj//fcxdepUpKeno3v37m7vc/PNN+PcuXP46KOP0Lt3b+Tn58NsNl9w44moDdkrI5zaS0TepZEkSWrJHUaPHo3hw4djyZIl9m39+/fH9ddfj0WLFjXYf82aNbjllltw8uRJdOnSpVWNLCsrQ1hYGEpLSxEaGtqqxyCiJrw+ECg7DfxxA5BwkdKtIaJOoLmf3y3qpjGZTNi1axemTJnisn3KlCnYtm2b2/usWrUKI0aMwKuvvoquXbuiT58+eOSRR1BdXe3xeWpra1FWVubyQ0TtTK6M6NhNQ0Te1aJumoKCAlgsFsTGxrpsj42NRV5entv7nDx5Elu2bIG/vz++/fZbFBQU4N5770VRUZHHcSOLFi3CCy+80JKmEdGF4pgRIlJIqwawajQal+uSJDXYJrNardBoNPjiiy8watQoXHXVVXj99dfxySefeKyOLFy4EKWlpfafnJyc1jSTiFqCY0aISCEtqoxERUVBp9M1qILk5+c3qJbI4uPj0bVrV4SFhdm39e/fH5Ik4fTp00hJSWlwH6PRCKOR386IvMZqBax14jLDCBF5WYsqI35+fkhNTcW6detctq9btw5jxoxxe5+xY8fi7NmzqKiosG87duwYtFotunXr1oomE1Gbs9Q6Luv9lGsHEalSi7tpFixYgKVLl2LZsmU4fPgw5s+fj+zsbMydOxeA6GKZOXOmff/bbrsNkZGR+MMf/oD09HRs2rQJjz76KO644w4EBHDZaSKfIHfRAKyMEJHXtXidkRkzZqCwsBAvvvgicnNzMWjQIKxevRpJSUkAgNzcXGRnZ9v3Dw4Oxrp16/DAAw9gxIgRiIyMxM0334yXXnqp7Y6CiC6MPHhVowW0Lf6zQER0QVq8zogSuM4IUTsrzgL+PgQwBAJP5SrdGiLqJNplnREi6qTkyoiO40WIyPsYRoiI03qJSFGq7hzOKqxEQYUJiREBiAnlH2FSMS54RkQKUnVl5K//O4rpS7Zh9QH2kZPKyVN7WRkhIgWoOozotWLVWLPV58fwErUvezcNx4wQkfepOozotOLwLQwjpHZmVkaISDmqDiOsjBDZcAArESlI1WFEpxNhhJURUj2zSfzmAFYiUoCqwwgrI0Q2cmVExzBCRN6n6jCi08qVEavCLSFSGKf2EpGCVB1GWBkhsuGYESJSkKrDiH02jYVhhFTOwjEjRKQcVYcRVkaIbOyVEYYRIvI+VYcRx5gRhhFSOY4ZISIFqTqMsDJCZMMxI0SkIFWHEcc6I5xNQyrHyggRKUjVYYSVESIbOYxwnREiUoCqwwjPTUNkwwGsRKQgVYcRVkaIbHiiPCJSkKrDiH02DdcZIbWzMIwQkXJUHUZYGSGysVdG/JRtBxGpkqrDCM9NQ2TDqb1EpCBVhxG9jpURIgCc2ktEilJ1GOFsGiIbDmAlIgWpOoxwzAiRjX2dEY4ZISLvU3UY4blpiGw4ZoSIFKTqMMLKCJENx4wQkYJUHUY4m4bIxsJuGiJSjqrDiN42gNXMRc9I7axm8VtnULYdRKRKqg4jHDNCBMBqBSRbdVDLMEJE3qfqMCKvM8IwQqomWRyXtTrl2kFEqqXqMKLVcAArkb2LBgC0euXaQUSqpeowomc3DRFgqXNcZhghIgWoOozo7FN7OZuGVIyVESJSmKrDCMeMEAGwcswIESlL3WGEi54ROSojWj1gG0dFRORNqg4j9hPlcZ0RUjPnMEJEpABVhxFWRogAWG0DWBlGiEghqg4jXPSMCI4xIxwvQkQKUXUY0XM2DZFTNw1XXyUiZag6jMiVEasEWFkdIbXimBEiUpiqw4h8ojwAsEgMI6RSDCNEpDBVhxGdzjGNkeNGSLUschjhmBEiUoaqw4g8ZgRgGCEVY2WEiBSm6jCicwojnN5LqiWHER0HsBKRMtQdRjSsjBCxMkJESlN1GNFqNZCLI5zeS6pl5ZgRIlKWqsMI4JhRw8oIqRYrI0SkMNWHEXnciJnnpyG14qJnRKQw1YcRPZeEJ7VjZYSIFKb6MCKvNcLZNKRaPDcNESlM9WGElRFSPQvP2ktEylJ9GNHxZHmkduymISKFqT6McDYNqR4XPSMihak+jDgqIwwjpFJcZ4SIFKb6MMIxI6R69gGs7KYhImWoPoxwnRFSPSsHsBKRshhGWBkhteMAViJSmOrDiF7H2TSkcgwjRKQw1YcRHWfTkNpxzAgRKUz1YUTP2TSkdqyMEJHCVB9GOGaEVI8rsBKRwlQfRlgZIdWzL3rGMEJEymhVGFm8eDGSk5Ph7++P1NRUbN68uVn327p1K/R6PYYNG9aap20XjsoIB7CSSnHMCBEprMVhZMWKFZg3bx6eeuop7NmzB+PGjcPUqVORnZ3d6P1KS0sxc+ZMTJ48udWNbQ96rjNCascxI0SksBaHkddffx133nkn5syZg/79++PNN99EYmIilixZ0uj97r77btx2221IS0tr8jlqa2tRVlbm8tNeOJuGVI+LnhGRwloURkwmE3bt2oUpU6a4bJ8yZQq2bdvm8X4ff/wxTpw4geeee65Zz7No0SKEhYXZfxITE1vSzBbhmBFSPZ6bhogU1qIwUlBQAIvFgtjYWJftsbGxyMvLc3ufjIwMPPHEE/jiiy+g1zfvm9fChQtRWlpq/8nJyWlJM1tEp+NsGlI5+5gRnrWXiJTRqrqsRqNxuS5JUoNtAGCxWHDbbbfhhRdeQJ8+fZr9+EajEUajsTVNazGdhpURUjmOGSEihbXor09UVBR0Ol2DKkh+fn6DagkAlJeX47fffsOePXtw//33AwCsViskSYJer8fatWsxadKkC2j+hdNzNg2pHcMIESmsRd00fn5+SE1Nxbp161y2r1u3DmPGjGmwf2hoKA4cOIC9e/faf+bOnYu+ffti7969GD169IW1vg3oOGaE1M6+6BnHjBCRMlr8VWjBggW4/fbbMWLECKSlpeGDDz5AdnY25s6dC0CM9zhz5gw+++wzaLVaDBo0yOX+MTEx8Pf3b7BdKfKJ8iyc2ktqxXVGiEhhLf7rM2PGDBQWFuLFF19Ebm4uBg0ahNWrVyMpKQkAkJub2+SaI76ElRFSPfsKrBzASkTKaNVXoXvvvRf33nuv29s++eSTRu/7/PPP4/nnn2/N07YLPdcZIbXjmBEiUpjqz03DygipHtcZISKFqT6McDYNqR4rI0SkMNWHEVZGSPUYRohIYaoPI47KCMMIqZQ9jHAAKxEpQ/VhRD5RHisjpFocM0JEClN9GOE6I6R6XGeEiBSm+jDCMSOkevYVWBlGiEgZqg8jnE1DqsdFz4hIYaoPI6yMkOrZu2k4ZoSIlKH6MMLZNKR6nNpLRApTfRgx6MRLUFNnUbglRAphGCEihak+jCR2CQQAZBVWKdwSIoVYOYCViJSl+jCSEhMMADhVWIlaM6sjpEKc2ktEClN9GIkOMSLEXw+rBGQWVCrdHCLvYzcNESlM9WFEo9HYqyPH8ysUbg2RAhhGiEhhqg8jAJASEwIAyDjHMEIqxEXPiEhhDCMAerMyQmpltQKwTWtnGCEihTCMAOgdK8JIRn65wi0h8jK5iwYAdAwjRKQMhhEAvaLkGTVVkCQufkYq4hxGWBkhIoUwjACICTUCAExmK8pqzE3sTdSJMIwQkQ9gGAHgb9AhxF/8IT5fXqtwa4i8iGGEiHwAw4hNTIiojjCMkKo4hxEN/xwQkTL418cm2hZG8strFG4JkRfZ1xgxABqNsm0hItViGLGJDvEHwMoIqQwXPCMiH8AwYhMdbOumqWAYIRVhGCEiH8AwYiPPqDlfxjBCKmKRw4hO2XYQkaoxjNiwMkKqJFdGdAZl20FEqsYwYhPN2TSkRuymISIfwDBiwzBCqsQwQkQ+gGHERl5npLDShDqLVeHWEHmJ1SJ+c40RIlIQ/wLZRAT6QacV6ywUVpgUbg2Rl7AyQkQ+gGHERqvV2Aexni2tVrg1RF4i2SojnE1DRApiGHHSO0acvfdYXrnCLSHyErmbhpURIlIQw4iTfnEhAIAjDCOkFnI3jYaVESJSDsOIk/7xoQCA9NwyhVtC5CWSbbC2ln8KiEg5/AvkRA4jh3PLIEmSwq0h8gL7bBpWRohIOQwjTnrFBEGv1aC8xoyzpTx7L6kAB7ASkQ9gGHFi1Ovsg1gPn2VXDakAKyNE5AMYRuoZkCC6anZnFyvcEiIvYGWEiHwAw0g941OiAQA/H8lXuCVEXsAVWInIB/AvUD0T+0ZDp9XgSF45coqqlG4OUfviOiNE5AMYRuoJD/TDiKQIAMCPh88p3BqidsZuGiLyAQwjblzWPxYAsPHYeYVbQtTOOICViHwAw4gbI5O7AAD2ny7leiPUubEyQkQ+gGHEjX5xIdBrNSiqNCGX641QZ8YBrETkA/gXyA1/gw4pseI8NQfOlCrcGqJ2ZF8OnpURIlIOw4gHg2zrjRxkGKHOjGNGiMgHMIx4MLhbGABWRqiT45gRIvIBDCMeDOoqwsjBMxzESp2Y1Sx+c50RIlIQw4gHA+JDodNqUFBhQl4ZB7FSJ8VuGiLyAQwjHvgbdEixnTTvwGl21VAnZe+m4Z8CIlKOuv8CfTsX+FMM8OuHbm+2d9XwDL7UWVlts2lYGSEiBak7jFjNgKUWsNS5vXmw07gRok6JA1iJyAeoO4zI3wblQXz1yJURzqihTotjRojIB6g7jMgzCORvh/UMiA+FVgOcL6/FOQ5ipc6IlREi8gEqDyONV0YC/HToGycWP9t5qshbrSLyHi4HT0Q+QN1/gexhxH1lBADG9ooEAGw+VuCNFhF5l/xvn+uMEJGCVB5GbH+AGwkjl6REAQC2HC/g4mfU+bCbhoh8gLrDSBMDWAFgdHIk/HRanCmpRmZBpZcaRuQlHMBKRD5A3WGkiTEjgBg3MqJHBABgcwa7aqiTYWWEiHyAysOIPJvG2uhuclcNwwh1OqyMEJEPUHkYaboyAgDjekcDAHacLESdpfHgQtShcDl4IvIB6v4LZB/A2ngYGZgQiohAAypqzdibU9L+7SLyFi4HT0Q+oFVhZPHixUhOToa/vz9SU1OxefNmj/uuXLkSl19+OaKjoxEaGoq0tDT873//a3WD21QzZtMAgFarwdje7KqhTkgO4hwzQkQKanEYWbFiBebNm4ennnoKe/bswbhx4zB16lRkZ2e73X/Tpk24/PLLsXr1auzatQuXXnoppk2bhj179lxw4y+YvNBTE5URABifIrpqvt93FhYrp/hSJyFxnREiUl6Lw8jrr7+OO++8E3PmzEH//v3x5ptvIjExEUuWLHG7/5tvvonHHnsMI0eOREpKCv7yl78gJSUF//nPfzw+R21tLcrKylx+2kUzKyMAMHVwHEL99ThZUIm1h/Lapz1E3sYBrETkA1oURkwmE3bt2oUpU6a4bJ8yZQq2bdvWrMewWq0oLy9Hly5dPO6zaNEihIWF2X8SExNb0szma+LcNM5C/A2YNaYHAGDxhhNcAI06Bw5gJSIf0KK/QAUFBbBYLIiNjXXZHhsbi7y85lULXnvtNVRWVuLmm2/2uM/ChQtRWlpq/8nJyWlJM5uvmbNpZLPH9IC/QYsDZ0o5doQ6Bw5gJSIf0KqvQxqNxuW6JEkNtrnz5Zdf4vnnn8eKFSsQExPjcT+j0YjQ0FCXn3bRgm4aAIgMNuLWUd0BAIs3HG+fNhF5Exc9IyIf0KIwEhUVBZ1O16AKkp+f36BaUt+KFStw55134l//+hcuu+yylre0PbSwMgIAd43rCYNOgx0ni7DhaH47NYzISzhmhIh8QIvCiJ+fH1JTU7Fu3TqX7evWrcOYMWM83u/LL7/E7Nmz8c9//hNXX31161raHjRNn7W3voTwAPx+dBIA4LGv96OkytQeLSPyDlZGiMgHtLibZsGCBVi6dCmWLVuGw4cPY/78+cjOzsbcuXMBiPEeM2fOtO//5ZdfYubMmXjttddw8cUXIy8vD3l5eSgtLW27o2itFgxgdfb4lf3QMyoI+eW1eOq7gxzMSh2XfZ0RTu0lIuW0OIzMmDEDb775Jl588UUMGzYMmzZtwurVq5GUJKoFubm5LmuOvP/++zCbzbjvvvsQHx9v/3nooYfa7ihaqxXdNIA4ed4bM4ZBp9Xgv/tzsWrf2XZoHJEX2AewcjYNESmnVV+H7r33Xtx7771ub/vkk09crm/YsKE1T+EdzVwO3p2hieF4YFJvvPljBp757iBG9uiChPCANm4gUTtjNw0R+QB1fx2yV0Zad/K7+y7tjaGJ4SirMePRr/fBypVZqaPhAFYi8gHqDiOa1nXTyAw6Ld64eSj8DVpsPV6IT7efaru2EXkDKyNE5APUHUYuoJtG1jM6GE9dPQAA8PIPR5BxrrwtWkbkHayMEJEPYBgBWjybpr7/G90dE/pEo9Zsxfx/7YXJ3LpuHyKv43LwROQD1P0XSNv8s/Y2RqPR4NUbhyA80ICDZ8rw1/8daYPGEXkBl4MnIh+g8jDSsuXgGxMb6o+XbxgCAPhwcya+3XP6gh+TqN1xnREi8gEMI0CbhBEAuHJQHO6Z2AsA8PC/9uFfv7XTCf6I2goHsBKRD1B3GLnA2TTuPDKlL25K7QarJJaLX7r5ZJs9NlGb4wBWIvIB6g4jbTCbpj6dVowfuWtcMgDgpf8eZiAh38XKCBH5AJWHEdsfYKltZ79oNBo8eVV/zL+sDwARSP61k1025IO4HDwR+QB1/wVq5blpmkOj0eDByb3xx/E9AQBPrNyPj7ZkoqK27Z+LqNVYGSEiH6DyMNL23TTONBoNFk7th1tGJsIqAX/6Ph2T/rYBx/Mr2uX5iFqMY0aIyAeoO4zYB7C2zWwat0+h0eDPvxuMZ64ZgG4RAcgvr8Xvl+5AVmFluz0nUbPZp/YyjBCRctQdRtp4aq8nOq0Gd16SjH/fNxZ9YoNxrqwWt334C37NLIIk8eR6pCB7Nw3XGSEi5ag8jMgDWNs3jMgig434fM5o9IwKwpmSatz8/nbc+N52VklIORzASkQ+QN1/gdpxAKsnMSH+WP7Hi3FTajcEGHTYlVWMa97egl8zi7D+SD5q6rwTjIgAcAArEfkElYeR9h3A6klMqD/+etNQrFswHsMSw1FeY8bN72/HHz7Zifu+2M2uG/IeDmAlIh/AMAK0+5gRT7pFBOLTO0ahX1yIfdtPR/Lx/qaT2JNdjFX7zsJiZTChdsTKCBH5AHWPWrN/G5RE37kCp1EPCzBg5b1jkHGuAttPFuLlH47g5R8cZ/1NP1uGJ6b283q7SCVYGSEiH6DuMOL8bdBqBrR+ijQj0E+PoYnhGNQ1DBarhI+3nkJFbR1q6qx4f9MJFFeacPPIRKQmRUCSJOzNKcGAhFAY9fwAoQtgtQKwVd5YGSEiBbGbRualGTWN0Wk1uO/S3tj51GSkv3AlZoxIhCQBK37Lwe+X7sCpgkr8e+9Z/G7xNvzf0l842JUujPO/ec6mISIFsTIi8/Ig1sZoNBpoNMBLvxuE0T274LPtWdibU4InVu5HWIABALDzVDHu+XwXBiSEYl36ObwyfQgu6h6hcMupQ3EeK8V1RohIQer+OuT8B9iHwojMoNPihuHd8NYtF8HfoMWOk0X46XC+/fb1R8/j3fUncOxcBeav2Itqk2ul5HBuGasn5JlzZYTdNESkIHWHEedBe9a2PXNvW+oeGYjrhnYFAJitEox6LVb88WLEh/nDqNciMsgPpwqrMG/FHpRW1wEAPt12ClP/vhl3ffYbrJyRQ+44V0Y4gJWIFKTu2qxWC0ADMZvG9yojzm4c0Q0rfssBAAzvHoHRPSOx4dGJqDFZsfd0Ce78ZCf+d+gc1h/9EcMSw7E3pwQAsDmjAG//fBwPTu4NANh3uhQ9o4Og12pQUWNGTKi/UodESmNlhIh8hLrDCCC6aqx1Ph9GRiRFICkyEFmFVbi4ZyQAwKjXwajXYUKfaPxrbhoe+WofTp6vxK+ZRQCAruEBOFNSjTd+PIb1R/MREWjA+qPnERVshMVqRaXJgn/dnYZhieEAgIxz5dibU4LLB8QiPNB1ZlGVyYxAv6b/udSaLfjT9+kYnRyJaUMT2vZFoLblXA1kZYSIFMQwotWJMOIDs2kao9Fo8MK1A/H5jizcOjqxwe3Du0fgpwUTcKqwCttPFOJ0cRXuGtcTX+7Mxls/ZdgrJQBQUFFrv/zifw7hm3vGIK+sBje/vx3FVXXwN2jx4cwRGJcSDQD4etdpPPLVPrw6fQhuHtnwuZ2tOZiHz3dkY83Bc7hmSDw0Gk3bvADU9lwqI+rusSUiZTGMKLQkfGtM7BuDiX1jPN6u0WiQHBWE5Kgg+7Z7J/bG9OHdsHL3GRw8U4rZY3tgX04JKmrNWLz+BHZnl2Dp5kz871Aeiqvq4KfToqbOiudXHcLa+ROg02rwj+2nAAB//ykDlw2IhVYDhPobkFVUhaQugdBqHYFj/RExwLagoha5pTVICA+w33aurAZVJotL+zzJLqzC9wfOYmZaDwQb+c+0XXDBMyLyEfwrbz9Znm9XRi5EbKg/7pnYy359ZI8uAAA/vRavrjmKP68+DAAINuqx/I8X4/dLf8GJ85Xo9eRqxIX6I6+sBgBwpqQaw/+0DgEGHVJig7H/dClSkyLw1xuHoGd0MCxWCRuPnbc/z/7TpfYwYrFKuGHxNuSX12DlPWOREhuMx77ejwEJoZg7wdE22dP/PohNx85jd1Yxls4a2WavhXzeH1Zs4AjgHC9CRApjbVbT+cOIJ/dM6IXbL04CAESHGPHFnNEY1DUMd41Ltu8jBxHnz+7qOgv2ny4FAOzKKsb1727FvV/sQq8nV6O4qs6+3/7TJfbL6WfLcKakGnUWCfP/tRf/3nsGq/adxStrjmD5r9l49Kt9OHhGPGadxYpNtlDz4+F8+/YLlVVYiSHPr8Wz/z5k31ZQUYsr39yEt37KsG/LL6/BxL+ux3P/Ptgmz+uz7Oel4XcSIlIW/wp1oG6atqbRaPDidQNx1eB49I0LQZcgMWh1zriesFgBi9WKt34+DgB4YFIKTuRXoG9cCOLC/LE3pwS/u6grFq0+jN3ZJVh9IM/+uP4G0dWz/uh5jEuJxtDEMGw9UWC//Xh+BZ5bJQKBJAFPrDwAAFi55wyevKo/hnQLc2nnI1/tw8p7xzRrAC0AFFWa8GP6OVwzNN7lPmsPnUN5rRmf/5KFOeOSkRQZhHXp53Akrxy5pTW4/9Le0Go1+H5fLk4VVuHU9ixMG5qAEbZKUqfDbhoi8hEMIyoOI4AIJGm9Il22+Rt0eOiyFABAZLAR64/m486xyQgLNNj3uXmEGMj6xZyL8dyqgyirNuPyAbE4eq4cw7tHYO7nu3A4twy3frgDgX46VNkWZBuaGI59OSWoqXNd1yUq2IiCilr86ft0+7ZRPbrgZEEljuSVY9wr6zEgIRQPTU7BP3/JxhWD4jC0WziKKk0YkBBqv09eaQ2mL9mGMyXVyCmuwsNT+gIATGYr9uQUAxAB6KMtmXjxukHYZxvYW1pdhwxb2Pols9D+eM+tOoQ3ZwxDSmyIfT9IcHktOizJ9h5w8CoRKYxhRO4v9/HZNEqZNaYHZo3p4fH2AD8dXr1xqMu2OosVwUY9KmrNLkEEAF6ZPhjP/fsQfskswkXdw9E/PhQnz1dgye9T8dn2LLzx4zH7vr8b3hW9Y4Jx24c7UFhpwuaMAmzOEBWWb/eegZ9OC7NVwr/vG4uYUCP8DTrMWvYrzpRUAwC+35+Lm1IT8cDyPcgqrESJUxfSV7+dxvzL+rjMMnr4q70ICzBg63FHGDl0tgxX/n0zPv3DKAzrHo6r39qM/PJaLPn9cEzuH9uq19RnsDJCRD6CYUQFA1i9zaDT4tM7RqKosg6T+sVg3oq9+M++s4gOMaJvbAj+dtNQvLLmCP4wtgdSkxxdIA9dloKuEQF4YdUh1FqsmNAnGgnhAfjRNmX5ka/24Xx5LUKMepTXmlFrFt/sn/z2AA6dLYNeq0Gt2Qo/nRYmixWZBZUY/9f1Lm3TaTXoFR2EY+cq8OHmk8jIr7DfdvBMmf2yn16Lr+em4ZU1R7D1eCGe+u4Arhocj9PFIujc/Y9d+Medo7H/dAn6xoU0OsvJZ9nHjDCMEJGyNJI8vcCHlZWVISwsDKWlpQgNDW36Di3xdipQeBz4wxogKa1tH5sAAGaLFZ9sO4WBCWENuoTcKakyobzGjMQugS7bMwsq8cPBXNw8IhHr0s9h56kirNx9xmUfjQb4/M7R+PtPGfbF35x1CfLDc9MG4KHlexttw8geEfhq7hhU1Jox+bUNOFfmWJulT2wwjp2rsIcenVaDz+4YhbG9o5o8Np+Sux94fxwQHAc8clTp1hBRJ9Tcz292Fttn06hzzIg36HVazBnXs1lBBADCA/0aBBEASI4Kwr0TeyMq2IhbR3XHK9OH2Afd+um1eGJqP3xw+wiM7R2FiX2j7fdbO388QmxrlYzsEYGrBscjIcyxDP7Y3pEI8hP/Dl68biCGdw+3jzUJNurxyvQh9vuPSIrAij+mITLIDyaLqMxYrBLu++dubMkowE3vbcOP6ecgSRJ8PuezMkJEPoLdNCofwNqRGXRa3DwiEe9tPIHHruiLOeN62m/7/egkHDpThqmD49AnNgSrHrgE7204gQcm94ZBp8UHM0fgxe/TsfNUEW4blYSFU/vDYpUwNDEcM9N6uDzPxL4x2P3s5cgtqbGPTXnxukF44Mvd+OP4Xth07DzSc8tw+7JfIElAVuEBDPwlFKcKq/DGjGH25fZ9DseMEJGPYDfNe+OAvP3A/30D9L6sbR+b2l2dxYrj+RXoFxfSqoXMJElq9QJoJrMVfnotdmcX44bF29zuY9RrsfyPF+Oi7hENbssvr4G/QYdQf4Vm5mT/AiybAkQkAw/tVaYNRNSpsZumuTiAtUMz6LToHx/a6kBxISux+unFf5/h3SMwMy0Jeq0G1w1LsD0uMLhrGGrNVrz4fbq922b7iUK8uuYIFq7cjzGLfsboP/+Ev/7vCKxWBb4TsJuGiHwEu2nYTUNt4IVrB+KxK/shwKBDr+hgJEcFYXRyF0z46wbsyS7BV7tOY83BPPxsO3ePzGy14N31J9A3LhTXevssx+ymISIfwTCi4uXgqe1oNBr7Cf0enJxi3z5nXDLe/vk4Hvt6PwBRTblmcDyC/fWY1C8Gv2QWYcmGE1iy4QSmefssx6yMEJGPYBhhZYTa0f2TeuNMcTVW7jkDf4MWn/xhFC7u6ZhVNCwxHJ9uO4XDuWWY/PpGzL+sD6Z5q0LCyggR+QiOGeGYEWpHRr0Or908FP+4cxT+c/8lLkEEENOY77xEnJjw5PlKPPyvfUg/KxZfyy+rwYqd2cgsqGyfxnE5eCLyEayMcDl4amcajQbjUqI93r7g8j64YXg3vPR9On46ko/pS7YhOsSInOIqSJI48eCz1wzEbaO7t23D5GogKyNEpDB+JWI3DSlMo9EgOSoIr944BCkxwaiusyC7SASRhDB/1NRZ8eS3B/D41/vxt/8dxaZj52Fpi9k3cjVQy+8kRKQs/hViGCEfERlsxJp543HifAWKK01Ijg5CVJARb/98HG/8eAwrfssRO64HRid3wbLZIxFkvID/whzASkQ+gmFEYysOccwI+QCdVoM+sSEu2x66LAU9o4Pw85F8aDUarDmYi18yi3DTe9vx6BV9ERNqxJMrD2Da0ASXVWibxAGsROQjGEbslRGGEfJd04Ym2GfZzExLwu0f/YL03DL84ZOd9n32nS7FHWOTIQHYnV2MYYnhMOga6YnlAFYi8hEMI+ymoQ5maGI4flwwAUu3ZOLzHVmoMjmCdHpuGT7ddgpf7TqN8X2iMSstCYF+evcnKWRlhIh8BMMIZ9NQBxQT6o8nr+qPeyf2ws5Txfhs+ylszijAU98ewL7TpQCATcfOY9Ox89BogHduHY7S6jocPFuKod3CcGNqInQcM0JEPoJhhJUR6sDCA/1w+YBYnC2pxuaMAnsQmdg3GtuOFwIATBYr7vvnbvt9/vkLsGzLKTwem4tJAPLK61CaV46+cSHunoKIqN2xs9i+6BnDCHVc41Ki7Jcv7RuND24fgV+enIzfnrkMAxPEmTL7xAZjVloSgo16HD1XjnWHzgAADpytwE3vbUNRpUmRthMRsTJiPzeNVdl2EF2A5Kgg3D2+J6pMFjx9TX/46bXw0/sBAP51dxqO51dgcNcwaLUaPHRZH/xwMBfH//sTAMAMLcpqzBj+p3UICzBAkiRc2i8G90zshX5xnk/5TUTUVhhG2E1DnYBGo8HCq/q7vS3IqMfQxHD79S5Bfvj96CRkFSUCvwA9Y0IBUSRBaXUdAODfe8/i+/25+P3o7ri4ZySqTBZkFVbiou7hmNQvtr0Ph4hUhmGEA1hJpZIi/AEAfePD8ULqQGQWVOLaYQmQJAkfbDqJ/x06h8+2Z+Gz7Vn2+2g1wJNX9UdyVBDG9o6Cv4GDX4nowjGMsDJCauU0tXfWmB4uN71/exdsOJqP7/fn4sT5CgQb9bBYJWw7UYiX/nsYABAeaMCQbuGID/WHXqdBdZ0Ft4zsjlHJXbx8IETU0TGM8Ky9pFZNTO2d2DcGE/vG2K9brRJeW3cUG4+dR1GFCWdLa7Dp2HmX+6zcfQY6rQZ9Y0Nw+YBYGA1azBiRiMhgo8dm1NRZsOHoeYztHYkQf8OFHxcRdTgMIxqGEVKpFi56ptVq8OgV/fDoFf1gsUrYeaoI2UVVOFdaA5PFivyyWny9+zQsVgnpuWVIzy0DAHy96zR6RgXh2LkKRAb74dXpQ5DitOT9U98exDe7T6NndBBuSk1Ev/gQXOoUgoio82MYYTcNqZWpUvzWe65aeKLTanBxz0hc3NN1Zddnpg1AcaUJPxzMxbFzFdh6vAAnz1fi5HnxXNlFVbhhyTZEhxhRWGFCiL8ep4urAQAnz1filTVHoNEAy2aNxMCEUPx0JB+l1XUINurRIzIIPaIC0TU8ABqNBvllNTiUW4YJKdHQajUX9loQkaIYRhhGSK0KM8TvyF5t9pDBRj2CjXr8cbx4zJyiKryy5gh6RAZhTO9IvPLDEew7XYryGvH/TZ69M6RbGIZ3j8Chs6XYeaoYd3y6E5Lk/jl6RgXh0n4x+HbPGRRVmvB/F3fHC9cOglYDrDmYh52nijEzLQk9ooJQa7bghwN5OJJXjrvH90REkB8kSYJG4xpeJEmCJIGhhkghDCPyScI4m4bU5vwx8Tuqb7s9RWKXQLxz23D79RV3p+GXzCL46bQw6DR4YuUBnC2pxl9+NxiDuobBZLbi/5b+gl9PFQEALuoejp5RwSipMiGrqApZhZU4WVCJk1sy7Y/5+Y5sfL4jG/4GLWrqxHpB/9hxCj0ig3C6uBrVdeL/9s9HzqG0ug4WKzCmVySGJoajvKYOeaU1WH80HyVVdRjdMxK3jkxEWq9IWCXAoNO4jGM5U1KNvNJq9IkNwWfbszA+JRqDu4W12+sHAN/ZQtfsMT0YlqjTYhjhWXtJjSxmoPC4uByV4rWn9TfoMKFPtP362nnjUVZTh/BAsUCbn16Lz+eMxtmSanQJ9kNovQGtFbVmrD+Sj7Xp5xDkp0O/uBC8vOYIauqsqKmzwt+gRf/4UOzJLkFGfgUAIC7UH9V1Fhw7V2F/nFX7zmLVvrMN2iefz0em12owoU80dFoNwgIM+M/+s6ipsyIhzB9nS2vwzs/HMbZ3JM6U1GD2mCTotFpoABRXmVBeY8bAhFCk9YrErqxilFTVwWKVsCenGDePSMSQbuEorarDufIapMQEQ6PR4ExJNf6z7yxyS6rx4OQUbMo4j/kr9gEQy/rPndALdRYrlu/MQXlNHWaMSMSPh88hrWcUukcGtvj9cFclIlKCRpI8FUN9R1lZGcLCwlBaWorQ0DZeEXLb28Dap4EhM4AbPmjbxybyVQXHgXdSAUMgsPCMo0LYAZnMVpTX1KGi1oyIID+EGPU4WVCJM8XViA31R0pMMA6eLcWra45iYt9oDOoahm3HC5CRX4HwQD9EB/thUNcwdI8MxH/2ncXXu07jXFltm7VPo0GDLie9VoNhieE4cKYUtWYrekYFoVdMMDYczUedRezcNTwA58trYbKIao9Oq8GstB7YcCzfPgZHp9XAYpXgp9PiD2N74LbR3ZFxrgK7souRGBEIg06Dc2U10Go1uHJgHHpGBwMA8kpr8I8dp/DZ9iyMS4nCnZf0RGm1CSVVdRjfJxpRwUaYzFZU11kQFuB5hlNZTR1KKutwuqQKR/PKcVn/WCR2EaGoymTG5owCBBh0GNotHGGBjc+UqqmzwE+n9Vr1x2KVoGOlqd019/O7VWFk8eLF+Otf/4rc3FwMHDgQb775JsaNG+dx/40bN2LBggU4dOgQEhIS8Nhjj2Hu3LnNfr52DSPbFwP/WwgMuhG48aO2fWwiX3Xkv8Dy24C4IcDczUq3xufU1Fmg02pw7Fw5tp8ohFGvxemSavSODobRoMPXu07j7vE98c2u0wCAbl0C8WP6OXQJ8oNGI8bOBBn12HmqCFmFVQj116NPbAhqzBZEBPphc0aB/bn0Wg3MVsef4VHJXZBVWGkPRFcNjkOgnx5f254LACKD/GC2SiitrkOIv94+Bqcpeq0GWq0GJrPn01/46bTo1iUAZ0uqUVNnRe+YYCR1CUR0iBH7Tpcir7QaF/eMxJG8cmQWVLrcN8Cgw/UXJSDU34Dv9+fiTIkYnOyn1+Ly/rEYldwFeWU1+PlwPuLD/TEwIRR5pbWICzPi462nEB/mj0v7xmD7yULUmq0Y0ysS3SICIEliXZtNGQUYmBCKvrEhyCyoxLmyGoQFGJDYJRDJUUEI9NMhr7QWGg2w81QRwgIMuGpwPExmK3RaDQw6LfRaDd76OQNf/pqNuRN6YUwvcV6nokoTvt1zGlcNjsfk/rEINOjwybZTeOvnDEwZEIvrhnVF1/AAGA1a7MspQe+YYPSKDnZbWbJaJZgs1gaLApotVhzJK0ePqCAEGxt2TNSaLTBbJAS5uU2WWVAJSZLswVJWZTKjstaC6BDHgPTKWjPyymrQKzoYZot4DbxdCWu3MLJixQrcfvvtWLx4McaOHYv3338fS5cuRXp6Orp3795g/8zMTAwaNAh33XUX7r77bmzduhX33nsvvvzyS0yfPr1ND6ZVfvkA+OFRYMD1wM2ftu1jE/mqza8DP70ADL4JmL5U6dZ0WpIk4XRxNSKD/RDop7dv251dgtzSanSLCESv6CBsOlaAsyXVGNItDKN7RiKnqArLtmZifEo0JvYV3Vqr9p3Fl79m45LeUZg5pgcqa834NbMIVwyMw7YTBVi0+ghOnK9Aj8ggpCZF4HRxNTQaoFtEAPLLa7Hx2Hl7hUajAYZ3j8D04d2wat8Z5BRVIzTAAA1gn5LdXAEGHYL99YgKNuJwvfvGhhrhb9Ahq7Dqgl9LbzPoNPYqlSd+Oi1CAwwAxH7y61teY4bJYkVKTDBCbed7AoCc4mqcL69FsFGP5Kgg1FmsCPU32Cs0+0+XoMYWwox6HfbmFEOn1aBPbAiMeh0qauuw46QYT9UzKgh+ei3OllQjITwAZ4qrUV5rxrgUsTJyoJ8OmzMKUFRpQp/YYJw4X4mu4QG4uGcXlFWbUWexYt/pEiSEB2Bw1zAUVNTivkt7Y0i38DZ9HdstjIwePRrDhw/HkiVL7Nv69++P66+/HosWLWqw/+OPP45Vq1bh8OHD9m1z587Fvn37sH37drfPUVtbi9paR5m0tLQU3bt3R05OTtuHkV2fAWufBPzDgUjv9Z0TKaokC6jMB8Y9ClzykNKtoTbSWNdDWU0dqmstsEgSgvz0HrtNMgsqcL7MhC7BBnQJ9MPB3DKcK63B+fJaRIb4ITEiELuzStAvPgQjkiLs432sVgnrj+bj4JlSVNdZkRQZgOuGdUWAQYdDZ0vx8+HzyDhfjjB/A0Yld0FGfgUKK0yICjHi2LkyjOsdhaN5Fcgrr8F1QxPgb9Bh7aE8mK0S6ixWnC2twcU9I7E3uwTVdRb0iAxEbKg/yqrrkFNcjVMFlaiuMyM+LABmq4SUmGCcKqzE8fxK+Om1kCSgztbl1SXQgKlD4rHi1xwEGXXQAKg0WXF5/xhsPn4e5TViDKFGA8we0wOZBZXILqrC6eJqmMxW9I4JQlZRNeoaqTB5YtBp7e1oDa0G0Gpcq2ltZdENgzBtaNc2fcyysjIkJiaipKQEYWGNDPaWWqC2tlbS6XTSypUrXbY/+OCD0vjx493eZ9y4cdKDDz7osm3lypWSXq+XTCaT2/s899xzEkTU5A9/+MMf/vCHPx38Jycnp9F80aLZNAUFBbBYLIiNdT1rZ2xsLPLy8tzeJy8vz+3+ZrMZBQUFiI+Pb3CfhQsXYsGCBfbrVqsVRUVFiIyMbNP+LjmxtUvFxcfx2HnsPHZ1UOtxAzx2Xzh2SZJQXl6OhISERvdr1dRedwsGNRYS3O3vbrvMaDTCaHRdFTI8PLwVLW2e0NBQ1f1DlfHYeexqo9ZjV+txAzx2pY+90e4ZmxbN54uKioJOp2tQBcnPz29Q/ZDFxcW53V+v1yMyMtLtfYiIiEg9WhRG/Pz8kJqainXr1rlsX7duHcaMGeP2PmlpaQ32X7t2LUaMGAGDgWfoJCIiUrsWr3S0YMECLF26FMuWLcPhw4cxf/58ZGdn29cNWbhwIWbOnGnff+7cucjKysKCBQtw+PBhLFu2DB999BEeeeSRtjuKVjIajXjuuecadAmpAY+dx642aj12tR43wGPvSMfe6kXPXn31VeTm5mLQoEF44403MH78eADA7NmzcerUKWzYsMG+/8aNGzF//nz7omePP/54ixY9IyIios6rQywHT0RERJ1Xxz0hBREREXUKDCNERESkKIYRIiIiUhTDCBERESlK1WFk8eLFSE5Ohr+/P1JTU7F5c+c6lfrzzz8PjUbj8hMXF2e/XZIkPP/880hISEBAQAAmTpyIQ4cOKdji1tu0aROmTZuGhIQEaDQafPfddy63N+dYa2tr8cADDyAqKgpBQUG49tprcfr0afi6po599uzZDf4dXHzxxS77dMRjX7RoEUaOHImQkBDExMTg+uuvx9GjR1326azve3OOvbO+70uWLMGQIUPsK4umpaXhhx9+sN/eWd9zoOlj78jvuWrDyIoVKzBv3jw89dRT2LNnD8aNG4epU6ciOztb6aa1qYEDByI3N9f+c+DAAfttr776Kl5//XW888472LlzJ+Li4nD55ZejvLxcwRa3TmVlJYYOHYp33nnH7e3NOdZ58+bh22+/xfLly7FlyxZUVFTgmmuugcVi8dZhtEpTxw4AV155pcu/g9WrV7vc3hGPfePGjbjvvvuwY8cOrFu3DmazGVOmTEFlZaV9n876vjfn2IHO+b5369YNL7/8Mn777Tf89ttvmDRpEq677jp74Ois7znQ9LEDHfg9b+pMvZ3VqFGjpLlz57ps69evn/TEE08o1KK299xzz0lDhw51e5vVapXi4uKkl19+2b6tpqZGCgsLk9577z0vtbB9AJC+/fZb+/XmHGtJSYlkMBik5cuX2/c5c+aMpNVqpTVr1nit7Req/rFLkiTNmjVLuu666zzep7Mce35+vgRA2rhxoyRJ6nrf6x+7JKnnfZckSYqIiJCWLl2qqvdcJh+7JHXs91yVlRGTyYRdu3ZhypQpLtunTJmCbdu2KdSq9pGRkYGEhAQkJyfjlltuwcmTJwEAmZmZyMvLc3kNjEYjJkyY0Oleg+Yc665du1BXV+eyT0JCAgYNGtQpXo8NGzYgJiYGffr0wV133YX8/Hz7bZ3l2EtLSwEAXbp0AaCu973+scs6+/tusViwfPlyVFZWIi0tTVXvef1jl3XU97xVZ+3t6AoKCmCxWBqc3C82NrbBSf06stGjR+Ozzz5Dnz59cO7cObz00ksYM2YMDh06ZD9Od69BVlaWEs1tN8051ry8PPj5+SEiIqLBPh3938TUqVNx0003ISkpCZmZmXjmmWcwadIk7Nq1C0ajsVMcuyRJWLBgAS655BIMGjQIgHred3fHDnTu9/3AgQNIS0tDTU0NgoOD8e2332LAgAH2D9TO/J57OnagY7/nqgwjMo1G43JdkqQG2zqyqVOn2i8PHjwYaWlp6NWrFz799FP7oKbO/ho4a82xdobXY8aMGfbLgwYNwogRI5CUlIT//ve/uOGGGzzeryMd+/3334/9+/djy5YtDW7r7O+7p2PvzO973759sXfvXpSUlOCbb77BrFmzsHHjRvvtnfk993TsAwYM6NDvuSq7aaKioqDT6Rokwfz8/AaJujMJCgrC4MGDkZGRYZ9Vo4bXoDnHGhcXB5PJhOLiYo/7dBbx8fFISkpCRkYGgI5/7A888ABWrVqF9evXo1u3bvbtanjfPR27O53pfffz80Pv3r0xYsQILFq0CEOHDsXf//53Vbznno7dnY70nqsyjPj5+SE1NRXr1q1z2b5u3TqMGTNGoVa1v9raWhw+fBjx8fFITk5GXFycy2tgMpmwcePGTvcaNOdYU1NTYTAYXPbJzc3FwYMHO93rUVhYiJycHMTHxwPouMcuSRLuv/9+rFy5Ej///DOSk5Ndbu/M73tTx+5OZ3nf3ZEkCbW1tZ36PfdEPnZ3OtR77vUhsz5i+fLlksFgkD766CMpPT1dmjdvnhQUFCSdOnVK6aa1mYcffljasGGDdPLkSWnHjh3SNddcI4WEhNiP8eWXX5bCwsKklStXSgcOHJBuvfVWKT4+XiorK1O45S1XXl4u7dmzR9qzZ48EQHr99delPXv2SFlZWZIkNe9Y586dK3Xr1k368ccfpd27d0uTJk2Shg4dKpnNZqUOq1kaO/by8nLp4YcflrZt2yZlZmZK69evl9LS0qSuXbt2+GO/5557pLCwMGnDhg1Sbm6u/aeqqsq+T2d935s69s78vi9cuFDatGmTlJmZKe3fv1968sknJa1WK61du1aSpM77nktS48fe0d9z1YYRSZKkd999V0pKSpL8/Pyk4cOHu0yL6wxmzJghxcfHSwaDQUpISJBuuOEG6dChQ/bbrVar9Nxzz0lxcXGS0WiUxo8fLx04cEDBFrfe+vXrJQANfmbNmiVJUvOOtbq6Wrr//vulLl26SAEBAdI111wjZWdnK3A0LdPYsVdVVUlTpkyRoqOjJYPBIHXv3l2aNWtWg+PqiMfu7pgBSB9//LF9n876vjd17J35fb/jjjvsf7ejo6OlyZMn24OIJHXe91ySGj/2jv6eayRJkrxXhyEiIiJypcoxI0REROQ7GEaIiIhIUQwjREREpCiGESIiIlIUwwgREREpimGEiIiIFMUwQkRERIpiGCEiIiJFMYwQERGRohhGiIiISFEMI0RERKSo/wd3+2wK0i2MgwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot training metrics\n", "\n", "plt.figure()\n", "plt.plot(history['loss'], label=\"loss\")\n", "plt.plot(history['recall'], label=\"recall\")\n", "plt.legend()\n", "plt.ylim(0,1)\n" ] }, { "cell_type": "markdown", "id": "03a3ae78", "metadata": {}, "source": [ "## Try the Model on an Example Clip" ] }, { "cell_type": "markdown", "id": "65c43bb6", "metadata": {}, "source": [ "To confirm that the model is working as expected, let's test it on an example audio file (obtained from Youtube) of someone talking and then saying the phrase \"turn on the office lights\" at the end of the clip. We'll simulate how the model would be used in production, by predicting every 80 ms (1280 samples) and plotting the predictions over time.\n", "\n", "This clip is a good sanity test to confirm the model is performing in the right way, as it contains about ~30 seconds of speech that does no contain the target phrase, but does contain related words (e.g., \"lights\") that should not result in an activation. So ideally, the model scores are low up to the end of the recording, where there should then be a spike right after the target spoken phrase." ] }, { "cell_type": "code", "execution_count": 16, "id": "d6ad350e", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:38:17.596854Z", "start_time": "2023-02-18T03:38:16.926888Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 394/394 [00:00<00:00, 19403.48it/s]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGiCAYAAAA1LsZRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAyfklEQVR4nO3df3RU9Z3/8ddMflKEqICBKGC01WpRtGG1waKttrFo3fXUrWzdI9piT1NUxKhr0bNFXb8b1+93WftjQbuC1FNbWQu27rdUSVv5oehWIygCa/lWNNEmpsEagmggmc/3j+TeyWTy496YyWfmc5+Pc3JCZu4kn8u9TF58frw/MWOMEQAAgCVx2w0AAADRRhgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVoUOI5s3b9Yll1yisrIyxWIx/eIXvxjyNZs2bVJFRYWKi4t1wgkn6P777x9OWwEAgINCh5H3339fM2fO1A9/+MNAx+/du1cXXXSR5syZo23btum2227TokWLtHbt2tCNBQAA7ol9lI3yYrGYHn/8cV166aUDHnPrrbfqiSee0O7du/3Hqqur9fLLL+u5554b7o8GAACOyM/0D3juuedUVVWV8tiFF16olStX6vDhwyooKEh7TUdHhzo6OvyvE4mE3n33XU2YMEGxWCzTTQYAACPAGKP29naVlZUpHh94MCbjYaS5uVmlpaUpj5WWlqqzs1Otra2aMmVK2mtqa2t15513ZrppAABgFDQ2Nuq4444b8PmMhxFJab0Z3sjQQL0cS5YsUU1Njf91W1ubpk2bpsbGRo0fPz5zDQUAACNm//79mjp1qsaNGzfocRkPI5MnT1Zzc3PKYy0tLcrPz9eECRP6fU1RUZGKiorSHh8/fjxhBACAHDPUFIuM1xmprKxUXV1dymMbNmzQrFmz+p0vAgAAoiV0GDlw4IC2b9+u7du3S+peurt9+3Y1NDRI6h5imT9/vn98dXW13nzzTdXU1Gj37t1atWqVVq5cqZtvvnlkzgAAgBz31l8O6j9fbNSHh7tsN8WK0GHkxRdf1JlnnqkzzzxTklRTU6MzzzxT3/3udyVJTU1NfjCRpPLycq1fv14bN27UGWecoX/6p3/S97//fV122WUjdAoAAOS22l//j/7h56/oypX/rbaDh9Oe/82ud3TtT19Sy/4PLbQu8z5SnZHRsn//fpWUlKitrY05IwAA51y58r+1ZU+rJOmk0iO0+utnqezIMZKkJ19tVvVP6iVJl886Tvf+7Uxr7Qwr6O9v9qYBAMAyr1sgFpP+8M4BXbZiqza+1qLv/3aPFj5S7x/38/q39NZfDlpqZeYQRgAAsMyoO43ccuHJOnHSWDW1fairH3pBy+r+oISR5s2aqrPLj1bCSGvr37bc2pFHGAEAwLJEovvzsUeO0dpvz9bls45TWUmxPlU2Xv/nqzN1z2Wn6YypR0qSDnSkzynJdaNS9AwAAAzM6xmJxWI68mOFOTUvZCTQMwIAgGX+nJHBDoqlHusSwggAAJZ5+SI+SKXSWE8acTCLEEYAALCu12qagcToGQEAAJnizxkZ5BjvOeNg3whhBAAAyxL0jAAAAJuSxdAH393WVYQRAAAs86PIYD0jDgcVwggAAJZ5HSODrqbxh2ncG6chjAAAYFmQQZrkBFb3EEYAALDM6+0YbJjGe9LBjhHCCAAAtpkAq2n8Yx3sGyGMAABgWbLOyGAVWN1FGAEAwLIgPSPUGQEAABmTDCPsTQMAACwItJqGnhEAAJApQVbTJJ9yL40QRgAAsMwfpnF6murACCMAAFjmr6ZhAisAALAh2Goaip4BAIAMSU5gHXqYhqJnAABgxCWCTGBlmAYAAGSMP4F1YNQZAQAAGeMP0wTZnMZBhBEAACwLVGeEYRoAAJApXr6IByh6xgRWAAAw4pK9HYPsTZNMI84hjAAAYFmg1TRMYAUAAJligqym8eeMuBdHCCMAAGQJVtMAAAArvN6OwSaw+sdmuC02EEYAALAsSDl49qYBAAAZE2wCazcHswhhBAAA24L0djCBFQAAZEyyHHzwY11CGAEAwLLk0t5B5oyMUltsIIwAAGBdz2qaQX4r+8t+HewaIYwAAGBZoJ4RP4u4l0YIIwAAWBZqNY17WYQwAgCAbck6I4OgzggAAMgUf5jG5VmqgyCMAABgmfGHaYZeTcOcEQAAMOKCDNMki55lujWjjzACAIBt/jDNYD0jsd6HOoUwAgCAZf5qmkGOoWcEAABkTJBy8Mmn3EsjhBEAACwLUvTMZYQRAAAs81bIDNozwjANAADIlCB1RpjACgAAMsYEWE0jv2fEvThCGAEAwDJ/mGaQY5JFz9xDGAEAwLJAwzTsTQMAADIlWYGV1TQAAMACbx5IPECdEQc7RggjAADY5geMQEt73YsjhBEAACwLUvRssPkkuY4wAgCARb17OgLVGXGvY4QwAgCATb3DRZDOD+PgrBHCCAAAFvWOFvFBukYYpgEAABkRdJgmeXwGG2PJsMLI8uXLVV5eruLiYlVUVGjLli2DHv/II49o5syZ+tjHPqYpU6bo61//uvbt2zesBgMA4JJEyjDNYD0jzBnxrVmzRosXL9btt9+ubdu2ac6cOZo7d64aGhr6Pf6ZZ57R/PnztWDBAu3cuVOPPfaYXnjhBV1zzTUfufEAAOS6lDkggeqMuJdGQoeRZcuWacGCBbrmmmt0yimn6L777tPUqVO1YsWKfo9//vnndfzxx2vRokUqLy/XZz/7WX3rW9/Siy+++JEbDwBArkuZwBqozkhm22NDqDBy6NAh1dfXq6qqKuXxqqoqbd26td/XzJ49W2+99ZbWr18vY4zeeecd/fznP9fFF1884M/p6OjQ/v37Uz4AAHCdw3NUBxUqjLS2tqqrq0ulpaUpj5eWlqq5ubnf18yePVuPPPKI5s2bp8LCQk2ePFlHHnmkfvCDHwz4c2pra1VSUuJ/TJ06NUwzAQDIGb17OgZdTePVGcl0gywY1gTWWJ+/LGNM2mOeXbt2adGiRfrud7+r+vp6Pfnkk9q7d6+qq6sH/P5LlixRW1ub/9HY2DicZgIAkPV6zwEJMkzjYhrJD3PwxIkTlZeXl9YL0tLSktZb4qmtrdU555yjW265RZJ0+umna+zYsZozZ47uvvtuTZkyJe01RUVFKioqCtM0AAByUuDVND2fIz+BtbCwUBUVFaqrq0t5vK6uTrNnz+73NQcPHlQ8nvpj8vLyJLm52Q8AAGEELgfPBNakmpoaPfjgg1q1apV2796tG2+8UQ0NDf6wy5IlSzR//nz/+EsuuUTr1q3TihUr9Prrr+vZZ5/VokWLdNZZZ6msrGzkzgQAgBwUPFu4O2ck1DCNJM2bN0/79u3TXXfdpaamJs2YMUPr16/X9OnTJUlNTU0pNUeuvvpqtbe364c//KFuuukmHXnkkTr//PP1L//yLyN3FgAA5KigE1hdFjqMSNLChQu1cOHCfp9bvXp12mPXX3+9rr/++uH8KAAA3Ba6zoh7fSPsTQMAgEWJ3nNGBjnO4cU0hBEAAGzqHS4GKpPR+zkHO0YIIwAA2GToGSGMAABgU2rPyMDHJYueuRdHCCMAAFiUulFeNFfTEEYAALDIq6g6VA7xV9NkuD02EEYAALDI6xkZqk/E3yjPwTRCGAEAwCI/jAzZNdJzvIN9I4QRAAAs8odphjjO4fmrhBEAAGxK9ozYbYdNhBEAACzyOjqGGqah6BkAAMiIRCLkME1GW2MHYQQAgCwQeGmvg10jhBEAACxKLu0dYphmyL6T3EUYAQDAotBFz9zrGCGMAABgkxcu4hFeTkMYAQDAIn81zRDHJSewutc1QhgBAMCiRPB68JIYpgEAACMs9N40mW2OFYQRAACs8iawDlX0rOdoB7tGCCMAAFgUtBw8Rc8AAEBGeOGC1TQAAMCKwHNG/HGajDbHCsIIAAAWeatpAhc9y3B7bCCMAABgUXI+6lDl4L3j3YsjhBEAACwKXQ4+w+2xgTACAIBFyXLwQx0ZSzneJYQRAACygMu78g6FMAIAgEXhJ7C61zVCGAEAwKKQW9MwTAMAAEaWv2vvkOXgmTMCAAAyIOhSXZdnlBBGAACwyC8HH+HfyBE+dQAA7EvOGWHXXgAAYIEJuprGqzOS6QZZQBgBAMAifwLrEMcle0Yy2Ro7CCMAAFjkD9MM1TXiHe9g3whhBAAAiwIP09AzAgAAMiHoMI3LCCMAAFiULAc/xGoaJrACAICMCFoOnmEaAACQCcly8IMfl3zevTRCGAEAwKLARc/E3jQAACADvKW6gVfTZLg9NhBGAACwKGydERcRRgAAsMhfTTPEcd7z7E0DAABGVNgJrO5FEcIIAAB2+cM0Qx3IBFYAAJAB/gTWoVbT+HVG3EsjhBEAACzyskV8qGEa7/iMtsYOwggAABYlgk4acRhhBAAAi0zQ1TQOz2AljAAAYFHg1TR9jncJYQQAAItM6I3y3IsjhBEAAKzqDhfxIbpG/L1pMt6e0UcYAQDAIhO4zoi7CCMAAFiUCLprrz9Mk+EGWUAYAQDAIqOAk0b6Hu8QwggAABaFn8Ca0eZYQRgBAMCi4BvlMYEVAABkgLdUd+jVNN4LMtseGwgjAABYxGoawggAAFaF3rXXwa6RYYWR5cuXq7y8XMXFxaqoqNCWLVsGPb6jo0O33367pk+frqKiIp144olatWrVsBoMAIBLgvaM+EXP3Msiyg/7gjVr1mjx4sVavny5zjnnHD3wwAOaO3eudu3apWnTpvX7mssvv1zvvPOOVq5cqY9//ONqaWlRZ2fnR248AAC5Lmi4cHifvPBhZNmyZVqwYIGuueYaSdJ9992np556SitWrFBtbW3a8U8++aQ2bdqk119/XUcffbQk6fjjj/9orQYAwBFeuAg6gTXye9McOnRI9fX1qqqqSnm8qqpKW7du7fc1TzzxhGbNmqV7771Xxx57rE466STdfPPN+uCDDwb8OR0dHdq/f3/KBwAALvLCxZATWOkZ6dba2qquri6VlpamPF5aWqrm5uZ+X/P666/rmWeeUXFxsR5//HG1trZq4cKFevfddwecN1JbW6s777wzTNMAAMhJQYueuWxYE1hjfeKbMSbtMU8ikVAsFtMjjzyis846SxdddJGWLVum1atXD9g7smTJErW1tfkfjY2Nw2kmAABZz19NE3TXXge7RkL1jEycOFF5eXlpvSAtLS1pvSWeKVOm6Nhjj1VJSYn/2CmnnCJjjN566y194hOfSHtNUVGRioqKwjQNAICcFLYcvItC9YwUFhaqoqJCdXV1KY/X1dVp9uzZ/b7mnHPO0Z/+9CcdOHDAf+wPf/iD4vG4jjvuuGE0GQAAdwQuB9/7NY51j4QepqmpqdGDDz6oVatWaffu3brxxhvV0NCg6upqSd1DLPPnz/ePv+KKKzRhwgR9/etf165du7R582bdcsst+sY3vqExY8aM3JkAAJCDknVGhip6lnzesSwSfmnvvHnztG/fPt11111qamrSjBkztH79ek2fPl2S1NTUpIaGBv/4I444QnV1dbr++us1a9YsTZgwQZdffrnuvvvukTsLAAByVMJbTTPEcSk9IxlrjR2hw4gkLVy4UAsXLuz3udWrV6c99slPfjJtaAcAAAQfpnEZe9MAAGCTCbc3TfdL3OobIYwAAGBR8AmsveaMZK45VhBGAACwKOhGeUrpGclYc6wgjAAAYFGyHHyIYRrH+kYIIwAAWJSgHDxhBAAAm5JzRoLt2isxTAMAAEaQCVpnxOG1v4QRAACyQLhy8BltyqgjjAAAYJEXLOJMYAUAADZ4wWLolb3u7k1DGAEAwKKEP4PVajOsIowAAGCRX/QsTDn4DLbHBsIIAAAW+cM0IXpG2JsGAACMGBOw6Bk9IwAAIKOGXE3DBFYAAJAJiUSwYZqU5wkjAABgpCTLwVtthlWEEQAALEoOuYTYm8axrhHCCAAAFgVdTdN7bxrmjAAAgBGTLAc/+HEOTxkhjAAAYFOyAGuIomeOdY0QRgAAsMgLFqGGaTLZIAsIIwAAWBS06JnLCCMAAFiUnMAaPI44NkpDGAEAwKYwwcLLKyztBQAAI8aLFUOVg5d6DeW4lUUIIwAA2JQIOIG1+5jugxzLIoQRAACsYgIrYQQAAJvC7E3jHcIEVgAAMGKSdUYCzBlhAisAABhpfp2RQD0jsZTXuIIwAgCARUHLwfcclPIaVxBGAACwKNRqmp7P7E0DAABGDOXgCSMAAGSFYHVGuj871jFCGAEAwCZ/NU2AvpFA80pyEGEEAACLkuXghz6WnhEAADDiEiHW9voTWB1bT0MYAQDAojATWP29adzKIoQRAABsClMO3lWEEQAALEr2jIQZpnELYQQAAKuCFz3zK7A6Nk5DGAEAwCIvVwRaTeO9JmOtsYMwAgCARYlQu/YygRUAAIywMMHC1UmuhBEAACwa3moat7pGCCMAAFg0rNU0bmURwggAADZ51VSDlYPvmTOSyQZZQBgBAMCiENXg6RkBAAAjL9SuvV6dEcf6RggjAABYRDl4wggAAFaFG3KhzggAABhhyZ6REMM0hBEAADBSvDkj4crBu5VGCCMAAFiUrDMyNHpGAADAiDMKsTdNoMiSewgjACKt7YPDtpuAiAtTZ8RVhBEAkXX/pj/qjLs2aMueP9tuCiKMYRrCCIAI2/Wn/TJGeq253XZTEGHhhmlSX+MKwgiAyEr0/PeyK+HWGztyS6hy8DHqjACAU7w3dLIIbEqE2LXX49otSxgBEFlej0jCtf9mIsd4wzRDH5mcM+LWPUsYARBZXghJ0DUCi8JMYHXVsMLI8uXLVV5eruLiYlVUVGjLli2BXvfss88qPz9fZ5xxxnB+LACMqATDNMgCYTbKS+7a65bQYWTNmjVavHixbr/9dm3btk1z5szR3Llz1dDQMOjr2traNH/+fF1wwQXDbiwAjCSvq5thGtjk3Ydhip65dsuGDiPLli3TggULdM011+iUU07Rfffdp6lTp2rFihWDvu5b3/qWrrjiClVWVg75Mzo6OrR///6UDwAYaQnCCLJAYhh1RlzrGwkVRg4dOqT6+npVVVWlPF5VVaWtW7cO+LqHHnpIf/zjH7V06dJAP6e2tlYlJSX+x9SpU8M0EwAC6fKHadx6Y0duCbVrr/cax27ZUGGktbVVXV1dKi0tTXm8tLRUzc3N/b5mz549+s53vqNHHnlE+fn5gX7OkiVL1NbW5n80NjaGaSYABGL8OiOWG4JI84dpLLfDpmDpoI++6c0Y02+i6+rq0hVXXKE777xTJ510UuDvX1RUpKKiouE0DQAC83pEXFsmidwUquhZhtsy2kKFkYkTJyovLy+tF6SlpSWtt0SS2tvb9eKLL2rbtm267rrrJEmJRELGGOXn52vDhg06//zzP0LzAWD4Ej09IgzTwKZQFVj7vMYVoYZpCgsLVVFRobq6upTH6+rqNHv27LTjx48frx07dmj79u3+R3V1tU4++WRt375dZ5999kdrPQB8BAmGaZAFvH1m4iHSiGu9eaGHaWpqanTllVdq1qxZqqys1I9+9CM1NDSourpaUvd8j7ffflsPP/yw4vG4ZsyYkfL6Y445RsXFxWmPA8BoYzUNskEiRBhObpTnltBhZN68edq3b5/uuusuNTU1acaMGVq/fr2mT58uSWpqahqy5ggAZIMEq2mQBULt2uvoRnnDmsC6cOFCLVy4sN/nVq9ePehr77jjDt1xxx3D+bEAMKLoGUE2oBw8e9MAiDDKwSMbhCoH77/GrZuWMAIgsgwb5SEb9Nx+QSawxhydNEIYARBZXQmGaWBfIkTRM39vmgy2xwbCCIDIYpgG2WBYu/Y6ds8SRgBEFsM0yAbJmiHBp7AyZwQAHMFqGmSDMD0jriKMAIgsr0OkiywCi8Is7XW1zghhBEBkJZjAiizg3X2BVtP0eY0rCCMAIotde5ENvPsv3ARWt+5ZwgiAyPKHaZjACotC7drrhZHMNccKwgiAyEpOYLXcEESavzdNgFkj/jGO3bOEEQCR5f2PlKW9sGkYK3udQxgBEFlUYEU28G6/MOXgqTMCAI5gmAbZIFw5+G6u5WfCCIDISpaDd+ydHTkpUNEz6owAgFsMFViRBZJFz6gzAgCR4w/TJCw3BJHmr6ZhAisARI83gbWLnhFYFK4cvPcat+5ZwgiAyPLez117Y0duSW6UxzANAESON0xDBVbYlAhVDp4JrADglORqGrvtQMSFGabp+yJHEEYARFYXG+UhC4QapvHnjGSuPTYQRgBElhdCmMAKm8Ls2usqwgiAyPKHaVjaC4vCbE3j1SJxLT4TRgBEVoKiZ8gCyQmsQWawdn9y7ZYljACIJGNMctde197ZkVP8OiPBswgb5QGAC3qvoGE1DWwa1q69jt2zhBEAkdS7NyRBGoFF3r2YF6joGXNGAMAZKWHEtf9mIqd4RfdYTQMAEWMYpkGW8HtG4mGGady6aQkjACKpd28I5eBhk3f7hQkjriGMAIik3gHEtf9lIrd492KALJKcM+LYLUsYARBJvTtDqMAKm7xeulCraRybwkoYARBJJmUCq8WGIPISieBhxONafiaMAIik3gGEYRrYFGbOiKsIIwAiiQmsyBbeMGE80ARW5owAgDN6Fzoji8CmRKgJrN1cu2UJIwAiKaUcPGkEFoWqwEqdEQBwBxVYkQ2MMX4wDrJrLz0jAOCQBKtpkAV652AmsAJAxBjqjCAL9L73As0ZSRYacQphBEAkUYEV2aD3fRhoNU3PZ4qeAYADGKZBNkgZpgk1gTVDDbKEMAIgklLKwZNGYEnqME2QOSM9dUYy1B5bCCMAIqnv0AxDNbChdw9dPMBvZHpGAMAhfTtD6B2BDb1r3ITZm8Y1hBEAkdQ3fJBFYEMi7JyRns9MYAUAB/QtdEbhM9gQejUNwzQA4I6+b+aEEdjg3XdB653FmMAKAO5I7xmx1BBEmr8vTcA04o/kOBaeCSMAIolhGmQDb5gmyL403cd1f3btbiWMAIiktDBC1wgs8G7DIJNXXUYYARBJfbMHWQQ2eD0joeeMOHa/EkYARFLfnhDqjMAGfwJr8DQiyb0ifYQRAJHUN3u49uaO3BB6AmvPZ9fuVsIIgEjqGz7oGIENXYnuz0Grr3oTXV3LzoQRAJHU1efdvO/XwGhI1hmhZwQAIidtAitdI7Ag7ARWVxFGAEQSdUaQDfylvSGLnrk2x4kwAiCSmDOCbNA1zGEa1xBGAERSItHna8f+p4nckFzaG+x4JrACgEP6Tlhlzghs8O67oBVYkxNY3bpfCSMAIolhGmSD5ARWVwdgghlWGFm+fLnKy8tVXFysiooKbdmyZcBj161bpy9+8YuaNGmSxo8fr8rKSj311FPDbjAAjIT0cvCkEYw+7z4MX4E1M+2xJXQYWbNmjRYvXqzbb79d27Zt05w5czR37lw1NDT0e/zmzZv1xS9+UevXr1d9fb0+//nP65JLLtG2bds+cuMBYLj6hg/KwcOGZJ2RYMf7e9NkqkGW5Id9wbJly7RgwQJdc801kqT77rtPTz31lFasWKHa2tq04++7776Ur//5n/9Zv/zlL/Vf//VfOvPMM/v9GR0dHero6PC/3r9/f9hmAsCg0svB22kHoi100TN6RqRDhw6pvr5eVVVVKY9XVVVp69atgb5HIpFQe3u7jj766AGPqa2tVUlJif8xderUMM0EgCGlbZTn2rs7coLXIxd+bxq37tdQYaS1tVVdXV0qLS1Neby0tFTNzc2Bvse//uu/6v3339fll18+4DFLlixRW1ub/9HY2BimmQAwJIqeIRt4t13Ue0ZCD9NIyXXOHmNM2mP9+dnPfqY77rhDv/zlL3XMMccMeFxRUZGKioqG0zQACIRde5EN/NU0Ea8HHyqMTJw4UXl5eWm9IC0tLWm9JX2tWbNGCxYs0GOPPaYvfOEL4VsKACMofQKrpYYg0rqGOYHVNaGGaQoLC1VRUaG6urqUx+vq6jR79uwBX/ezn/1MV199tX7605/q4osvHl5LAWAEpdcZoWcEo8+7DwMXPXN0b5rQwzQ1NTW68sorNWvWLFVWVupHP/qRGhoaVF1dLal7vsfbb7+thx9+WFJ3EJk/f76+973v6TOf+YzfqzJmzBiVlJSM4KkAQHB9e0KowAobvPsw6DANc0Z6zJs3T/v27dNdd92lpqYmzZgxQ+vXr9f06dMlSU1NTSk1Rx544AF1dnbq2muv1bXXXus/ftVVV2n16tUf/QwAYBjSJ7BaaggiLWydEVFnJGnhwoVauHBhv8/1DRgbN24czo8AgIximAbZwLvvAi/tdbRnhL1pAERS354Q6ozABvam6UYYARBJfcu/uzYhELkhEbbOSM/nSBc9AwBXpM0ZYWkvLEgkQi7tZZgGANzR982cYRrYEHrOiKMTWAkjACKpb88IwzSwoWuYG+W51jVCGAEQSWkTWBmmgQXDnTPiGsIIgEhiozxkg0TIXXs9rt2thBEAkdS34iphBDZ4q7qCruz1NqV17XYljACIpL7DNIQR2BB2AquHpb0A4ACW9iIbJIa9UV6mWmQHYQRAJPVdPcPSXtjg9dDFAk9gZWkvADijb/hgaS9s6PInsFpuiGURP30AUZU+Z8ROOxBtZph1RlzLzoQRAJHUd85I371qgNHg1beJB67A2o0JrADggL7/s2SYBjYkK7AGOz5ZgTUz7bGFMAIgktLrjFhqCCLNhF5NwwRWAHBG3wmsDNPABu++Cz1M41hPHmEEQCT1fS+n6BlsCLs3jasIIwAiib1pkA1CV2BlNQ0AuCM9jFhqCCItEXZvGoqeAYA72JsG2aCLcvCSCCMAIiptNQ1dI7AgkQg3TEOdEQBwCMM0yAah96ahZwQA3NE3fLC0FzaEHaZxFWEEQCT17RlxrW4DcoMJW4FVboYWwgiASEqvM2KnHYi20EXP/GEat25YwgiASOo7LNO3IiswGsIWPXN0axrCCIBo8oZp8nv+R8rSXtiQXE0T8AXe3jSO3a6EEQCR5L2Z5+e5+eaO3JDctTdcz4hrCCMAIsnrCSmId78NspoGNgx3bxrqjACAA/w9QfIYpoE9oYueUWcEANzRlej+7M8ZoWcEFiSGubTXtbuVMAIgkow/gbX7bZAsAhuGv7Q3Uy2ygzACIJL8OSP53e/unYmEzeYgorwQHHijPP9PbqURwgiASPJ+CYwtzJckfXiYMILRlwi5msZVhBEAkeT9EvhYYZ4k6cPDXTabg4himKYbYQRAJCXDSHfPyAeEEVgQegIrRc8AwB3eFBGvZ+SDQ4QRjD5/iXnQNNKDOiMA4ADvl8DYop45I53MGcHo80Jx4AqsDNMAgDu8N3N/zgg9I7AgfDl46owAgDPSe0YIIxh9oTfKc1TETx9AVHn/Ix1TwJwR2OOF4hjDNAAQPX6dkSKW9sKermEWPWMCKwA4oLNnc5ojigokUfQsG3V0dmnxo9v0ny822m5KxnjbEsQD/jaOJdOIUwgjACLp/Y5OSdLEIwolSYe6En4BKmSHn/53g36x/U/6h5+/YrspGeMXPWMCKwBEzwEvjIwr8h9jqCa7/OGddttNyDh/b5rQFVjdiiOEEQCR5IeRsckwQhXW7PLn9kP+n1375etJhOwZcRVhBEDkdHYl/Dki44rzVZTf/VZIz0h2+fOBDv/PHY4WpQtbZ8TjWjQjjACInPc7kqFjbFG+igtYUZON3mn70P/z/g8PW2xJ5rA3TTfCCIDIae/o/sVWmB9XYX7crzXCiprscfBQp5r3J8PIgQ87LbYmc5JFz8Iu7XULYQRA5Hg9I+N6qq8WF3S/FTJnJHu80Xow5et2V8NIT6qIM4EVAKLlQE/PyFg/jDBMk232tr6f8rU34dg14Zf2uokwAiByDvT0jBzRE0bGFFISPts0tX2Q8nW7o3NGvB6OoBVY/ddlojEWEUYARI43/8ALI8X5PWGEnpGs8U6v+SKStN/RYZouf2+aYMf7e9g4lkYIIwAix6u+ekRxas9IBxNYs8Y7+ztSvnZ1AmvPrgThi545lkYIIwAip72jT88IE1izTkt7d8+Id41cncBqQtYZ8VfTuJVFCCMAosfrGWECa/Zq6ekZOXHSWEnJSceu8YZp8oL+NqbOCAC4wVuZMc4bpilgzki2aWn3wsgRktztGaEcfDfCCIDI8cLI2MK+PSPMGckGBzo6/Wt04jGOhxGvzkjYYRrmjABAbvNX0/TpGWGYJju09KykGVuYp9LxxZKS83xckzAhK7D6Rc8y1SI78m03AABGm7+apqg7hHgTWAkjdm3+w5/1u/9p0dFjCyVJpeOL/aE0V+uMeEXPAi/t7ekbcSyLEEYAJBlj9JvdLZpx7HhNKRljuzkZk1xNUyApOUzDnBF7Egmjmx57WX9uTy7pnTSuyA8jri7tHW7PiBdiXMEwDQDfUzub9c2HX9TXfvS8Ojrd/cWcXE3j9YxQgdW2HW+3pQQRqft//+N6AqPrc0aCVmA9fkL36qJtDX9xKpAMK4wsX75c5eXlKi4uVkVFhbZs2TLo8Zs2bVJFRYWKi4t1wgkn6P777x9WY5H7dv1pv1Y/u9f/ZYDs8uOtb0qS3th3UA9u2Wu5NZkz0GqalvYO5zYgyxW/3f2OJOlLn5qsU6eMlySdc+LElGEaF69NcpgmWBj5q+OPUsmYAv3l4GHVv/mXTDZtVIUeplmzZo0WL16s5cuX65xzztEDDzyguXPnateuXZo2bVra8Xv37tVFF12kb37zm/rJT36iZ599VgsXLtSkSZN02WWXjchJDNeGnc3a03JAsVj3OFz3Z6V+HeseoUsYo86EUVfCqLPL+F1r6nW892cpOeO592SjhFHK63o/3/2aWD+PpX8f43+/nvnUxihhumdXJ4x3TPeBefGY8uMx5cXjPeeQUFeie017PBbzP/Li3btGxmMx5cVigccv+xrovcLIaG/rQT32YqM6E0Y/f+ktXXjqZBXkx1WQF1dM3nmZXudpUs7X+1o9x/X3nPf34TWj+3rGlBePKd5zPXvPWu89I72/tqe2J3lMPNb99+V9374z4d/v6NJv/6f7zfWUyeN13FFjAu/K6RnoGsT6bJWVMEaHuxLqShgd7jLq7EqoM9F9rTu7uv9e8vK8+yB5P6R8nReTMdLze/f53/d/P/Wa9ra+r2lHf6zX9en5e1Hy70VK/r37fx7g2IGeU8pzqd/bE+9pa2HPPVOUHx+wa3uw31mHuxJ+qXGvzsinpx+l/HhM2xvf0//61W6VHZkcour7rQb7hdj3qb4rHvp7aSzW/XjrgQ6NKcjT+DEFSvT8m+5KGHV0JtTR2aWOw92fEwkp3uvfb168+99r99fdn2O9/tz7vk95j/O/9l6fbGPfez0vnvx34/077f0elPD+PfqPe48Z/88J0/1z82LJ+867prGYtPaltyVJXzi1VF8+fYp+u7tF5508SZJUmBfX+4e6dOvaV3TipCP89zvT83fUmTBKJLz36IT/Xu19GOOds6Q+7/Xxnvd4Lwz0/X0Qj/f8ixvg90S854u03x29/p69+6b330VRflyFvYqLBB2myc+L6/xPHqPHt72tH/xuj86YeqQOdxnlxb2/27j/fp4fjw24Sqd3+PH+9LmTJ+mEnqXUoy1mQkbNs88+W5/+9Ke1YsUK/7FTTjlFl156qWpra9OOv/XWW/XEE09o9+7d/mPV1dV6+eWX9dxzz/X7Mzo6OtTRkeyua2tr07Rp09TY2Kjx48eHae6gbnnsZf361eYR+34IpiAvrsNdLKHMVhXTj9JJpUfoZ79vtN2UjJowtlDrb5jjB5L7N/5RP3z6/1luVbQVF8S14cbz/AmsnkdfaNDd/3f3AK/KffnxmLYuOV8fKwzWP7BhZ7Nq/vPlEW/HvX97ui46bcqIfs/9+/dr6tSpeu+991RSUjLwgSaEjo4Ok5eXZ9atW5fy+KJFi8y5557b72vmzJljFi1alPLYunXrTH5+vjl06FC/r1m6dKlRTwDngw8++OCDDz5y+6OxsXHQfBFqmKa1tVVdXV0qLS1Neby0tFTNzf33MDQ3N/d7fGdnp1pbWzVlSnoKW7JkiWpqavyvE4mE3n33XU2YMCHwuFoQXmIb6R6XbBOF8+Qc3RGF84zCOUrROE/OcXDGGLW3t6usrGzQ44a1tLdvIDDGDBoS+ju+v8c9RUVFKioqSnnsyCOPHEZLgxk/fryzN1FvUThPztEdUTjPKJyjFI3z5BwHNujwTI9Qq2kmTpyovLy8tF6QlpaWtN4Pz+TJk/s9Pj8/XxMmTAjz4wEAgINChZHCwkJVVFSorq4u5fG6ujrNnj2739dUVlamHb9hwwbNmjVLBQUFIZsLAABcE7rOSE1NjR588EGtWrVKu3fv1o033qiGhgZVV1dL6p7vMX/+fP/46upqvfnmm6qpqdHu3bu1atUqrVy5UjfffPPIncUwFRUVaenSpWlDQq6Jwnlyju6IwnlG4RylaJwn5zgyQi/tlbqLnt17771qamrSjBkz9G//9m8699xzJUlXX3213njjDW3cuNE/ftOmTbrxxhu1c+dOlZWV6dZbb/XDCwAAiLZhhREAAICRwt40AADAKsIIAACwijACAACsIowAAACrIh1Gli9frvLychUXF6uiokJbtmyx3aRhu+OOO3p24Ux+TJ482X/eGKM77rhDZWVlGjNmjD73uc9p586dFls8tM2bN+uSSy5RWVmZYrGYfvGLX6Q8H+ScOjo6dP3112vixIkaO3as/vqv/1pvvfXWKJ7F0IY6z6uvvjrt2n7mM59JOSbbz7O2tlZ/9Vd/pXHjxumYY47RpZdeqtdeey3lmFy/nkHOMdev5YoVK3T66af7lTgrKyv161//2n8+16+hZ6jzzPXr2J/a2lrFYjEtXrzYf2xUr+cQe+M569FHHzUFBQXmP/7jP8yuXbvMDTfcYMaOHWvefPNN200blqVLl5pPfepTpqmpyf9oaWnxn7/nnnvMuHHjzNq1a82OHTvMvHnzzJQpU8z+/fsttnpw69evN7fffrtZu3atkWQef/zxlOeDnFN1dbU59thjTV1dnXnppZfM5z//eTNz5kzT2dk5ymczsKHO86qrrjJf+tKXUq7tvn37Uo7J9vO88MILzUMPPWReffVVs337dnPxxRebadOmmQMHDvjH5Pr1DHKOuX4tn3jiCfOrX/3KvPbaa+a1114zt912mykoKDCvvvqqMSb3r6FnqPPM9evY1+9//3tz/PHHm9NPP93ccMMN/uOjeT0jG0bOOussU11dnfLYJz/5SfOd73zHUos+mqVLl5qZM2f2+1wikTCTJ08299xzj//Yhx9+aEpKSsz9998/Si38aPr+kg5yTu+9954pKCgwjz76qH/M22+/beLxuHnyySdHre1hDBRG/uZv/mbA1+Tieba0tBhJZtOmTcYYN69n33M0xs1redRRR5kHH3zQyWvYm3eexrh1Hdvb280nPvEJU1dXZ8477zw/jIz29YzkMM2hQ4dUX1+vqqqqlMerqqq0detWS6366Pbs2aOysjKVl5fr7/7u7/T6669Lkvbu3avm5uaU8y0qKtJ5552Xs+cb5Jzq6+t1+PDhlGPKyso0Y8aMnDvvjRs36phjjtFJJ52kb37zm2ppafGfy8XzbGtrkyQdffTRkty8nn3P0ePKtezq6tKjjz6q999/X5WVlU5eQyn9PD2uXMdrr71WF198sb7whS+kPD7a13NYu/bmutbWVnV1daVt7ldaWpq2qV+uOPvss/Xwww/rpJNO0jvvvKO7775bs2fP1s6dO/1z6u9833zzTRvN/ciCnFNzc7MKCwt11FFHpR2TS9d57ty5+upXv6rp06dr7969+sd//Eedf/75qq+vV1FRUc6dpzFGNTU1+uxnP6sZM2ZIcu969neOkhvXcseOHaqsrNSHH36oI444Qo8//rhOPfVU/5ePK9dwoPOU3LiOkvToo4/qpZde0gsvvJD23Gj/m4xkGPHEYrGUr40xaY/lirlz5/p/Pu2001RZWakTTzxRP/7xj/2JVS6dr2c455Rr5z1v3jz/zzNmzNCsWbM0ffp0/epXv9JXvvKVAV+Xred53XXX6ZVXXtEzzzyT9pwr13Ogc3ThWp588snavn273nvvPa1du1ZXXXWVNm3a5D/vyjUc6DxPPfVUJ65jY2OjbrjhBm3YsEHFxcUDHjda1zOSwzQTJ05UXl5eWnJraWlJS4G5auzYsTrttNO0Z88ef1WNS+cb5JwmT56sQ4cO6S9/+cuAx+SiKVOmaPr06dqzZ4+k3DrP66+/Xk888YSefvppHXfccf7jLl3Pgc6xP7l4LQsLC/Xxj39cs2bNUm1trWbOnKnvfe97Tl1DaeDz7E8uXsf6+nq1tLSooqJC+fn5ys/P16ZNm/T9739f+fn5fjtH63pGMowUFhaqoqJCdXV1KY/X1dVp9uzZllo1sjo6OrR7925NmTJF5eXlmjx5csr5Hjp0SJs2bcrZ8w1yThUVFSooKEg5pqmpSa+++mrOnrck7du3T42NjZoyZYqk3DhPY4yuu+46rVu3Tr/73e9UXl6e8rwL13Ooc+xPLl7Lvowx6ujocOIaDsY7z/7k4nW84IILtGPHDm3fvt3/mDVrlv7+7/9e27dv1wknnDC61zPkxFtneEt7V65caXbt2mUWL15sxo4da9544w3bTRuWm266yWzcuNG8/vrr5vnnnzdf/vKXzbhx4/zzueeee0xJSYlZt26d2bFjh/na176W9Ut729vbzbZt28y2bduMJLNs2TKzbds2f/l1kHOqrq42xx13nPnNb35jXnrpJXP++edn3fK6wc6zvb3d3HTTTWbr1q1m79695umnnzaVlZXm2GOPzanz/Pa3v21KSkrMxo0bU5ZDHjx40D8m16/nUOfowrVcsmSJ2bx5s9m7d6955ZVXzG233Wbi8bjZsGGDMSb3r6FnsPN04ToOpPdqGmNG93pGNowYY8y///u/m+nTp5vCwkLz6U9/OmUJXq7x1n8XFBSYsrIy85WvfMXs3LnTfz6RSJilS5eayZMnm6KiInPuueeaHTt2WGzx0J5++mkjKe3jqquuMsYEO6cPPvjAXHfddeboo482Y8aMMV/+8pdNQ0ODhbMZ2GDnefDgQVNVVWUmTZpkCgoKzLRp08xVV12Vdg7Zfp79nZ8k89BDD/nH5Pr1HOocXbiW3/jGN/z3zEmTJpkLLrjADyLG5P419Ax2ni5cx4H0DSOjeT1jxhgTri8FAABg5ERyzggAAMgehBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABY9f8B7h6mjcwkwbsAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Load data\n", "sr, dat = scipy.io.wavfile.read(\"training_tutorial_data/turn_on_the_office_lights_test_clip.wav\")\n", "\n", "# Pre-compute audio features using helper function\n", "features = F._get_embeddings(dat)\n", "\n", "# Get predictions for each window\n", "scores = []\n", "for i in tqdm(range(0, features.shape[0]-28)): # 28 is the number of timestep frames for this model\n", " window = features[i:i+28][None,]\n", " with torch.no_grad():\n", " scores.append(float(fcn(torch.from_numpy(window)).detach().numpy()))\n", " \n", "plt.figure()\n", "_ = plt.plot(scores)\n", "_ = plt.ylim(0,1)\n" ] }, { "cell_type": "markdown", "id": "6531eefd", "metadata": {}, "source": [ "Overall, the model is working well on this test clip. There are a few spikes around the word \"lights\" spoken in other contexts, but the clear activation is around the entire phrase.\n", "\n", "To make this test a little more difficult, let's arbitrarily mix the test clip with some background music from the fma-large dataset at a low signal-to-noise ratio to simulate a more realistic (and challenging) scenario. Listen to the clip below to get a more intuitive feel of what the type of audio environment this represents." ] }, { "cell_type": "code", "execution_count": 19, "id": "2f72631b", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:40:11.733430Z", "start_time": "2023-02-18T03:40:11.721719Z" } }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Load two clips and mix them\n", "_, dat = scipy.io.wavfile.read(\"turn_on_the_office_lights_test_clip.wav\")\n", "_, dat_music = scipy.io.wavfile.read(\"fma_sample/000182.wav\")\n", "dat[-20*16000:] = (dat[-20*16000:] + dat_music[0:20*16000]*.7)/2 #quick manual mixing\n", "\n", "ipd.display(ipd.Audio(dat[-16000*6:], rate=16000, normalize=True, autoplay=False))" ] }, { "cell_type": "code", "execution_count": 20, "id": "344383c7", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:40:23.392725Z", "start_time": "2023-02-18T03:40:22.770119Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 394/394 [00:00<00:00, 19051.83it/s]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGiCAYAAAA1LsZRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApJ0lEQVR4nO3df3RV1Z338c+5N8kNRRMVMD8UYrTSaqPMNKk2Uez4o3HQOuNqpzLjPAUtuMwoIkRdFllT1OWaOD5rKP0F2hF0XGMr04qVeaRK+oz8UOwzNSYWgcfyFDTRJmSCYxJAE3Lvfv5Izs39GbiBnHPuue/XWndBzj2X7M1B81l7f/feljHGCAAAwCUBtxsAAAByG2EEAAC4ijACAABcRRgBAACuIowAAABXEUYAAICrCCMAAMBVhBEAAOAqwggAAHAVYQQAALgq4zCybds23XDDDSovL5dlWfrlL395zM9s3bpV1dXVKiws1LnnnqvHH398PG0FAAA+lHEYOXz4sGbNmqUf/ehHx3X//v37dd1112n27NlqbW3VAw88oMWLF+v555/PuLEAAMB/rBM5KM+yLL3wwgu68cYb095z//33a+PGjdqzZ0/0WkNDg95++2298cYb4/3WAADAJ/Im+hu88cYbqq+vj7t27bXXau3atTp69Kjy8/OTPjMwMKCBgYHo15FIRB999JGmTJkiy7ImuskAAOAkMMaov79f5eXlCgTST8ZMeBjp6upSSUlJ3LWSkhINDQ2pp6dHZWVlSZ9pamrSQw89NNFNAwAADujo6NDZZ5+d9v0JDyOSkkYz7JmhdKMcy5YtU2NjY/Tr3t5ezZgxQx0dHSoqKpq4hgIAgJOmr69P06dP16mnnjrmfRMeRkpLS9XV1RV3rbu7W3l5eZoyZUrKz4RCIYVCoaTrRUVFhBEAALLMsUosJnyfkdraWjU3N8dd27x5s2pqalLWiwAAgNyScRg5dOiQ2tra1NbWJml46W5bW5va29slDU+xzJs3L3p/Q0OD3n//fTU2NmrPnj1at26d1q5dq3vvvffk9AAAAGS1jKdp3nzzTV155ZXRr+3ajvnz5+vpp59WZ2dnNJhIUmVlpTZt2qSlS5fqxz/+scrLy/WDH/xA3/jGN05C8wEAQLY7oX1GnNLX16fi4mL19vZSMwIAQJY43p/fnE0DAABcRRgBAACuIowAAABXEUYAAICrCCMAAMBVhBEAAOAqwggAAHAVYQQAALiKMAIAAFxFGAEAAK4ijAAAAFcRRgAAgKsIIwAAwFWEEQAA4CrCCAAAcBVhBAAAuIowAgAAXEUYAQAAriKMAAAAVxFGAACAqwgjAADAVYQRAADgKsIIAABwFWEEAAC4ijACAABcRRgBAACuIowAAABXEUYAAICrCCMAAMBVhBEAAOAqwggAAHAVYQQAALiKMAIAAFxFGAEAAK4ijAAAAFcRRgAAgKsIIwAAwFWEEQAA4CrCCAAAcBVhBAAAuIowAgAAXEUYAQAAriKMAAAAVxFGAACAqwgjAADAVYQRAADgKsIIAABwFWEEAAC4ijACAABcRRgBAACuIowAAABXEUYAAICrCCMAAMBVhBEAAOAqwggAAHAVYQQAALiKMAIAAFxFGAEAAK4ijAAAAFcRRgAAgKsIIwAAwFWEEQAA4CrCCAAAHhSJGBlj3G6GI8YVRlavXq3KykoVFhaqurpa27dvH/P+Z599VrNmzdJnPvMZlZWV6dZbb9XBgwfH1WAAAPxuYCisa763Vbc90+J2UxyRcRhZv369lixZouXLl6u1tVWzZ8/WnDlz1N7envL+1157TfPmzdOCBQu0a9cu/fznP9dvf/tbLVy48IQbDwCAH+34fwe1778O69d7DrjdFEdkHEZWrlypBQsWaOHChbrgggu0atUqTZ8+XWvWrEl5/29+8xudc845Wrx4sSorK3X55Zfr9ttv15tvvnnCjQcAwI8ODQy53QRHZRRGBgcH1dLSovr6+rjr9fX12rFjR8rP1NXV6YMPPtCmTZtkjNGBAwf0i1/8Qtdff33a7zMwMKC+vr64FwAAueIwYSS9np4ehcNhlZSUxF0vKSlRV1dXys/U1dXp2Wef1dy5c1VQUKDS0lKddtpp+uEPf5j2+zQ1Nam4uDj6mj59eibNBAAgqx0eDLvdBEeNq4DVsqy4r40xSddsu3fv1uLFi/Xd735XLS0tevnll7V//341NDSk/fOXLVum3t7e6Kujo2M8zQQAICsdybGRkbxMbp46daqCwWDSKEh3d3fSaImtqalJl112me677z5J0sUXX6zJkydr9uzZeuSRR1RWVpb0mVAopFAolEnTAADwjUODuRVGMhoZKSgoUHV1tZqbm+OuNzc3q66uLuVnjhw5okAg/tsEg0FJypn10wAAZOLIANM0Y2psbNSTTz6pdevWac+ePVq6dKna29uj0y7Lli3TvHnzovffcMMN2rBhg9asWaN9+/bp9ddf1+LFi3XJJZeovLz85PUEAACfOJxjIyMZTdNI0ty5c3Xw4EE9/PDD6uzsVFVVlTZt2qSKigpJUmdnZ9yeI7fccov6+/v1ox/9SPfcc49OO+00XXXVVfrHf/zHk9cLAAB8JNdW01gmC+ZK+vr6VFxcrN7eXhUVFbndHAAAJtT/ePL/6LX/1yNJeu/R9FtheN3x/vzmbBoAADwmdpomC8YMThhhBAAAj4ktYI34P4sQRgAA8JrY7eAZGQEAAI47EjtN42I7nEIYAQDAYw7HTdP4P44QRgAA8JjBcCT6+xzIIoQRAAC8ZHAocuybfIYwAgCAhyRueMY0DQAAcNShhDCSA1mEMAIAgJf0f5oQRlxqh5MIIwAAeEjyyIj/4whhBAAADzkymFgz4lJDHEQYAQDAQ5IKVgkjAADASeGElb0mB9IIYQQAAA8JJ8zLME0DAAAclThNQwErAABwVOLIiP+jCGEEAABPSR4ZcakhDiKMAADgIUkjIzmQRggjAAB4CNM0AADAVUzTAAAAVyXuM8KpvQAAwFHhxJERl9rhJMIIAAAeEqGAFQAAuCl5NY1LDXEQYQQAAA+hgBUAALgqeWmv/9MIYQQAAA9JPBiPkREAAOCoxGkalvYCAABHsQMrAABwFatpAACAq5JX0/g/jRBGAADwEKZpAACAq5K2g8+BNEIYAQDAQ5K2g8+BsRHCCAAAHpJ0am8k9X1+QhgBAMBDkgpYGRkBAABOYmkvAABwFQWsAADAVRSwAgAAVzFNAwAAXJU4TcNBeQAAwFGJ2cP/UYQwAgCApzBNAwAAXJU4TZMLYyOEEQAAPCRxNU3E/1mEMAIAgJcwTQMAAFyVtB18DqQRwggAAB6SODLCNA0AAHBUOGlpr//TCGEEAAAPSSxgzYEsQhgBAMBLkgpYXWqHkwgjAAB4CNvBAwAAVyWd2uv/LEIYAQDASxJHRnIgixBGAADwkuQdWP0fRwgjAAB4SNLZNP7PIoQRAAC8JBKJ/5p9RgAAgKMSp2USw4kfEUYAAPAQ9hkBAACuSlpNQwErAABwUtI+Iy61w0mEEQAAPMQeGQkGLEmMjKS1evVqVVZWqrCwUNXV1dq+ffuY9w8MDGj58uWqqKhQKBTSeeedp3Xr1o2rwQAA+JldsDoaRlxsjEPyMv3A+vXrtWTJEq1evVqXXXaZnnjiCc2ZM0e7d+/WjBkzUn7mpptu0oEDB7R27Vp99rOfVXd3t4aGhk648QAA+I1dwJoXsDSo3JimyTiMrFy5UgsWLNDChQslSatWrdIrr7yiNWvWqKmpKen+l19+WVu3btW+fft0xhlnSJLOOeecE2s1AAA+ZU/T5I2MjLADa4LBwUG1tLSovr4+7np9fb127NiR8jMbN25UTU2NHnvsMZ111lmaOXOm7r33Xn3yySdpv8/AwID6+vriXgAA5AK7gDUvOPwjOgeySGYjIz09PQqHwyopKYm7XlJSoq6urpSf2bdvn1577TUVFhbqhRdeUE9Pj+644w599NFHaetGmpqa9NBDD2XSNAAAfCGpgNXNxjhkXAWslmXFfW2MSbpmi0QisixLzz77rC655BJdd911WrlypZ5++um0oyPLli1Tb29v9NXR0TGeZgIAkHVia0ak3FhNk9HIyNSpUxUMBpNGQbq7u5NGS2xlZWU666yzVFxcHL12wQUXyBijDz74QOeff37SZ0KhkEKhUCZNAwDAF0anaXJnNU1GIyMFBQWqrq5Wc3Nz3PXm5mbV1dWl/Mxll12mP/7xjzp06FD02u9//3sFAgGdffbZ42gyAAD+NVrAOlIzkgMTNRlP0zQ2NurJJ5/UunXrtGfPHi1dulTt7e1qaGiQNDzFMm/evOj9N998s6ZMmaJbb71Vu3fv1rZt23Tffffp29/+tiZNmnTyegIAgA/YG7Cyz8gY5s6dq4MHD+rhhx9WZ2enqqqqtGnTJlVUVEiSOjs71d7eHr3/lFNOUXNzs+666y7V1NRoypQpuummm/TII4+cvF4AAOATkUji0l43W+MMy2RBZUxfX5+Ki4vV29uroqIit5sDAMCEqVz2koyRqs4q0jsf9ul//tXF+mbNdLebNS7H+/Obs2kAAPAIY0x0WiYYrRnxP8IIAAAeEY6Zk8nPoaW9hBEAADwiHBM8cqmAlTACAIBH2Cf2SjH7jLjUFicRRgAA8Ij4kZHcOZuGMAIAgEekqhnh1F4AAOCYSEwYCXBQHgAAcFrsNI296VkuzNMQRgAA8Ah7ZMSypICVOzuwEkYAAPAIe2QkaFlSdGDE/2mEMAIAgEfYBayBgGVnEWpGAACAc+x9RoKWFZ2myYGBEcIIAABeYS/jDQYsjWQRlvYCAADn2DUjgdGSkZxAGAEAwCPs1TTBANM0AADABeGYaRoxTQMAAJwWXU1jWbLX0/g/ihBGAADwjOhqmoClHNqAlTACAIBXjBawspoGAAC4IBxTwGrl0HoawggAAB6Rap8RtoMHAACOGS1glSyW9gIAAKfF7jMyWjPiYoMcQhgBAMAj4gpYR66ZHFjcSxgBAMAjwuzACgAA3BRJsbSXAlYAAOCY8MimZ4FA7DSN/xFGAADwiOjSXlbTAAAAN6ReTeP/NEIYAQDAI+JX03BQHgAAcFg4xchIDgyMEEYAAPCK2O3go6f25sDYCGEEAACPiK6msSwKWAEAgPPiClhHrrHPCAAAcExcASsjIwAAwGmjBazioDwAAOC82AJWDsoDAACOs0dG4s+mcbFBDiGMAADgEfaUTOypvbmAMAIAgEdEYkdG7Gs5MDRCGAEAwCMiMatpxGoaAADgtNGlvWIHVgAA4Lz4Tc+G0whLewEAgGPs4BHgoDwAAOCG0aW90uhaGv+nEcIIAAAeEd30zLIUGCkaiUTcbJEzCCMAAHhEdDVNIHZchJERAADgkPDIKAg7sAIAAFfEnk1j78CaA1mEMAIAgFfYS3utmAJWdmAFAACOCccUsFqjx/b6HmEEAACPMCkOysuBLEIYAQDAK8LRaZrR1TRM0wAAAMfET9NwUB4AAHCYia6mGS1gzYEsQhgBAMArYqdpoqf25sDQCGEEAACPsDc9CwaYpgEAAC6wR0EClkZ3YM2BiRrCCAAAHhGOhhFGRgAAgAsiMfuMsAMrAABwnL0dPAflAQAAV9iraQIBS/bYSA5kEcIIAABeEYnZ9IylvcewevVqVVZWqrCwUNXV1dq+fftxfe71119XXl6e/uRP/mQ83xYAAF+LpFpN4/8sknkYWb9+vZYsWaLly5ertbVVs2fP1pw5c9Te3j7m53p7ezVv3jxdffXV424sAAB+xjTNcVq5cqUWLFighQsX6oILLtCqVas0ffp0rVmzZszP3X777br55ptVW1t7zO8xMDCgvr6+uBcAAH4XXU0TV8Dq/ziSURgZHBxUS0uL6uvr467X19drx44daT/31FNP6Q9/+INWrFhxXN+nqalJxcXF0df06dMzaSYAAFkpOk0TGD25N+L/LJJZGOnp6VE4HFZJSUnc9ZKSEnV1daX8zN69e/Wd73xHzz77rPLy8o7r+yxbtky9vb3RV0dHRybNBAAgK4Vjl/aOXMuBLKLjSwcJ7LRmM8YkXZOkcDism2++WQ899JBmzpx53H9+KBRSKBQaT9MAAMhakZgdWAOB4d/nwjRNRmFk6tSpCgaDSaMg3d3dSaMlktTf368333xTra2tWrRokSQpEonIGKO8vDxt3rxZV1111Qk0HwAA/4jEHJQXjrAdfEoFBQWqrq5Wc3Nz3PXm5mbV1dUl3V9UVKSdO3eqra0t+mpoaNDnPvc5tbW16dJLLz2x1gMA4CMRk2IH1hyYqMl4mqaxsVHf+ta3VFNTo9raWv3kJz9Re3u7GhoaJA3Xe3z44Yd65plnFAgEVFVVFff5M888U4WFhUnXAQDIdeGYfUZsuTAyknEYmTt3rg4ePKiHH35YnZ2dqqqq0qZNm1RRUSFJ6uzsPOaeIwAAIJl9Nk0wYCmQQ6f2WiYLKmP6+vpUXFys3t5eFRUVud0cAAAmxA0/fE07P+zVU7d+SYcHhrTop626tPIMrb/92Ht0edHx/vzmbBoAADwifmkvO7ACAACHpTooLxfSCGEEAACPSHVQXsT71RQnjDACAIBHxB6UJ6ZpAACA0+xBkGCAg/IAAIALYvcZCXBQHgAAcFquHpRHGAEAwCNSTdPkwq5nhBEAADwidmQkugOrmw1yCGEEAACPCMcclCeW9gIAAKfZK2eCgZiaEf9nEcIIAABeMTpNo5w6KI8wAgCAR9jLeAMxBaxM0wAAAMdEUhyUlwsIIwAAeEQ45qC80R1YXWyQQwgjAAB4RPSgvMDoQXkmBxb3EkYAAPCISGT419hpGraDBwAAjgnHLu3loDwAAOC0iGEHVgAA4BJjTLRYNWCJAlYAAOCscExxSPwOrP5PI4QRAAA8ILZQ1bIsWUzTAAAAJ8XutBpkB1YAAOC0uDBicVAeAABwWGzNiGVpdJqGMAIAAJxgb3gmDU/TBHLnaBrCCAAAXpA8TWMlXfcrwggAAB4QNonTNMO/z4EsQhgBAMALIhF791V7ae/wdQ7KAwAAjrDrV4MjxSIclAcAABxlT9PYq2iYpgEAAI6yp2mCCWEkF/ZgJYwAAOABoyf2auRX9hkBAAAOsjc9C0RrRoaxtBcAADgiqYA1uprG/wgjAAB4wOg0jR1GmKYBAAAOik7TWEzTAAAAF9ihIzjyk9nKoXkawggAAB5gH5Rnj4wEcieLEEYAAPCCcGLNCAflAQAAJ0ULWKPTNMO/5kAWIYwAAOAFiTuw2jgoDwAAOMLeZ8Te9Mz+lZERAADgiHRLewkjAADAEdGlvYmn9jJNAwAAnDBawGov7WWaBgAAOGh0mmb4a3ZgBQAAjhrdgTU+jfg/ihBGAADwhMQdWO1Nz3JgYIQwAgCAF4zuwKq4XyXJ+DyREEYAAPAAkzBNY8VsfubzLEIYAQDAC8Ij0zRWwj4jkv/rRggjAAB4QDhhn5FA3MiIv+MIYQQAAA9InKaJHRqJ+DuLEEYAAPACe58Re0Ak9rw8v+/CShgBAMAD7DASDKSapnGlSY4hjAAA4AF24AimKmAljAAAgIlmF7BaCQflSUzTAAAAB4xO0wx/bYlpGgAA4CAT3YE11ciIvxFGAADwgOipvYHkMOL3k3sJIwAAeEAkqYCVaRoAAOCgyBgH5fl9nmZcYWT16tWqrKxUYWGhqqurtX379rT3btiwQV/96lc1bdo0FRUVqba2Vq+88sq4GwwAgB8lT9OMphGmaRKsX79eS5Ys0fLly9Xa2qrZs2drzpw5am9vT3n/tm3b9NWvflWbNm1SS0uLrrzySt1www1qbW094cYDAOAXydM0o/wdRaS8TD+wcuVKLViwQAsXLpQkrVq1Sq+88orWrFmjpqampPtXrVoV9/U//MM/6MUXX9S///u/60//9E9Tfo+BgQENDAxEv+7r68u0mQAAZJXIWKtpGBkZNTg4qJaWFtXX18ddr6+v144dO47rz4hEIurv79cZZ5yR9p6mpiYVFxdHX9OnT8+kmQAAZJ2xpmn8HUUyDCM9PT0Kh8MqKSmJu15SUqKurq7j+jP+6Z/+SYcPH9ZNN92U9p5ly5apt7c3+uro6MikmQAAZJ2Iid/0TBodHfF7zUjG0zRSfFqThoePEq+l8rOf/UwPPvigXnzxRZ155plp7wuFQgqFQuNpGgAAWSkSiZ+mkYbrRozk+6GRjEZGpk6dqmAwmDQK0t3dnTRakmj9+vVasGCB/u3f/k3XXHNN5i0FAMDH7ALW2DBin+Ab9vnISEZhpKCgQNXV1Wpubo673tzcrLq6urSf+9nPfqZbbrlFP/3pT3X99dePr6UAAPhY2CSPjNhhZCjs7zCS8TRNY2OjvvWtb6mmpka1tbX6yU9+ovb2djU0NEgarvf48MMP9cwzz0gaDiLz5s3T97//fX35y1+OjqpMmjRJxcXFJ7ErAABkr0gkuWYkPxDQp4pEi1v9KuMwMnfuXB08eFAPP/ywOjs7VVVVpU2bNqmiokKS1NnZGbfnyBNPPKGhoSHdeeeduvPOO6PX58+fr6effvrEewAAgA9El/bGbL0aDI6MjBBGkt1xxx264447Ur6XGDC2bNkynm8BAEBOCUeGf42dpsmza0Z8HkY4mwYAAA+ILu1NUTNy1E4qPkUYAQDAA4Yiw4EjGIgdGRn+Mc3ICAAAmHD2ipn8YEwYyZGaEcIIAAAecHQkjOTFLKcZXdrLNA0AAJhg9jRNXoACVgAA4ILRaZrRH812zQjTNAAAYMLZK2byUtSMMDICAAAmnD36kR9IrhlhaS8AAJhwduDIz6NmBAAAuCA6TROgZgQAALhgrH1GGBkBAAAT7uhI4MijZgQAALhhKNVqGmpGAACAU9hnBAAAuOpoih1Yg0G2gwcAAA4ZSnE2jR1MGBkBAAATzh79KEhxUB41IwAAYMINRkdGRqdp8qkZAQAATrFP7Y3dZ2S0ZoQwAgAAJli0ZiSQXDMSjlDACgAAJljKU3uZpgEAAE6Jntobu5qG7eABAIATjDHRwBG3z0h0O3jCCAAAmECxYSPVPiPUjAAAgAk1FBM2CtgOHgAAOC1+ZMRK+j1LewEAwIQ6GnP2TKqaEUZGAADAhBrdY8SSZcUu7aVmBAAAOCDVHiMSB+UBAACHRPcYCcT/WA6OFLNSMwIAACbUECMjAADATUejJ/YmjIxQMwIAAJxg7zNSkBBG7BN8GRkBAAATanRkJH6aJhigZgQAADggWjMSSF0zwkF5AABgQtkjI/lpakaGqBkBAAAT6Wgk9WoaakYAAIAjRndgTRwZoWYEAAA4wK4ZyU+zzwg1IwAAYEIdjaQeGcmjZgQAADgh7Q6s1IwAAAAn2DUhiZueUTMCAAAckW41DTUjAADAEUNpzqZhmgYAADjiqL2aJs0OrBSwAgCACZX+1N7hr8PUjAAAgIl0rH1GmKYBAAATKu0+I0GmaQAAgAPS7TMSZGQEAAA4wQ4bifuM2CMlxkgRHwcSwggAAC47eoyREcnfoyOEEQAAXJbu1N7YglY/140QRgAAcNnRNKtpGBkBAACOSLfPSOxIiZ/3GiGMAADgMnsKJi9hB9bYLxkZAQAAE8auGclPGBmxLCs6dUPNCAAAmDDpVtNIMXuNME0DAAAmij0FkzgyIo3WjYSZpgEAABOl75OjkqRJ+cGk93JhS3jCCAAALjLGaG/3IUnSudMmJ72fC4flEUYAAHBRd/+Aej85qoAlnTftlKT3qRkBAAAT6vcH+iVJ50yZrMJU0zTUjAAAgIn0btdwGJlZcmrK96kZSWP16tWqrKxUYWGhqqurtX379jHv37p1q6qrq1VYWKhzzz1Xjz/++Lgai+y3+499evr1/To8MOR2UwB4zNFwRP/VP+B2Mxy398BwvcjM0jRhZGSa5vcj9/lRXqYfWL9+vZYsWaLVq1frsssu0xNPPKE5c+Zo9+7dmjFjRtL9+/fv13XXXafbbrtN//qv/6rXX39dd9xxh6ZNm6ZvfOMbJ6UT47V5V5f2dh+SZUmWrJFfFf+1ZcmSFDFGQxGjcMRoKGwUMaPDZfb99u8lyYp5Txo5/tko7nOx7w9/xkpxLfnPMdE/z8iMXIwYyWjk15HfywzPNeYFLAUDgZE+RBSOSMGAFLCs6CsYkAKBkd9bVlwbMmHSjCIaGe3vOaKfv9mhoYjRL976QNdeWKr8vIDygwFZsvtlYvpp4vprf62R+1K9Z/992M0Yfp6WggFLgZHnGYjp3Oidqdse357RewLW8N+X/ecGEv7CDg+E9b//7wFJ0gWlRTr79EkKBJL/UjN9/rH3pfoz4ts+dh9NymujXwQs+9+OFXc+RtJnEv7iTsZAcib//Kzx/mOdYLF/LybuulJeT/xM/PXYz5gx3kt9PVG6/09F3z/Of2OJ7Yj7bzTVtTH+uw1HItrw1ofq7P1U50z5jGafP01TTwmpIC+gUF5AASv2c7F/Vvz/94yJ/z6Whv9btSz7/3nDv1ojv3eL3b5IxGjb3v+SJM0sSa4XkaT6L5RqzZY/6LsvvqN3u/pVflph3M+UobCJ/rtI+lkW8/DS/ayz37vy82emrFlxgmXS/etP49JLL9UXv/hFrVmzJnrtggsu0I033qimpqak+++//35t3LhRe/bsiV5raGjQ22+/rTfeeCPl9xgYGNDAwGg67u3t1YwZM9TR0aGioqJMmjum+37+tn71TtdJ+/NwfPKDgegGPwAAKZQf0P9adLnKTpuU9F44YnT/L97Wy7sOTGgbHvuri3XdRWUn9c/s6+vT9OnT9fHHH6u4uDj9jSYDAwMDJhgMmg0bNsRdX7x4sbniiitSfmb27Nlm8eLFcdc2bNhg8vLyzODgYMrPrFixwmgk/PLixYsXL168svvV0dExZr7IaJqmp6dH4XBYJSUlcddLSkrU1ZV6hKGrqyvl/UNDQ+rp6VFZWXIKW7ZsmRobG6NfRyIRffTRR5oyZcpJHZK1E9vJHnHxmlzoJ330j1zoZy70UcqNftLHsRlj1N/fr/Ly8jHvy7hmREqeozXGjBkSUt2f6rotFAopFArFXTvttNPG0dLjU1RU5Nt/RLFyoZ/00T9yoZ+50EcpN/pJH9Mbc3pmREaraaZOnapgMJg0CtLd3Z00+mErLS1NeX9eXp6mTJmSybcHAAA+lFEYKSgoUHV1tZqbm+OuNzc3q66uLuVnamtrk+7fvHmzampqlJ+fn2FzAQCA32S8z0hjY6OefPJJrVu3Tnv27NHSpUvV3t6uhoYGScP1HvPmzYve39DQoPfff1+NjY3as2eP1q1bp7Vr1+ree+89eb0Yp1AopBUrViRNCflNLvSTPvpHLvQzF/oo5UY/6ePJkfHSXml407PHHntMnZ2dqqqq0ve+9z1dccUVkqRbbrlF7733nrZs2RK9f+vWrVq6dKl27dql8vJy3X///dHwAgAActu4wggAAMDJwtk0AADAVYQRAADgKsIIAABwFWEEAAC4KqfDyOrVq1VZWanCwkJVV1dr+/btbjdp3B588MHhE4ZjXqWlpdH3jTF68MEHVV5erkmTJunP/uzPtGvXLhdbfGzbtm3TDTfcoPLyclmWpV/+8pdx7x9PnwYGBnTXXXdp6tSpmjx5sv7iL/5CH3zwgYO9OLZj9fOWW25JerZf/vKX4+7xej+bmpr0pS99SaeeeqrOPPNM3XjjjXr33Xfj7sn253k8fcz2Z7lmzRpdfPHF0Z04a2tr9atf/Sr6frY/Q9ux+pntzzGVpqYmWZalJUuWRK85+jyPcTaebz333HMmPz/f/PM//7PZvXu3ufvuu83kyZPN+++/73bTxmXFihXmC1/4guns7Iy+uru7o+8/+uij5tRTTzXPP/+82blzp5k7d64pKyszfX19LrZ6bJs2bTLLly83zz//vJFkXnjhhbj3j6dPDQ0N5qyzzjLNzc3mrbfeMldeeaWZNWuWGRoacrg36R2rn/Pnzzd//ud/HvdsDx48GHeP1/t57bXXmqeeesq88847pq2tzVx//fVmxowZ5tChQ9F7sv15Hk8fs/1Zbty40bz00kvm3XffNe+++6554IEHTH5+vnnnnXeMMdn/DG3H6me2P8dE//mf/2nOOeccc/HFF5u77747et3J55mzYeSSSy4xDQ0Ncdc+//nPm+985zsutejErFixwsyaNSvle5FIxJSWlppHH300eu3TTz81xcXF5vHHH3eohScm8Yf08fTp448/Nvn5+ea5556L3vPhhx+aQCBgXn75Zcfanol0YeQv//Iv034mG/vZ3d1tJJmtW7caY/z5PBP7aIw/n+Xpp59unnzySV8+w1h2P43x13Ps7+83559/vmlubjZf+cpXomHE6eeZk9M0g4ODamlpUX19fdz1+vp67dixw6VWnbi9e/eqvLxclZWV+uu//mvt27dPkrR//351dXXF9TcUCukrX/lK1vb3ePrU0tKio0ePxt1TXl6uqqqqrOv3li1bdOaZZ2rmzJm67bbb1N3dHX0vG/vZ29srSTrjjDMk+fN5JvbR5pdnGQ6H9dxzz+nw4cOqra315TOUkvtp88tzvPPOO3X99dfrmmuuibvu9PMc16m92a6np0fhcDjpcL+SkpKkQ/2yxaWXXqpnnnlGM2fO1IEDB/TII4+orq5Ou3btivYpVX/ff/99N5p7wo6nT11dXSooKNDpp5+edE82Pec5c+bom9/8pioqKrR//379/d//va666iq1tLQoFAplXT+NMWpsbNTll1+uqqoqSf57nqn6KPnjWe7cuVO1tbX69NNPdcopp+iFF17QhRdeGP3h45dnmK6fkj+eoyQ999xzeuutt/Tb3/426T2n/5vMyTBisywr7mtjTNK1bDFnzpzo7y+66CLV1tbqvPPO07/8y79EC6v81F/bePqUbf2eO3du9PdVVVWqqalRRUWFXnrpJX39619P+zmv9nPRokX63e9+p9deey3pPb88z3R99MOz/NznPqe2tjZ9/PHHev755zV//nxt3bo1+r5fnmG6fl544YW+eI4dHR26++67tXnzZhUWFqa9z6nnmZPTNFOnTlUwGExKbt3d3UkpMFtNnjxZF110kfbu3RtdVeOn/h5Pn0pLSzU4OKj//u//TntPNiorK1NFRYX27t0rKbv6edddd2njxo169dVXdfbZZ0ev++l5putjKtn4LAsKCvTZz35WNTU1ampq0qxZs/T973/fV89QSt/PVLLxOba0tKi7u1vV1dXKy8tTXl6etm7dqh/84AfKy8uLttOp55mTYaSgoEDV1dVqbm6Ou97c3Ky6ujqXWnVyDQwMaM+ePSorK1NlZaVKS0vj+js4OKitW7dmbX+Pp0/V1dXKz8+Pu6ezs1PvvPNO1vZbkg4ePKiOjg6VlZVJyo5+GmO0aNEibdiwQf/xH/+hysrKuPf98DyP1cdUsvFZJjLGaGBgwBfPcCx2P1PJxud49dVXa+fOnWpra4u+ampq9Ld/+7dqa2vTueee6+zzzLDw1jfspb1r1641u3fvNkuWLDGTJ0827733nttNG5d77rnHbNmyxezbt8/85je/MV/72tfMqaeeGu3Po48+aoqLi82GDRvMzp07zd/8zd94fmlvf3+/aW1tNa2trUaSWblypWltbY0uvz6ePjU0NJizzz7b/PrXvzZvvfWWueqqqzy3vG6sfvb395t77rnH7Nixw+zfv9+8+uqrpra21px11llZ1c+/+7u/M8XFxWbLli1xyyGPHDkSvSfbn+ex+uiHZ7ls2TKzbds2s3//fvO73/3OPPDAAyYQCJjNmzcbY7L/GdrG6qcfnmM6satpjHH2eeZsGDHGmB//+MemoqLCFBQUmC9+8YtxS/Cyjb3+Oz8/35SXl5uvf/3rZteuXdH3I5GIWbFihSktLTWhUMhcccUVZufOnS62+NheffVVIynpNX/+fGPM8fXpk08+MYsWLTJnnHGGmTRpkvna175m2tvbXehNemP188iRI6a+vt5MmzbN5OfnmxkzZpj58+cn9cHr/UzVP0nmqaeeit6T7c/zWH30w7P89re/Hf1/5rRp08zVV18dDSLGZP8ztI3VTz88x3QSw4iTz9MyxpjMxlIAAABOnpysGQEAAN5BGAEAAK4ijAAAAFcRRgAAgKsIIwAAwFWEEQAA4CrCCAAAcBVhBAAAuIowAgAAXEUYAQAAriKMAAAAV/1/NEoHiB/UT8gAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Pre-compute audio features using helper function\n", "features = F._get_embeddings(dat)\n", "\n", "# Get predictions for each window\n", "scores = []\n", "for i in tqdm(range(0, features.shape[0]-28)): # 28 is the number of timestep frames for this model\n", " window = features[i:i+28][None,]\n", " with torch.no_grad():\n", " scores.append(float(fcn(torch.from_numpy(window)).detach().numpy()))\n", " \n", "plt.figure()\n", "_ = plt.plot(scores)\n", "_ = plt.ylim(0,1)" ] }, { "cell_type": "markdown", "id": "078a1bc2", "metadata": {}, "source": [ "The model is now less confidant in it's prediction than before, but the score is still above a default score of 0.5 which confirms that the model at least represents a good starting point." ] }, { "cell_type": "markdown", "id": "0367ee93", "metadata": {}, "source": [ "# Export the Model" ] }, { "cell_type": "markdown", "id": "f4eb66ce", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T20:00:58.877229Z", "start_time": "2023-02-12T20:00:58.868037Z" } }, "source": [ "Now that the model is trained and passes basic performance validation tests, it can be exported to ONNX so that it can be used by the openWakeWord inference engine. With Torch, this process is quite simple." ] }, { "cell_type": "code", "execution_count": 21, "id": "75acc3bb", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:40:40.474020Z", "start_time": "2023-02-18T03:40:40.401258Z" } }, "outputs": [], "source": [ "# Export model to ONNX format\n", "\n", "output_path = \"turn_on_the_office_lights.onnx\"\n", "torch.onnx.export(fcn, args=torch.zeros((1, 28, 96)), f=output_path) # the 'args' is the shape of a single example" ] }, { "cell_type": "markdown", "id": "03bd9f1e", "metadata": {}, "source": [ "# Evaluate the Model" ] }, { "cell_type": "markdown", "id": "8c671f55", "metadata": {}, "source": [ "Let's now load in the ONNX model with openWakeWord, and use that to run some more rigorous testing. First, let's just confirm that the ONNX model works as expected." ] }, { "cell_type": "code", "execution_count": 22, "id": "295fd55c", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:41:08.824352Z", "start_time": "2023-02-18T03:41:08.615724Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/dscripka/anaconda3/envs/torch_gpu/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n", " warnings.warn(\n" ] } ], "source": [ "# Create openWakeWord instance\n", "\n", "oww = openwakeword.Model(\n", " wakeword_model_paths=[\"turn_on_the_office_lights.onnx\"],\n", " enable_speex_noise_suppression=True,\n", " vad_threshold=0.5\n", ")\n" ] }, { "cell_type": "code", "execution_count": 23, "id": "6f009d79", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:41:13.924834Z", "start_time": "2023-02-18T03:41:12.995975Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAw3UlEQVR4nO3df3BU9b3/8dduNj/4lSAggUDAWH/RcqU1aAsWW380Fq39tuMMfGun2BZmmqIg5Np+RWaqZTqN7bQM2gq0V5BxxquMF+y1t6mS3lZAsVMJSUWhaiuSAAkxCEn4lR+7n+8fm3M2mx+754Sc7IbzfMzsJJz97OazORvyyufzPp9PwBhjBAAAkCLBVHcAAAD4G2EEAACkFGEEAACkFGEEAACkFGEEAACkFGEEAACkFGEEAACkFGEEAACkVCjVHXAiEono2LFjGjNmjAKBQKq7AwAAHDDGqLW1VQUFBQoG+x//GBZh5NixYyosLEx1NwAAwADU1dVp6tSp/d4/LMLImDFjJEVfTG5ubop7AwAAnGhpaVFhYaH9e7w/wyKMWFMzubm5hBEAAIaZZCUWFLACAICUIowAAICUIowAAICUIowAAICUch1Gdu3apbvuuksFBQUKBAL63e9+l/QxO3fuVHFxsXJycnT55Zdr48aNA+krAAC4CLkOI2fOnNGsWbP061//2lH7Q4cO6Y477tC8efNUXV2thx9+WMuXL9e2bdtcdxYAAFx8XF/aO3/+fM2fP99x+40bN2ratGlat26dJGnGjBnau3evfvGLX+juu+92++UBAMBFxvOakTfeeEMlJSVxx26//Xbt3btXHR0dfT6mra1NLS0tcTcAAHBx8jyMNDQ0KD8/P+5Yfn6+Ojs71dTU1OdjysvLlZeXZ99YCh4AgIvXkFxN03PlNWNMn8ctq1atUnNzs32rq6vzvI8AACA1PF8OftKkSWpoaIg71tjYqFAopPHjx/f5mOzsbGVnZ3vdNQAAkAY8HxmZM2eOKisr447t2LFDs2fPVmZmptdfHgAApDnXYeT06dOqqalRTU2NpOiluzU1NaqtrZUUnWJZtGiR3b60tFSHDx9WWVmZDh48qM2bN2vTpk168MEHB+cVAACQxowx+mfjaXWEI73uaz7bod/u+pfqPj6bgp6lD9fTNHv37tXNN99s/7usrEySdO+992rLli2qr6+3g4kkFRUVqaKiQitXrtSTTz6pgoICPfHEE1zWCwC46IUjRg9te0svVB3RrKl5embxZ5U3Ijor0BGO6PM/+7Na2zr14Ymz+unX/y3FvU2dgLGqSdNYS0uL8vLy1NzcrNzc3FR3BwAAR/78j+P67pa99r+tQJIdCmr5c9XaceC4JOm6aWO1femNqeqmZ5z+/va8gBUAAL9qOt0uSZo4Jlsd4Yj+fqRZt63dqayMoI6eOme3m3LJyFR1MS2wUR4AAB77VEGunl3yOU0ZO0Iftbbp6KlzGj8qS1/+1CRJUiT9Jyk8xcgIAABe6coYgUBAnyzI1Y6VN+kP++tljNGtM/JVsb9eL7/TYLfzK8IIAAAeMV0pw1ric1R2SAtmx1YVtxb/9PvICNM0AAB4xNgjI33fbx0mjAAAgJQIdqUUn2cRwggAAF6JZYy+h0aCXYcjhBEAAOCFpNM0Aaudv9MIYQQAAI/0LGDtiQLWKMIIAAAeSZYx7JqRIehLOiOMAADgseRX0wxZV9ISYQQAAI9YGSPQXwFr129hakYAAIA3ukJGfyMjXNobRRgBAMAjyTIGBaxRhBEAADzGCqyJEUYAAPCIvc5Iv4ueMU0jEUYAAPCMiaWRPgXtRc+Gpj/pijACAIBHYlfT9C1gLwfv7zRCGAEAwCPJMkaARc8kEUYAAPBcoJ8K1iBX00gijAAA4Jmk0zRdH1mBFQAAeMIkW/QsaDccmg6lKcIIAAApElv0LMUdSTHCCAAAHks+TePvNEIYAQDAI/YyI0kKWH2eRQgjAAB4xXSVsPY3MsLVNFGEEQAAPGKSXE4TYAVWSYQRAAA8k3zXXqudv9MIYQQAAI8l2yiPq2kAAIAnYgWsfd/P1TRRhBEAADyStIA1yNU0EmEEAADPJAsZQbuA1d9phDACAIDH+pumscZMqBkBAACe6r+ANfqRmhEAAOCJpBvlsQKrJMIIAACeSXo1DTUjkggjAAB4JlnEYJ2RKMIIAACe63tohBVYowgjAAB4JPmiZ4yMSIQRAAA8k3zRs6521IwAAAAvJF/0jKtpJMIIAACeY2+axAgjAAB4xIoY/S16FuBqGkmEEQAAvJN00TOrmb/TCGEEAACPJIsYAWpGJBFGAADwjH1pbz/3szdNFGEEAACPBfqZp7GvphnKzqQhwggAAB5xurIqIyMAAMATyVZgDQa5mkYijAAA4JnkG+V1tWNkBAAAeKm/dUZYgTWKMAIAgEeSb5QXRc0IAADwRLKN8liBNYowAgCAV5JulNetqY9HRwgjAAB4rN9pmm53+DiLEEYAAPCKvVFev4uexT73c90IYQQAAI9YUy/JakYkf6/COqAwsn79ehUVFSknJ0fFxcXavXt3wvbPPvusZs2apZEjR2ry5Mn6zne+oxMnTgyowwAADBf2YEe/0zSxzxkZcWHr1q1asWKFVq9ererqas2bN0/z589XbW1tn+1fe+01LVq0SIsXL9Y777yjF154QW+++aaWLFlywZ0HACCdJV/0jJoRaQBhZO3atVq8eLGWLFmiGTNmaN26dSosLNSGDRv6bP/Xv/5Vl112mZYvX66ioiJ9/vOf1/e+9z3t3bv3gjsPAMBw0P+iZ7HPCSMOtbe3q6qqSiUlJXHHS0pKtGfPnj4fM3fuXB05ckQVFRUyxuj48eP6r//6L9155539fp22tja1tLTE3QAAGG6SL3oWu4NpGoeampoUDoeVn58fdzw/P18NDQ19Pmbu3Ll69tlntXDhQmVlZWnSpEkaO3asfvWrX/X7dcrLy5WXl2ffCgsL3XQTAIC0kHzRs9jnhBGXel6iZIzp97KlAwcOaPny5frRj36kqqoqvfzyyzp06JBKS0v7ff5Vq1apubnZvtXV1Q2kmwAApFSyfBHkahpJUshN4wkTJigjI6PXKEhjY2Ov0RJLeXm5brzxRv3gBz+QJF177bUaNWqU5s2bp5/85CeaPHlyr8dkZ2crOzvbTdcAAEhb/S96FvvcRIamL+nI1chIVlaWiouLVVlZGXe8srJSc+fO7fMxZ8+eVTAY/2UyMjIk+XvpWwCAfyTbtVdimsaVsrIyPfXUU9q8ebMOHjyolStXqra21p52WbVqlRYtWmS3v+uuu7R9+3Zt2LBBH3zwgV5//XUtX75cN9xwgwoKCgbvlQAAkGbsRc/6GRmJu5pmCPqTrlxN00jSwoULdeLECa1Zs0b19fWaOXOmKioqNH36dElSfX193Joj3/72t9Xa2qpf//rX+vd//3eNHTtWt9xyi372s58N3qsAACANJVnzLK7e0s8jI67DiCQtXbpUS5cu7fO+LVu29Dq2bNkyLVu2bCBfCgCAYctJvggEou38HEbYmwYAAK/1N0+jbnUj/s0ihBEAALySbJ2R7vdFCCMAAGCwJVuBVYqNjDBNAwAABp2TeGEFFcIIAADwTH/rjEixMOLjLEIYAQDAK26maQgjAADAA8kLWKkZIYwAAOAZR+uMWG097Ul6I4wAAOARJ9M0FLASRgAA8Fwg0aJnQatmhDACAAAGmXEw+WJP0/g3ixBGAADwirtFz4agQ2mKMAIAgEecLXrG1TSEEQAAPMaiZ4kRRgAA8IizaZroR0ZGAADAoHOyay8rsBJGAADwjqtFz/ybRggjAAB4xIoXiRc942oawggAAB5LVMAa7PpNTM0IAAAYdNaqquzamxhhBAAAjzhaZ8Rq6+M0QhgBAMAjTvIFK7ASRgAA8FyijfJii575N40QRgAA8Ih9NU2CNlxNQxgBAMAzzgpY49v6EWEEAACPOIkX1IwQRgAA8I61N42jpv5NI4QRAAA8lqiAlZERwggAAJ6xN8pLVDPCCqyEEQAAvOIkX9hLxfs3ixBGAADwinFQM2JdTcPICAAA8E7CRc+oGSGMAADgEbtmJEEbVmAljAAA4Bl7msbBrr2MjAAAgEHnbNGzrraMjAAAgMEWK2BNUDPSdZ9/owhhBAAAzyWapglwNQ1hBAAA7yQvYKVmhDACAIBnHC16Rs0IYQQAAK9Y8cLJ1TQ+ziKEEQAAvJawgJWaEcIIAABeMQ7Wgw8wMkIYAQDAK/Y0TYI27E1DGAEAwDNO8gU1I4QRAAA8F0i0UV7XR0ZGAADAoHMyTWPXjHjem/RFGAEAwCNWAWviS3ujHxkZAQAAKRG7tDe1/UglwggAAB6xr+x1sOiZnytYCSMAAHgs0aJn7E1DGAEAwDNGyWtGRM0IYQQAAK+wzogzhBEAADziLIxEPzIyAgAAPJNo0TNGRggjAAB4xq4ZSdDGus/4eNkzwggAAB5xcmlvgKtpCCMAAHjFSb6gZmSAYWT9+vUqKipSTk6OiouLtXv37oTt29ratHr1ak2fPl3Z2dn6xCc+oc2bNw+owwAADBvWyEiCiRrWPJNCbh+wdetWrVixQuvXr9eNN96o3/zmN5o/f74OHDigadOm9fmYBQsW6Pjx49q0aZOuuOIKNTY2qrOz84I7DwDAcOBkBVbj4zTiOoysXbtWixcv1pIlSyRJ69at0yuvvKINGzaovLy8V/uXX35ZO3fu1AcffKBx48ZJki677LIL6zUAAMOAowJWakbcTdO0t7erqqpKJSUlccdLSkq0Z8+ePh/z0ksvafbs2fr5z3+uKVOm6KqrrtKDDz6oc+fODbzXAAAMA04GO5imcTky0tTUpHA4rPz8/Ljj+fn5amho6PMxH3zwgV577TXl5OToxRdfVFNTk5YuXaqPP/6437qRtrY2tbW12f9uaWlx000AANKClS8ST9NEP1LA6lLPxVuMMf0u6BKJRBQIBPTss8/qhhtu0B133KG1a9dqy5Yt/Y6OlJeXKy8vz74VFhYOpJsAAKQJJ4ueEUYcmTBhgjIyMnqNgjQ2NvYaLbFMnjxZU6ZMUV5enn1sxowZMsboyJEjfT5m1apVam5utm91dXVuugkAQFqwAkbCdUastt53J225CiNZWVkqLi5WZWVl3PHKykrNnTu3z8fceOONOnbsmE6fPm0fe++99xQMBjV16tQ+H5Odna3c3Ny4GwAAw409TZOgTayA1b9xxPU0TVlZmZ566ilt3rxZBw8e1MqVK1VbW6vS0lJJ0VGNRYsW2e3vuecejR8/Xt/5znd04MAB7dq1Sz/4wQ/03e9+VyNGjBi8VwIAQJpxs2uvn6+mcX1p78KFC3XixAmtWbNG9fX1mjlzpioqKjR9+nRJUn19vWpra+32o0ePVmVlpZYtW6bZs2dr/PjxWrBggX7yk58M3qsAACANxQpYky965ueREddhRJKWLl2qpUuX9nnfli1beh275pprek3tAADgF4mmaYIUjbA3DQAAnnFQwBqkZoQwAgCAVxzFC3uaxsuepDfCCAAAHrEGO5ztTTMEHUpThBEAADyWaNdeVmAljAAA4Blro7xEFaxWUGEFVgAAMOhM8ixij4z4N4oQRgAA8IyzXXu5moYwAgCAR9wteuZ9f9IVYQQAAI8lnqbhahrCCAAAHnGya69dM+LjNEIYAQAghagZIYwAAOCZ2NU0yWtGfJxFCCMAAHjN2d40Q9SZNEQYAQDAI9aiZ4kKWO1Ne308NEIYAQDAI07yhX01jcd9SWeEEQAAPGIHjETLwbM3DWEEAACv2Jf2Jpyosdp63Zv0RRgBAMBjTgpYfZxFCCMAAHjFXg4+QRumaQgjAAB4x8lGeS7aXqwIIwAAeMTJRnnBoDVN4980QhgBAMBjiWpGrLsikSHpSloijAAA4JHY1TT9CwQYGSGMAADgESfxIlbA6mlX0hphBAAAj9gb5SWcpgnEtfUjwggAAJ5LUMDK5TSEEQAAvGJvlOdoOfgh6FCaIowAAOARe5omQZvYNI1/0whhBAAAjzjJF9bIiH+jCGEEAADPJVr0zLqPaRoAAOCZRNM0VgEr0zQAAGDQ2YueOShg9XEWIYwAAOAVR4ueiRVYCSMAAHgkdjVNopqR+LZ+RBgBAMBjiadprAJW/6YRwggAAB5xMvVi5RQfZxHCCAAAXnGyN03Q3rXXvwgjAAB4xM2uvVzaCwAABp2TAtYgBayEEQAAvJZomsaqGqGAFQAAeMDFomdD0Jt0RRgBAMAjTgY77AJWH6cRwggAAB6x8kXCRc+stj5OI4QRAAA8xjRNYoQRAAA8Ym+Ul6AN0zSEEQAAPOMoX3QlFa6mAQAAg87VCqz+zSKEEQAAvBIrSnVQwOp5b9IXYQQAAI85KmD18dAIYQQAAI8kHxdhmkYijAAA4B0HAcMKKhSwAgCAQWePjCScp4lv60eEEQAAPOZsmsa/cYQwAgCAR+xFz5IPjFAzAgAABp+TfBEMBhy3vVgRRgAA8Ii96Bkb5SVEGAEAwCNGDqZp7OXgh6BDaYowAgBACllX2hgfT9QMKIysX79eRUVFysnJUXFxsXbv3u3oca+//rpCoZA+/elPD+TLAgAwrDjZm8ZeZyTieXfSluswsnXrVq1YsUKrV69WdXW15s2bp/nz56u2tjbh45qbm7Vo0SLdeuutA+4sAADDiZOxjoRrkPiE6zCydu1aLV68WEuWLNGMGTO0bt06FRYWasOGDQkf973vfU/33HOP5syZM+DOAgAwrNgjI/0HjiB707gLI+3t7aqqqlJJSUnc8ZKSEu3Zs6ffxz399NP617/+pUceecTR12lra1NLS0vcDQCA4SrR2Id1pQ0FrA41NTUpHA4rPz8/7nh+fr4aGhr6fMz777+vhx56SM8++6xCoZCjr1NeXq68vDz7VlhY6KabAACkBTdX01DA6lLP4SZjTJ9DUOFwWPfcc49+/OMf66qrrnL8/KtWrVJzc7N9q6urG0g3AQBIKSczL3YY8W8WkbOhii4TJkxQRkZGr1GQxsbGXqMlktTa2qq9e/equrpa999/vyQpEonIGKNQKKQdO3bolltu6fW47OxsZWdnu+kaAABpx94oL+GiZ0zTuBoZycrKUnFxsSorK+OOV1ZWau7cub3a5+bmav/+/aqpqbFvpaWluvrqq1VTU6PPfvazF9Z7AADSmJO9aYL2b2L/phFXIyOSVFZWpm9961uaPXu25syZo9/+9reqra1VaWmppOgUy9GjR/XMM88oGAxq5syZcY+fOHGicnJyeh0HAOBi5aSAlWkaFxYuXKgTJ05ozZo1qq+v18yZM1VRUaHp06dLkurr65OuOQIAgB/Y+cLRcvD+TSMBMwwubG5paVFeXp6am5uVm5ub6u4AAODIZQ/9QZL0t9W3auKYnD7b/LOxVbet3aWxIzNV86OSPtsMV05/f7M3DQAAHktUwGoNm0R8XMFKGAEAwGMJC1jtdUb8izACAIAHuldBJBwXCZBGCCMAAHjAaUWmvWtv+pdweoYwAgCAB7pHi8Qb5QV6tfcbwggAAB5wPk1jtfe2P+mMMAIAgMcSFbBamKYBAACDymm0CAaZpiGMAADgge4DHYk3yrPa+zeOEEYAAPCAUVwa6Rc1I4QRAAA8l3jRM6ZpCCMAAHjAOBsYYZ0REUYAAEgpaw0SH2cRwggAAF6IGxlJME/T/S6/FrESRgAA8JiTaRrJv6MjhBEAADzQ/WoaJwWs0cf4E2EEAAAPON4or1tQ8WsRK2EEAAAPxG2Ul3DRs24jI/7MIoQRAAC8ELdRXqJFz7r9JjY+naghjAAAkEIUsBJGAADwRNw0jdMCVsIIAAAYLAMpYGWaBgAADB7Hu/bG7ov4M4sQRgAA8FrCAlZWYCWMAADghbhFzxK0i5+m8SfCCAAAHnBcM9J9nZGIR51Jc4QRAAA8EH81Tf9jI0EKWAkjAAB4IW7RswTtugcVClgBAIAnEq8zEvucAlYAADBonMaKALv2EkYAAPBC90GORDUj3bFrLwAAGDRuilHtqRp/ZhHCCAAAXnIyKGKNnPg0ixBGAADwRFeycDJBY7VhmgYAAAwaN7HC2rnXp1mEMAIAgBesYOGoeLWrCSMjAABg0FgFrE6maawCVp9mEcIIAABecjYw4uzS34sVYQQAAA+4GeUIME0DAAAGmxUrnIx6UMAKAAAGnb3PjPP6VdYZAQAAg89RNQjTNAAAYLDFLu1N3pZpGgAAkFKxwOLPNEIYAQDAA7GSEecFrBF/ZhHCCAAAXrAXPXNTwEoYAQAAg83RRnkUsAIAgMHmbtEzClgBAMAgsxc9czBPE1tnxJ9phDACAIAHrEXP3EzTMDICAAAGH+uMJEUYAQDAA7G9aZJjmgYA4Fg4YrTgN2/ogeerU90VpLmBFLCyzggAIKm3jpzS3w59rP+uOZbqriDtWeuMOChgtWtG/JlGCCMA4MK5jrD9uV9/ccAZN3vTxNYZ8a4/6YwwAgAutHVG7M/9+osD7jipGQkGYlUjfkQYAQAX2rqNjHRGIglawu/cxAqWgx+A9evXq6ioSDk5OSouLtbu3bv7bbt9+3Z96Utf0qWXXqrc3FzNmTNHr7zyyoA7DACp1H1kJMzQCBKITdM4qRmhgNWVrVu3asWKFVq9erWqq6s1b948zZ8/X7W1tX2237Vrl770pS+poqJCVVVVuvnmm3XXXXepuppKdADDD2EETtkb5TloSwGrS2vXrtXixYu1ZMkSzZgxQ+vWrVNhYaE2bNjQZ/t169bphz/8oa6//npdeeWV+ulPf6orr7xSv//97y+48wAw1AgjcMvVrr2e9iR9uQoj7e3tqqqqUklJSdzxkpIS7dmzx9FzRCIRtba2aty4cf22aWtrU0tLS9wNANJB95oRwggSGdg6I/58T7kKI01NTQqHw8rPz487np+fr4aGBkfP8ctf/lJnzpzRggUL+m1TXl6uvLw8+1ZYWOimmwDgmbPthBE4E8sVyYdGgj4fGhlQAWvPYhxjjKMCneeee06PPvqotm7dqokTJ/bbbtWqVWpubrZvdXV1A+kmAAy6M+2d9udhn/4VC2fsmhFH0zT+LmANuWk8YcIEZWRk9BoFaWxs7DVa0tPWrVu1ePFivfDCC7rtttsSts3OzlZ2drabrgHAkDjXbWSkM+zT3xxwxL6axkFbu4DVp0MjrkZGsrKyVFxcrMrKyrjjlZWVmjt3br+Pe+655/Ttb39b//mf/6k777xzYD0FgDRwpo1pGrjjbAVWf+/a62pkRJLKysr0rW99S7Nnz9acOXP029/+VrW1tSotLZUUnWI5evSonnnmGUnRILJo0SI9/vjj+tznPmePqowYMUJ5eXmD+FIAwHtnmaaBB6y84tcCVtdhZOHChTpx4oTWrFmj+vp6zZw5UxUVFZo+fbokqb6+Pm7Nkd/85jfq7OzUfffdp/vuu88+fu+992rLli0X/goAYAhRwAqnYtM0DgpYu+Yp/PqOch1GJGnp0qVaunRpn/f1DBivvvrqQL4EAKSluJERwggSGEgBq1/TCHvTAIALjIzALTcFrH6dpiGMAIAL3cNIJ2EECQxk0TOfZhHCCAC4caaNaRo4Y707HG2U1+MxfkMYAQAXzjFNA4fcbHoXZJoGAOCEMSZ+BVbCCBxgnZHkCCMA4FBbZyRuuW7CCBKJTdMkb2tP0/g0jRBGAMCh7sWrEoueITE3b4+gNTLiUV/SHWEEABxq6+wRRiKRFPUEw0PXOiNOLu61lhnxaRohjACAQz03xmOjPCRir8DqYpqGAlYAQEId4fiREL/+4oA7ThY9Y5oGAOBIz0XOWPQMibh5dwTsaRp/vqcIIwDgUM+REa6mQSKxaRoHG+VxaS8AwImeNSKEESRijXK42ZvG+HSihjACAA51RhgZwQA4SSNd/HqBFmEEABzqYGQELrh5d1DACgBwpGf4oIAVidg1Iw7aUsAKAHCES3vhhlX/4WrXXp++pQgjAOAQi57BFRcjI7FpGn++pwgjAOBQzwJWRkbghLNde6Mf/fqWIowAgEM9C1ipGUEi7hY9i6YRv76lCCMA4BCX9sKNWAGri5oRpmkAAIlwaS/ciBWwJm/LNA0AwJFeBayEEQyS2HLw/nxPEUYAwKFeBayEESTgJlfEloP3J8IIADhEASvcsN4dztYZ6Spg9el7ijACAA51sugZXBjYRnn+RBgBAId6joSw6BkSiY2MJG8bsGtGvOtPOiOMAIBDPZeDD/t1i1W44iSMBLva+HW0jTACAA71HAkJ+/QXBxxyU8DqXS+GBcIIADjUwaJncMFeZ8TJomdM0wAAnOg1MkIYQQL2CqwuFj1jmgYAkJB1NU1WKPpfJ5f2wglHV9PI2rXXnwgjAOCQFT6yu8KIX9eEgDMDWfSMkREAQELWNE1OZkb034QRJGC/OxzM0wTZmwYA4IRVwGqNjFAzgkRcLXrm8+tpCCMA4FDPkRHCCBJxs+hZsOu3sV+n/ggjAOBQJyMjGABnYx4UsAIAHOjoOTLi1wl+ODKgXXt9+pYijACAQ9alvTmZjIzAia6aERcFrFxNAwBIKHZpLzUjSM5e9MxBW9YZAQA40tFjZIRLe+GEmxVYDSMjAIBErKtpGBmBE27eHUH2pgEAONERsQpYqRlBcrFpGudriBifTtQQRgDAIauAlZEROGEHC1cb5XnXn3RGGAEAh+xpGkZG4ICbAlamaQAAjljLwecwMoJBZgUWpmkAAAn1HBmxVmQF+uJuOXhGRgAADvSsGWFgBInENspLnkbskRGfphHCCAA41GEvesbICJxzMjIiloMHADgRWw6+a2SELOKpi2WUwNE0TVcjv462EUYAwKFORkaGzAPPV+uWX+7UmbbOVHdlwFxtlGc9hgJWAEAinT137SWLeKKtM6z/rjmmQ01ntOu9j1LdnQGzgoWjmhGfT9OEUt0BABgurJGQ2AqspJHB9LOX/6F9h0/qh1++2j528mxHCnt0Yex1RlxM01wsU1NuEUYAwAFjjDp67E3DRnmDJxwx2vDqvyRJj/3xH/bxD0+cSVWXLtjApmn8iWkaAHCg+wJn1shIhDAyaOo+Pmt//uaHJ+3PP/ho+IaRk2fbJUljcpL/3R+wC1j9+Z4ijACAA91HQayRkY6I8e2w+mB773hrn8eH88iI1ffLxo9K2tbvNSOEEQBwoKNbteqkvByFggG1d0Z0rPl8Cnt18Xi/8XSfx2tPnB22y+5/2BQd7Sma4CCMdE3UDM9XeuEGFEbWr1+voqIi5eTkqLi4WLt3707YfufOnSouLlZOTo4uv/xybdy4cUCdBYBUsa6kkaTR2SFdPWmMJOmtulMp6tHFpefIyFX5oyVJ7eGIDg/T0ZFDTdF+OwkjQXtkxJ9xxHUY2bp1q1asWKHVq1erurpa8+bN0/z581VbW9tn+0OHDumOO+7QvHnzVF1drYcffljLly/Xtm3bLrjzAGKoX/DWu12/LMdkh5QRDOjaqXmSpL8faU5lty4KxhgdONYiSfrVNz6j1/7fzXplxU363OXjJEl//kdjKrs3IOc7wjrWfE6SdJmTkRGfT9O4vppm7dq1Wrx4sZYsWSJJWrdunV555RVt2LBB5eXlvdpv3LhR06ZN07p16yRJM2bM0N69e/WLX/xCd99994X1Pg2c7wjr/eOnFTHxS9VY6Tb+mP1Z3DFjfex6juix6B0mrp2xnyMYCCiUEVBmRkAZwaCCgeixQNfH6eNHamRWSE2n23T05Dl1hCNqD0dkTPRNH1CsbV+XnUUiRuGIsefJrXYBRQutrM+DwUDXsa7jPZ6n589V/A+aSXBf/GtP9j3t/t3v+dqCASkjGFRGIKBgMHa/9X1QV/+7P1fsa8aeu3v/+mzbo489z7fpeT77aGu6nfSe5773c8Re9ZGPz+oXO97TFRNHa9Gc6bp0THbc9yL6edfHHue8+zmLfy/E/hEx0fdDpOs9ETaxz40xyggGFQpG35MZwYAyM4LKCAYUCgbU0Hxe/3uwUSfOtOuuWZM19ZKRXe/b6C36vNHLZiPGqDMcff6xI7I0bfxI/b3ulIyJnsdg12OCgYCCgWi/IiZaXBox0eeJmOjjjTGaMnakrp40RvXN5/TWkWZlZQSVHQoqKxRUdihDGcGAfQ67/4xZ5zJ2bow2vXZIkvSVWZMlSddOHavn/lanbfuO6GufKbBfR7jre2XdpGjfA93ej1LAPhYM9H5PBpP8mRgKBnTZ+FE6WN+qpjNtau+MqK0zovbOiDKCUigYVGZGQKFgUKGM6PcrIxj9GiOzQvq3KXnKCMbOb2c4ov1Hm9URNsoIKta+2/tnTE5I0xPUPrR3RnSwvkWdkej/M5Gu97j10Sh2vowx9vc7EpHea2zV+42nlR0Kas4nxmvC6Oj7945/m6y/fvCx/rvmmD53+XgFAwF1RiLqCBt1hCPqDBt1RKIfO8MRXZk/RldMHG336Z+NrXr/+Onoe7Hr/8pQ13vI+hg9u9IVE0crNycz8Te+h8aW8/q4q0i1p7qPz8mY6Pdt/KispM/l9wJWV2Gkvb1dVVVVeuihh+KOl5SUaM+ePX0+5o033lBJSUncsdtvv12bNm1SR0eHMjN7n/y2tja1tbXZ/25paXHTTce2VR3R28cG/leNMVLlgeM6eurcIPZqcFwyMlOfLRqv//3HcftyRFz8qg6fVNXhk8kbpsifDh4f8q859xPjta/2pM53DM6aIP/3+mmSZI+MfNTapi+vSzxVnW5mFY7VddPG2v9+88OP9fbR5P/PfvHqS/udcnj13Y/saYmB+t5Nl9tBRJJu/9QkPfLSO9p/tFlf+dVrSR8fDEh3XzdVo3NCOn2+U9v2HXG8vHreiEz9n08XxIW0RJpOt+t/3jqWdCSjaMIoO2gkYjWprj2lH//+HUd9GGx3XzdVM6fkpeRruwojTU1NCofDys/Pjzuen5+vhoaGPh/T0NDQZ/vOzk41NTVp8uTJvR5TXl6uH//4x266NiA73/tIL/392AU/T25OSGO6Jeq4vzrj/gIN9Hk8aI0odB95UPwIRuxx0fsipusvg66/vky3vw7PdYR18myHXn4nek4m5eYoJzOoUEZ0BCXuL78+fpCMoj/UoWCw27bWJu4vR3skqNvn1vGeP3c9x0v6+6vcen3xj43/JNBHu9ixrv730adwt5Ge7iMW3Uc1Auret75GFOJHUvrsSyB52+6vqedzdO9DX+e+V3+6/pERCGhW4ViFgkHtqz2ps+2dcefZeo3dR9vU7T77fvt4t89lukaVon9NWn81h4LRY8FAwP7+Wu9JewQlYhTKCOiWaybKGOn1fzaprTOiTqudMfbz2beur3Wo6YzCEaNLx2Rr/Kis2OhMt/e61d4aCYv+9W+NmkgH61u0518nJEmXTxilkdkZau8aQWjrjPah589a3Ohfj+Nzr5hgh5BPTs7Vt+depj//o1Fn2jr7/f5Y3+fu5yPSbVRA6nq/xr1vrXdk3063deh8R0RZGUFdmT9a2V0jPZmhoCLdzkNnODqKEPu6RkdOntPf607p7z1qXUZlZSg/N8c+b9Zf6NZ74aPTbXr13Y/06rv9r4g6JiekcaOyoqOm9vcwYI/cquu4NULZfcRo6iUjVfrFT8Q9X35ujr7/hU/of96qV3tnRGFjlBkMKJQRHfHJ7Br5CWUE1dEZ0YH6Fr1QdSTuOT5VkKvsUND++e/+0SpKPtce1okz7XrmjcP9vrb+TBidpf7OVSgY0Dc/O83R81i/Q95vPN1vMa/XPjPtkuERRiw9f2EYYxImv77a93XcsmrVKpWVldn/bmlpUWFh4UC6mtCXPpmvwnEjLug5JozO1sLrCzUyK33Wj+sIR7R93xHVfnxWs6eP0xevvtRRMgfSydtHm7X3w4+14AJ+vl7/Z5P2/KtJ08eP0t3XTXX8V68TgUBAj371U3r0q58atOd06uMz7dq+74huuupSXZU/xtVjj506p+37juhcR9g+NjIrpLuvm6pJeTn9Pu7dhlZV7K/vdz+esSOytPCGQtdTHcn88MvX6IdfviZpO2OMdhw4rreOnLKPXTt1rEo+mZ/0/7+OcETbqo6o7uTZhO26Cyigm666VDcUjXP8mES+/pkpOt8e1qlzfU/7DIUru01xDbWAcVG6297erpEjR+qFF17Q17/+dfv4Aw88oJqaGu3cubPXY2666SZ95jOf0eOPP24fe/HFF7VgwQKdPXu2z2manlpaWpSXl6fm5mbl5uY67S4AAEghp7+/XV1Nk5WVpeLiYlVWVsYdr6ys1Ny5c/t8zJw5c3q137Fjh2bPnu0oiAAAgIub60t7y8rK9NRTT2nz5s06ePCgVq5cqdraWpWWlkqKTrEsWrTIbl9aWqrDhw+rrKxMBw8e1ObNm7Vp0yY9+OCDg/cqAADAsOV6InbhwoU6ceKE1qxZo/r6es2cOVMVFRWaPn26JKm+vj5uzZGioiJVVFRo5cqVevLJJ1VQUKAnnnjiorisFwAAXDhXNSOpQs0IAADDjyc1IwAAAIONMAIAAFKKMAIAAFKKMAIAAFKKMAIAAFKKMAIAAFKKMAIAAFKKMAIAAFKKMAIAAFIqffa9T8BaJLalpSXFPQEAAE5Zv7eTLfY+LMJIa2urJKmwsDDFPQEAAG61trYqLy+v3/uHxd40kUhEx44d05gxYxQIBAbteVtaWlRYWKi6ujr2vEkxzkV64XykD85F+uBcuGeMUWtrqwoKChQM9l8ZMixGRoLBoKZOnerZ8+fm5vLGShOci/TC+UgfnIv0wblwJ9GIiIUCVgAAkFKEEQAAkFK+DiPZ2dl65JFHlJ2dnequ+B7nIr1wPtIH5yJ9cC68MywKWAEAwMXL1yMjAAAg9QgjAAAgpQgjAAAgpQgjAAAgpXwdRtavX6+ioiLl5OSouLhYu3fvTnWXLjq7du3SXXfdpYKCAgUCAf3ud7+Lu98Yo0cffVQFBQUaMWKEvvjFL+qdd96Ja9PW1qZly5ZpwoQJGjVqlL761a/qyJEjQ/gqhr/y8nJdf/31GjNmjCZOnKivfe1revfdd+PacC6GxoYNG3TttdfaC2fNmTNHf/zjH+37OQ+pU15erkAgoBUrVtjHOB9DxPjU888/bzIzM81//Md/mAMHDpgHHnjAjBo1yhw+fDjVXbuoVFRUmNWrV5tt27YZSebFF1+Mu/+xxx4zY8aMMdu2bTP79+83CxcuNJMnTzYtLS12m9LSUjNlyhRTWVlp9u3bZ26++WYza9Ys09nZOcSvZvi6/fbbzdNPP23efvttU1NTY+68804zbdo0c/r0absN52JovPTSS+YPf/iDeffdd827775rHn74YZOZmWnefvttYwznIVX+9re/mcsuu8xce+215oEHHrCPcz6Ghm/DyA033GBKS0vjjl1zzTXmoYceSlGPLn49w0gkEjGTJk0yjz32mH3s/PnzJi8vz2zcuNEYY8ypU6dMZmamef755+02R48eNcFg0Lz88stD1veLTWNjo5Fkdu7caYzhXKTaJZdcYp566inOQ4q0traaK6+80lRWVpovfOELdhjhfAwdX07TtLe3q6qqSiUlJXHHS0pKtGfPnhT1yn8OHTqkhoaGuPOQnZ2tL3zhC/Z5qKqqUkdHR1ybgoICzZw5k3N1AZqbmyVJ48aNk8S5SJVwOKznn39eZ86c0Zw5czgPKXLffffpzjvv1G233RZ3nPMxdIbFRnmDrampSeFwWPn5+XHH8/Pz1dDQkKJe+Y/1ve7rPBw+fNhuk5WVpUsuuaRXG87VwBhjVFZWps9//vOaOXOmJM7FUNu/f7/mzJmj8+fPa/To0XrxxRf1yU9+0v7lxXkYOs8//7z27dunN998s9d9/FwMHV+GEUsgEIj7tzGm1zF4byDngXM1cPfff7/eeustvfbaa73u41wMjauvvlo1NTU6deqUtm3bpnvvvVc7d+607+c8DI26ujo98MAD2rFjh3Jycvptx/nwni+naSZMmKCMjIxeqbWxsbFXAoZ3Jk2aJEkJz8OkSZPU3t6ukydP9tsGzi1btkwvvfSS/vKXv2jq1Kn2cc7F0MrKytIVV1yh2bNnq7y8XLNmzdLjjz/OeRhiVVVVamxsVHFxsUKhkEKhkHbu3KknnnhCoVDI/n5yPrznyzCSlZWl4uJiVVZWxh2vrKzU3LlzU9Qr/ykqKtKkSZPizkN7e7t27txpn4fi4mJlZmbGtamvr9fbb7/NuXLBGKP7779f27dv15///GcVFRXF3c+5SC1jjNra2jgPQ+zWW2/V/v37VVNTY99mz56tb37zm6qpqdHll1/O+RgqqambTT3r0t5NmzaZAwcOmBUrVphRo0aZDz/8MNVdu6i0traa6upqU11dbSSZtWvXmurqavsS6scee8zk5eWZ7du3m/3795tvfOMbfV42N3XqVPOnP/3J7Nu3z9xyyy1cNufS97//fZOXl2deffVVU19fb9/Onj1rt+FcDI1Vq1aZXbt2mUOHDpm33nrLPPzwwyYYDJodO3YYYzgPqdb9ahpjOB9DxbdhxBhjnnzySTN9+nSTlZVlrrvuOvsyRwyev/zlL0ZSr9u9995rjIleOvfII4+YSZMmmezsbHPTTTeZ/fv3xz3HuXPnzP3332/GjRtnRowYYb7yla+Y2traFLya4auvcyDJPP3003YbzsXQ+O53v2v/v3PppZeaW2+91Q4ixnAeUq1nGOF8DI2AMcakZkwGAADApzUjAAAgfRBGAABAShFGAABAShFGAABAShFGAABAShFGAABAShFGAABAShFGAABAShFGAABAShFGAABAShFGAABAShFGAABASv1/mvbc4SbC9sQAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Do a quick test prediction on the test clip to confirm that the behavior is still as expected\n", "\n", "scores = oww.predict_clip(\"turn_on_the_office_lights_test_clip.wav\")\n", "\n", "plt.figure()\n", "_ = plt.plot([i[\"turn_on_the_office_lights\"] for i in scores])" ] }, { "cell_type": "markdown", "id": "d26f14af", "metadata": {}, "source": [ "Since that looks fine, we can now conduct a more rigorous test to evaluate the false-accept rate in something closer to a production scenario. Specifically, we want the openWakeWord models to respond consistently when a user speaks the target wake word/phrase, but also does not activate even in the presence of many hours of continuous background noise and un-related speech.\n", "\n", "To test that, we'll use a few clips (for a total of ~ 1 hour) from the [Santa Barbara Corpus of Spoken American English](https://www.linguistics.ucsb.edu/research/santa-barbara-corpus) to produce a more realistic metric for the false-activation rate per hour.\n", "\n", "The combined clip (already converted to a single-channel, 16khz, 16-bit WAV file) can be downloaded [here](https://f002.backblazeb2.com/file/openwakeword-resources/data/santa_barbara_corpus_test_clip.wav).\n" ] }, { "cell_type": "code", "execution_count": 24, "id": "756d7100", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:43:13.392821Z", "start_time": "2023-02-18T03:41:57.091857Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGiCAYAAAA1LsZRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9bElEQVR4nO3de3hU1aH//0+4BY/FqCABFCltj61t1NpQNVhs1ZaWIj3+6u/IqX0AW+hXSpGDaL8WbQtaj7H1SNEqeEGktqjYgooVhaDcAyoh3AOCXBIgISRAEi65r+8fmJDLTDKXPfv6fj1PfGRmz8xae+9Z67PXXntPkjHGCAAAwCEdnC4AAAAINsIIAABwFGEEAAA4ijACAAAcRRgBAACOIowAAABHEUYAAICjCCMAAMBRhBEAAOAowggAAHBU1GFk5cqVGjZsmPr06aOkpCS9+eab7b5mxYoVSk9PV9euXfWFL3xBzz77bCxlBQAAPhR1GDl58qSuuuoqPf300xEtv3fvXv3whz/UoEGDlJubqwceeEATJkzQ/Pnzoy4sAADwn6R4figvKSlJb7zxhm699dawy9x///1auHCh8vLyGh8bO3asNm3apLVr18b60QAAwCc6JfoD1q5dq8GDBzd77Pvf/75efPFF1dTUqHPnzq1eU1VVpaqqqsZ/19fX6+jRo+revbuSkpISXWQAAGABY4wqKirUp08fdegQ/mRMwsNIUVGRUlNTmz2Wmpqq2tpalZSUqHfv3q1ek5mZqYceeijRRQMAADYoKCjQJZdcEvb5hIcRSa1GMxrODIUb5Zg8ebImTZrU+O+ysjJdeumlKigo0HnnnZe4ggIAAMuUl5erb9++6tatW5vLJTyM9OrVS0VFRc0eKy4uVqdOndS9e/eQr0lOTlZycnKrx8877zzCCAAAHtPeFIuE32ckIyNDWVlZzR5bsmSJBgwYEHK+CAAACJaow8iJEye0ceNGbdy4UdKZS3c3btyo/Px8SWdOsYwcObJx+bFjx2r//v2aNGmS8vLyNHv2bL344ou67777rKkBgJgt2lKonP3HnC6GrxWWnVZ1bb3TxQBcLeowsn79el199dW6+uqrJUmTJk3S1Vdfrd///veSpMLCwsZgIkn9+/fXokWLtHz5cn3961/XH/7wBz311FO67bbbLKoCgFjsKCrXuLkbdNvMbKeLEpWjJ6sVxx0JbLXtUJkyMj/Q7c/ZexuD4vJK1dd7Yx0BUgxzRr7zne+02RDMmTOn1WPf/va3tWHDhmg/CkAC5ZeecroIUVu0pVDj5m7QnQM/r6k/+prTxWnX/JyDkqSNBcdt+8zlO4t150sf6wdf66VnR6Tb9rlAPPhtGgCe8eiiMzdPnJO9z9mCRKiDA7dFen7lHknSe9uK2lkScA/CCICw9peeVHF5pdPFAOBzhBH4wp4jJ1RT579JggVHT2nG8t0qO11j+2cfO1mtbz++XNc8+r7tn+0X5ZX2bzfAi2y56RmQSP/afEjjX8nV9V/qrrljrnO6OJb60dOrdexUjfIKK/SXn1xt62fvKTlp6+f5zUd7j+r19QecLgbgCYyMwPP++tn8gTW7S50tSAIcO3XmyHrtpyUOlwTR+ssHu5wuAuAZhBEAjjhdXaefvfSR5n643+miAHAYYQSAI/66dp+W7TyiB9/Y6nRRADiMMALAERVM7gTwGcII4IDD5ZXKzec27IAb1dbV6z+eXq2Jr+U6Vob9pSf1zLLdgbkiizACOODaR9/X/zcjW1sOlDldFAAtrN9/TJsOlOnNjYccK8OQJ1fp8cU7NXXhNsfKYCfCCOCgj/cddboIAFqod8FvH52qrpN05hLxICCMIGFq6+r13IpPtfnAcaeLAgBwMcIIEuaVj/KV+e4O/ejpNU4XBQ7bXXxCs1btUWVNnWXvmbkoTzc/sVwnqmote08AzuAOrEiYvMIKp4sAl/jutBWSpBNVtZr43cssec/nPvtBuNc+yteYQV+w5D0BOIOREQC2yc0/bvl7uuD0PoA4EUYAAICjCCMA4COMFMGLCCOAj0xf+onu+tt61dXTIwHwDsKIR3x65ISqa+udLgZcbvrSXVq87bBW7+ZXfgF4B2HEAxZtKdTNT6zQiBc/dLoo8IgqCy+hBYBEI4x4wMtr90mSPgzInfjQmhPzAJKS7P9MAMFEGAEAAI4ijAAAAEcRRgAAgKMIIwAAwFGEEQAA4CjCCCBpb8lJ7Szih/0AwAn8ai88L97LXo0xuvF/l0uSNk0ZrJRzOsdfKABAxBgZQeA1vXP6kYpK5wqCdnHvE8CfCCOAg4L2CzJN65skkgXciX3TfoQRAADgKMIIEoYhdQBAJAgj8DxCD3CWCdzJP/gBYQQAADiKMAIAABxFGAECKonzWwBcgjDiAVxmhkQw7dwtzit7HXMk4Gfx3tTRKwgjgAcEpD0CEFCEEQAA4CjCCADPCMqQNRA0hBEAAOAowggAAHAUYQQAADiKMAIAABxFGAF8iHmeQOy4d439CCOAg7xyYzGrtHejNQDBRBhB4DUNBPSVAGA/wggAAC4Vy09I1dcb/TPngHYXn7C+QAnSyekCAEHGQAysxuge3t58SPf9Y5Mkad9jQx0uTWQYGQEAoAmv/zhpbv5xp4sQNcIIEsaurzNHggDgbYQRD4jlnCHgduzXABoQRgCElERaAGATwgiAkLgnCAC7EEaAAKmsqSNkAHAdwggQELuLK/SV372ne1/f5HRRAKAZwggQo7kf7tdLa/Y6XYyIzVp1pqwLcg86XJLYMY0F8CduegbEoKq2Tg++sVWSNOyqPurxuWQZY5j0CQAxYGQEiEFd/dl5F5U1dXpm2W5983+WquDoKQdLBQDeRBiB57lhMOLxxTtVcqJaj723w+miBA7zcQHvI4wAgI+4IZwD0SKMAAAARxFGAACAo2IKIzNmzFD//v3VtWtXpaena9WqVW0uP3fuXF111VX6t3/7N/Xu3Vs/+9nPVFpaGlOBAfgDcz2A9gXlexJ1GJk3b54mTpyoBx98ULm5uRo0aJCGDBmi/Pz8kMuvXr1aI0eO1OjRo7Vt2zb94x//0Mcff6wxY8bEXXjAagH53gOAq0QdRqZNm6bRo0drzJgxuvzyyzV9+nT17dtXM2fODLn8unXr9PnPf14TJkxQ//799a1vfUt33XWX1q9fH3fhAQDNBeVIGv4SVRiprq5WTk6OBg8e3OzxwYMHKzs7O+RrBg4cqAMHDmjRokUyxujw4cP65z//qaFDh4b9nKqqKpWXlzf7AwAA/hRVGCkpKVFdXZ1SU1ObPZ6amqqioqKQrxk4cKDmzp2r4cOHq0uXLurVq5fOP/98/eUvfwn7OZmZmUpJSWn869u3bzTF9B0u1QMA+FlME1hb3vK6rdtgb9++XRMmTNDvf/975eTk6L333tPevXs1duzYsO8/efJklZWVNf4VFBTEUkw4jBDVPjf/gi63tkdQsevbL6rfpunRo4c6duzYahSkuLi41WhJg8zMTF1//fX69a9/LUm68sorde6552rQoEF65JFH1Lt371avSU5OVnJycjRFAxBQdByA90U1MtKlSxelp6crKyur2eNZWVkaOHBgyNecOnVKHTo0/5iOHTtKcvdRIQAAsEfUp2kmTZqkWbNmafbs2crLy9M999yj/Pz8xtMukydP1siRIxuXHzZsmBYsWKCZM2dqz549WrNmjSZMmKBrrrlGffr0sa4mgEPsyNTRBndyPgAvieo0jSQNHz5cpaWlevjhh1VYWKi0tDQtWrRI/fr1kyQVFhY2u+fInXfeqYqKCj399NO69957df755+umm27SH//4R+tqAdgsSWHODbg4BLQ8ncGcEABuEXUYkaRx48Zp3LhxIZ+bM2dOq8fuvvtu3X333bF8FACfIgsBaMBv08DzOCUBAN5GGAF8KJJRByaQA3ALwogH0GfYhzMHALzOiwcahBFExRij3cUVqqmrd7ooAACfIIygXcYYLdpSqPzSU1qw4aC+O22lRv+VHzoEAFgjpqtpECzvbS3SuLkbJElpF58nSVr5yREniwQACePBsxyex8gI2vXxvmNOFwGQRCcB+BVhBHAQNx6D1chr8CLCiAfQXwHhMVoCeB9hBIFnwvw/AMAehBEAnsEoIeBPhBEAIdHvA84LSgAnjCBhwv6yrQ+EayAMJ3oAIGqEEXiel48cvHjbZgCwGmEE8AAiC2AfLx/geBVhBAgMWlgA7kQYAQDApWI5k+vFkVTCCAAAcBRhBIBtmK8LIBTCCAAAcBRhBAAAOIowAgAAHEUYAeAILjQGQjtdXed0EWxHGEFU3HiLdyZFtsY6Abxp8bYiXf7795wuhu0IIx7gxgCAs+j44Srsj572639scroIjiCMAAAARxFGAAsxSgLAaV4cSyeMAE0QJs7ix8IA2IUwAgAAHEUYAQAAjiKMAAiJU1aAN3nxq0sYQcIw58Bd2B4A3IowAgSEX0c6fFotIFAIIx5j/NqjAIBLMIhoP8KIx9zw+DLl7D+mQX/6QAs2HHC6OPCwJAfO2xjGMQCEQBjxmIKjp3XbzGwVHD2tSa8H87bBbtO0T2deBgBEjzACAAAcRRhBuzjaBwAkEmEEiBNziu3DL1jD75yYy+UGhBEgoLgyC4BbEEY8IKBBOWJuWj/073AaVyzFjzVoP8IIouKmjh8A4A+EEQAA4CjCCICQGAUDYBfCCBKG+RNoE2kHSAgvtr2EEbTLizs2EFRc/gwvIowATXAlgo1iSLlsH8CfCCPwPEZuIsNZEQBuRRgBPIDA5T1sMyByhBEkDEfiTqInBOAdhBFExe9He0z+Syy/7z9NEcaByBFGAHgafT7gfYQRIE5c4QEA8SGMADEINwQfpNMQgF8x2mY/wkiC1dUblZ6ocroYAAC4FmEkwX7y/DqlP7JUWw+WOV0UwJcYjAK8jzCSYB/tOypJ+sf6AodLAjiPK0wSjzlM8CLCCOBL9PoAvIMwgqhwZAsA7ubF0THCSIROVtXKWHCpxKnqWu0uPmFBiQD7WLHvA0A4hJEI7DlyQl+bsli/eDkn7ve6+YkV+u60Ffr4s7kkXlZ6okoPvrFFWw4wOdePuBstALsQRiLw93X5kqSleYfjfq/CskpJ0rtbiuJ+L6c98MYWzf0wX8OeXu10UQAAHkYYQcx2HeZ0EwAgfjGFkRkzZqh///7q2rWr0tPTtWrVqjaXr6qq0oMPPqh+/fopOTlZX/ziFzV79uyYCgz/qq6t1/t5h1VRWeN0UXyJky4A3KpTtC+YN2+eJk6cqBkzZuj666/Xc889pyFDhmj79u269NJLQ77m9ttv1+HDh/Xiiy/qS1/6koqLi1VbWxt34eEv/7tkp55fuUff/PwF+sfYgbZ9btPJmV6chQ4AXhd1GJk2bZpGjx6tMWPGSJKmT5+uxYsXa+bMmcrMzGy1/HvvvacVK1Zoz549uvDCCyVJn//85+MrNXzp9c9uDPfxvmMOl8R9uJoFgJ9FdZqmurpaOTk5Gjx4cLPHBw8erOzs7JCvWbhwoQYMGKA//elPuvjii3XZZZfpvvvu0+nTp8N+TlVVlcrLy5v9AW5FTohcs3UVw01ruMIHfhfUezlFNTJSUlKiuro6paamNns8NTVVRUWhrw7Zs2ePVq9era5du+qNN95QSUmJxo0bp6NHj4adN5KZmamHHnoomqIBrsBpHgCIXkwTWJNaRDdjTKvHGtTX1yspKUlz587VNddcox/+8IeaNm2a5syZE3Z0ZPLkySorK2v8Kyjgd128yK6AH9QjCQDwi6hGRnr06KGOHTu2GgUpLi5uNVrSoHfv3rr44ouVkpLS+Njll18uY4wOHDigf//3f2/1muTkZCUnJ0dTNCQQnT0AIJGiGhnp0qWL0tPTlZWV1ezxrKwsDRwY+uqH66+/XocOHdKJE2fvSfHJJ5+oQ4cOuuSSS2IoMgArhBvNtMKeIyd0ooor5gBEJurTNJMmTdKsWbM0e/Zs5eXl6Z577lF+fr7Gjh0r6cwplpEjRzYuf8cdd6h79+762c9+pu3bt2vlypX69a9/rZ///Oc655xzrKuJxzC3wDpMIHWXbYfKdNMTKzQw832niwIEkhfbxKgv7R0+fLhKS0v18MMPq7CwUGlpaVq0aJH69esnSSosLFR+fn7j8p/73OeUlZWlu+++WwMGDFD37t11++2365FHHrGuFoBHebHRaM+yHcWSpPJKRkYARCbqMCJJ48aN07hx40I+N2fOnFaPfeUrX2l1agcAYD0/Bly7JfIUJkLjt2kAAICjCCMA2sXRNoKEOx7bjzCCqCSF/QfcJf7GlJFqAHYhjAAOcrLD5+gP8CcvHkgQRgB4GpkK8D7CiE1oL9vnwTDvKV48WgIQDIQRIE4ETQCID2EEsFCiThk4cd8DK+pyuLxSlTV1lr4nAP+J6aZniB+NMqLh1cmm1z76vrqf28XpYgBwOUZGgCY82udHxKm6lZ6sduaDAXgGYQTt8nMHHSvWCQBYhzASAX5hF21pOpuDK1YAOM2LB0uEEQ/w6o82ebXciE8itzsHBoA/EUYikMQdMAAASBjCCABHEPEBNCCMAICPcCILXkQYQcw4sgUAWIEwAqBdHG0DSCTCCOAgN1+Cx8VQCCquBLQfYQSu4dUGwMV5wne4sg3wJ8IIouPRwAD7ePV3dAA3CGoLSxixSVB3sKBxcz9MjgTgVoQRRMfNvS0AwJMII0ATjB4kFrdzBxAKYQTwqeraem0/VM4cDgCuRxgBfOquv63XD59apb+t22/r53r1qijAL7x4+EEYQbvoW5wXS+OybOcRSdKcNfssLQsAWI0wAtfgdAJiwTwUwPsIIzaJp7lkYMI+fslDfqkHgGAgjABwBHkJQAPCCGLGREUAgBUIIwAAwFGEEQAAXCKopy8JI3ANr5728epVQNGsbzfX0a2/5OviVQa4DmEEsBD9D4B4uDNaJx5hBAAAOIowEgFuqgQAQOIQRhAdj87rCLKGKO22uRXuKg0AJxFGIuC2RhzOKz1Z7XQRAMA3CCNADO5+ZYPTRQCAkLx4JRdhxCFuvlQS7duQf9zpIrSJs2kAvIQw4gHEFiQCgRiAWxBGAACAowgjHsCIOxgeA+BnhBHEzC0hiX4aALyNMBJAWw+W6bF3d+hEVa3TRQEAQJ2cLgDsd8tfVkuSTlbV6g+3pjlcGuc1HVlhTicA2I+RkQDbUVTudBGacctpn2iRX+zDJcuAPxFGAAsxsgKnccm2tyUFNHETRpAwdn2ngvnVjRydEwC3I4ygXfRlYBcAkEiEEZvQocNrvDJabIhKgOcRRgB4BqEe8CfCCBAQXhnpABA8hBEAAOAowgiiwsG1N7j1dIZbywXAWYSRAIu3Y6BfAQBYgTCCdjHXIHG4EgRwH9o8+xFGAC+Io3EM6h0dAXgHYcQm8fQHbu1LXFoshBHtablEz+9ob7+uratPbAEAuAZhJAIMpcNxceyCXrwd/G/f3KKvTVmsg8dPO10UwFZe/L5agTAC13DrCFA0/FAHN/j7unxV1dbrxVV7nS4KABsQRhwSzOzrfwE9qAFgkaDO8YopjMyYMUP9+/dX165dlZ6erlWrVkX0ujVr1qhTp076+te/HsvHOiaJ2RG2oCMHgGCKOozMmzdPEydO1IMPPqjc3FwNGjRIQ4YMUX5+fpuvKysr08iRI3XzzTfHXFgAQNvI9PCiqMPItGnTNHr0aI0ZM0aXX365pk+frr59+2rmzJltvu6uu+7SHXfcoYyMjHY/o6qqSuXl5c3+ADswWRkA7BdVGKmurlZOTo4GDx7c7PHBgwcrOzs77Oteeuklffrpp5oyZUpEn5OZmamUlJTGv759+0ZTTAAWi+YUWkBPeQOIQ1RhpKSkRHV1dUpNTW32eGpqqoqKikK+ZteuXfrNb36juXPnqlOnThF9zuTJk1VWVtb4V1BQEE0xEaFEjwHYNdeGsYzEiDVUMPcHcJr3voSRpYMWWs72NcaEnAFcV1enO+64Qw899JAuu+yyiN8/OTlZycnJsRQNCea9XRxexmkzIBiiCiM9evRQx44dW42CFBcXtxotkaSKigqtX79eubm5Gj9+vCSpvr5exhh16tRJS5Ys0U033RRH8QFEquXxQlAvIQTgPlGdpunSpYvS09OVlZXV7PGsrCwNHDiw1fLnnXeetmzZoo0bNzb+jR07Vl/+8pe1ceNGXXvttfGVHgAAeF7Up2kmTZqkESNGaMCAAcrIyNDzzz+v/Px8jR07VtKZ+R4HDx7Uyy+/rA4dOigtLa3Z63v27KmuXbu2ejxoSk9WO10EhMA9ZRKLky4AQok6jAwfPlylpaV6+OGHVVhYqLS0NC1atEj9+vWTJBUWFrZ7zxFI72wu1DN3OF0KBAEBAIgOk7DtF9ME1nHjxmncuHEhn5szZ06br506daqmTp0ay8fCBZqOGzDlIH6MxAAAv01jm3iSNt0VEB5HsYD3EUbgGl4daaEztI9X9xEAbSOMIGHsukeEu/qn6OrMfTQANOWu9sw+hJEAMxzSe0ZQtxS7KJzACJz9CCMAALhUUA4aCSMALMVRJYBoEUaAgArKERcA9yOMAD7khZjBPVYSg4wJL+4DMd30DN5kjNG2Q+VOFwMexFU/ABKJkZEAmb1mn275y2qni+FqdLpnMXIBcDrTLoSRAHlpzV6niwAHuKktPVxeGdXyTIYFgoEwEgGOlmPDkbW7uGFrHI3y16rdFKQAJA5hBIAjCPkAGhBGEDNGPhC1z/IH5+GB0Fp+M5ICcq6Sq2ki4JdOt2X7T3cAJxypqNKtz6zRweOnnS4KAJdgZATtsi+KeTX0nY11bjzgd9uB1bMrPrU0iLitfkA8gro7E0Y8wOlhuqb9Kw0/2tNeIKurd2FiA+Aowgg8j64NOIsDBngRYQSAZ7jxNBiA+BFGAFiKI/MzuHQZiBxhBPAALoUF4GeEEQBIAL/cEgDe48VjF8II4KBENRpNu0EvNkwAgoUw4gFWDdEz1O9vz6741OkiOILdujnWR/wY07IfYSTAaLT85bF3dzhdBACICWEEQLsIrgASiTASYO67BJMez03ct38A8CvCCGJGZ+UtTv+sQCyY5wQEA2EkQGjWQ2va38XS99FfAv7F99sehBG4iPeO3Fui3UosDw7uAFEJ6j5OGEHC2PWlCuh3FwB8gzACBJQX55AAQROUeVOEEQ+wqtMIyD4Nh3EbdMBZXjzOIIwAPmSa/b93UyijN3CCd78x3kUYQVToGqwVaV9LpwzAzwgjAcZpG7hdUM6XA1by4teGMBIBLw9zw15uHr+ItmN3qkE7dPy0Mx8MwDGEkQAJFapOVtWqpq7egdIgGkEZITh2sloDH/vA6WIAsBlhJMBOVtXqa1MW61t/pPGHO+wqPuF0EWCB2rp6zf1wv3YXVzhdFHhEJ6cL4AV+vVSxoeE/XF7lcEm8LZ65pcdO1VhXEJdo67RmMMZ3nOWGdfzqxwX63ZtbJUn7HhvqcGngBYyMICpuaOhacmOZIjVz+acqOHrK6WIgCpU1dcreXcLpzTbk5h9zugjwGMII4LCleYedLgKiMP6VDbpj1of647s7nC4KfCgg08NaIYwAcfJr48GtTUJbmlcsSXp57X6HS4JEYde3H2EEruGHzs/NucSLN05z8/oEEsGDX1NLEEaAJpxoCJxqe5y4XDgolygDiA5hBIBreOmokJshAtYhjNjEDQ1XrAelHuof4saBe/z8eik8gokmwR6EEQct3X5Y2btL2l2Opt3fvDiXI1ZBqiuAyHHTMweNeXm9JP/eFIhuJzL0z/AbRscQLUZGACAGnNJDIlixX7lhWkC0CCMAAMBRhJEA8V5WDgYGtMNj9ME5xhgVV1SGfO5EVa0qa+psLlEwBWWeFWHEBbx074VgfC1sFpDGBt7y2Ls7dM3/vK+5Hza/02xlTZ3SpizWFVMXO1Qy+BFhBIBtvBS82+OfmoT23Mo9kqSH397e7PF9pSclSTV1fl8DsBNhBHBYJOMiQW32391aqBNVtU4XA0CCEUYCxEcHpQiIkhPVmvBqrtPFAJBghBEXcHtIcHnxHBfv+mHKSNs+2FHsdBG8xe0NChACYQSuEWuf7Ka+PJI5EZnv5jX7d6JvEGXJfQtc0r95bVsDiAxhBLBRZU2dnluxx+liRMSNIzZ0/wiaoIRewghgo6fe3+V0ERIunhDjwvwDwAaEkQgk+ta69uXeYCRsN/t439FWjyViBMKtB1MuLRba0HKbNT2tGJSjdiQeYcQm/HAU4C90w4B1CCNwjaA27sRUwF3cOF/K72IKIzNmzFD//v3VtWtXpaena9WqVWGXXbBggb73ve/poosu0nnnnaeMjAwtXsxthKPBF8Pf/LZ9GbmH3/Zpr/HidzDqMDJv3jxNnDhRDz74oHJzczVo0CANGTJE+fn5IZdfuXKlvve972nRokXKycnRjTfeqGHDhik3lxsZwRpe+t5xug4AWusU7QumTZum0aNHa8yYMZKk6dOna/HixZo5c6YyMzNbLT99+vRm/3700Uf11ltv6e2339bVV18d8jOqqqpUVVXV+O/y8vJoi2mpxN8HwsiOwXovpmUEi5eOqPk+BYNd7XPj59n2Se4S1chIdXW1cnJyNHjw4GaPDx48WNnZ2RG9R319vSoqKnThhReGXSYzM1MpKSmNf3379o2mmICnBGm0pL2a0sEDwRRVGCkpKVFdXZ1SU1ObPZ6amqqioqKI3uOJJ57QyZMndfvtt4ddZvLkySorK2v8KygoiKaYcAkrj3Jr6+r1yeEK719KGGqdBCeLBPaoz9Pa2Ghe/zrCPaI+TSNJSS16GWNMq8dCefXVVzV16lS99dZb6tmzZ9jlkpOTlZycHEvR4FOTXt+khZsO6bdDL9eYQV9wuji+EMl3FggiQpb9ohoZ6dGjhzp27NhqFKS4uLjVaElL8+bN0+jRo/X666/ru9/9bvQl9TG79nsrPqdpB2ZnZ7Zw0yFJ0rMrPrXtM/2ChtU+2w6V6TuPL9M7mwudLgrgKVGFkS5duig9PV1ZWVnNHs/KytLAgQPDvu7VV1/VnXfeqVdeeUVDhw6NraQel+i7uPqBV4/Tm3b2sWxlr9Ybrf1q7gbtKz2lX72ywemiWIcdFDaI+jTNpEmTNGLECA0YMEAZGRl6/vnnlZ+fr7Fjx0o6M9/j4MGDevnllyWdCSIjR47Uk08+qeuuu65xVOWcc85RSkqKhVUB3C/Wdp3RDW84XVPndBFcgfwSu6Cuu6jDyPDhw1VaWqqHH35YhYWFSktL06JFi9SvXz9JUmFhYbN7jjz33HOqra3Vr371K/3qV79qfHzUqFGaM2dO/DVAoPixU2buxlmsivi54SvihjLAW2KawDpu3DiNGzcu5HMtA8by5ctj+YhA8WMHi8j5rf8lUPgb2zexgtod8Ns0ANrFnKfI+PKeMW1d2mtfKeBzhBHARk4eVUb72b7sWAG4EmEEnuLHI7EgDXtzSjLxArQ7wUcIIy7g9iFwGrfESnQYCVLYaVBbV6+SE1XtL2ixIK5ruI+7e5TQCCOeQAvnF3ad+gh6p/jjmdka8MhS7SyqcKwMbj/IANyEMOIJ1jRqnv9dF/mjDi0xN6Mpa9bF5gNlkqQ3Nx605P0QHfZoRIswAvict/KbpwrbCp0wEBvCiAt4q7MAECRtnW7y40ilxGlOJxBGABuFauScavj82pE0FYAqwmeC8L0MhTDiQnX1RsXllZa/rx928UTUoemRX0DbAViEW/v7jxebBC/uhYQRFxr91491zaPvK3t3idNFaZPVO7xX23HCTOTi3cYe3UUAtIMwEgG7L9FbvvOIJGlO9j5bP9dqdh0ler2DimQ9eTWoOY3La+E1LduDoOzBhBHAQm4JDaFGaLwxauOSFYi4uOV7AO8gjLhYtH3HkYoq/eFf27W72LkbPSVayE7W/mIEAqMKkFrfB4ecgUQgjPjIvf/YpBdX79UPn1wd8nlvHBkHTySNO9sOkXLDrsL+ap2ghD/CiAtY9cXdfOC4JKm6rt6aN/S4tzYe1JS3tqq+PrEtYzR3UHXTpb1AJNq8z4iN5YC/dXK6AF7g1O26ObqIz3+/tlGSdE3/7hp6ZW9nC9OGispap4vQLvbFyBAsES/uMwL41NFT1U4XoU1zP9xvy+dE21H64jdzbG7XA9qPwGW8uBsSRgIkqIk7mh7CiSPb+gCdVQvqLgigbYQRm9AIQwo92sDQ/lmxrAs3hewgbMsg1BH2I4y4QPuXUPLtbxBLx+NkV5X9aYmeWbY74ZNo/cLqXMFat14k24jAgmgRRnwkUd9/GvTY3fHCh3p88U69s6Uw7DI03AiyT4+c0A+mr9S/Nh9yuihwEGHE1aKLAYSG0Nwwip9/9JQkgoffb6Tmx+3b1vfHiu/Wva9v0o6iCo1/JTf+N4NnEUYCxN/dgHf54qoVIEYnq9x/aTsSjzDiAm44co+F1UeBkXTKsayqRE9wjPdXe91yNF1ZU6f/f2a2pi/9xOmihBXt6nXT5FYA4RFGbBLbVQLWl8PN/DyE31an6JIsogUbDmr9/mOavnSXY2VwSzCLFaNcQGwIIz5CMxiafyOOtapr6yx5Hzfthy1/jh3xY5UiEQgjgA3aHOVySevux46b0zSIjf++C25HGHEB25pLP7TLMdQhmv7Iib4rSM2e309j+DDPxcRP25k8aw/CCBLGP81RYlnRgVlxUzU6UkTLv/O8/Fov9yKM+Ahfn9ASvV5ORPGru4k6FbJwU/gbRoU7VdGyLH7MIhzVIoi8uN8TRlzMjfuTHzuseN0x68N2l2lzykgEn9HeEei+0pMRvAsSzQ3fDzd0RP4dMUGiEEZcwKpJdm5oCBPNjU1cda1Pfna3jVEbu9Z7EPZhNOfG77STgro+CCMuxpUA1nDDemwoQqjO1opTN5FMGIx2UmEizijZfcTs/Jb3IyIjrEcYCRKb2xA6gtZCrRMrNosVwYEuBpHhmw3rEUY8wOmrHNzUSblhlCMWTp5Dj3Tkxar9LJ6aenPrnuXHe7XEwk+X9sIehJEIJLojCffuVn+q3c2DV4OD3YI0qsEu4T1tbTK2J6xCGEHC2NVQtRd63NRghpwz4pIo4ZZyxMKK+6zAGd7d62Alwghi5sSv9rodXWJihdpDnnx/l656aIn2lnB5M/wnKG0KYSRA7D6fbdvloO3Uyw33PGj7t2nif/+Wq8DROSo2v7amzqiiqlaPL94Rxycjct4/aHCzoK5dwogL2HUawe65dW45PfLooh2aufxTp4sRllsaH6/PvQy1v7llHwQiFdRdljASAT+cPnCCG0YkGvzxPWePmhvWRKgO35IJrF5PEhZwQ/BgKwCxIYzYJJaG0urG9fipGmvfEJ4Q6VVNdKQAnEIY8ZG2jo5r6+y/ZXkijlRdcPBrObeMvDXdfcpO+yO4uml0LkiiGahjC0EijLiDDd/GagfCSKRKT1RpR1G508WwRejbwdtejHZ9vPeo00WImiuChwu3Zby4X5D3uOK7EKVOThcA4Xlvd4pN+iNLnS5C4rXRoEcSRipr2g6T1tw47eybeHHfo88EvIuRER9p6wjGiVMBiTii8mOH45bTNE0l6mi4vdDERFz3c9MmWvHJEd36zBrtOlzhdFEQJ8IIYubHYJCo4U03rqpWgaNJJ+PG8lqpvLJGd770kRZsOGDp+7qhn3bDEL1dbcOo2R9pY8Fx/XLuBovfuekoofPrMwgII4gKE9Pi4+Yj/6Yla9mZOD1v4I3c9kNDNCV8ZtluLd95RJNe3xR7oQKi5T7rxoOQYyernS4C4kQYcQGrkrebOzrpzNHoik+OOHJlj9PaasCtnu9hzeeYhBzlt9eRhQs998yLLTSE+7xyn1wthDNcmI9i5sawZwfCCBKm5ZfqjhfWadTsj/Tcyj2Njx0ur4zuPX3V7FgnlkDTck26Pcy2J7pGPDF19fo69JvKmjrd/8/NWrKtyOmioB2EERdrOEq0onmzs42s+Wzko2Vw2HrwzOW7Tc/TX/vo+/YVzAaxnM5wQwd2urquzdM0VklslIz83SNd5WWnarTtUFmM5fG/cPuJ/T89Ebogc7L3ad76Av2fv+XYWyBEjTCCmIVqcGat2qN/f/BdrdtTGvZ1HRzqfJ2e9yCFuc9Igt43GnM/3G9BKbwj0vX1rT9+oKFPrdbH+7x33xWvsGL/D/fNLiqLbuRVOhNAtxe6575Hzh+q2IMwkiDGGM1evTfCZRNcmAQJVe5H3smTJP36n5vC1su5MOLIx5757ASPCRwur2rjs8M83uSJypq6iF4TiXg2r12jRJF+TEVVrSTp/bziyN431gJ5iF1f34WbDun19QURLWvlCM31f/xAv3tza/QvtIgLBkodwU3PEuSdLYV6+F/bnS5G3E5V16owynkd0pnGIVyHFs+XrWWjs+twhXL2H4vstbF/bEJZ0fjMXhNZ8A2nvtXVM3G9nSO8WGYvSHTfGGqz1dbVa8KruZKk73z5IvXs1rXt9wiz8WO5h8+JzwIo7EUYSZBPivxxE55rH31fFZXRfzmNCd85WDky8r0/rwz5eF6IYdYzDZazozIhf7XX3qKEVG9Ms7J5caJwNCV2443mcFbTbVlRWaue3SJfvqmgjjJ4EadpEqTOJ4dpsQQRqe35GR1s2Ovun7+51WORnK5oakP+MeWXnrKuUK5ztuLGNG+4fbL7huXFTqq4vFIlJ8KfjvMrv++L7XHDBHc7EEYSJJpbafjxu2aa/LclO+aMhGrAQj32zLLd+srv3mv1+N6Sk/rxjGzd8PgyS8rz1sZDlrxPorQMj/tKTuq2mdm2fX5dvVFdy3NFUSo/XaNZq/ZEdLl4ovbASHbt46eqo55Mfbq6Ttc8+r4GPLJU9XGup2g53xdGUN9wc0asLYgtghq+OE2TIPU+3aOatoNtVTFRp2kiXauhPiLUqYfHF+8M+fodFs+mP3j8dNjnEn3kUxjBFQVnTmCdLccTWZ9E/P4VlTWqN1LKOZ3PvFcMu/53p61QvTH68/CvR//iz6zff0zr9x/TKx/lt7tstOs80tNWnxw+0ebzH+w4rJ/PWa8R1/XTH25Ni/jzj1ScHRGprqtX1w4dI35tvNr8nodZL/Hu0ibCduZsOUJzPkg5xIPdDyMjCRLvUZ4U2ZewvNLeO0lGWq+2wlgHixqI7N0lYZ9zqg3aWHBcv39rq8qiuMOnU2XdU3Ky8f9jzc719UZXTF2iqx5aoqraM1fkxPJWe0tOan/pKR09Ef9tvfccaVqv0KVpKKuVdkYwT+yP754Jv39bF92l1B07nt1LrGhb4mH3/hpPdb1yisPyu1J7o9rNEEYiEMtkPrtGRh5YsKXx/9va/6wqT6QNoVHzDu72Z9c2/r9Vp2numPVh+CdDfEY0qyDWtVVRWauX1+7XA29s0d2fXQ3QnkS0l5FM0PzTe2dHhVpOYI3U6SaXBDccvcezq7271do7ZYbbXZftOGLp50hnRoiaqq5t3cHU1jd/bMuBMj2zbHfIZZvq2GTj2D0fzYn+vGmbG0n7G/5qGveqrKnT7uITOlJRpasfztL//ad1v5PU2Y6JeRbzXokdEMt3P5rzuvHcjKut+x98euSEbvjTMs37ON+y85C1kYYR07wR+ajJTaMiaUxfWrM35CV27TXaDSJphEojnAy4YMOBqG969c7mQr29KbJ5IkvzinX7c2sTOheg5X1EWrLyo+O5Eme+xb+iWxPmiLNjtMNzbVRp4mu5GvrUqlb75sf7Wl9y3nQ93/v6Jg17erUeX7xTL6/d1+bHNy1ue/uJnVcK/WO9tdurgVWnaRLJGKNth8oibpNa+q/n1+m701Zo9F8/VkVVrV63cF02HUnzCsLIZ/6+br/ufX2T6uqNjDHK2X8s5FD7HS+s0x0vrGv3/SpCdKQvNPlNlpaMMSosaz6vYPXuEp2qrm2zo2jaqLZc7HdvblX+0VO6f/4WC0dGIv3imbCNSG7+cf32zS1tDk0+9PZ2TV24LfoC6swR+saC461L1KI86Y8sDfseTZed9Pom/eeza/XWxoMJGd6XpI/2HtW+0pOqqz9zs7zth+Kbs9LyaPbKh5boZBv3TzAyEf/yqTFGr36Ur7zC8mb7nBuHxDuECR1Rh5E2vLnxkLYdKteaT8OfNmzQdGSxafBqb3s3rUd7o5MWVq2Ve1/f1OzW6lNi/I5GI662K4Hr4u/r9mvoU6v1y7+3f6t5Y0yrU+oNbdTmA5H93EBboefN3INa3OT3dzo32Ql+/9ZWvbByj3735lZX3IU6nJjCyIwZM9S/f3917dpV6enpWrVqVZvLr1ixQunp6eratau+8IUv6Nlnn42psIlSW1ev3765VfM3HNB7W4u0eFuRbpuZraFPnalX0y9D9qelyv60VLf8ZZXe3VKowrLTqqo9M9zW1LpPm98Ofe6H+fqfRXlhy5D57g5lZH6gv2bva/b4/JwDbX4Z641RZU2djp+qbrWzNj0qjGQXLK6o1OnqM51t2emakJMum46M7Cg624D+7+KdWrbj7ChNvZGOtDHy8Pd1+Vqw4WCb5Vm+M/SoT3vnV296YnnIxxuO2GP9Qv73axv19Ae74zq/29ZH1xvp1Y/y9fC/tuuHT7X+Tu0uPqFnlu3WqeozoaKypk7FYa4caTnqU11br9c+Dn83S2OkmSs+Dfv86l1nO9q3Nxdq8oItGvLkqpDr0k3tXaiOubKmTvlHQ1+y3TSwNaxnKbLvT21d20vtLKoIGyQavuPvbC7U7NV7tbZF+9EhitM0oULhql1H9L1pK/TR3qOqqzcqO13T7mhZU/M+zteCDQc0f8OBVm1dqH1g5Sfhg9nbmw5p68HIf+/n/vmb271C6lR1XchyNB0lMsao7FT4uVw1dfWa+Fpkp1YlafaafZKk93e0f3feCa9t1JVTl0R0kBHqgGfOmr267Lfv6smlu1o9d6SiShPnbdRdf8tpHDXr2OQ0zctr9+t/FuXpb+v2hzxIc4skE2XLPG/ePI0YMUIzZszQ9ddfr+eee06zZs3S9u3bdemll7Zafu/evUpLS9MvfvEL3XXXXVqzZo3GjRunV199VbfddltEn1leXq6UlBSVlZXpvPPOi6a47frZSx9p2c7w5497dktWcUXwru13i32PDdWWA2Ua9vRqp4sCOOLa/hfqw72J/22ci88/R4/cmqY52fs0amA//XzO+oR/5tRhX9UTSz5pHEm+69tf0HMrwo8g/3bo5Xp0UZ6lpxUj9bnkTo2njtvaJt/7aqqeH5GulbtKNGr2R3YWMWa//M4Xdf8PvpKQ9460/446jFx77bX6xje+oZkzZzY+dvnll+vWW29VZmZmq+Xvv/9+LVy4UHl5Z0cFxo4dq02bNmnt2rWtlpekqqoqVVWdDQBlZWW69NJLVVBQYGkYmbxgs97eVGjZ+wEA4FUvjBygjC92t/Q9y8vL1bdvXx0/flwpKSnhFzRRqKqqMh07djQLFixo9viECRPMDTfcEPI1gwYNMhMmTGj22IIFC0ynTp1MdXV1yNdMmTLF6LMLMvjjjz/++OOPP2//FRQUtJkvorrpWUlJierq6pSamtrs8dTUVBUVhb4kr6ioKOTytbW1KikpUe/evVu9ZvLkyZo0aVLjv+vr63X06FF1797d0klyDYnN6hEXN6PO1NmvqDN19isv19kYo4qKCvXp06fN5WK6A2vLQGCMaTMkhFo+1OMNkpOTlZyc3Oyx888/P4aSRua8887z3AaOF3UOBuocDNQ5GLxa5zZPz3wmqqtpevTooY4dO7YaBSkuLm41+tGgV69eIZfv1KmTune39twUAADwnqjCSJcuXZSenq6srKxmj2dlZWngwIEhX5ORkdFq+SVLlmjAgAHq3LlzlMUFAAB+E/V9RiZNmqRZs2Zp9uzZysvL0z333KP8/HyNHTtW0pn5HiNHjmxcfuzYsdq/f78mTZqkvLw8zZ49Wy+++KLuu+8+62oRo+TkZE2ZMqXVKSE/o87BQJ2DgToHQxDqHPWlvdKZm5796U9/UmFhodLS0vTnP/9ZN9xwgyTpzjvv1L59+7R8+fLG5VesWKF77rlH27ZtU58+fXT//fc3hhcAABBsMYURAAAAq/DbNAAAwFGEEQAA4CjCCAAAcBRhBAAAOCrQYWTGjBnq37+/unbtqvT0dK1a1fpn291o6tSpSkpKavbXq1evxueNMZo6dar69Omjc845R9/5zne0bdu2Zu9RVVWlu+++Wz169NC5556rH/3oRzpw4ECzZY4dO6YRI0YoJSVFKSkpGjFihI4fP25HFbVy5UoNGzZMffr0UVJSkt58881mz9tZx/z8fA0bNkznnnuuevTooQkTJqi6utr2Ot95552ttvt1113n2TpnZmbqm9/8prp166aePXvq1ltv1c6dO5st47ftHEmd/badZ86cqSuvvLLx7qEZGRl69913G5/32zaOpM5+28aWaO/H8fzqtddeM507dzYvvPCC2b59u/nv//5vc+6555r9+/c7XbR2TZkyxXzta18zhYWFjX/FxcWNzz/22GOmW7duZv78+WbLli1m+PDhpnfv3qa8vLxxmbFjx5qLL77YZGVlmQ0bNpgbb7zRXHXVVaa2trZxmR/84AcmLS3NZGdnm+zsbJOWlmZuueUWW+q4aNEi8+CDD5r58+cbSeaNN95o9rxddaytrTVpaWnmxhtvNBs2bDBZWVmmT58+Zvz48bbXedSoUeYHP/hBs+1eWlrabBkv1fn73/++eemll8zWrVvNxo0bzdChQ82ll15qTpw40biM37ZzJHX223ZeuHCheeedd8zOnTvNzp07zQMPPGA6d+5stm7daozx3zaOpM5+28ZWCGwYueaaa8zYsWObPfaVr3zF/OY3v3GoRJGbMmWKueqqq0I+V19fb3r16mUee+yxxscqKytNSkqKefbZZ40xxhw/ftx07tzZvPbaa43LHDx40HTo0MG89957xhhjtm/fbiSZdevWNS6zdu1aI8ns2LEjAbUKr2XHbGcdFy1aZDp06GAOHjzYuMyrr75qkpOTTVlZWULqa0zrOhtzpgH7j//4j7Cv8Xqdi4uLjSSzYsUKY0wwtnPLOhvj/+1sjDEXXHCBmTVrViC2cYOGOhsTjG0crUCepqmurlZOTo4GDx7c7PHBgwcrOzvboVJFZ9euXerTp4/69++v//qv/9KePXskSXv37lVRUVGzuiUnJ+vb3/52Y91ycnJUU1PTbJk+ffooLS2tcZm1a9cqJSVF1157beMy1113nVJSUhxfR3bWce3atUpLS2v2i5Pf//73VVVVpZycnITWM5Tly5erZ8+euuyyy/SLX/xCxcXFjc95vc5lZWWSpAsvvFBSMLZzyzo38Ot2rqur02uvvaaTJ08qIyMjENu4ZZ0b+HUbxyqmX+31upKSEtXV1bX6cb/U1NRWP+rnRtdee61efvllXXbZZTp8+LAeeeQRDRw4UNu2bWssf6i67d+/X5JUVFSkLl266IILLmi1TMPri4qK1LNnz1af3bNnT8fXkZ11LCoqavU5F1xwgbp06WL7ehgyZIj+8z//U/369dPevXv1u9/9TjfddJNycnKUnJzs6TobYzRp0iR961vfUlpaWmM5GsrflF+2c6g6S/7czlu2bFFGRoYqKyv1uc99Tm+88Ya++tWvNnaaftzG4eos+XMbxyuQYaRBUlJSs38bY1o95kZDhgxp/P8rrrhCGRkZ+uIXv6i//vWvjZOgYqlby2VCLe+mdWRXHd2yHoYPH974/2lpaRowYID69eund955Rz/+8Y/Dvs4LdR4/frw2b96s1atXt3rOr9s5XJ39uJ2//OUva+PGjTp+/Ljmz5+vUaNGacWKFWHL4YdtHK7OX/3qV325jeMVyNM0PXr0UMeOHVslw+Li4lYp0gvOPfdcXXHFFdq1a1fjVTVt1a1Xr16qrq7WsWPH2lzm8OHDrT7ryJEjjq8jO+vYq1evVp9z7Ngx1dTUOL4eevfurX79+mnXrl2SvFvnu+++WwsXLtSyZct0ySWXND7u5+0crs6h+GE7d+nSRV/60pc0YMAAZWZm6qqrrtKTTz7p620crs6h+GEbxyuQYaRLly5KT09XVlZWs8ezsrI0cOBAh0oVu6qqKuXl5al3797q37+/evXq1axu1dXVWrFiRWPd0tPT1blz52bLFBYWauvWrY3LZGRkqKysTB999FHjMh9++KHKysocX0d21jEjI0Nbt25VYWFh4zJLlixRcnKy0tPTE1rP9pSWlqqgoEC9e/eW5L06G2M0fvx4LViwQB988IH69+/f7Hk/buf26hyK17dzKMYYVVVV+XIbh9NQ51D8uI2jZsMkWVdquLT3xRdfNNu3bzcTJ0405557rtm3b5/TRWvXvffea5YvX2727Nlj1q1bZ2655RbTrVu3xrI/9thjJiUlxSxYsMBs2bLF/OQnPwl5qdwll1xili5dajZs2GBuuummkJeNXXnllWbt2rVm7dq15oorrrDt0t6KigqTm5trcnNzjSQzbdo0k5ub23jptV11bLg07uabbzYbNmwwS5cuNZdccklCLo1rq84VFRXm3nvvNdnZ2Wbv3r1m2bJlJiMjw1x88cWerfMvf/lLk5KSYpYvX97sEsdTp041LuO37dxenf24nSdPnmxWrlxp9u7dazZv3mweeOAB06FDB7NkyRJjjP+2cXt19uM2tkJgw4gxxjzzzDOmX79+pkuXLuYb3/hGs8vr3KzhOvzOnTubPn36mB//+Mdm27Ztjc/X19ebKVOmmF69epnk5GRzww03mC1btjR7j9OnT5vx48ebCy+80JxzzjnmlltuMfn5+c2WKS0tNT/96U9Nt27dTLdu3cxPf/pTc+zYMTuqaJYtW2YktfobNWqU7XXcv3+/GTp0qDnnnHPMhRdeaMaPH28qKyttrfOpU6fM4MGDzUUXXWQ6d+5sLr30UjNq1KhW9fFSnUPVVZJ56aWXGpfx23Zur85+3M4///nPG9vZiy66yNx8882NQcQY/23j9ursx21shSRjjLFvHAYAAKC5QM4ZAQAA7kEYAQAAjiKMAAAARxFGAACAowgjAADAUYQRAADgKMIIAABwFGEEAAA4ijACAAAcRRgBAACOIowAAABH/T8N5CwyCqzeawAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Estimate the false-accept rate on realistic test data (will take up to several minutes an a desktop-grade CPU)\n", "\n", "scores = oww.predict_clip(\"santa_barbara_corpus_test_clip.wav\")\n", "\n", "plt.figure()\n", "_ = plt.plot([i[\"turn_on_the_office_lights\"] for i in scores])\n", "_ = plt.ylim(0,1)" ] }, { "cell_type": "code", "execution_count": 25, "id": "b9e2f282", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:43:20.326550Z", "start_time": "2023-02-18T03:43:20.309552Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "False-accept rate per hour: 94.0\n" ] } ], "source": [ "# Calculate the false-accept rate per hour from this result\n", "\n", "false_accepts = openwakeword.metrics.get_false_positives(\n", " [i[\"turn_on_the_office_lights\"] for i in scores], threshold=0.5\n", ")\n", "\n", "print(f\"False-accept rate per hour: {false_accepts/1}\")" ] }, { "cell_type": "markdown", "id": "04d294cf", "metadata": {}, "source": [ "It looks like the false-accept rate for this model is very high, and would need to be reduced quite significantly to be viable for a production deployment. Of course, this was expected as the model was trained on a very small amount of positive and negative data." ] }, { "cell_type": "markdown", "id": "c758890a", "metadata": {}, "source": [ "# Create a User-specific Verifier Model" ] }, { "cell_type": "markdown", "id": "362c6812", "metadata": {}, "source": [ "As we saw, the simple model trained on this dataset performs quite well at detecting the presence of the wakeword/phrase, but often activates when it shouldn't, leading to an unnacceptably high false-accept rate.\n", "\n", "In practice, there are two ways to improve the performance of the model:\n", "\n", "1) Train on much larger amounts of positive and negative examples. The models released with openWakeWord are often trained on >100,000 positive examples, and over 30,000 hours of negative data.\n", "\n", "2) Create a user-specific \"verifier\" model based on examples of a specific person speaking the both the wake word/phrase and unrelated speech. The openWakeWord inference engine uses this verifier model to filter out likely false activations by focusing on only known speakers.\n", "\n", "We'll demonstrate the 2nd option here, as it's a very quick way to significantly improve performance at the cost of making the model far less likely to work well with other voices. The approach behind this verifier model are relatively simple, and are discussed in more detail in the openWakeWord documentation [here](https://github.com/dscripka/openWakeWord/docs/custom_verifier_models.md).\n", "\n", "For test data, we'll use 20 examples of the wake phrase generated with the [Tortoise](https://github.com/neonbjb/tortoise-tts) TTS model. While this is also synthetic data, it's very high quality and trained on different data than the TTS models used to generate training data. For unrelated speech, an English [phonetic pangram](https://www.liquisearch.com/list_of_pangrams/english_phonetic_pangrams) sentence was generated with the same TTS voice.\n", "\n", "This example wake phrase clips and reference negative speech (phonetic pangram) are included in the openWakeWord repo (in the `notebooks/training_tutorial_data` directory)." ] }, { "cell_type": "code", "execution_count": 26, "id": "f6b88046", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:45:18.538259Z", "start_time": "2023-02-18T03:45:18.525190Z" } }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Provide paths to positive and negative speech from the target speaker for training a custom\n", "# verifier model\n", "\n", "reference_clips = [str(i) for i in Path(\"training_tutorial_data/positive/\").glob(\"*.wav\")]\n", "negative_clips = [str(i) for i in Path(\"training_tutorial_data/negative/\").glob(\"*.wav\")]\n", "\n", "# Listen to one of the clips\n", "ipd.display(ipd.Audio(reference_clips[0], rate=16000, normalize=True, autoplay=False))" ] }, { "cell_type": "markdown", "id": "937e0df5", "metadata": {}, "source": [ "Now that we have the data (note that all of the clips *must* be 16 khz, 16-bit PCM WAV files), we can train a custom verifier model. This is simply a scikit-learn logistic regression model, using the same audio features from the normal openWakeWord pre-processor, so it is very fast to train and adds negligble time to the openWakeWord inference engine." ] }, { "cell_type": "code", "execution_count": 27, "id": "7f965293", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:45:57.606472Z", "start_time": "2023-02-18T03:45:56.587146Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Processing positive reference clips: 100%|██████████| 3/3 [00:00<00:00, 4.77it/s]\n", "Processing negative reference clips: 100%|██████████| 1/1 [00:00<00:00, 5.53it/s]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Training and saving verifier model...\n", "Done!\n" ] } ], "source": [ "# Train verifier model on the reference clips\n", "\n", "output_model_path = \"turn_on_the_office_lights_verifier.pkl\"\n", "openwakeword.train_custom_verifier(\n", " positive_reference_clips = reference_clips[0:3], # use 3 reference examples for the wake phrase\n", " negative_reference_clips = negative_clips,\n", " output_path = output_model_path,\n", " model_name = \"turn_on_the_office_lights.onnx\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "4ad4d2ec", "metadata": {}, "source": [ "After the model is trained, we can instantiate a new openWakeWord instance and include the path to the trained verifier model, as well as set the threshold score from the base model required to invoke the verifier. In practice, you can set this threshold score a bit lower than normal, though as usual actual testing in the deployment environment is recommended." ] }, { "cell_type": "code", "execution_count": 28, "id": "50b99677", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:46:01.357976Z", "start_time": "2023-02-18T03:46:01.146646Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/dscripka/anaconda3/envs/torch_gpu/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:54: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'\n", " warnings.warn(\n" ] } ], "source": [ "# Create openWakeWord instance with verifier model\n", "\n", "oww = openwakeword.Model(\n", " wakeword_model_paths=[\"turn_on_the_office_lights.onnx\"],\n", " enable_speex_noise_suppression=True,\n", " vad_threshold=0.5,\n", " custom_verifier_models={\"turn_on_the_office_lights\": \"turn_on_the_office_lights_verifier.pkl\"},\n", " custom_verifier_threshold=0.3,\n", ")\n" ] }, { "cell_type": "markdown", "id": "6666dadb", "metadata": {}, "source": [ "Finally, we can run the model on our test clip from the Santa Barbara corpus and see if the false activation rate has decreased to an acceptable level." ] }, { "cell_type": "code", "execution_count": 29, "id": "19fd5f51", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:47:25.076640Z", "start_time": "2023-02-18T03:46:07.021368Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGiCAYAAAA1LsZRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7XUlEQVR4nO3deXxU5aH/8S9r8FqJVTSAIpe219beqK3hqsGtasVS9dbb/iqt/QlW7c/UIlW016JtUes1aiu1rYIoIPWWIlpAbUUkVNnDFgKyhEUIScCEkEAmgZD9+f1BM2SZySw5M885M5/365WXMnOW5znLc77nOcv0MMYYAQAAWNLTdgEAAEByI4wAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqyIOI8uXL9ctt9yiwYMHq0ePHnr77bdDjrNs2TJlZGSoX79++tznPqeXX345mrICAIAEFHEYOXbsmC6++GK9+OKLYQ1fWFiob37zm7rqqquUn5+vRx99VOPHj9e8efMiLiwAAEg8PbrzQ3k9evTQggULdOuttwYd5pFHHtG7776rgoIC/2dZWVnavHmzcnNzo501AABIEL1jPYPc3FyNHDmy3Wc33nijZsyYocbGRvXp06fTOPX19aqvr/f/u6WlRYcPH9aZZ56pHj16xLrIAADAAcYY1dTUaPDgwerZM/jFmJiHkbKyMqWlpbX7LC0tTU1NTaqoqNCgQYM6jZOdna0nnngi1kUDAABxUFJSonPPPTfo9zEPI5I69Wa0XhkK1ssxceJETZgwwf9vn8+n8847TyUlJerfv3/sCgoAABxTXV2tIUOG6LTTTutyuJiHkYEDB6qsrKzdZ+Xl5erdu7fOPPPMgOOkpKQoJSWl0+f9+/cnjAAA4DGhbrGI+XtGMjMzlZOT0+6zxYsXa/jw4QHvFwEAAMkl4jBy9OhRbdq0SZs2bZJ04tHdTZs2qbi4WNKJSyxjxozxD5+VlaWioiJNmDBBBQUFmjlzpmbMmKGHH37YmRoAAABPi/gyzYYNG3Tttdf6/916b8fYsWM1a9YslZaW+oOJJA0bNkwLFy7Ugw8+qJdeekmDBw/WH/7wB33nO99xoPgAAMDruvWekXiprq5WamqqfD4f94wAAOAR4R6/+W0aAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWRRVGpkyZomHDhqlfv37KyMjQihUruhx+9uzZuvjii/Uv//IvGjRokH74wx+qsrIyqgIDAIDEEnEYmTt3rh544AE99thjys/P11VXXaVRo0apuLg44PArV67UmDFjdPfdd2vbtm166623tH79et1zzz3dLjwAAPC+iMPI5MmTdffdd+uee+7RBRdcoBdeeEFDhgzR1KlTAw6/Zs0a/eu//qvGjx+vYcOG6corr9S9996rDRs2dLvwAADA+yIKIw0NDcrLy9PIkSPbfT5y5EitXr064DgjRozQ/v37tXDhQhljdPDgQf31r3/VTTfdFHQ+9fX1qq6ubvcHAAASU0RhpKKiQs3NzUpLS2v3eVpamsrKygKOM2LECM2ePVujR49W3759NXDgQJ1++un64x//GHQ+2dnZSk1N9f8NGTIkkmICAAAPieoG1h49erT7tzGm02ettm/frvHjx+tXv/qV8vLytGjRIhUWFiorKyvo9CdOnCifz+f/KykpiaaYAADAA3pHMvCAAQPUq1evTr0g5eXlnXpLWmVnZ+uKK67Qz372M0nSRRddpFNPPVVXXXWVnnrqKQ0aNKjTOCkpKUpJSYmkaAAAwKMi6hnp27evMjIylJOT0+7znJwcjRgxIuA4tbW16tmz/Wx69eol6USPCgAASG4RX6aZMGGCpk+frpkzZ6qgoEAPPvigiouL/ZddJk6cqDFjxviHv+WWWzR//nxNnTpVe/fu1apVqzR+/HhdeumlGjx4sHM1AQAAnhTRZRpJGj16tCorK/Xkk0+qtLRU6enpWrhwoYYOHSpJKi0tbffOkTvvvFM1NTV68cUX9dBDD+n000/Xddddp2effda5WgAAAM/qYTxwraS6ulqpqany+Xzq37+/7eIAAIAwhHv85rdpAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYFVUYmTJlioYNG6Z+/fopIyNDK1as6HL4+vp6PfbYYxo6dKhSUlL0+c9/XjNnzoyqwAAAILH0jnSEuXPn6oEHHtCUKVN0xRVXaNq0aRo1apS2b9+u8847L+A4t912mw4ePKgZM2boC1/4gsrLy9XU1NTtwgMAAO/rYYwxkYxw2WWX6ZJLLtHUqVP9n11wwQW69dZblZ2d3Wn4RYsW6Xvf+5727t2rM844I6pCVldXKzU1VT6fT/37949qGgAAIL7CPX5HdJmmoaFBeXl5GjlyZLvPR44cqdWrVwcc591339Xw4cP13HPP6ZxzztH555+vhx9+WMePHw86n/r6elVXV7f7AwAAiSmiyzQVFRVqbm5WWlpau8/T0tJUVlYWcJy9e/dq5cqV6tevnxYsWKCKigrdd999Onz4cND7RrKzs/XEE09EUjQAAOBRUd3A2qNHj3b/NsZ0+qxVS0uLevToodmzZ+vSSy/VN7/5TU2ePFmzZs0K2jsyceJE+Xw+/19JSUk0xQQAAB4QUc/IgAED1KtXr069IOXl5Z16S1oNGjRI55xzjlJTU/2fXXDBBTLGaP/+/fq3f/u3TuOkpKQoJSUlkqIBAACPiqhnpG/fvsrIyFBOTk67z3NycjRixIiA41xxxRX69NNPdfToUf9nu3btUs+ePXXuuedGUWQAAJBIIr5MM2HCBE2fPl0zZ85UQUGBHnzwQRUXFysrK0vSiUssY8aM8Q9/++2368wzz9QPf/hDbd++XcuXL9fPfvYz3XXXXTrllFOcqwkAAPCkiN8zMnr0aFVWVurJJ59UaWmp0tPTtXDhQg0dOlSSVFpaquLiYv/wn/nMZ5STk6P7779fw4cP15lnnqnbbrtNTz31lHO1AAAAnhXxe0Zs4D0jAAB4T0zeMwIAAOA0wggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCCJS29Ckacv2qLDimO2iAAASBGEEEfnNBzuV/f4OXfvbpbaLAgBIEIQRRGT9vsO2iwAASDCEEQCuc6y+SbsO1tguBoA4IYwAcJ3rn1+mkb9brnWF9MQByYAwAsB1yqrrJEmLtpZZLgmAeCCMAAAAqwgjQIKYvmKvxsxcp7rGZttFAYCIEEaABPHUewVavuuQ3tpQYrsoABARwgiQYGob6BkB4C2EEQAAYBVhBAAAWEUYgecdb2jWn9cUqdR33HZRAABRIIzA855dtEO/eHurbvnjSttFAQBEgTACz1u6s1ySVHG0wXJJ4DQjY7sIAOKAMAIAAKwijCSRT8prdOtLq/TRjnLbRQEAwI8wkkTG/SVfm0qq9MNZ66OeRg/1cLBEAAAQRpLKkVruqQAAuA9hBAAAWEUYAQAAVhFG4Hk9enAfCwB4GWEEAABYRRgB4FqGd54BSYEwkkR4LBcA4EaEEQCIob2Hjqq2ocl2MQBXI4wAQIxsLqnSdc8v07W/XWq7KICrEUaABMNtFu6xaFuZJOlgdb3lkgDuRhiB53EnDNyKbRMID2EEAABYRRhBRAwXAYCw8T4+IDyEkSRCwwjEF4/TA+EhjHhEfVOz7SLAIzj8uQc9iUB4CCMesPWAT1/8xSL9z3vbbReFMz0AgOOiCiNTpkzRsGHD1K9fP2VkZGjFihVhjbdq1Sr17t1bX/nKV6KZbdL6zQc7JUmvrii0XBIAAJwXcRiZO3euHnjgAT322GPKz8/XVVddpVGjRqm4uLjL8Xw+n8aMGaPrr78+6sICAdFZ43rrCg/rZ29t1pFjDbaLAsCFIg4jkydP1t1336177rlHF1xwgV544QUNGTJEU6dO7XK8e++9V7fffrsyMzNDzqO+vl7V1dXt/gB4123TcvVW3n79+u/2LzUCcJ+IwkhDQ4Py8vI0cuTIdp+PHDlSq1evDjrea6+9pj179mjSpElhzSc7O1upqan+vyFDhkRSTAAuVXS41nYRALhQRGGkoqJCzc3NSktLa/d5WlqaysrKAo6ze/du/fznP9fs2bPVu3fvsOYzceJE+Xw+/19JSUkkxUQQXM0A4svwMA0QlvDSQQc9OrywwhjT6TNJam5u1u23364nnnhC559/ftjTT0lJUUpKSjRFAwAAHhNRGBkwYIB69erVqRekvLy8U2+JJNXU1GjDhg3Kz8/XuHHjJEktLS0yxqh3795avHixrrvuum4UHwAAeF1El2n69u2rjIwM5eTktPs8JydHI0aM6DR8//79tWXLFm3atMn/l5WVpS9+8YvatGmTLrvssu6VHhCXnwDA6yK+TDNhwgTdcccdGj58uDIzM/XKK6+ouLhYWVlZkk7c73HgwAG9/vrr6tmzp9LT09uNf/bZZ6tfv36dPgcAAMkp4jAyevRoVVZW6sknn1RpaanS09O1cOFCDR06VJJUWloa8p0jiAy/KQN4E/suEJ6o3sB63333ad++faqvr1deXp6uvvpq/3ezZs3S0qVLg477+OOPa9OmTdHMFt0U6CZjOKPUd1zvfVyq5hYenwCASPHbNIADrvnNUv3kLxv1l3X0CuIkHu0FwkMYARzQ0NQiSVqx65Dlkrib4egMIADCCJBgONwnN9/xRk1evFOflB+1XRQgbIQRD3DTyaQbbzvhXhjvYF3F3qR3tuoPH36iG363zHZRgLARRgAggWwsrpLkrpMYIBTCCAAAsIow4gH0bAPeROcEEB7CCOAgDj7O4ukbIDkQRgAE9fTCAj27aEdMpn28oVnfeyVX05btiWr8P/5jt257OVd1jc0OlwxAvBFGYmzu+mL93+lrVV3XaLso6MLxhuaEWUdOXdWrPFqvV5bv1dSle1QTg2Uzd32x1uw9rOz3ows7z+fs0rp9hzV/4wGHSwYg3ggjMfbIvC1a+UlF1Gd/CM2Jg++XJy3SRY8vVm1DkwNTSwyNzScvkbS0OD/9443OTLShyb09IzZu9+IeM3gRYSROjtZxkHOz1lsT9lXU2i1IDDW3GM1ZV2z1ZVjcAwIgkIh/tReAN81ZV6xfvL1VkrTvmZsslyY5EL2A8NAzkkTovk1uG4uP2C4CAAREGAEAAFYRRjzATR0aXPJPHqYbFxmaWwKPG+lv0/BbNkByIIwgZmrqGnXHjLV6Y12x7aKguyLIBBVH63XJr3P02IIt3Z5tODe8ko/bI77BiwgjiJlXl+/Vit0V+vn87h+UusLJs7u8nlsk3/FGzV7bdQjtTs8LgMRCGPEANzXZkRz4q3mcGUmOy5pAeAgjQBuJfLbegw58AC5FGEkiXM5AJGJxVs9LzwAEQhgBEkwsDvcEWQCxRBgBHMSJP9qyEeJ4HBpeRBiJk+4co2LRtKzdW6mWIO+CALyEAAh4H2EkibRttEe/skZz1vP+j464ybP7wl2CybCsCUpAeAgjSezt/AO2i4A4clvvfSI/uWQTNwnDiwgjSSTeByO3HfwQmdjcCBvZRsFhFUgOhBEAIUWbK+dv3N/pM9/xRt360irNWrWvW2UCkDgII0AbXDo4yYmOrQlvbu702bRle7SppErlNfUOzAEd8TQNvIgwAjiKMBNKbUOz7SIAcBnCCGKG++jcpTvny5Gsyq5OzGNxc6WbNzN62oDwEEY8IFbdrokSFtz1iKibygIA3kAYiRMOUYiXSLa1xuaWoC+/c2NWrThar8cWbNHWAz7bRQHgIMJInDU0tWj8nHy9ub4k7HGc6tp2ogeBUOUdeUVH9MAb+Sqvrgv4fV1jsy79nyX6Py+vDjktt6z3n8/7WLPXFuvmP660XZSwuKvXDuEyxujphQWavbbIdlGSRm/bBUg28zbu17ubP9W7mz/Vbf8xxHZxVNvQpD69eqpPL+dzKTf12/WdqSdChu94o1774aWdvs8rOqIjtY06UlwV55JFb0dZje0iuB67XfdtLK7SK8v3SpJ+cNlQy6VJDvSMxFlVbaPtIvgdrW/Sl3/1ga557iPbRUEMFVXW2i5C1BLlviZ4S3Wde9rpZEEYiRM3tqmtZ5mf+gJ34yO2ahuatP9I/IICPVXxl8xP0/Ba+uj5XHTSGi+EEXieuw6y4TfAI575UFc++5H2HDoaw/K4Cy/kSg6fVh1XZvaHevHD3baL4jl/2/ypLn5ysbLfL7BdlLgijACWtF6yW77rkOWSBOaVM1uvlDOZTM7ZpbLqOv128S7bRfGcJ/62XZI0bdleyyWJL8IIkl6yHswiqbZTPRplXBJMCi1Juk8heoSROPvzmsgfFXPqQEAPeXJy02Hh8LEG20VIfOzn3WZ3Ebppj40fwkicHag6brsISHDJ2ZQB8DLCCGKGFz6FJ15LKRHWh+d690iGiFCyXuEijCBmkvmxxlDyi4/YLkLMOBl6PBc+gG5K1laTN7AipMbmFr2wZJeu/MJZSbujhCvcs5r/mnLyFexufdzVDWdobigDko9b98lERs8IQpqzrlgvfbRH3391je2ixJyNg5+tdi+Sniu3NM2eCyduWXDwDKee7mtoanFkOvFCGEki0baL+yq8+zrxeOOEqntaWozumLHWdjE8zQ2bYCLcn+Rl2z716fxfvO+pF6cRRoAE13qmFWlQshGs8kuqtGJ3RfxnDLiEE/0izy3aKclbL04jjACWcQ55UnOL167DhGChOgm2BJOO5y5FOoQwEoaWFqO564u1+2D0P1+eKBsYB84TVn9SoX/9+Xt66M3N3Z9YEl3bSYSqltfU6fXcffyyawJLgM3UcwgjYZiff0CPzNuiG3633NHpGmMS+lXkXrxuHO7B8vbpJ+5rmLdxv6pqY/dW0T2HjurZRTusvLk02k2zq2XY3c3dDWHm/05fq1+9s00T522xXRQkuDtmrFVR5THbxYgLwkgYNpdUxWS6//niKo19bX1Mpu1FLS1GeUVHVNfYHNF4th/DO1rf5P//aA62XZV+1O9XaOrSPfrvv34c+YRDiOy3aRyaZwJcRNh18MSvLOdsPxhy2KZEu+yUJGyutbYnqCt2V+j+OfkRT8MNoT1SvGfEoi0HfLaL4CovL9+j5xbt1DXnn6U/3XWp7eK4QuvjeZtKon9JmhNhLYE78GKmoLRaM1YWxn2+HjwOoY2Ou9rB6uT4cUl6RuLEDUnVdg9CKH9avU+StGzXobjOt+2BNpEPui5f/Qnn6YXeeawSLpLAbVBXCCNAN7n9PojWbl83BK1Q9xF1XBZuvqzj5rKhe7ye271YfsKICyTyTazorOP6DudG34ju74i0QC725zXFWh7nnjIA8UcYQUjBztyLKo/p21NW6YNtZQG/58yxs0/Ka3R59j9iOo+6Rm+9BjqUMTPX+f+f3I5El6ybOGHEA9x6pvuzv36sjcVVuvd/82wXxTMeW7BVB6vrYzqP3y3ZJV9t6HdgRNLocb9JZwQjxEKy9pQTRsKQ7A1xsH0j1AEvXu8Z8dLqaQmwMGOxfa38JDFfqZ7s+2I43H6jOiITTTvqxW2AMOICSRqEE1KsVqUT0/Vg+wSX+nDHQa3ZW2m7GDFjc19J1sMB7xlJIm4/Fnnxja1OiHWtbTdux9q8FC4WCPPxVV5Tp7tmbZAk7XvmJsulSTwdt+dkufeOnpE4ocEMLRF2ulDBIlDgsnUWFq/ZXvo/S/z/nwjruFXi1CQyNn6aIJk4sY948bSOMOIBydroeYVXg2aoYjtVr2MNkb3eH97HJUFEKqowMmXKFA0bNkz9+vVTRkaGVqxYEXTY+fPn64YbbtBZZ52l/v37KzMzUx988EHUBU5EHj2WAQFxHAKckyyXryMOI3PnztUDDzygxx57TPn5+brqqqs0atQoFRcXBxx++fLluuGGG7Rw4ULl5eXp2muv1S233KL8/Mh//McWr575xkSbUx7OfpyRLI1NonHrI5hsTd1nc5906WYVcxHfwDp58mTdfffduueeeyRJL7zwgj744ANNnTpV2dnZnYZ/4YUX2v376aef1jvvvKO//e1v+upXvxpwHvX19aqvP/kuhurq6kiLmVBoXLrmqVAUqKxxK39kM/LUcgUSRLI+ORdRz0hDQ4Py8vI0cuTIdp+PHDlSq1evDmsaLS0tqqmp0RlnnBF0mOzsbKWmpvr/hgwZEkkxHefFFRuQw/VI1gTfFRZJ9yTKrgYgMhGFkYqKCjU3NystLa3d52lpaSorC/xK8I6ef/55HTt2TLfddlvQYSZOnCifz+f/KykpiaSYnuPW7t5W0YYxJ0Ncqe+4Zq4sVE1d6DeLRsr20o/5o71RVtCNm2Wg7vNEekLHC9quA7e3XV7U0JRYP+cQrqjeM9Lx7W7GmLDe+DZnzhw9/vjjeuedd3T22WcHHS4lJUUpKSnRFA0e1tV12u9MWa1PfXXa+qlPk2/7SvwK5RKxaPRtHEi6ey3eTcHDPSWB07zfG+69CkTUMzJgwAD16tWrUy9IeXl5p96SjubOnau7775bb775pr7+9a9HXlJ4TqTHuq4ONJ/66iTJ87/g6r0mInLeb8ijt2W/T5nZ/9A7mw7YLgq6gQ6f+IsojPTt21cZGRnKyclp93lOTo5GjBgRdLw5c+bozjvv1F/+8hfddBNv7OuuY/VNmrZsj4oqj1ktRzIfdNrq7tl6rH9HonXyblhfburZiIUfz85Tqa9OP31jk+2iWOWCTQ0eE/GjvRMmTND06dM1c+ZMFRQU6MEHH1RxcbGysrIknbjfY8yYMf7h58yZozFjxuj555/X5ZdfrrKyMpWVlcnn8zlXC4+LtHnOfr9A2e/v0A2Tl8ekPB1xlhBbNNzeFGi/aGxOzuv9HdFkIFIR3zMyevRoVVZW6sknn1RpaanS09O1cOFCDR06VJJUWlra7p0j06ZNU1NTk37yk5/oJz/5if/zsWPHatasWd2vQRJau/ewJKkhCRu+QAcAN5zxx1qyNO7JsC6BWPPifhTVDaz33Xef7rvvvoDfdQwYS5cujWYWrmJ7vTq1YTkyGbpJuhTqplAbjUTUT9NYiEBsXt5ijDcPfKEkYp3cjt+mAbqpuwdQWw1fJMWOpI7xDhTB5tfc4r2nhYBkRRhxgXg13k7PJlS5Iz3IhtOQc+Kc3MI92P/y7a36yhOLVV5dF+MSAbGVLL00hBEPSpaN04tCPRkT6GAar/UZr83GDdvn/64pUk19k15bvc92UeLODcsfiBRhxANsX0cP1rg53egl+mOfwdC1nzgIAnCa7fY/XggjYUiSbQFRYvtwDgfzxOD11ej18nsRYcQFPNUjYPlowW9hxE8iLOpEqIPbEBjdz4uriDDiQjnbD+re/92gqtqGgN9H263vxQ000QRqyMNp3LtzUA0adi0cqLsbDtwe3JNtH3P32oCXRPWekWQT7wbmR69vkCQN+MxO/c9/XdjuuzfXl7i+QU42bbcPem7ijyWORJYsPVGEERc7VFMvqf3G+N/zPrZUmtizdSNndwNErN7X0R2BAmvcntrpYkbxbliTMbhzQzS8GGC4TOMCnEyHj0WV3Nx+oI31jx4iTliNcUcYcTG3HHgJS84JfM+I8y2fEwdtjqsA4oUwAs8LduDdf6RW6woPx7k0sWHjXhQ7IZQElAgIsogUYSTBHK1vsl2EqDl9ff/KZz/SbdNytWW/z9HpdtTdoEC73RbdcF7CDdvu5PbLmYEQRuIkmgNtpPv5wi2lSp/0gV5Ysivg94lwPTuatm/T/irHy+GkBFgtANAthJEweOUg/uiCLZKkF5bsdnS6Hql+cHE8ews1p2iXpRPboBfPlhA5N+yvdJggUoQRuEayHiybmkO33HSHB9flomGxIQm5IZBGijDiCR7csmIkmoNyrI9H3Z3+mxtKHClHW259v0aiZyovHgQANyCMuEDoBtqbLXgytsuh6hyo9+dIbWNsChNCqMBiZ6vz9laT6GFL8voaglsRRsJAF7l7BLp3ItTZqNtXX09a95hx+aqHS3n9krEXe+gII67mbFPqwe0zKYT1Q3mRTrPN2m4NY15soMLhpnq5qSzxEGy7TLblEEvJsigJIwkkHhttLOcRzn0O9FJ5GwcpAIEQRuKkq24/t95s2F1uqVU8A0ysHu11QncWg1vWpZeQm+GEZNmMCCNhsPWekUgbs2TZaJ3Wdrk5fQA5UHVcqz6p6HIYr1+ftq2rdRbvnjR6fiJXXl2n++fka/2+xPjpBjfwYptCGIFrxGoHshnSrnjmQ/1g+lqt3VsZdJiwbmCNQSVCHadtXBLjYJ58Hl2wRX/b/Km++3Ku7aL4sR3GH2EEnuLV3p8NRUfiOr9EvfQXKa+8PTmZFVXW2i4CXIAw4gLBTkBPfuzNBjVepQ41n1if4Hd7+uEcMB1YmF44LodalpHWIe6XaTy6rwaSu6dS/2fqau0sq7FdlKQW1Rblwc2QMBIn8ThTDfnCLQ9uoMkg1qvFTTdSuqks6Nr3X12jDUVHdNes9UGHYX3CKYQRF3P6ILXr4FGHp+gNkbSXsQpsXZ2hhzXPEJXYst/XfppRbD31Tc2687V1emX5nojHbTdvQm9CqThaH/E4idRDhPggjLiYG086YlmmsHqP3LhQusmJZvsfOw52exrv5H+qpTsP6emFOxwokTvE+8ydIJYYWI3xRxhxgQQ8viaZk2sw6P0//regBnqdvTseHa9taLJSjraiebU/N+vGFksX8UAYSSBebzS82rW7sbiqW+M7UWsnlh1PniAc4f18gbdbI2+X3psIIy7Gq887i2aJRLIco1nk//3XjyMfqY1YZ4DW6RM1Yo9lDESHMJJAkrYhdNEZfaiiBPo6nF6NSF8zb8K4dBRqGokgGeO84z1cXb3hNgmWsBdPCr24KxNGXMCLG3s4EvHgFi3T4b9u5IXVxTaVeNy4T7CZxR9hxMXcuJO2FeoMLEEzlvNc2vJFu/rcdO9PvEuSkPfdJGCV4D6EEXhKNL1IbghFXZXBmRtYI9epSC44kDpdApurPhkuYQTjpkAKbyCMICKRNDFeaYqdDCvBX+1/4ouA94zEoN2O5mDg1cNH4aFjmr22SI3NLbaLkpi8siPHiBtOZiLlxR663rYLAOf29XhvgKF6KdyyEyfDGWpYP2/jwQYqHG/l7ddbeft1vKFZ91z1OdvF8aN3wLvctK+4qSyxRM9IAnHbjbDJEAKc4JaDlhvavO5sMXkBfhnZZbtEXLhgNUbEa+VFbBBG4iQZG8VwVR6t146y6qjHD9WYPb1whya9szXq6ceaEyHAibMnt4QiuEtXJxW0a7EXzUmmF/dkLtMgZsLdhzKeWhLbgkj6U26RnvhWesznE8zJ18F3/i6sN1rGoNUPNVuvHWi8Vl4AJ9Ez4gKhf88kfmWJRLJcy4y1WPRIhHOJzI3H7u4sCTdcFnTDHmF/KSQWlmd8EEbihON2cuuqQXPLtuGWcjjJDQEl8STghtKBm/aFZDnpI4wkkGTYaN1+aHF7+aLltpuj4W5J0BS5mheXP2EkDLZWrNebfw5gnUW7KdU2Nnc93Q4T7u57RnzHGyMe3zZXbG4uOAi4oAgRccNqg32EETdw+d4YbePm8mq5Rjg9WqEOtF2Fj3BDYdtirCs8HNY4kXLLNmGM0brCwzp8rMF2UdANxhhVHK23XQw4gDASBltnXJGGgK4OOk0J8nbKQFX0RJdkF+vGVvGTuedqSUG5bpuWq2ue+8h2UVzPzZvJ0wsLNPypJVqQvz9m8/DifuKFJrEjwoiL+X/p1YF9ocFCGPHgPtxtoRoBN9/X07Z3pTsNcCyquH5f6J6aSEr8j4KDkqSa+qYuh9tXcUzvbykNv3cpgjIksnjt+6+uKJQkPfX3AsemeaDquFbsOuTY9BAewghCirZd8eKTDLEqc1dT7RnjI1hrAAoZEtp8b2vNBQtr3305N6rpBTsohhuYvvbbpfrx7I3K2X4wqvknIjfmaSe31yue+VB/+PATB6eIcBBGXMCpA2BXZ91ueLvmO5sO6NrfLtWugzW2i+Iqjrw9tcMkuvueES/2akVW5siW+cbiqoiGl7wZxsPhxW0D7kcYcbHWrmE3nolE46dvbFJhxTE98MYm/2fhdL+35fYGPljp3N6At7SYDofn2BS4u5ty1+OHX+ZY7VNuuAzngiLEvQxdXUbbst8nX633ng7rDjdsh5EijITBg+s1ZiJZFsHah7b3r0Tb/R6NYA1WPANOoMXnxObV3Wm8t6XUFQ1YvG4WtF9TOCnYVrP6kwrd8uJKXfPb6G9Udvl5RMIgjMSJ28+MYyFYlZ2+RyLcybl1HcQ6A4RzgC+qPNZhHCmRm2EX5C7PiPXlPCdWRbByLP7nvT5VSdYz4kWEkRgxxmjmysIwh41xYRSbxjfaSfZ08ZHADffWRGPvofZhIpx6tN3ujGm/PruzScZyCbr9Phdvbj3udM+f1mv0tFzXPdnlBV6sNmHkn/53TZEmvLlJzS3OtGjvbSnVk3/f7si0bDpW36RS3/Goxg3+JEP0u0rHac5YWRjw5sJAbxC1eazq+lJQ95uOuRtKQg7TVUDpuNlbe7dOnI4ekYZOt9+rFE9Or6JAS7ahqUVLCsq1tvCwCiuOBRiiwzSCbLBePblIRoSRf/rl21s1f+MBLd5W5sj0dpUlxhMjl2f/Qwu3BF4mgXb/o/VNWvVJxT9fsha4gXDyMs2vgwS+pwJ8HqzBqq3v+lXrTgrUkLvh7K3jwdbGI87ddaDquB5+a7N2lFWHHNYNyxzhCWebCTYM69k7etsugNs49ZsczW7oM3ZATV3XL4XqaOzMdcorOqKHR54fdJh4XKYpCHBACnYG9tVf5wScxqKtpRqYeoq+MuT0bpenq83BXnt5slAtpn3D3dDUor9/XGqhTNHbUVajHWU1ejv/QMhhY7XMj4Z4iVo8ON5zYaEta1uH7sw+lvejwFn0jHTg0FUaRfLC00Ta1vOKjkiS3lgf/LJBrF/ydWIenWcSqFH5wfQ17Yf559ooKK1W1p836taXVjlSnrfygr+uOhbZLOKeDWPalWNyzi69sGS3s4XqQkuLUYtDO19TGNOJxeWgo/VNKvXVOT7dVk3NLY5dRo6Ejaes2t/PFEadE6kRdYIHe4QIIx20OBSDnZqO5K7tKty20JgY3TMS5nCB5hDoAL1+35GA4+8L4zp1IMt3HdIdM9bqYHX7g9KhmuA/5hXr69rhrLOOw+w/Ev59QjV1je16FKPZ8kf9foVufGF5yP3Gyd9rimxCoQfZdsDnzLwCaGpu0ZXPfqSvT14W956KaLZOJ/MLl2mi4MFwRhjpINCOHs3BwokzmHDanGNx7hZubgmvy8cYE7T8vRzqGnnm/R0RDR+vNnzF7gr96p2tYQ8f6wazrDrw2fqavSdfOGdkotrOW1qMLnx8sS5+YrHqm6K/92bnwRrtLj+qyqPx+RXdzfurHJ9mLDevT6vqVFZdp8KKY6pvcs+PXsbu3qKT0+3WZZpkTSMerDZhpINAGSKaHc7JnpGu/LLNQa+r7c+p4jQ1n5xQV/PruBy/1eZyh1P7ycvL9gT/0nIjtPfQMd0/Jz+sYRubTbcO5OHquEhmrd7n//+O94yE63jjyXK39v50Z1ub1tU6jUKwe7cOxuBySjT7/J5DRzV3fXHIk5devU6unHAuQ8VS281k+orwXl/QHWH9tEHQp2niM/9o7Ks4pr+sLVZjc4t8tY2OXoLrHY9r4Q4jjHQQqEFp+1FLiwmrmzSS69/d6XZ9P8iTLpJ05FiDfvbWZq0rPOxYOAp3h2kxpt1OvLmkqt13ocxcWRjwZsCGMM8Kw9kV6xqDB4C2JXz83W2au744rPm22l1+VH/b/GlYwy4pOKhLnszx182Jeyc69nLsP1Lb5fBOZufuNN4bigJfNotWsO217cG9uybM3aSb/7hCjc3t57Xqk8qAw9c3Neubv1+hCXM36frnl+mReVv0Rojtq1ebpBhqH3T6sl9XIXVyzi5H59Wq4ztwQg4fk1KEmKcx2n2w5p9PDkbua79dqkcXbNETf9umi59crNumOfc26l49vXdo916JY6C67uT17tb9vO2Bqm1Y+PrkZbpv9kZJUnFlbdAD2tEAj4uWB+kub51HUeWxdvNa+UmF6hqbu7zm3zYAdxzsqfcK9Fbeft02LdexnTX8MBK8EdlYXKWfz/tYjV3sxE/+fbsef3dbNEVUU3OLygKc+XYsz4hnPgw6jbbDzlq9T4/M26Lluw5F3fCEcqyhWfuP1OpA1XFlPJWjZxdFdgkqlCuf/UjlXdy3YmRUHeaTZC0tRq8s36ONxUfabVetXeJuevog2JNbkT7R1VWV5ucf0NYD1Vq9pyLkdOoam/X/Xs/T9tJqzW/z1E9eiBDW9tJmqLAay5PiT8qP6v2tzrz+IFzduiwVw2UxZ12Jbvjdco37S+ge0JYWoyPHAl+C/POaE0E01DZQ2xD8kvyirWVatuuQ/9992oTth97crN/l7NIDb+RbeTIqXFGFkSlTpmjYsGHq16+fMjIytGLFii6HX7ZsmTIyMtSvXz997nOf08svvxxVYWMhd0+lLnp8sf/fxhjtKKvWl365SI8u2HLiszbD7604pve3lumhNzfr6t98pFtfWqW6xmbt7vBLtLkdGqbZa4t16dP/CFqOJ/62Xdf8ZqlmdHhr6/yNB7rsSWgxJ856N5dUdeo1aPuK73B6I44ca/BfLvAdb9SBqs43MbbtIm77+Ozb+Qe0/dOT/zbGqOhw8LPxN9aXaP7G4E+YSNLSneUBPw8ViO7604aA90m0nrE3NbfIGKPDQRqHYMbMXKfpKwu77FEJpavV0GKkPyzZrSO1jZq6tPPlit0Ha/T7Jbv99wkdb2gOGLqkwA1XVz01xkgvL9sb9PtXlp/8bn7+AT29cIe+PWW1qxs3KfBZfV1js4oqu+4pkiK/HytUSNh1sEbPL97Z7qDRcdxdB2u0dm9lpxue24aRUJdpAt0n8Ul5jf77r5u199BRfxtX0sX+2ZYx0uu5+5RffERfn7xMv/lgZ4fvO5dnxe7gwWzOumJtiOAHMm99aZWWB1hmbdU2NAdc/m17iZpbTJd1NsZo+org+0BHryw/sY8uCuPdVFl/ztNXf52jLfuju8n5pY8+0Zd/9UHA9ypVHq1X1p/zNHbmOv8yaLu9zNu4X7//x269velTbSx2tufRST1MhK3J3Llzdccdd2jKlCm64oorNG3aNE2fPl3bt2/Xeeed12n4wsJCpaen60c/+pHuvfderVq1Svfdd5/mzJmj73znO2HNs7q6WqmpqfL5fOrfv38kxe1SS4vR5x5d2OUwaf1TdLA6+BklYmvfMzdpy36fbnlxpe2iAFZcNuwMrS2M7Neto3HO6afoqf9K159W79OYzKG6a9aGmM/z19/6dz33wU7/+4yyrvl8l/eC/eKmC/TUewXdnu+/nf0Z7S4/GtE4/9K3l2obTpyIZH7uTOXuDXwZ7usXpOnVMRlauvOQfjhrfbfLGg8//trn9cg3vhSTaYd7/I44jFx22WW65JJLNHXqVP9nF1xwgW699VZlZ2d3Gv6RRx7Ru+++q4KCkxtQVlaWNm/erNzcwNfI6uvrVV9/MgD4fD6dd955KikpcTSMfGfqau1MkDelAgDQHa+OGa7Mz5/p6DSrq6s1ZMgQVVVVKTU1NfiAJgL19fWmV69eZv78+e0+Hz9+vLn66qsDjnPVVVeZ8ePHt/ts/vz5pnfv3qahoSHgOJMmTTI6cXWEP/74448//vjz+F9JSUmX+SKi18FXVFSoublZaWlp7T5PS0tTWVng62ZlZWUBh29qalJFRYUGDRrUaZyJEydqwoQJ/n+3tLTo8OHDOvPMMx19brw1sTnd4+Jm1Jk6JyrqTJ0TlZfrbIxRTU2NBg8e3OVwUf02TcdAYIzpMiQEGj7Q561SUlKUkpLS7rPTTz89ipKGp3///p5bwd1FnZMDdU4O1Dk5eLXOXV6e+aeInqYZMGCAevXq1akXpLy8vFPvR6uBAwcGHL53794680xnr00BAADviSiM9O3bVxkZGcrJaf8rpzk5ORoxYkTAcTIzMzsNv3jxYg0fPlx9+vSJsLgAACDRRPyekQkTJmj69OmaOXOmCgoK9OCDD6q4uFhZWVmSTtzvMWbMGP/wWVlZKioq0oQJE1RQUKCZM2dqxowZevjhh52rRZRSUlI0adKkTpeEEhl1Tg7UOTlQ5+SQDHWO+NFe6cRLz5577jmVlpYqPT1dv/vd73T11VdLku68807t27dPS5cu9Q+/bNkyPfjgg9q2bZsGDx6sRx55xB9eAABAcosqjAAAADiF36YBAABWEUYAAIBVhBEAAGAVYQQAAFiV1GFkypQpGjZsmPr166eMjAytWLHCdpHC8vjjj6tHjx7t/gYOHOj/3hijxx9/XIMHD9Ypp5yir33ta9q2bVu7adTX1+v+++/XgAEDdOqpp+o///M/tX///nbDHDlyRHfccYdSU1OVmpqqO+64Q1VVVfGoopYvX65bbrlFgwcPVo8ePfT222+3+z6edSwuLtYtt9yiU089VQMGDND48ePV0NAQ9zrfeeedndb75Zdf7tk6Z2dn6z/+4z902mmn6eyzz9att96qnTs7/zx9Iq3ncOqcaOt56tSpuuiii/xvD83MzNT777/v/z7R1nE4dU60deyIUD+Ol6jeeOMN06dPH/Pqq6+a7du3m5/+9Kfm1FNPNUVFRbaLFtKkSZPMv//7v5vS0lL/X3l5uf/7Z555xpx22mlm3rx5ZsuWLWb06NFm0KBBprq62j9MVlaWOeecc0xOTo7ZuHGjufbaa83FF19smpqa/MN84xvfMOnp6Wb16tVm9erVJj093dx8881xqePChQvNY489ZubNm2ckmQULFrT7Pl51bGpqMunp6ebaa681GzduNDk5OWbw4MFm3Lhxca/z2LFjzTe+8Y12672ysrLdMF6q84033mhee+01s3XrVrNp0yZz0003mfPOO88cPXrUP0yiredw6pxo6/ndd9817733ntm5c6fZuXOnefTRR02fPn3M1q1bjTGJt47DqXOirWMnJG0YufTSS01WVla7z770pS+Zn//855ZKFL5JkyaZiy++OOB3LS0tZuDAgeaZZ57xf1ZXV2dSU1PNyy+/bIwxpqqqyvTp08e88cYb/mEOHDhgevbsaRYtWmSMMWb79u1GklmzZo1/mNzcXCPJ7NixIwa1Cq7jgTmedVy4cKHp2bOnOXDggH+YOXPmmJSUFOPz+WJSX2M619mYEw3Yt771raDjeL3O5eXlRpJZtmyZMSY51nPHOhuT+OvZGGM++9nPmunTpyfFOm7VWmdjkmMdRyopL9M0NDQoLy9PI0eObPf5yJEjtXr1akuliszu3bs1ePBgDRs2TN/73ve0d+9eSVJhYaHKysra1S0lJUXXXHONv255eXlqbGxsN8zgwYOVnp7uHyY3N1epqam67LLL/MNcfvnlSk1Ntb6M4lnH3Nxcpaent/vFyRtvvFH19fXKy8uLaT0DWbp0qc4++2ydf/75+tGPfqTy8nL/d16vs8/nkySdccYZkpJjPXesc6tEXc/Nzc164403dOzYMWVmZibFOu5Y51aJuo6jFdWv9npdRUWFmpubO/24X1paWqcf9XOjyy67TK+//rrOP/98HTx4UE899ZRGjBihbdu2+csfqG5FRUWSpLKyMvXt21ef/exnOw3TOn5ZWZnOPvvsTvM+++yzrS+jeNaxrKys03w++9nPqm/fvnFfDqNGjdJ3v/tdDR06VIWFhfrlL3+p6667Tnl5eUpJSfF0nY0xmjBhgq688kqlp6f7y9Fa/rYSZT0HqrOUmOt5y5YtyszMVF1dnT7zmc9owYIF+vKXv+w/aCbiOg5WZykx13F3JWUYadWjR492/zbGdPrMjUaNGuX//wsvvFCZmZn6/Oc/rz/96U/+m6CiqVvHYQIN76ZlFK86umU5jB492v//6enpGj58uIYOHar33ntP3/72t4OO54U6jxs3Th9//LFWrlzZ6btEXc/B6pyI6/mLX/yiNm3apKqqKs2bN09jx47VsmXLgpYjEdZxsDp/+ctfTsh13F1JeZlmwIAB6tWrV6dkWF5e3ilFesGpp56qCy+8ULt37/Y/VdNV3QYOHKiGhgYdOXKky2EOHjzYaV6HDh2yvoziWceBAwd2ms+RI0fU2NhofTkMGjRIQ4cO1e7duyV5t87333+/3n33XX300Uc699xz/Z8n8noOVudAEmE99+3bV1/4whc0fPhwZWdn6+KLL9bvf//7hF7HweocSCKs4+5KyjDSt29fZWRkKCcnp93nOTk5GjFihKVSRa++vl4FBQUaNGiQhg0bpoEDB7arW0NDg5YtW+avW0ZGhvr06dNumNLSUm3dutU/TGZmpnw+n9atW+cfZu3atfL5fNaXUTzrmJmZqa1bt6q0tNQ/zOLFi5WSkqKMjIyY1jOUyspKlZSUaNCgQZK8V2djjMaNG6f58+frww8/1LBhw9p9n4jrOVSdA/H6eg7EGKP6+vqEXMfBtNY5kERcxxGLw02yrtT6aO+MGTPM9u3bzQMPPGBOPfVUs2/fPttFC+mhhx4yS5cuNXv37jVr1qwxN998sznttNP8ZX/mmWdMamqqmT9/vtmyZYv5/ve/H/BRuXPPPdcsWbLEbNy40Vx33XUBHxu76KKLTG5ursnNzTUXXnhh3B7trampMfn5+SY/P99IMpMnTzb5+fn+R6/jVcfWR+Ouv/56s3HjRrNkyRJz7rnnxuTRuK7qXFNTYx566CGzevVqU1hYaD766COTmZlpzjnnHM/W+cc//rFJTU01S5cubfeIY21trX+YRFvPoeqciOt54sSJZvny5aawsNB8/PHH5tFHHzU9e/Y0ixcvNsYk3joOVedEXMdOSNowYowxL730khk6dKjp27evueSSS9o9Xudmrc/h9+nTxwwePNh8+9vfNtu2bfN/39LSYiZNmmQGDhxoUlJSzNVXX222bNnSbhrHjx8348aNM2eccYY55ZRTzM0332yKi4vbDVNZWWl+8IMfmNNOO82cdtpp5gc/+IE5cuRIPKpoPvroIyOp09/YsWPjXseioiJz0003mVNOOcWcccYZZty4caauri6uda6trTUjR440Z511lunTp48577zzzNixYzvVx0t1DlRXSea1117zD5No6zlUnRNxPd91113+dvass84y119/vT+IGJN46zhUnRNxHTuhhzHGxK8fBgAAoL2kvGcEAAC4B2EEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVv1/syesQD3AFYkAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Run false-accept rate test again, now with the verifier model\n", "\n", "scores = oww.predict_clip(\"santa_barbara_corpus_test_clip.wav\")\n", "\n", "plt.figure()\n", "plt.plot([i[\"turn_on_the_office_lights\"] for i in scores])\n", "_ = plt.ylim(0,1)\n" ] }, { "cell_type": "code", "execution_count": 30, "id": "c59007d2", "metadata": { "ExecuteTime": { "end_time": "2023-02-18T03:47:29.856142Z", "start_time": "2023-02-18T03:47:29.836354Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "False-accept rate per hour: 0.0\n" ] } ], "source": [ "# Calculate the false-accept rate per hour from this new result\n", "\n", "false_accepts = openwakeword.metrics.get_false_positives(\n", " [i[\"turn_on_the_office_lights\"] for i in scores], threshold=0.5\n", ")\n", "\n", "print(f\"False-accept rate per hour: {false_accepts/1}\")" ] }, { "cell_type": "markdown", "id": "7c7375c6", "metadata": {}, "source": [ "Sucess! Now the false-activation rate is at most <1 per hour given that there weren't any false-positives in our ~1 hour test clip, which is an orders of magnitude decrease! This model is now much closer to being ready for a production deployment, assuming that each user is known and can provide the neccessary data to train the verifier model." ] } ], "metadata": { "kernelspec": { "display_name": "torch_gpu", "language": "python", "name": "torch_gpu" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "384px" }, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 5 }