How to Build a Basic Speech Recognition Network with Tensorflow (Demo Video Included)

This tutorial will show you how to build a basic speech recognition network that recognizes simple speech commands. Speech recognition is a subfield of computer science and linguistics that identifies spoken words and converts them into text.

When speech is recorded using a voice recording device like a microphone, it converts physical sound to electrical energy. Then, using an analog-to-digital converter, this is converted to digital data, which can be fed to a neural network or hidden Markov model to convert them to text.

We are going to train such a neural network here, which, after training, will be able to recognize small speech commands.

Speech recognition is also known as:

• Automatic Speech Recognition (ASR)
• Computer Speech Recognition
• Speech to Text (STT)

The steps involved are:

1. Import required libraries
2. Download dataset
3. Data Exploration and Visualization
4. Preprocessing
5. Training
6. Testing

Here is a colab notebook with all of the codes if you want to follow along.

Let us start the implementation.

Download and extract the mini_speech_commands.zip, file containing the smaller Speech Commands datasets.

The dataset's audio clips are stored in eight folders corresponding to each speech command: no, yes,down,go,left, up,right,and stop.

Dataset: TensorFlow recently released the Speech Commands Datasets. It includes 65,000 one-second long utterances of 30 short words by thousands of different people. We will be working with a smaller version of the Speech Commands dataset called mini speech command datasets.

Download the mini Speech Commands dataset and unzip it.

The dataset's audio clips are stored in eight folders corresponding to each speech command: no yes down  go  left  up  right and stop

Now that we have the dataset, let us understand and visualize it.

Data Exploration and Visualization is an approach that helps us understand what's in a dataset and the characteristics of the dataset. Let us visualize the audio signal in the time series domain.

Here is what the audio looks in a waveform.

To listen to the above command up :

Check the list of commands for which we will be training our speech recognition model. These audio clips are stored in eight folders corresponding to each speech command: no, yes, down, go, left, up, right, and stop.

Remove unnecessary files:

Let us plot a bar graph to understand the number of recordings for each of the eight voice commands:

As we can see, we have almost the same number of recordings for each command.

Let us define these preprocessing steps in the code snippet below:

Convert the output labels to integer encoded labels and then to a one-hot vector since it is a multi-classification problem. Then reshape the 2D array to 3D since the input to the conv1d must be a 3D array:

Train test split - train test split is a model validation procedure that allows you to simulate how a model would perform on new/unseen data. We are doing an 80:20 split of data for training and testing.

Create a model and compile it. Now, we define a model:

Define callbacks:

Start the training:

Plot the training loss vs validation loss:

Now, we have a trained model. We need to load it and use it to predict our commands.

Load the model for prediction:

Start predicting:

You can always create your own dataset with creative ways like clap sounds and whistles or your own custom words and train your model to recognize them.

Let us now check out the demo video for this experiment.

We just completed a tutorial on building a speech recognition system! Here is a quick recap:

We began by exploring the dataset, giving us a good feel for what is inside. Then, we prepped the data, converting it into a format suitable for training. After a train-test split, we designed a Conv1D neural network for the task.

By following these steps, we have laid the foundation for a speech recognition system. With further tweaks and your own data, you can expand its capabilities. Keep exploring the world of speech recognition!