Dataset

This dataset was created by ehddnr301

Contents

The Google Speech Commands Dataset v0.02 is a curated collection of short (approximately one-second) audio recordings of spoken words, specifically designed for training and benchmarking keyword spotting systems. Each recording captures a single spoken command uttered by a diverse set of speakers, making the dataset highly valuable for developing robust, real-world voice-controlled applications. The commands include common terms such as "yes", "no", "up", "down", "left", "right", "on", "off", "stop", and "go", among others.

In addition to the primary command recordings, the dataset also provides a set of background noise audio files. These files, stored in a dedicated folder, are intended to support data augmentation techniques and help improve model performance in noisy environments. The dataset has been widely adopted in both academic research and industry applications, serving as a benchmark for lightweight and efficient speech recognition systems.

Context

Variational Autoencoder (VAE) run on Tensorflow Speech Recognition challenge data.

The code to train the VAE is here: https://www.kaggle.com/holzner/variational-autoencoder-for-speech-dataset .

The VAE was trained on the amplitude part of the spectrograms using a convolutional encoder and a decoder consisting of convolutional and upscaling layers. It was trained only on those classes of the train dataset which should be predicted in the challenge (excluding the background noise samples).

The motivation behind this was to produce features which allow to better distinguish 'unknown' classes which are not present in the train dataset but are in the test dataset.

Content

The dataset contains the following columns:

• fname: the filename (in the train or test dataset) of the sample in this row
• muXX: the predicted mu parameters (XX = 00-11) of the latent Gaussians
• sigmaXX: the predicted sigma parameters (XX = 00-11) of the latent Gaussians

158538 public test audio files corresponding to files in sample_submission.csv

I don't know why kaggle removed the public test data set in input folder,
anyway, I re-uploaded this file for anyone who may want to test her/his model.
"7z x -oxxxx/" is the command of extracting files to xxxx folder.
Note: xxxx is the output folder name, there's no space between -o and xxxx/
Good Luck.

Context

• We would like to have good open source speech recognition
• Commercial companies try to solve a hard problem: map arbitrary, open-ended speech to text and identify meaning
• The easier problem should be: detect a predefined sequence of sounds and map it to a predefined action.
• Lets tackle the simplest problem first: Classifying single, short words (commands)
• Audio training data is difficult to obtain.

Approaches

• The parent project (spoken verbs) created synthetic speech datasets using text-to-speech programs. The focus there is on single-syllable verbs (commands).
• The Speech Commands dataset (by Pete Warden, see the TensorFlow Speech Recognition Challenge) asked volunteers to pronounce a small set of words: (yes, no, up, down, left, right, on, off, stop, go, and 0-9).
• This data set provides synthetic counterparts to this real world dataset.

Open questions

One can use these two datasets in various ways. Here are some things I am interested in seeing answered:

1. What is it in an audio sample that makes it "sound similar"? Our ears can easily classify both synthetic and real speech, but for algorithms this is still hard. Extending the real dataset with the synthetic data yields a larger training sample and more diversity.
2. How well does an algorithm trained on one data set perform on the other? (transfer learning) If it works poorly, the algorithm probably has not found the key to audio similarity.
3. Are synthetic data sufficient for classifying real datasets? If this is the case, the implications are huge. You would not need to ask thousands of volunteers for hours of time. Instead, you could easily create arbitrary synthetic datasets for your target words.

A interesting challenge (idea for competition) would be to train on this data set and evaluate on the real dataset.

Synthetic data creation

Here I describe how the synthetic audio samples were created. Code is available at https://github.com/JohannesBuchner/spoken-command-recognition, in the "tensorflow-speech-words" folder.

1. The list of words is in "inputwords". "marvin" was changed to "marvel", because "marvin" does not have a pronounciation coding yet.
2. Pronounciations were taken from the British English Example Pronciation dictionary (BEEP, http://svr-www.eng.cam.ac.uk/comp.speech/Section1/Lexical/beep.html ). The phonemes were translated for the next step with a translation table (see compile.py for details). This creates the file "words". There are multiple pronounciations and stresses for each word.
3. A text-to-speech program (espeak) was used to pronounce these words (see generatetfspeech.sh for details). The pronounciation, stress, pitch, speed and speaker were varied. This gives >1000 clean examples for each word.
4. Noise samples were obtained. Noise samples (airport babble car exhibition restaurant street subway train) come from AURORA (https://www.ee.columbia.edu/~dpwe/sounds/noise/), and additional noise samples were synthetically created (ocean white brown pink). (see ../generatenoise.sh for details)
5. Noise and speech were mixed. The speech volume and offset were varied. The noise source, volume was also varied. See addnoise.py for details. addnoise2.py is the same, but with lower speech volume and higher noise volume. All audio files are one second (1s) long and are in wav format (16 bit, mono, 16000 Hz).
6. Finally, the data was compressed into an archive and uploaded to kaggle.

Acknowledgements

This work built upon

• Pronounciation dictionary: BEEP: http://svr-www.eng.cam.ac.uk/comp.speech/Section1/Lexical/beep.html
• Noise samples: AURORA: https://www.ee.columbia.edu/~dpwe/sounds/noise/
• eSPEAK: http://espeak.sourceforge.net/ and mbrola voices http://www.tcts.fpms.ac.be/synthesis/mbrola/mbrcopybin.html

Please provide appropriate citations to the above when using this work.

To cite the resulting dataset, you can use:

APA-style citation: "Buchner J. Synthetic Speech Commands: A public dataset for single-word speech recognition, 2017. Available from https://www.kaggle.com/jbuchner/synthetic-speech-commands-dataset/".

BibTeX @article{speechcommands, title={Synthetic Speech Commands: A public dataset for single-word speech recognition.}, author={Buchner, Johannes}, journal={Dataset available from https://www.kaggle.com/jbuchner/synthetic-speech-commands-dataset/}, year={2017} }

Thanks to everyone trying to improve open source voice detection and speech recognition.

Links

• https://www.kaggle.com/jbuchner/spokenverbs
• https://www.kaggle.com/c/tensorflow-speech-recognition-challenge
• https://github.com/JohannesBuchner/spoken-command-recognition/

Context

Original audio files were preprocessed as numeric arrays.

Content

The original data can be downloaded here :

http://storage.googleapis.com/download.tensorflow.org/data/mini_speech_commands.zip

Command Names: ['stop' 'up' 'yes' 'left' 'right' 'go' 'down' 'no']

Acknowledgments

This catalog really impressed me => TensorFlow Datasets

Inspiration

It could be fun to compare with the same content in other languages, don't you think?

Credits: http://download.tensorflow.org/data/speech_commands_v0.01.tar.gz%7D

There are 610 audio files divided as: - 193 for birds - 207 for cats - 210 for dogs

This is a small part of a public dataset for single-word speech recognition, 2017.

Citing:

@article{speechcommands, title={Speech Commands: A public dataset for single-word speech recognition.}, author={Warden, Pete}, journal={Dataset available from http://download.tensorflow.org/data/speech_commands_v0.01.tar.gz}, year={2017} }

Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition

SCLAB 제직자 교육을 위해 가져온 교육용 데이터입니다. 출처 : https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/audio/speech_commands.py