The DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus (TIMIT) Training and Test Data The TIMIT corpus of read speech has been designed to provide speech data for the acquisition of acoustic-phonetic knowledge and for the development and evaluation of automatic speech recognition systems. TIMIT has resulted from the joint efforts of several sites under sponsorship from the Defense Advanced Research Projects Agency - Information Science and Technology Office (DARPA-ISTO). Text corpus design was a joint effort among the Massachusetts Institute of Technology (MIT), Stanford Research Institute (SRI), and Texas Instruments (TI). The speech was recorded at TI, transcribed at MIT, and has been maintained, verified, and prepared for CD-ROM production by the National Institute of Standards and Technology (NIST). This file contains a brief description of the TIMIT Speech Corpus. Additional information including the referenced material and some relevant reprints of articles may be found

The TIMIT corpus is a widely used benchmark for speech recognition tasks. It contains 3,696 training utterances from 462 speakers, excluding the SA sentences. The core test set was used to report the results. It contains 192 utterances from 24 speakers, excluding the validation set.

With the rapid development of deep learning techniques, the generation and counterfeiting of multimedia material are becoming increasingly straightforward to perform. At the same time, sharing fake content on the web has become so simple that malicious users can create unpleasant situations with minimal effort. Also, forged media are getting more and more complex, with manipulated videos (e.g., deepfakes where both the visual and audio contents can be counterfeited) that are taking the scene over still images. The multimedia forensic community has addressed the possible threats that this situation could imply by developing detectors that verify the authenticity of multimedia objects. However, the vast majority of these tools only analyze one modality at a time. This was not a problem as long as still images were considered the most widely edited media, but now, since manipulated videos are becoming customary, performing monomodal analyses could be reductive. Nonetheless, there is a lack in the literature regarding multimodal detectors (systems that consider both audio and video components). This is due to the difficulty of developing them but also to the scarsity of datasets containing forged multimodal data to train and test the designed algorithms.

In this paper we focus on the generation of an audio-visual deepfake dataset. First, we present a general pipeline for synthesizing speech deepfake content from a given real or fake video, facilitating the creation of counterfeit multimodal material. The proposed method uses Text-to-Speech (TTS) and Dynamic Time Warping (DTW) techniques to achieve realistic speech tracks. Then, we use the pipeline to generate and release TIMIT-TTS, a synthetic speech dataset containing the most cutting-edge methods in the TTS field. This can be used as a standalone audio dataset, or combined with DeepfakeTIMIT and VidTIMIT video datasets to perform multimodal research. Finally, we present numerous experiments to benchmark the proposed dataset in both monomodal (i.e., audio) and multimodal (i.e., audio and video) conditions. This highlights the need for multimodal forensic detectors and more multimodal deepfake data.

For the initial version of TIMIT-TTS v1.0

Arxiv: https://arxiv.org/abs/2209.08000

TIMIT-TTS Database v1.0: https://zenodo.org/record/6560159

kimetsu/Timit dataset hosted on Hugging Face and contributed by the HF Datasets community

The dataset used in this paper is a collection of phonetically and phonologically local allophonic distribution in English, where voiceless stops surface as aspirated word-initially before a stressed vowel (e.g. in ["phIt] ‘pit’), except if a sibilant [s] precedes the stop (e.g. in ["spIt] ‘spit’).

The TIMIT acoustic-phonetic continuous speech corpusCD-ROM contains a large collection of speech samples from 250 male and 250 female speakers.

The QUT-NOISE Databases and Protocols

Overview

This distribution contains the QUT-NOISE database and the code required to create the QUT-NOISE-TIMIT database from the QUT-NOISE database and a locally installed copy of the TIMIT database. It also contains code to create the QUT-NOISE-SRE protocol on top of an existing speaker recognition evaluation database (such as NIST evaluations). Further information on the QUT-NOISE and QUT-NOISE-TIMIT databases is available in our paper:

D. Dean, S. Sridharan, R. Vogt, M. Mason (2010) , in Proceedings of Interspeech 2010, Makuhari Messe International Convention Complex, Makuhari, Japan.

This paper is also available in the file: docs/Dean2010, The QUT-NOISE-TIMIT corpus for the evaluation of voice activity detection algorithm.pdf, distributed with this database.

Further information on the QUT-NOISE-SRE protocol is available in our paper:
D. Dean, A. Kanagasundaram, H. Ghaemmaghami, M. Hafizur, S. Sridharan (2015) . In Proceedings of Interspeech 2015, September, Dresden, Germany.

Licensing

The QUT-NOISE data itself is licensed CC-BY-SA, and the code required to create the QUT-NOISE-TIMIT database and QUT-NOISE-SRE protocols is licensed under the BSD license. Please consult the approriate LICENSE.txt files (in the code and QUT-NOISE directories) for more information. To attribute this database, please include the following citation:

D. Dean, S. Sridharan, R. Vogt, M. Mason (2010) , in Proceedings of Interspeech 2010, Makuhari Messe International Convention Complex, Makuhari, Japan.

If your work is based upon the QUT-NOISE-SRE, please also include this citation:
D. Dean, A. Kanagasundaram, H. Ghaemmaghami, M. Hafizur, S. Sridharan (2015) . In Proceedings of Interspeech 2015, September, Dresden, Germany.

Download and Installation

Download the following QUT-NOISE*.zip files:

• (26.7 MB, md5sum: 672461fd88782e9ea10d5c2cb7a84196)
• (1.6 GB, md5sum: f87fb213c0e1c439e1b727fb258ef2cd)
• (1.7 GB, md5sum: d680118b4517e1257a9263b99d1ac401)
• (1.4 GB, md5sum: d99572ae1c118b749c1ffdb2e0cf0d2e)
• (1.4 GB, md5sum: fe107ab341e6bc75de3a32c69344190e)
• (1.6 GB, md5sum: 68d5ebc2e60cb07927cc4d33cdf2f017)

Creating QUT-NOISE-TIMIT

Obtaining TIMIT

In order to construct the QUT-NOISE-TIMIT database from the QUT-NOISE data supplied here you will need to obtain a copy of the TIMIT database from the . If you just want to use the QUT-NOISE database, or you wish to combine it with different speech data, TIMIT is not required.

Creating QUT-NOISE-TIMIT

• Once you have obtained TIMIT, download and install a copy of and install it in your MATLABPATH.
• Run matlab in the QUT-NOISE/code directory, and run the function: createQUTNOISETIMIT('/location/of/timit-cd/timit'). This will create the QUT-NOISE-TIMIT database in the QUT-NOISE/QUT-NOISE-TIMIT directory.
• If you wish to verify that the QUT-NOISE-TIMIT database matches that evaluated in our original paper, please check that the md5sums (use md5sum on unix-based OSes) match those in the QUT-NOISE-TIMIT/md5sum.txt file.
• Using the QUT-NOISE-SRE protocol
  • The code related to the QUT-NOISE-SRE protocol can be used in two ways:
    1. To create a collection of noisy audio files across the scenarios in the QUT-NOISE database at different noise levels, or,
    2. To recreate a list of file names based on the QUT-NOISE-SRE protocl produced by another researcher, having already done (1). This allows existing research to be reproduced without having to send large volumes of audio around.
  • If you are interested in creating your own noisy database from an existing SRE database (1 above), please look at the example script exampleQUTNOISESRE.sh in the QUT-NOISE/code directory. You will need to make some modifications, but it should give you the right idea.
  • If you are interested in creating our QUT-NOISE-NIST2008 database published at Interspeech 2015, you can find the list of created noisy files in the QUT-NOISE-NIST2008.train.short2.list and QUT-NOISE-NIST2008.test.short3.list files in the QUT-NOISE/code directory.
  • These files can be recreated as follows (provided you have access to the NIST2008 SRE data):
    
    Run matlab in the QUT-NOISE/code directory, and run the following functions:
    
    createQUTNOISESREfiles('NIST2008.train.short2.list', ...
    'QUT-NOISE-NIST2008.train.short2.list', ...
    '

The TIMIT corpus is a large database of speech recordings used for speaker recognition and speech synthesis tasks.

Context

The canonical metadata on NLTK:

The TIMIT Acoustic-Phonetic Continuous Speech Corpus CDROM is a widely used dataset for speech recognition tasks.

TCD-Timit is an audio-visual corpus of continuous speech.

Dataset

This dataset was created by Tommy NgX

Released under CC0: Public Domain

Contents

Speaker Verification Corpus

The TIMIT corpus of read speech has been designed to provide speech data for the acquisition of acoustic-phonetic knowledge and for the development and evaluation of automatic speech recognition systems. TIMIT has resulted from the joint efforts of several sites under sponsorship from the Defense Advanced Research Projects Agency - Information Science and Technology Office (DARPA-ISTO). Text corpus design was a joint effort among the Massachusetts Institute of Technology (MIT), Stanford Research Institute (SRI), and Texas Instruments (TI). The speech was recorded at TI, transcribed at MIT, and has been maintained, verified, and prepared for CD-ROM production by the National Institute of Standards and Technology (NIST). This file contains a brief description of the TIMIT Speech Corpus. Additional information including the referenced material and some relevant reprints of articles may be found in the printed documentation which is also available from NTIS (NTIS# PB91-100354).

See http://conradsanderson.id.au/vidtimit/ for details.

Summary: Video and corresponding audio recordings of 43 people, reciting short sentences. Useful for research on topics such as automatic lip reading, multi-view face recognition, multi-modal speech recognition and person identification.

This dataset contains all data, source code, pre-trained computational predictive models and experimental results related to:

Hueber T., Tatulli E., Girin L., Schwatz, J-L "How predictive can be predictions in the neurocognitive processing of auditory and audiovisual speech? A deep learning study." (biorXiv preprint https://doi.org/10.1101/471581).

• Raw data are extracted from the publicly available database NTCD-TIMIT (10.5281/zenodo.260228).
  • Audio recordings are available in the audio_clean/ directory
  • Post-processed lip image sequences are available in the lips_roi/ directory (67x67 pixels, 8bits, obtained by lossless inverse DCT-2D transform from the DCT feature available in the original repository of NTCD-TIMIT)
  • Phonetic segmentation (extracted from NTCD-TIMIT original zenodo repository) is available in the HTK MLF file volunteer_labelfiles.mlf
• Audio features (MFCC-spectrogram and log-spectrogram) are available in the mfcc_16k/ and fft_16k/ directories.
• Models (audio-only, video-only and audiovisual, based on deep feed-forward neural networks and/or convolutional neural network, in .h5 format, trained with Keras 2.0 toolkit) and data normalization parameters (in .dat scikit-learn format) are available in models_mfcc/ and models_logspectro/ directories
• Predicted and target (ground truth) MFCC-spectro (resp. log-spectro) for the test databases (1909 sentences), and for the different values of \(\tau_p\) or \(\tau_f\) are available in pred_testdb_mfccspectro/ (resp. pred_testdb_logspectro/) directory

Source code for extracting audio features, training and evaluating the models is available on GitHub https://github.com/thueber/DeepPredSpeech/

All directories have been zipped before upload.

Feel free to contact me for more details.

Thomas Hueber, Ph. D., CNRS research fellow, GIPSA-lab, Grenoble, France, thomas.hueber@gipsa-lab.fr

This dataset contains common speech and noise corpora for evaluating fundamental frequency estimation algorithms as convenient JBOF dataframes. Each corpus is available freely on its own, and allows redistribution:

• CMU-ARCTIC (BSD license) [1]
• FDA (free to download) [2]
• KEELE (free for noncommercial use) [3]
• MOCHA-TIMIT (free for noncommercial use) [4]
• PTDB-TUG (ODBL license) [5]
• NOISEX (free to download) [7]
• QUT-NOISE (CC-BY-SA license) [8]

These files are published as part of my dissertation, "Pitch of Voiced Speech in the Short-Time Fourier Transform: Algorithms, Ground Truths, and Evaluation Methods", and in support of the Replication Dataset for Fundamental Frequency Estimation.

References:

1. John Kominek and Alan W Black. CMU ARCTIC database for speech synthesis, 2003.
2. Paul C Bagshaw, Steven Hiller, and Mervyn A Jack. Enhanced Pitch Tracking and the Processing of F0 Contours for Computer Aided Intonation Teaching. In EUROSPEECH, 1993.
3. F Plante, Georg F Meyer, and William A Ainsworth. A Pitch Extraction Reference Database. In Fourth European Conference on Speech Communication and Technology, pages 837–840, Madrid, Spain, 1995.
4. Alan Wrench. MOCHA MultiCHannel Articulatory database: English, November 1999.
5. Gregor Pirker, Michael Wohlmayr, Stefan Petrik, and Franz Pernkopf. A Pitch Tracking Corpus with Evaluation on Multipitch Tracking Scenario. page 4, 2011.
6. John S. Garofolo, Lori F. Lamel, William M. Fisher, Jonathan G. Fiscus, David S. Pallett, Nancy L. Dahlgren, and Victor Zue. TIMIT Acoustic-Phonetic Continuous Speech Corpus, 1993.
7. Andrew Varga and Herman J.M. Steeneken. Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recog- nition systems. Speech Communication, 12(3):247–251, July 1993.
8. David B. Dean, Sridha Sridharan, Robert J. Vogt, and Michael W. Mason. The QUT-NOISE-TIMIT corpus for the evaluation of voice activity detection algorithms. Proceedings of Interspeech 2010, 2010.

The QUT-NOISE-TIMIT dataset is a dataset for speech enhancement. It consists of clean speech and noise.

nh0znoisung/timit dataset hosted on Hugging Face and contributed by the HF Datasets community

Dataset

This dataset was created by Tommy NgX

Released under CC0: Public Domain

Contents

DeepfakeTIMIT is a database of videos where faces are swapped using the open source GAN-based approach (adapted from here: https://github.com/shaoanlu/faceswap-GAN), which, in turn, was developed from the original autoencoder-based Deepfake algorithm.

When creating the database, we manually selected 16 similar looking pairs of people from publicly available VidTIMIT database. For each of 32 subjects, we trained two different models: a lower quality (LQ) with 64 x 64 input/output size model, and higher quality (HQ) with 128 x 128 size model (see the available images for the illustration). Since there are 10 videos per person in VidTIMIT database, we generated 320 videos corresponding to each version, resulting in 620 total videos with faces swapped. For the audio, we kept the original audio track of each video, i.e., no manipulation was done to the audio channel.

Any publication (eg. conference paper, journal article, technical report, book chapter, etc) resulting from the usage of DeepfakeTIMIT must cite the following paper:

P. Korshunov and S. Marcel,
DeepFakes: a New Threat to Face Recognition? Assessment and Detection.
arXiv and Idiap Research Report

Any publication (eg. conference paper, journal article, technical report, book chapter, etc) resulting from the usage of VidTIMIT and subsequently DeepfakeTIMIT must also cite the following paper:

C. Sanderson and B.C. Lovell,
Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference.
Lecture Notes in Computer Science (LNCS), Vol. 5558, pp. 199-208, 2009.