Abstract: Indian Sign Language (ISL) is a complete language with its own grammar, syntax, vocabulary and several unique linguistic attributes. It is used by over 5 million deaf people in India. Currently, there is no publicly available dataset on ISL to evaluate Sign Language Recognition (SLR) approaches. In this work, we present the Indian Lexicon Sign Language Dataset - INCLUDE - an ISL dataset that contains 0.27 million frames across 4,287 videos over 263 word signs from 15 different word categories. INCLUDE is recorded with the help of experienced signers to provide close resemblance to natural conditions. A subset of 50 word signs is chosen across word categories to define INCLUDE-50 for rapid evaluation of SLR methods with hyperparameter tuning. The best performing model achieves an accuracy of 94.5% on the INCLUDE-50 dataset and 85.6% on the INCLUDE dataset. Download Instructions: For ease of access, we have prepared a Shell Script to download all the parts of the dataset and extract them to form the complete INCLUDE dataset.You can find the script here: http://bit.ly/include_dl

Sign language is a cardinal element for communication between deaf and dumb community. Sign language has its own grammatical structure and gesticulation nature. Research on SLRT focuses a lot of attention in gesture identification. Sign language comprises of manual gestures performed by hand poses and non-manual features expressed through eye, mouth and gaze movements. The sentence-level completely labelled Indian Sign Language dataset for Sign Language Translation and Recognition (SLTR) research is developed. The ISL-CSLTR dataset assists the research community to explore intuitive insights and to build the SLTR framework for establishing communication with the deaf and dumb community using advanced deep learning and computer vision methods for SLTR purposes. This ISL-CSLTR dataset aims in contributing to the sentence level dataset created with two native signers from Navajeevan, Residential School for the Deaf, College of Spl. D.Ed & B.Ed, Vocational Centre, and Child Care & Learning Centre, Ayyalurimetta, Andhra Pradesh, India and four student volunteers from SASTRA Deemed University, Thanjavur, Tamilnadu. The ISL-CSLTR corpus consists of a large vocabulary of 700 fully annotated videos, 18863 Sentence level frames, and 1036 word level images for 100 Spoken language Sentences performed by 7 different Signers. This corpus is arranged based on signer variants and time boundaries with fully annotated details and it is made available publicly. The main objective of creating this sentence level ISL-CSLRT corpus is to explore more research outcomes in the area of SLTR. This completely labelled video corpus assists the researchers to build framework for converting spoken language sentences into sign language and vice versa. This corpus has been created to address the various challenges faced by the researchers in SLRT and significantly improves translation and recognition performance. The videos are annotated with relevant spoken language sentences provide clear and easy understanding of the corpus data. Acknowledgements: The research was funded by the Science and Engineering Research Board (SERB), India under Start-up Research Grant (SRG)/2019–2021 (Grant no. SRG/2019/001338). And also, we thank all the signers for their contribution in collecting the sign videos and the successful completion of the ISL-CSLTR corpus. We would like to thank Navajeevan, Residential School for the Deaf, College of Spl. D.Ed & B.Ed, Vocational Centre, and Child Care & Learning Centre, Ayyalurimetta, Andhra Pradesh, India for their support and contribution.

Sign Language Recognition is a breakthrough for helping deaf-mute people and has been researched for many years. It is still a challenge for deaf and dumb people to normally have a conversation since most people aren’t acquainted with sign language. But owing to recent technology, we are better able to understand people using sign language.This dataset contains images of the most common phrases in Indian Sign Language. The images of ISL are captured using a standard webcam of a laptop with a resolution of 1280*720. These include 30 images of 40 different signs from ISL. Images are not filtered or resized, are captured under varying light intensities and different backgrounds, and stored in NumPy arrays. These images are of static sign language phrases and do not include any dynamic sign.

CISLR: Corpus for Indian Sign Language Recognition

This repository contains the Indian Sign Language Dataset proposed in the following paper

Paper: CISLR: Corpus for Indian Sign Language Recognition https://preview.aclanthology.org/emnlp-22-ingestion/2022.emnlp-main.707/

Authors: Abhinav Joshi, Ashwani Bhat, Pradeep S, Priya Gole, Shashwat Gupta, Shreyansh Agarwal, Ashutosh Modi Abstract: Indian Sign Language, though used by a diverse community, still lacks well-annotated… See the full description on the dataset page: https://huggingface.co/datasets/IIT-K/CISLR.

Indian Sign Language images along with corresponding XML files that can be used for Artificial Intelligence, Machine Learning and deep learning projects such as hand gesture to speech converter.

The dataset contains the RGB images of hand gestures of twenty ISL words, namely, ‘afraid’,’agree’,’assistance’,’bad’,’become’,’college’,’doctor’,’from’,’pain’,’pray’, ’secondary’, ’skin’, ’small’, ‘specific’, ‘stand’, ’today’, ‘warn’, ‘which’, ‘work’, ‘you’’ which are commonly used to convey messages or seek support during medical situations. All the words included in this dataset are static. The images were captured from 8 individuals including 6 males and 2 females in the age group of 9 years to 30 years. The dataset contains a 18000 images in jpg format. The images are labelled using the format ISLword_X_YYYY_Z, where:•ISLword corresponds to the words ‘afraid’, ‘agree’, ‘assistance’, ‘bad’, ‘become’, ‘college’, ‘doctor’ ,‘from’, ’pray’, ‘pain’, ‘secondary’, ‘skin’, ‘small’, ‘specific’, ‘stand’, ‘today’, ‘warn’, ‘which’, ‘work’, ‘you’.•X is an image number in the range 1 to 900.•YYYY is an identifier of the participant and is in the range of 1 to 6.•Z corresponds to 01 or 02 that identifies the sample number for each subject.For example, the file named afraid_1_user1_1 is the image sequence of the first sample of the ISL gesture of the word ‘afraid’ presented by the 1st user.

Indian Sign Language Image Dataset Description The Indian Sign Language (ISL) image dataset is a valuable collection of images capturing various gestures and expressions used in Indian Sign Language. Unlike video datasets, this compilation focuses on individual frames, providing detailed snapshots of hand movements, facial expressions, and body language crucial for accurate sign interpretation.

Dataset Details:

Content: The dataset encompasses a diverse range of signs and expressions integral to Indian Sign Language, including alphabet signs, common phrases, and expressive gestures.

Format: Images are presented in a standardized format, often with high resolution, ensuring a clear representation of the intricate details in each sign.

Annotation: Each image may be annotated with corresponding glosses or labels, facilitating supervised learning tasks. Annotations could include details about the signed content, aiding in the development of sign language recognition systems.

Use Cases: The ISL image dataset is designed to support various applications, including sign language recognition, translation, and assistive technologies to enhance communication and interaction for individuals within the deaf and hard-of-hearing community.

Research Opportunities: Researchers and developers can leverage this dataset to explore and implement innovative solutions for automatic sign language recognition, bridging the communication gap and fostering inclusivity.

Note: Access to the dataset may be subject to specific terms and conditions outlined by the dataset providers or creators.

By providing this image dataset, the intention is to contribute to advancements in technology that empower and improve the quality of life for individuals who rely on Indian Sign Language for communication.

This dataset consists of static hand gestures and lips movement for each character in the English alphabet, eight Hindi Vowels and ten Numerals as represented in Indian sign language (ISL). The dataset consists of 1,02,470 images of subjects from different age groups presenting static gestures under varied backgrounds and illumination conditions. The dataset is structured into three folders namely Kids, Teenagers and Adults. Each folder consists of sub-folders namely Full Sleeves and Half Sleeves indicating the type of clothing that the subject has worn at the time of image acquisition. In each sub-folders, images for the English alphabet, Hindi Vowels and Numerals are stored respectively in the sub-folders named with that specific character. However, for the English alphabet 'E' and Numeral '9' we have captured two different signs for each (that are used interchangeably), and it is contained in the folder namely E1 and E2 for alphabet 'E' and 9a and 9b for Numeral '9'. For the English alphabet, wherever a character is represented by a dynamic sign, the last frame of the sign is captured. For example, this is typically a case with English characters like 'J', 'H' and 'Y'. The images are stored in .jpeg format and have resolutions varying from 300 x 500 to 800 x 600, and the size is less than 100KB. The dataset is captured by a team pursuing research at Chandigarh College of Engineering and Technology, Chandigarh. The subjects have been informed about the research and Informed Participant Consent has been obtained prior to image acquisition. This dataset can be used only for research purposes either as it is or after cropping the static gestures from the image after duly referencing it. Any other use of the dataset is strictly prohibited and any illegal use is subject to the Indian court of law.

Dataset

This dataset was created by Gaurav Dutta

Contents

Sign languages are the primary means of communication for a large number of people worldwide. Recently, the availability of Sign language translation datasets have facilitated the incorporation of Sign language research in the NLP community. Though a wide variety of research focuses on improving translation systems for sign language, the lack of ample annotated resources hinders research in the data driven natural language processing community. In this resource paper, we introduce ISLTranslate, a translation dataset for continuous Indian Sign Language (ISL), consisting of 30k ISL-English sentence pairs. To the best of our knowledge, it is the first and largest translation dataset for continuous Indian Sign Language with corresponding English transcripts. We provide a detailed analysis of the dataset and examine the distribution of words and phrases covered in the proposed dataset. To validate the performance of existing end-to-end Sign language to spoken language translation systems, we benchmark the created dataset with multiple existing state-of-the-art systems for sign languages.

Dataset

This dataset was created by chinmay d

Contents

The dataset includes the videos of the dynamic hand gestures for the agricultural words from Indian sign language (ISL). This dataset is used in a work on the recognition of hand gestures for the Indian sign language (ISL) words commonly used by deaf farmers. A hybrid deep learning model with convolutional long short term memory (LSTM) network has been exploited for gesture classification. The model has attained an average classification accuracy of 76.21% on the proposed dataset of ISL words from the agricultural domain. The work is published in the journal Expert Systems with Applications https://www.sciencedirect.com/science/article/abs/pii/S0957417421010009.

The dataset includes videos files of the hand gestures of eight words (accident, call, doctor, help, hot, lose, pain, thief) from Indian sign language (ISL), commonly used to communicate during emergency situations. The data is useful for the researchers working on vision based automatic sign language recognition as well as hand gesture recognition. All the words included in the dataset, except the word doctor are dynamic hand gestures. The videos in this dataset were collected by asking the participants to stand comfortably behind a black colored board and present the hand gestures, in front of the board. A Sony cyber shot DSC-W810 digital camera with 20.1 mega pixel resolution has been used for capturing the videos. The videos have been collected from 26 individuals including 12 males and 14 females in the age group of 22 to 26 years. Two sample videos have been captured from each participant in an indoor environment under normal lighting conditions by placing the camera at a fixed distance. The dataset is presented in two folders with the original raw video sequences in one folder, and the cropped and downsampled video sequences in the other folder.

Dataset

This dataset was created by Priyaziya

Contents

This dataset contains depth data collected through Intel RealSense Depth Camera D435i. Data corresponding to Indian Sign Language (ISL) gesture of Weekdays (Sunday-Saturday) is used.Data is stored as comma separated values. Each line corresponds to a sign.

Dataset

This dataset was created by Vaishnavi Sonawane

Contents

A new large-scale Kazakh-Russian Sign Language dataset (FluentSigners-50) as a new Continuous Sign Language Recognition benchmark. FluentSigners-50 proposes to address three shortcomings of commonly used datasets: continuous signing, signer variety, and native signers. FluentSigners-50's main advantage is in its large signer variety: age (ranging from 8 to 57 years old), gender (18 male and 32 female), clothing, skin tone, body proportions, disability (deaf or hard of hearing), and fluency. Additionally, as the dataset was crowd-sourced: the participants were using a variety of their own recording devices (such as smartphones and web cameras), it resulted in a large variety of backgrounds, lighting conditions, camera quality, frame rates, camera aspect ratios, and angles. Finally, FluentSigners-50 contains recordings of 50 contributors that use sign language on a daily basis: either deaf, hard of hearing, hearing CODA (Child of Deaf Adults), and hearing SODA (Sibling of a Deaf Adult). As a result, the dataset contains a high degree of linguistic variability, including phonetic, phonological, lexical, and syntactic variations. It thus is a better training set for recognition of natural signing.

The FluentSigners-50 dataset consists of everyday conversational phrases and sentences in KRSL, the sign language used in the Republic of Kazakhstan. KRSL is closely related to Russian Sign Language (RSL) and some other sign languages of the ex-Soviet Union. While no official research comparing KRSL with RSL exists, our observations based on our experience researching both languages are that they show a substantial lexical overlap and are entirely mutually intelligible. The sentences and phrases of FluentSigners-50 represent the following sentence types: statements, polar questions, wh-questions, and requests.

All FluentSigners-50 contributors use sign language on a daily basis as they are either deaf (N=32), hard of hearing (N=6), hearing SODA (N=3), or hearing CODA (N=9). Native signers are signers who have been exposed to signed languages since birth because their parents are deaf. While the early acquisition may be necessary for the development of native language abilities, other factors, particularly the quality of language input, may play a role. According to this distinction, FluentSigners-50 has 30 CODA contributors (including nine hearing signers) and 20 who are not CODA (16 deaf, one hard of hearing, and three hearing SODA). Nevertheless, we decided to name our dataset FluentSigners-50 because all of our contributors use sign language daily, and it is their primary language of communication. They all came from various regions of Kazakhstan and are of different age and gender groups.

This video and gloss-based dataset has been meticulously crafted to enhance the precision and resilience of ISL (Indian Sign Language) gesture recognition and generation systems. Our goal in sharing this dataset is to contribute to the research community, providing a valuable resource for fellow researchers to explore and innovate in the realm of sign language recognition and generation.Overview of the Dataset: Comprising a diverse array of ISL gesture videos and gloss datasets. The term "gloss" in this context often refers to a written or spoken description of the meaning of a sign, allowing for the representation of sign language in a written form. The dataset includes information about the corresponding spoken or written language and the gloss for each sign. Key components of a sign language gloss dataset include ISL grammar that follows a layered approach, incorporating specific spatial indices for tense and a lexicon with compounds. It follows a unique word order based on noun, verb, object, adjective, or part of a question. Marathi sign language follows the subject-object-verb (SOV) form, facilitating comprehension and adaptation to regional languages. This Marathi sign language gloss aims to become a medium for everyday communication among deaf individuals. This dataset reflects a careful curation process, simulating real-world scenarios. The original videos showcase a variety of gestures performed by a professional signer capturing a broad spectrum of sign language expressions. Incorporating Realism with green screen with controlled lighting conditions. All videos within this dataset adhere to pixels, ensuring uniformity for data presentation and facilitating streamlined pre-processing and model development stored in a format compatible with various machine and Deep learning frameworks, these videos seamlessly integrate into the research pipeline

Dataset

This dataset was create by Kartik Shenoy

Released under Database: Open Database, Contents: © Original Authors

Contents

This dataset contains a comprehensive collection of Indian Sign Language (ISL) hand gestures representing the 26 letters of the English alphabet, captured using the MediaPipe pose estimation technology. The dataset includes over 50,000 high-quality images of hand gestures captured from various angles, lighting conditions, and skin tones. The dataset is ideal for researchers and developers working in the field of computer vision, machine learning, and sign language recognition.