Full Modality Dataset Statistics

  Video Statistics

Total Videos: 28,472 Total Duration: 1422.33 hours Average Duration: 179.84 seconds Median Duration: 160.08 seconds Duration Range: 10.04s - 1780.03s

  QA Statistics

Total Questions: 1,444,526 Average Questions per Video: 50.7 Questions per Video Range: 14 - 450

  Question Type Distribution

OE: 1,444,526 (100.0%)

  Question Category Distribution

temporal: 96,873 (6.7%) causal: 96,873… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab/full-modality-data.

The 2021-2022 School Learning Modalities dataset provides weekly estimates of school learning modality (including in-person, remote, or hybrid learning) for U.S. K-12 public and independent charter school districts for the 2021-2022 school year and the Fall 2022 semester, from August 2021 – December 2022.

These data were modeled using multiple sources of input data (see below) to infer the most likely learning modality of a school district for a given week. These data should be considered district-level estimates and may not always reflect true learning modality, particularly for districts in which data are unavailable. If a district reports multiple modality types within the same week, the modality offered for the majority of those days is reflected in the weekly estimate. All school district metadata are sourced from the https://nces.ed.gov/ccd/files.asp#Fiscal:2,LevelId:5,SchoolYearId:35,Page:1">National Center for Educational Statistics (NCES) for 2020-2021.

School learning modality types are defined as follows:

• In-Person: All schools within the district offer face-to-face instruction 5 days per week to all students at all available grade levels.
• Remote: Schools within the district do not offer face-to-face instruction; all learning is conducted online/remotely to all students at all available grade levels.
• Hybrid: Schools within the district offer a combination of in-person and remote learning; face-to-face instruction is offered less than 5 days per week, or only to a subset of students.

• School learning modality data provided here are model estimates using combined input data and are not guaranteed to be 100% accurate. This learning modality dataset was generated by combining data from four different sources: Burbio [1], MCH Strategic Data [2], the AEI/Return to Learn Tracker [3], and state dashboards [4-20]. These data were combined using a Hidden Markov model which infers the sequence of learning modalities (In-Person, Hybrid, or Remote) for each district that is most likely to produce the modalities reported by these sources. This model was trained using data from the 2020-2021 school year. Metadata describing the location, number of schools and number of students in each district comes from NCES [21].
• You can read more about the model in the CDC MMWR: https://www.cdc.gov/mmwr/volumes/70/wr/mm7039e2.htm" target="_blank">COVID-19–Related School Closures and Learning Modality Changes — United States, August 1–September 17, 2021.
• The metrics listed for each school learning modality reflect totals by district and the number of enrolled students per district for which data are available. School districts represented here exclude private schools and include the following NCES subtypes:
  • Public school district that is NOT a component of a supervisory union
  • Public school district that is a component of a supervisory union
  • Independent charter district
• “BI” in the state column refers to school districts funded by the Bureau of Indian Education.

• Data from August 1, 2021 to June 24, 2022 correspond to the 2021-2022 school year. During this time frame, data from the AEI/Return to Learn Tracker and most state dashboards were not available. Inferred modalities with a probability below 0.6 were deemed inconclusive and were omitted. During the Fall 2022 semester, modalities for districts with a school closure reported by Burbio were updated to either “Remote”, if the closure spanned the entire week, or “Hybrid”, if the closure spanned 1-4 days of the week.
• Data from August

Algorithms that classify hyper-scale multi-modal datasets, comprising of millions of images, into constituent modality types can help researchers quickly retrieve and classify diagnostic imaging data, accelerating clinical outcomes. This research aims to demonstrate that a deep neural network that is trained on a hyper-scale dataset (4.5 million images) composed of heterogeneous multi-modal data can be used to obtain significant modality classification accuracy (96%). By combining 102 medical imaging datasets, a dataset of 4.5 million images was created. A ResNet-50, ResNet-18, and VGG16 were trained to classify these images by the imaging modality used to capture them (Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and X-ray) across many body locations. The classification accuracy of the models was then tested on unseen data. The best performing model achieved classification accuracy of 96% on unseen data, which is on-par, or exceeds the accuracy of more complex implementations using EfficientNets or Vision Transformers (ViTs). The model achieved a balanced accuracy of 86%. This research shows it is possible to train Deep Learning (DL) Convolutional Neural Networks (CNNs) with hyper-scale multimodal datasets, composed of millions of images. Such models can find use in real-world applications with volumes of image data in the hyper-scale range, such as medical imaging repositories, or national healthcare institutions. Further research can expand this classification capability to include 3D-scans.

This dataset contsin the raw fMRI data of a preregistered study. Dataset includes:

session pre 1. anat/ anatomical scans (T1-weighted images) for each subject 2. func/ whole-brain EPI data from all task runs (8x single task, 2x dual task, 1x resting state and 2x localizer task) 3. fmap/ fieldmaps with magnitude1, magnitude2 and phasediff

session post 2. func/ whole-brain EPI data from all task runs (8x single task, 2x dual task) 3. fmap/ fieldmaps with magnitude1, magnitude2 and phasediff

Please note, some participants did not complete the post session. We updated our consent form to get explicit permission to publish the individual data, although not all participants resigned the new version. Those participants are excluded here but part of the t-maps on neurovault (compare participants.tsv).

Tasks were always included either visual or/and auditory input and required either manual or/and vocal responses (visual+manual and auditory+vocal are modality compatible and visual+vocal and auditory+manual are modality incompatible). Tasks were presented as either single task, or dual task. Participants completed a practice intervention prior to session post in which one group worked for 80 minutes outside the scanner on modality incompatible dual-tasks, one on modality compatible dual-task and the third one paused for 80 min.

For exact tasks description and material and scripts, please see the preregistration: https://osf.io/whpz8

Cross-modal retrieval takes one modality data as a query and retrieves semantically relevant data in another modality. Most existing cross-modal retrieval methods are designed for scenarios with complete modality data. However, in real-world applications, incomplete modality data often exists, which these methods struggle to handle effectively. In this paper, we propose a typical concept-driven modality-missing deep cross-modal retrieval model. Specifically, we first propose a multi-modal Transformer integrated with multi-modal pretraining networks, which can fully capture the multi-modal fine-grained semantic interaction in the incomplete modality data, extract multi-modal fusion semantics and construct cross-modal subspace, and at the same time supervise the learning process to generate typical concepts. In addition, the typical concepts are used as the cross-attention key and value to drive the training of the modal mapping network, so that it can adaptively preserve the implicit multi-modal semantic concepts of the query modality data, generate cross-modal retrieval features, and fully preserve the pre-extracted multi-modal fusion semantics. More information about the source code: https://gitee.com/MrSummer123/CPCMR

During the COVID-19 pandemic, many public schools across the United States shifted from fully in-person learning to alternative learning modalities such as hybrid and fully remote learning. In this study, data from 14,688 unique school districts from August 2020 to June 2021 were collected to track changes in the proportion of schools offering fully in-person, hybrid and fully remote learning over time. These data were provided by Burbio, MCH Strategic Data, the American Enterprise Institute’s Return to Learn Tracker and individual state dashboards. Because the modalities reported by these sources were incomplete and occasionally misaligned, a model was needed to combine and deconflict these data to provide a more comprehensive description of modalities nationwide. A hidden Markov model (HMM) was used to infer the most likely learning modality for each district on a weekly basis. This method yielded higher spatiotemporal coverage than any individual data source and higher agreement with three of the four data sources than any other single source. The model output revealed that the percentage of districts offering fully in-person learning rose from 40.3% in September 2020 to 54.7% in June of 2021 with increases across 45 states and in both urban and rural districts. This type of probabilistic model can serve as a tool for fusion of incomplete and contradictory data sources in order to obtain more reliable data in support of public health surveillance and research efforts.

The 2020-2021 School Learning Modalities dataset provides weekly estimates of school learning modality (including in-person, remote, or hybrid learning) for U.S. K-12 public and independent charter school districts for the 2020-2021 school year, from August 2020 – June 2021. These data were modeled using multiple sources of input data (see below) to infer the most likely learning modality of a school district for a given week. These data should be considered district-level estimates and may not always reflect true learning modality, particularly for districts in which data are unavailable. If a district reports multiple modality types within the same week, the modality offered for the majority of those days is reflected in the weekly estimate. All school district metadata are sourced from the National Center for Educational Statistics (NCES) for 2020-2021. School learning modality types are defined as follows: In-Person: All schools within the district offer face-to-face instruction 5 days per week to all students at all available grade levels. Remote: Schools within the district do not offer face-to-face instruction; all learning is conducted online/remotely to all students at all available grade levels. Hybrid: Schools within the district offer a combination of in-person and remote learning; face-to-face instruction is offered less than 5 days per week, or only to a subset of students. Data Information School learning modality data provided here are model estimates using combined input data and are not guaranteed to be 100% accurate. This learning modality dataset was generated by combining data from four different sources: Burbio [1], MCH Strategic Data [2], the AEI/Return to Learn Tracker [3], and state dashboards [4-20]. These data were combined using a Hidden Markov model which infers the sequence of learning modalities (In-Person, Hybrid, or Remote) for each district that is most likely to produce the modalities reported by these sources. This model was trained using data from the 2020-2021 school year. Metadata describing the location, number of schools and number of students in each district comes from NCES [21]. You can read more about the model in the CDC MMWR: COVID-19–Related School Closures and Learning Modality Changes — United States, August 1–September 17, 2021. The metrics listed for each school learning modality reflect totals by district and the number of enrolled students per district for which data are available. School districts represented here exclude private schools and include the following NCES subtypes: Public school district that is NOT a component of a supervisory union Public school district that is a component of a supervisory union Independent charter district “BI” in the state column refers to school districts funded by the Bureau of Indian Education. Technical Notes Data from September 1, 2020 to June 25, 2021 correspond to the 2020-2021 school year. During this timeframe, all four sources of data were available. Inferred modalities with a probability below 0.75 were deemed inconclusive and were omitted. Data for the month of July may show “In Person” status although most school districts are effectively closed during this time for summer break. Users may wish to exclude July data from use for this reason where applicable. Sources K-12 School Opening Tracker. Burbio 2021; https

Student enrollment data disaggregated by students from low-income families, students from each racial and ethnic group, gender, English learners, children with disabilities, children experiencing homelessness, children in foster care, and migratory students for each mode of instruction.

This dataset contains a collection of medical imaging files for use in the "Medical Image Processing with Python" lesson, developed by the Netherlands eScience Center.

The dataset includes:

1. SimpleITK compatible files: MRI T1 and CT scans (training_001_mr_T1.mha, training_001_ct.mha), digital X-ray (digital_xray.dcm in DICOM format), neuroimaging data (A1_grayT1.nrrd, A1_grayT2.nrrd). Data have been downloaded from here.
2. MRI data: a T2-weighted image (OBJECT_phantom_T2W_TSE_Cor_14_1.nii in NIfTI-1 format). Data have been downloaded from here.
3. Example images for the machine learning lesson: chest X-rays (rotatechest.png, other_op.png), cardiomegaly example (cardiomegaly_cc0.png).
4. Additional anonymized data: TBA

These files represent various medical imaging modalities and formats commonly used in clinical research and practice. They are intended for educational purposes, allowing students to practice image processing techniques, machine learning applications, and statistical analysis of medical images using Python libraries such as scikit-image, pydicom, and SimpleITK.

The proliferation of edge devices has generated an unprecedented volume of time series data across different domains, motivating various well-customized methods. Recently, Large Language Models (LLMs) have emerged as a new paradigm for time series analytics by leveraging the shared sequential nature of textual data and time series. However, a fundamental cross-modality gap between time series and LLMs exists, as LLMs are pre-trained on textual corpora and are not inherently optimized for time series. Many recent proposals are designed to address this issue. In this survey, we provide an up-to-date overview of LLMs-based cross-modality modeling for time series analytics. We first introduce a taxonomy that classifies existing approaches into four groups based on the type of textual data employed for time series modeling. We then summarize key cross-modality strategies, e.g., alignment and fusion, and discuss their applications across a range of downstream tasks. Furthermore, we conduct experiments on multimodal datasets from different application domains to investigate effective combinations of textual data and cross-modality strategies for enhancing time series analytics. Finally, we suggest several promising directions for future research. This survey is designed for a range of professionals, researchers, and practitioners interested in LLM-based time series modeling.

Multi-Modal Imaging Data-Integration Market Outlook

According to our latest research, the global Multi-Modal Imaging Data-Integration market size reached USD 1.67 billion in 2024. The market is expected to expand at a robust CAGR of 9.5% during the forecast period, reaching a projected value of USD 3.78 billion by 2033. This impressive growth is driven by the increasing demand for integrated imaging solutions in clinical diagnostics and research, as well as technological advancements in imaging modalities and data analytics platforms. As per our detailed analysis, the integration of multiple imaging modalities is revolutionizing the way healthcare professionals diagnose and treat complex diseases, offering comprehensive insights that single-modality imaging cannot provide.

One of the primary growth factors propelling the Multi-Modal Imaging Data-Integration market is the rising prevalence of chronic diseases such as cancer, cardiovascular disorders, and neurological conditions. These diseases often require precise and multifaceted diagnostic approaches, which multi-modal imaging excels at delivering. By combining data from modalities like MRI, CT, PET, and ultrasound, clinicians can achieve a more holistic view of patient pathology, leading to improved treatment planning and patient outcomes. Moreover, the increasing adoption of personalized medicine is further driving the need for integrated imaging data, as tailored therapeutic strategies rely heavily on accurate, multi-dimensional diagnostic information.

Another significant driver is the rapid technological evolution in both imaging hardware and software. Innovations such as artificial intelligence (AI) and machine learning are enabling more effective integration and interpretation of complex imaging datasets. Advanced integration techniques, including software-based and hybrid solutions, are making it feasible to seamlessly combine anatomical, functional, and molecular information from various imaging platforms. This technological leap is not only enhancing diagnostic precision but also reducing the time and cost associated with traditional, single-modality imaging workflows. The ongoing investment in research and development by both public and private sectors is ensuring a steady pipeline of improvements in multi-modal imaging data-integration.

The growing adoption of digital health solutions, including cloud-based imaging data repositories and telemedicine platforms, is also contributing to market expansion. Healthcare institutions are increasingly recognizing the value of integrated imaging data in facilitating remote consultations, multidisciplinary team discussions, and collaborative research. The shift toward value-based care models emphasizes outcomes and efficiency, making multi-modal data-integration an attractive proposition for hospitals, diagnostic centers, and research institutes. Additionally, regulatory support for interoperability and data standardization is gradually lowering barriers to adoption, fostering a more conducive environment for market growth.

From a regional perspective, North America continues to dominate the Multi-Modal Imaging Data-Integration market, accounting for the largest revenue share in 2024. This leadership is attributed to the region’s advanced healthcare infrastructure, high adoption rates of cutting-edge imaging technologies, and significant investments in healthcare IT. Europe follows closely, benefiting from robust government initiatives and a strong focus on research collaborations. The Asia Pacific region is emerging as the fastest-growing market, driven by expanding healthcare access, rising investments in medical technology, and an increasing burden of chronic diseases. Latin America and the Middle East & Africa, while currently holding smaller shares, are expected to witness steady growth due to improving healthcare systems and rising awareness of integrated imaging benefits.

Imaging Modality Analysis

The Imaging Modality segment forms the b

Image-based ship target analysis is an important task in the field of ship monitoring. Previous studies have achieved remarkable results in ship detection and recognition tasks. However, these related studies mainly rely on unimodal datasets, and there is still no publicly available ship individual re-identification dataset released, which restricts the research in the field of cross-modal individual re-identification of ship targets. To address this issue, we have constructed the first cross-modal ship re-identification dataset, CMShipReID. This dataset contains data from three modalities, namely visible light, near-infrared, and thermal infrared, which are collected by drones. It covers 10 categories, approximately 138 individual ships, and 8,337 images, thus providing data support for the research on cross-modal individual re-identification of ships. We have tested the mainstream re-identification algorithms as the performance benchmark for this dataset, which can serve as a fundamental reference for relevant scholars.

Multimodal AI Market size was valued at USD 1.74 Billion in 2024 and is projected to reach USD 15.89 Billion by 2032, growing at a CAGR of 4.8% from 2026 to 2032.

Multimodal AI Market Drivers

Enhanced Understanding of Complex Data: Multimodal AI allows systems to process and integrate information from various sources (text, images, audio, video), leading to a richer understanding of complex data and real-world scenarios. Improved Human-Computer Interaction: By understanding multiple modalities, AI systems can interact with humans in a more natural and intuitive way, mimicking human communication. Increased Accuracy and Robustness: Combining information from different modalities can improve the accuracy and robustness of AI models, especially in noisy or ambiguous environments. Expansion of AI Applications: Multimodal AI is enabling new applications in various fields, including: Healthcare (e.g., diagnosis from medical images and patient records) Education (e.g., personalized learning with interactive multimedia) Entertainment (e.g., immersive virtual reality experiences) Retail (e.g., enhanced product search and recommendations) Autonomous vehicles (combining visual and lidar data) Advancements in Deep Learning: Deep learning techniques, particularly transformer models, have proven highly effective in processing and integrating multimodal data. Availability of Large Multimodal Datasets: The increasing availability of large, diverse datasets containing information from multiple modalities is fueling the development of multimodal AI models. Growing Demand for Contextual AI: Multimodal AI allows AI to gain a much better understanding of context, which is vital for many applications.

🇺🇸 미국 English The 2020-2021 School Learning Modalities dataset provides weekly estimates of school learning modality (including in-person, remote, or hybrid learning) for U.S. K-12 public and independent charter school districts for the 2020-2021 school year, from August 2020 – June 2021. These data were modeled using multiple sources of input data (see below) to infer the most likely learning modality of a school district for a given week. These data should be considered district-level estimates and may not always reflect true learning modality, particularly for districts in which data are unavailable. If a district reports multiple modality types within the same week, the modality offered for the majority of those days is reflected in the weekly estimate. All school district metadata are sourced from the National Center for Educational Statistics (NCES) for 2020-2021. School learning modality types are defined as follows: In-Person: All schools within the district offer face-to-face instruction 5 days per week to all students at all available grade levels.

This data repository for MedMNIST v1 is out of date! Please check the latest version of MedMNIST v2.

Abstract

We present MedMNIST, a collection of 10 pre-processed medical open datasets. MedMNIST is standardized to perform classification tasks on lightweight 28x28 images, which requires no background knowledge. Covering the primary data modalities in medical image analysis, it is diverse on data scale (from 100 to 100,000) and tasks (binary/multi-class, ordinal regression and multi-label). MedMNIST could be used for educational purpose, rapid prototyping, multi-modal machine learning or AutoML in medical image analysis. Moreover, MedMNIST Classification Decathlon is designed to benchmark AutoML algorithms on all 10 datasets; We have compared several baseline methods, including open-source or commercial AutoML tools. The datasets, evaluation code and baseline methods for MedMNIST are publicly available at https://medmnist.github.io/.

Please note that this dataset is NOT intended for clinical use.

We recommend our official code to download, parse and use the MedMNIST dataset:

pip install medmnist

Citation and Licenses

If you find this project useful, please cite our ISBI'21 paper as: Jiancheng Yang, Rui Shi, Bingbing Ni. "MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis," arXiv preprint arXiv:2010.14925, 2020.

or using bibtex: @article{medmnist, title={MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis}, author={Yang, Jiancheng and Shi, Rui and Ni, Bingbing}, journal={arXiv preprint arXiv:2010.14925}, year={2020} }

Besides, please cite the corresponding paper if you use any subset of MedMNIST. Each subset uses the same license as that of the source dataset.

PathMNIST

Jakob Nikolas Kather, Johannes Krisam, et al., "Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study," PLOS Medicine, vol. 16, no. 1, pp. 1–22, 01 2019.

License: CC BY 4.0

ChestMNIST

Xiaosong Wang, Yifan Peng, et al., "Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in CVPR, 2017, pp. 3462–3471.

License: CC0 1.0

DermaMNIST

Philipp Tschandl, Cliff Rosendahl, and Harald Kittler, "The ham10000 dataset, a large collection of multisource dermatoscopic images of common pigmented skin lesions," Scientific data, vol. 5, pp. 180161, 2018.

Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, and Allan Halpern: “Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)”, 2018; arXiv:1902.03368.

License: CC BY-NC 4.0

OCTMNIST/PneumoniaMNIST

Daniel S. Kermany, Michael Goldbaum, et al., "Identifying medical diagnoses and treatable diseases by image-based deep learning," Cell, vol. 172, no. 5, pp. 1122 – 1131.e9, 2018.

License: CC BY 4.0

RetinaMNIST

DeepDR Diabetic Retinopathy Image Dataset (DeepDRiD), "The 2nd diabetic retinopathy – grading and image quality estimation challenge," https://isbi.deepdr.org/data.html, 2020.

License: CC BY 4.0

BreastMNIST

Walid Al-Dhabyani, Mohammed Gomaa, Hussien Khaled, and Aly Fahmy, "Dataset of breast ultrasound images," Data in Brief, vol. 28, pp. 104863, 2020.

License: CC BY 4.0

OrganMNIST_{Axial,Coronal,Sagittal}

Patrick Bilic, Patrick Ferdinand Christ, et al., "The liver tumor segmentation benchmark (lits)," arXiv preprint arXiv:1901.04056, 2019.

Xuanang Xu, Fugen Zhou, et al., "Efficient multiple organ localization in ct image using 3d region proposal network," IEEE Transactions on Medical Imaging, vol. 38, no. 8, pp. 1885–1898, 2019.

License: CC BY 4.0

Dataset for the blind submission of Leverage Points in Modality shifts: Comparing Language-only and Multimodal Word Representations paper

Dataset description: This is a study of examples of Russian impersonal constructions with the modal word možno ‘can, be possible’ with and without the future copula budet ‘will be,’ i.e., možno + budet + INF and možno + INF. The data was collected in 2020-2021 from the old version of the Russian National Corpus (ruscorpora.ru). In the spreadsheet 01DataMoznoBudet, the data merges the results of four searches conducted to extract examples of sentences with the following construction types: možno + budet + INF.PFV, možno + budet + INF.IPFV, možno + INF.PFV and možno + INF.IPFV. The results for each search were downloaded, pseudorandomized, and the first 200 examples were manually annotated, based on the syntactic analyses given in the corpus. The syntactic and morphological categories used in the corpus are explained here: https://ruscorpora.ru/corpus/main. In the spreadsheet 01DataZavtraMoznoBudet, the data merges the results of four searches conducted to extract examples of sentences with the following structure: zavtra + možno + budet + INF.PFV, zavtra + možno + budet + INF.IPFV, zavtra + možno + INF.PFV and zavtra + možno + INF.IPFV. All of the examples (103 sentences) were imported to a spreadsheet and annotated manually, based on the syntactic analyses given in the corpus. The syntactic and morphological categories used in the corpus are explained here: https://ruscorpora.ru/corpus/main. Article abstract: This paper examines Russian impersonal constructions with the modal word možno ‘can, be possible’ with and without the future copula budet ‘will be,’ i.e., možno + budet + INF and možno + INF. My contribution can be summarized as follows. First, corpus-based evidence reveals that možno + INF constructions are vastly more frequent than constructions with copula. Second, the meaning of constructions without the future copula is more flexible: while the possibility is typically located in the present, the situation denoted by the infinitive may be located in the present or the future. Third, I show that the možno + INF construction is more ambiguous and can denote present, gnomic or future situations. Fourth, I identify a number of contextual factors that unambiguously locate the situation in the future. I demonstrate that such factors are more frequently used with the future copula, and thus motivate the choice between the two constructions. Finally, I illustrate the interpretations in a straightforward manner by means of schemas of the type used in cognitive linguistics.

luisrui/Modality-Interference-in-MLLMs-DATA dataset hosted on Hugging Face and contributed by the HF Datasets community

A collection of 7 brain maps. Each brain map is a 3D array of values representing properties of the brain at different locations.

Collection description

This collection contains group-level activation maps for the fMRI study "Task-Dependent Recruitment of Modality-Specific and Multimodal Regions during Conceptual Processing" by Kuhnke et al. (2020). The study investigated task dependency of conceptual knowledge retrieval, focusing on sound and action features of word meaning.

Modality exclusivity norms have been developed in different languages for research on the relationship between perceptual and conceptual systems. This paper sets up the first modality exclusivity norms for Chinese, a Sino-Tibetan language with semantics as its orthographically relevant level. The norms are collected through two studies based on Chinese sensory words. The experimental designs take into consideration the morpho-lexical and orthographic structures of Chinese. Study 1 provides a set of norms for Mandarin Chinese single-morpheme words in mean ratings of the extent to which a word is experienced through the five sense modalities. The degrees of modality exclusivity are also provided. The collected norms are further analyzed to examine how sub-lexical orthographic representations of sense modalities in Chinese characters affect speakers’ interpretation of the sensory words. In particular, we found higher modality exclusivity rating for the sense modality explicitly represented by a semantic radical component, as well as higher auditory dominant modality rating for characters with transparent phonetic symbol components. Study 2 presents the mean ratings and modality exclusivity of coordinate disyllabic compounds involving multiple sense modalities. These studies open new perspectives in the study of modality exclusivity. First, links between modality exclusivity and writing systems have been established which has strengthened previous accounts of the influence of orthography in the processing of visual information in reading. Second, a new set of modality exclusivity norms of compounds is proposed to show the competition of influence on modality exclusivity from different linguistic factors and potentially allow such norms to be linked to studies on synesthesia and semantic transparency.