The open-source data annotation tool market is experiencing robust growth, driven by the increasing demand for high-quality training data in artificial intelligence (AI) and machine learning (ML) applications. The market's expansion is fueled by several key factors: the rising adoption of AI across various industries (including automotive, healthcare, and finance), the need for efficient and cost-effective data annotation solutions, and a growing preference for flexible, customizable tools offered by open-source platforms. While cloud-based solutions currently dominate the market due to scalability and accessibility, on-premise deployments remain significant for organizations with stringent data security requirements. The competitive landscape is dynamic, with numerous established players and emerging startups vying for market share. The market is segmented geographically, with North America and Europe currently holding the largest shares due to early adoption of AI technologies and robust research & development activities. However, the Asia-Pacific region is projected to witness significant growth in the coming years, driven by increasing investments in AI infrastructure and talent development. Challenges remain, such as the need for skilled annotators and the ongoing evolution of annotation techniques to handle increasingly complex data types. The forecast period (2025-2033) suggests continued expansion, with a projected Compound Annual Growth Rate (CAGR) – let's conservatively estimate this at 15% based on typical growth in related software sectors. This growth will be influenced by advancements in automation and semi-automated annotation tools, as well as the emergence of novel annotation paradigms. The market is expected to see further consolidation, with larger players potentially acquiring smaller, specialized companies. The increasing focus on data privacy and security will necessitate the development of more robust and compliant open-source annotation tools. Specific application segments like healthcare, with its stringent regulatory landscape, and the automotive industry, with its reliance on autonomous driving technology, will continue to be major drivers of market growth. The increasing availability of open-source datasets and pre-trained models will indirectly contribute to the market’s expansion by lowering the barrier to entry for AI development.

The open-source data labeling tool market is experiencing robust growth, driven by the increasing demand for high-quality training data in various AI applications. The market's expansion is fueled by several key factors: the rising adoption of machine learning and deep learning algorithms across industries, the need for efficient and cost-effective data annotation solutions, and a growing preference for customizable and flexible tools that can adapt to diverse data types and project requirements. While proprietary solutions exist, the open-source ecosystem offers advantages including community support, transparency, cost-effectiveness, and the ability to tailor tools to specific needs, fostering innovation and accessibility. The market is segmented by tool type (image, text, video, audio), deployment model (cloud, on-premise), and industry (automotive, healthcare, finance). We project a market size of approximately $500 million in 2025, with a compound annual growth rate (CAGR) of 25% from 2025 to 2033, reaching approximately $2.7 billion by 2033. This growth is tempered by challenges such as the complexities associated with data security, the need for skilled personnel to manage and use these tools effectively, and the inherent limitations of certain open-source solutions compared to their commercial counterparts. Despite these restraints, the open-source model's inherent flexibility and cost advantages will continue to attract a significant user base. The market's competitive landscape includes established players like Alecion and Appen, alongside numerous smaller companies and open-source communities actively contributing to the development and improvement of these tools. Geographical expansion is expected across North America, Europe, and Asia-Pacific, with the latter projected to witness significant growth due to the increasing adoption of AI and machine learning in developing economies. Future market trends point towards increased integration of automated labeling techniques within open-source tools, enhanced collaborative features to improve efficiency, and further specialization to cater to specific data types and industry-specific requirements. Continuous innovation and community contributions will remain crucial drivers of growth in this dynamic market segment.

The open-source data labeling tool market is experiencing robust growth, driven by the increasing demand for high-quality training data in machine learning and artificial intelligence applications. The market's expansion is fueled by several factors: the rising adoption of AI across various sectors (including IT, automotive, healthcare, and finance), the need for cost-effective data annotation solutions, and the inherent flexibility and customization offered by open-source tools. While cloud-based solutions currently dominate the market due to scalability and accessibility, on-premise deployments remain significant, particularly for organizations with stringent data security requirements. The market's growth is further propelled by advancements in automation and semi-supervised learning techniques within data labeling, leading to increased efficiency and reduced annotation costs. Geographic distribution shows a strong concentration in North America and Europe, reflecting the higher adoption of AI technologies in these regions; however, Asia-Pacific is emerging as a rapidly growing market due to increasing investment in AI and the availability of a large workforce for data annotation. Despite the promising outlook, certain challenges restrain market growth. The complexity of implementing and maintaining open-source tools, along with the need for specialized technical expertise, can pose barriers to entry for smaller organizations. Furthermore, the quality control and data governance aspects of open-source annotation require careful consideration. The potential for data bias and the need for robust validation processes necessitate a strategic approach to ensure data accuracy and reliability. Competition is intensifying with both established and emerging players vying for market share, forcing companies to focus on differentiation through innovation and specialized functionalities within their tools. The market is anticipated to maintain a healthy growth trajectory in the coming years, with increasing adoption across diverse sectors and geographical regions. The continued advancements in automation and the growing emphasis on data quality will be key drivers of future market expansion.

Leaves from genetically unique Juglans regia plants were scanned using X-ray micro-computed tomography (microCT) on the X-ray μCT beamline (8.3.2) at the Advanced Light Source (ALS) in Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA USA). Soil samples were collected in Fall of 2017 from the riparian oak forest located at the Russell Ranch Sustainable Agricultural Institute at the University of California Davis. The soil was sieved through a 2 mm mesh and was air dried before imaging. A single soil aggregate was scanned at 23 keV using the 10x objective lens with a pixel resolution of 650 nanometers on beamline 8.3.2 at the ALS. Additionally, a drought stressed almond flower bud (Prunus dulcis) from a plant housed at the University of California, Davis, was scanned using a 4x lens with a pixel resolution of 1.72 µm on beamline 8.3.2 at the ALS Raw tomographic image data was reconstructed using TomoPy. Reconstructions were converted to 8-bit tif or png format using ImageJ or the PIL package in Python before further processing. Images were annotated using Intel’s Computer Vision Annotation Tool (CVAT) and ImageJ. Both CVAT and ImageJ are free to use and open source. Leaf images were annotated in following Théroux-Rancourt et al. (2020). Specifically, Hand labeling was done directly in ImageJ by drawing around each tissue; with 5 images annotated per leaf. Care was taken to cover a range of anatomical variation to help improve the generalizability of the models to other leaves. All slices were labeled by Dr. Mina Momayyezi and Fiona Duong.To annotate the flower bud and soil aggregate, images were imported into CVAT. The exterior border of the bud (i.e. bud scales) and flower were annotated in CVAT and exported as masks. Similarly, the exterior of the soil aggregate and particulate organic matter identified by eye were annotated in CVAT and exported as masks. To annotate air spaces in both the bud and soil aggregate, images were imported into ImageJ. A gaussian blur was applied to the image to decrease noise and then the air space was segmented using thresholding. After applying the threshold, the selected air space region was converted to a binary image with white representing the air space and black representing everything else. This binary image was overlaid upon the original image and the air space within the flower bud and aggregate was selected using the “free hand” tool. Air space outside of the region of interest for both image sets was eliminated. The quality of the air space annotation was then visually inspected for accuracy against the underlying original image; incomplete annotations were corrected using the brush or pencil tool to paint missing air space white and incorrectly identified air space black. Once the annotation was satisfactorily corrected, the binary image of the air space was saved. Finally, the annotations of the bud and flower or aggregate and organic matter were opened in ImageJ and the associated air space mask was overlaid on top of them forming a three-layer mask suitable for training the fully convolutional network. All labeling of the soil aggregate and soil aggregate images was done by Dr. Devin Rippner. These images and annotations are for training deep learning models to identify different constituents in leaves, almond buds, and soil aggregates Limitations: For the walnut leaves, some tissues (stomata, etc.) are not labeled and only represent a small portion of a full leaf. Similarly, both the almond bud and the aggregate represent just one single sample of each. The bud tissues are only divided up into buds scales, flower, and air space. Many other tissues remain unlabeled. For the soil aggregate annotated labels are done by eye with no actual chemical information. Therefore particulate organic matter identification may be incorrect. Resources in this dataset:Resource Title: Annotated X-ray CT images and masks of a Forest Soil Aggregate. File Name: forest_soil_images_masks_for_testing_training.zipResource Description: This aggregate was collected from the riparian oak forest at the Russell Ranch Sustainable Agricultural Facility. The aggreagate was scanned using X-ray micro-computed tomography (microCT) on the X-ray μCT beamline (8.3.2) at the Advanced Light Source (ALS) in Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA USA) using the 10x objective lens with a pixel resolution of 650 nanometers. For masks, the background has a value of 0,0,0; pores spaces have a value of 250,250, 250; mineral solids have a value= 128,0,0; and particulate organic matter has a value of = 000,128,000. These files were used for training a model to segment the forest soil aggregate and for testing the accuracy, precision, recall, and f1 score of the model.Resource Title: Annotated X-ray CT images and masks of an Almond bud (P. Dulcis). File Name: Almond_bud_tube_D_P6_training_testing_images_and_masks.zipResource Description: Drought stressed almond flower bud (Prunis dulcis) from a plant housed at the University of California, Davis, was scanned by X-ray micro-computed tomography (microCT) on the X-ray μCT beamline (8.3.2) at the Advanced Light Source (ALS) in Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA USA) using the 4x lens with a pixel resolution of 1.72 µm using. For masks, the background has a value of 0,0,0; air spaces have a value of 255,255, 255; bud scales have a value= 128,0,0; and flower tissues have a value of = 000,128,000. These files were used for training a model to segment the almond bud and for testing the accuracy, precision, recall, and f1 score of the model.Resource Software Recommended: Fiji (ImageJ),url: https://imagej.net/software/fiji/downloads Resource Title: Annotated X-ray CT images and masks of Walnut leaves (J. Regia) . File Name: 6_leaf_training_testing_images_and_masks_for_paper.zipResource Description: Stems were collected from genetically unique J. regia accessions at the 117 USDA-ARS-NCGR in Wolfskill Experimental Orchard, Winters, California USA to use as scion, and were grafted by Sierra Gold Nursery onto a commonly used commercial rootstock, RX1 (J. microcarpa × J. regia). We used a common rootstock to eliminate any own-root effects and to simulate conditions for a commercial walnut orchard setting, where rootstocks are commonly used. The grafted saplings were repotted and transferred to the Armstrong lathe house facility at the University of California, Davis in June 2019, and kept under natural light and temperature. Leaves from each accession and treatment were scanned using X-ray micro-computed tomography (microCT) on the X-ray μCT beamline (8.3.2) at the Advanced Light Source (ALS) in Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA USA) using the 10x objective lens with a pixel resolution of 650 nanometers. For masks, the background has a value of 170,170,170; Epidermis value= 85,85,85; Mesophyll value= 0,0,0; Bundle Sheath Extension value= 152,152,152; Vein value= 220,220,220; Air value = 255,255,255.Resource Software Recommended: Fiji (ImageJ),url: https://imagej.net/software/fiji/downloads

Image Annotation Tool Market Outlook

The global image annotation tool market size is projected to grow from approximately $700 million in 2023 to an estimated $2.5 billion by 2032, exhibiting a remarkable compound annual growth rate (CAGR) of 15.2% over the forecast period. The surging demand for machine learning and artificial intelligence applications is driving this robust market expansion. Image annotation tools are crucial for training AI models to recognize and interpret images, a necessity across diverse industries.

One of the key growth factors fueling the image annotation tool market is the rapid adoption of AI and machine learning technologies across various sectors. Organizations in healthcare, automotive, retail, and many other industries are increasingly leveraging AI to enhance operational efficiency, improve customer experiences, and drive innovation. Accurate image annotation is essential for developing sophisticated AI models, thereby boosting the demand for these tools. Additionally, the proliferation of big data analytics and the growing necessity to manage large volumes of unstructured data have amplified the need for efficient image annotation solutions.

Another significant driver is the increasing use of autonomous systems and applications. In the automotive industry, for instance, the development of autonomous vehicles relies heavily on annotated images to train algorithms for object detection, lane discipline, and navigation. Similarly, in the healthcare sector, annotated medical images are indispensable for developing diagnostic tools and treatment planning systems powered by AI. This widespread application of image annotation tools in the development of autonomous systems is a critical factor propelling market growth.

The rise of e-commerce and the digital retail landscape has also spurred demand for image annotation tools. Retailers are using these tools to optimize visual search features, personalize shopping experiences, and enhance inventory management through automated recognition of products and categories. Furthermore, advancements in computer vision technology have expanded the capabilities of image annotation tools, making them more accurate and efficient, which in turn encourages their adoption across various industries.

Data Annotation Software plays a pivotal role in the image annotation tool market by providing the necessary infrastructure for labeling and categorizing images efficiently. These software solutions are designed to handle various annotation tasks, from simple bounding boxes to complex semantic segmentation, enabling organizations to generate high-quality training datasets for AI models. The continuous advancements in data annotation software, including the integration of machine learning algorithms for automated labeling, have significantly enhanced the accuracy and speed of the annotation process. As the demand for AI-driven applications grows, the reliance on robust data annotation software becomes increasingly critical, supporting the development of sophisticated models across industries.

Regionally, North America holds the largest share of the image annotation tool market, driven by significant investments in AI and machine learning technologies and the presence of leading technology companies. Europe follows, with strong growth supported by government initiatives promoting AI research and development. The Asia Pacific region presents substantial growth opportunities due to the rapid digital transformation in emerging economies and increasing investments in technology infrastructure. Latin America and the Middle East & Africa are also expected to witness steady growth, albeit at a slower pace, due to the gradual adoption of advanced technologies.

Component Analysis

The image annotation tool market by component is segmented into software and services. The software segment dominates the market, encompassing a variety of tools designed for different annotation tasks, from simple image labeling to complex polygonal, semantic, or instance segmentation. The continuous evolution of software platforms, integrating advanced features such as automated annotation and machine learning algorithms, has significantly enhanced the accuracy and efficiency of image annotations. Furthermore, the availability of open-source annotation tools has lowered the entry barrier, allowing more organizations to adopt these technologies.

Services associated with image ann

To develop robust algorithms for automated diagnosis of medical conditions such as cardiac arrhythmias, researchers require large collections of data with human expert annotations. Currently, there is a lack of accessible, open-source platforms for human experts to collaboratively develop these annotated datasets through a web interface. In this work, we developed a flexible, generalizable, web-based framework to enable multiple users to create and share annotations on multi-channel physiological waveforms. The software is simple to install and offers a range of features, including: user management and task customization; a programmatic interface for data import and export; and a leaderboard for annotation progress tracking.

Tool Bar Annotation

## Overview

Tool Bar Annotation is a dataset for object detection tasks - it contains 8MM 16MM annotations for 2,835 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

The Open Source Data Annotation Tool market is rapidly evolving as businesses and researchers increasingly recognize the significance of high-quality, labeled data for training machine learning models. These tools facilitate the efficient tagging and classification of various data types, including images, text, and

Files used for computing dog body parts and their annotation file.In every folder the original .oni video files the annotation file .ann and segmentation parameters .spthe redme file describes the annotation file formatAnnotation software is also included to possibly annotate new sequences

LARa Version 03 is a freely accessible logistics-dataset for human activity recognition. In the “Innovationlab Hybrid Services in Logistics” at TU Dortmund University, two picking and one packing scenarios with 16 subjects were recorded using an optical marker-based Motion Capturing system (OMoCap), Inertial Measurement Units (IMUs), and an RGB camera. Each subject was recorded for one hour (960 minutes in total). All the given data have been labelled and categorised into eight activity classes and 19 binary coarse-semantic descriptions, also called attributes. In total, the dataset contains 221 unique attribute representations.

The dataset was created according to the guideline of the following paper: “A Tutorial on Dataset Creation for Sensor-based Human Activity Recognition”, PerCom, 2023 DOI: 10.1109/PerComWorkshops56833.2023.10150401

The LARa Version 03 contains a new Annotation tool for OMoCap and RGB Videos, namely, the Sequence Attribute Retrieval Annotator (SARA). SARA, developed and modified based on the LARa Version 02 annotation tool, includes desirable features and attempts to overcome limitations as found in the LARa annotation tool. Furthermore, few features were included based on the explorative study of previously developed annotation tools, see journal. In alignment with the LARa annotation tool, SARA focuses on OMoCap and video annotations. However, it is to be noted that SARA was not intended to be a video annotation tool with features such as subject tracking and multiple subject annotations. Here, the video is considered to be a supporting input to the OMoCap annotation. We would recommend other tools for pure video-based multiple-human activity annotation, including subject tracking, segmentation, and pose estimation. There are different ways of installing the annotation tool: Compiled binaries (executable files) for Windows and Mac can be directly downloaded from here. Python users can install the tool from https://pypi.org/project/annotation-tool/ (PyPi): “pip install annotation-tool”. For more information, please refer to the “Annotation Tool - Installation and User Manual”.

Upgrade:

Annotation tool (SARA) added (for Windows and MacOS, including an installation and user manual)

Neural Networks updated (can be used with the annotation tool)

OMoCap data:

Annotation errors corrected

Annotations reformatted, fitting the SARA annotation tool

“additional annotated data” extended

“Markers_Exports” added

IMU data (MbientLab and MotionMiners Sensors)

Annotation errors corrected

README file (protocol) updated and extended

If you use this dataset for research, please cite the following paper: “LARa: Creating a Dataset for Human Activity Recognition in Logistics Using Semantic Attributes”, Sensors 2020, DOI: 10.3390/s20154083.

If you use the Mbientlab Networks, please cite the following paper: “From Human Pose to On-Body Devices for Human-Activity Recognition”, 25th International Conference on Pattern Recognition (ICPR), 2021, DOI: 10.1109/ICPR48806.2021.9412283.

For any questions about the dataset, please contact Friedrich Niemann at friedrich.niemann@tu-dortmund.de.

Open Source Data Labelling Tool Market Outlook

The global market size for Open Source Data Labelling Tools was valued at USD 1.5 billion in 2023 and is projected to reach USD 4.6 billion by 2032, growing at a compound annual growth rate (CAGR) of 13.2% during the forecast period. This significant growth can be attributed to the increasing adoption of artificial intelligence (AI) and machine learning (ML) across various industries, which drives the need for accurately labelled data to train these technologies effectively.

The rapid advancement and integration of AI and ML in numerous sectors serve as a primary growth factor for the Open Source Data Labelling Tool market. With the proliferation of big data, organizations are increasingly recognizing the importance of high-quality, annotated data sets to enhance the accuracy and efficiency of their AI models. The open-source nature of these tools offers flexibility and cost-effectiveness, making them an attractive choice for businesses of all sizes, especially startups and SMEs, which further fuels market growth.

Another key driver is the rising demand for automated data labelling solutions. Manual data labelling is a time-consuming and error-prone task, leading many organizations to seek automated tools that can swiftly and accurately label large datasets. Open source data labelling tools, often augmented with advanced features like natural language processing (NLP) and computer vision, provide a scalable solution to this challenge. This trend is particularly pronounced in data-intensive industries such as healthcare, automotive, and finance, where the precision of data labelling can significantly impact operational outcomes.

Additionally, the collaborative nature of open-source communities contributes to the market's growth. Continuous improvements and updates are driven by a global community of developers and researchers, ensuring that these tools remain at the cutting edge of technology. This ongoing innovation not only boosts the functionality and reliability of open-source data labelling tools but also fosters a sense of community and shared knowledge, encouraging more organizations to adopt these solutions.

In the realm of data labelling, Premium Annotation Tools have emerged as a significant player, offering advanced features that cater to the needs of enterprises seeking high-quality data annotation. These tools often come equipped with enhanced functionalities such as collaborative interfaces, real-time updates, and integration capabilities with existing AI systems. The premium nature of these tools ensures that they are designed to handle complex datasets with precision, thereby reducing the margin of error in data labelling processes. As businesses increasingly prioritize accuracy and efficiency, the demand for premium solutions is on the rise, providing a competitive edge in sectors where data quality is paramount.

From a regional perspective, North America holds a significant share of the market due to the robust presence of tech giants and a well-established IT infrastructure. The region's strong focus on AI research and development, coupled with substantial investments in technology, drives the demand for data labelling tools. Meanwhile, the Asia Pacific region is expected to exhibit the highest growth rate during the forecast period, attributed to the rapid digital transformation and increasing AI adoption across countries like China, India, and Japan.

Component Analysis

When dissecting the Open Source Data Labelling Tool market by component, it is evident that the segment is bifurcated into software and services. The software segment dominates the market, primarily due to the extensive range of features and functionalities that open-source data labelling software offers. These tools are customizable and can be tailored to meet specific needs, making them highly versatile and efficient. The software segment is expected to continue its dominance as more organizations seek comprehensive solutions that integrate seamlessly with their existing systems.

The services segment, while smaller in comparison, plays a crucial role in the overall market landscape. Services include support, training, and consulting, which are vital for organizations to effectively implement and utilize open-source data labelling tools. As the adoption of these tools grows, so does the demand for professional services that can aid in deployment, customization

a Traffic data Visualization and Annotation tool - version 1.1 as published in the Journal of Open-Source Software

LabelMe database is a large collection of images with ground truth labels for object detection and recognition. The annotations come from two different sources, including the LabelMe online annotation tool.

This dataset contains 35 of 39 taxonomies that were the result of a systematic review. The systematic review was conducted with the goal of identifying taxonomies suitable for semantically annotating research data. A special focus was set on research data from the hybrid societies domain.

The following taxonomies were identified as part of the systematic review:

Filename

Taxonomy Title

acm_ccs

ACM Computing Classification System [1]

amec

A Taxonomy of Evaluation Towards Standards [2]

bibo

A BIBO Ontology Extension for Evaluation of Scientific Research Results [3]

cdt

Cross-Device Taxonomy [4]

cso

Computer Science Ontology [5]

ddbm

What Makes a Data-driven Business Model? A Consolidated Taxonomy [6]

ddi_am

DDI Aggregation Method [7]

ddi_moc

DDI Mode of Collection [8]

n/a

DemoVoc [9]

discretization

Building a New Taxonomy for Data Discretization Techniques [10]

dp

Demopaedia [11]

dsg

Data Science Glossary [12]

ease

A Taxonomy of Evaluation Approaches in Software Engineering [13]

eco

Evidence & Conclusion Ontology [14]

edam

EDAM: The Bioscientific Data Analysis Ontology [15]

n/a

European Language Social Science Thesaurus [16]

et

Evaluation Thesaurus [17]

glos_hci

The Glossary of Human Computer Interaction [18]

n/a

Humanities and Social Science Electronic Thesaurus [19]

hcio

A Core Ontology on the Human-Computer Interaction Phenomenon [20]

hft

Human-Factors Taxonomy [21]

hri

A Taxonomy to Structure and Analyze Human–Robot Interaction [22]

iim

A Taxonomy of Interaction for Instructional Multimedia [23]

interrogation

A Taxonomy of Interrogation Methods [24]

iot

Design Vocabulary for Human–IoT Systems Communication [25]

kinect

Understanding Movement and Interaction: An Ontology for Kinect-Based 3D Depth Sensors [26]

maco

Thesaurus Mass Communication [27]

n/a

Thesaurus Cognitive Psychology of Human Memory [28]

mixed_initiative

Mixed-Initiative Human-Robot Interaction: Definition, Taxonomy, and Survey [29]

qos_qoe

A Taxonomy of Quality of Service and Quality of Experience of Multimodal Human-Machine Interaction [30]

ro

The Research Object Ontology [31]

senses_sensors

A Human-Centered Taxonomy of Interaction Modalities and Devices [32]

sipat

A Taxonomy of Spatial Interaction Patterns and Techniques [33]

social_errors

A Taxonomy of Social Errors in Human-Robot Interaction [34]

sosa

Semantic Sensor Network Ontology [35]

swo

The Software Ontology [36]

tadirah

Taxonomy of Digital Research Activities in the Humanities [37]

vrs

Virtual Reality and the CAVE: Taxonomy, Interaction Challenges and Research Directions [38]

xdi

Cross-Device Interaction [39]

We converted the taxonomies into SKOS (Simple Knowledge Organisation System) representation. The following 4 taxonomies were not converted as they were already available in SKOS and were for this reason excluded from this dataset:

1) DemoVoc, cf. http://thesaurus.web.ined.fr/navigateur/ available at https://thesaurus.web.ined.fr/exports/demovoc/demovoc.rdf

2) European Language Social Science Thesaurus, cf. https://thesauri.cessda.eu/elsst/en/ available at https://zenodo.org/record/5506929

3) Humanities and Social Science Electronic Thesaurus, cf. https://hasset.ukdataservice.ac.uk/hasset/en/ available at https://zenodo.org/record/7568355

4) Thesaurus Cognitive Psychology of Human Memory, cf. https://www.loterre.fr/presentation/ available at https://skosmos.loterre.fr/P66/en/

References

[1] “The 2012 ACM Computing Classification System,” ACM Digital Library, 2012. https://dl.acm.org/ccs (accessed May 08, 2023).

[2] AMEC, “A Taxonomy of Evaluation Towards Standards.” Aug. 31, 2016. Accessed: May 08, 2023. [Online]. Available: https://amecorg.com/amecframework/home/supporting-material/taxonomy/

[3] B. Dimić Surla, M. Segedinac, and D. Ivanović, “A BIBO ontology extension for evaluation of scientific research results,” in Proceedings of the Fifth Balkan Conference in Informatics, in BCI ’12. New York, NY, USA: Association for Computing Machinery, Sep. 2012, pp. 275–278. doi: 10.1145/2371316.2371376.

[4] F. Brudy et al., “Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, in CHI ’19. New York, NY, USA: Association for Computing Machinery, Mai 2019, pp. 1–28. doi: 10.1145/3290605.3300792.

[5] A. A. Salatino, T. Thanapalasingam, A. Mannocci, F. Osborne, and E. Motta, “The Computer Science Ontology: A Large-Scale Taxonomy of Research Areas,” in Lecture Notes in Computer Science 1137, D. Vrandečić, K. Bontcheva, M. C. Suárez-Figueroa, V. Presutti, I. Celino, M. Sabou, L.-A. Kaffee, and E. Simperl, Eds., Monterey, California, USA: Springer, Oct. 2018, pp. 187–205. Accessed: May 08, 2023. [Online]. Available: http://oro.open.ac.uk/55484/

[6] M. Dehnert, A. Gleiss, and F. Reiss, “What makes a data-driven business model? A consolidated taxonomy,” presented at the European Conference on Information Systems, 2021.

[7] DDI Alliance, “DDI Controlled Vocabulary for Aggregation Method,” 2014. https://ddialliance.org/Specification/DDI-CV/AggregationMethod_1.0.html (accessed May 08, 2023).

[8] DDI Alliance, “DDI Controlled Vocabulary for Mode Of Collection,” 2015. https://ddialliance.org/Specification/DDI-CV/ModeOfCollection_2.0.html (accessed May 08, 2023).

[9] INED - French Institute for Demographic Studies, “Thésaurus DemoVoc,” Feb. 26, 2020. https://thesaurus.web.ined.fr/navigateur/en/about (accessed May 08, 2023).

[10] A. A. Bakar, Z. A. Othman, and N. L. M. Shuib, “Building a new taxonomy for data discretization techniques,” in 2009 2nd Conference on Data Mining and Optimization, Oct. 2009, pp. 132–140. doi: 10.1109/DMO.2009.5341896.

[11] N. Brouard and C. Giudici, “Unified second edition of the Multilingual Demographic Dictionary (Demopaedia.org project),” presented at the 2017 International Population Conference, IUSSP, Oct. 2017. Accessed: May 08, 2023. [Online]. Available: https://iussp.confex.com/iussp/ipc2017/meetingapp.cgi/Paper/5713

[12] DuCharme, Bob, “Data Science Glossary.” https://www.datascienceglossary.org/ (accessed May 08, 2023).

[13] A. Chatzigeorgiou, T. Chaikalis, G. Paschalidou, N. Vesyropoulos, C. K. Georgiadis, and E. Stiakakis, “A Taxonomy of Evaluation Approaches in Software Engineering,” in Proceedings of the 7th Balkan Conference on Informatics Conference, in BCI ’15. New York, NY, USA: Association for Computing Machinery, Sep. 2015, pp. 1–8. doi: 10.1145/2801081.2801084.

[14] M. C. Chibucos, D. A. Siegele, J. C. Hu, and M. Giglio, “The Evidence and Conclusion Ontology (ECO): Supporting GO Annotations,” in The Gene Ontology Handbook, C. Dessimoz and N. Škunca, Eds., in Methods in Molecular Biology. New York, NY: Springer, 2017, pp. 245–259. doi: 10.1007/978-1-4939-3743-1_18.

[15] M. Black et al., “EDAM: the bioscientific data analysis ontology,” F1000Research, vol. 11, Jan. 2021, doi: 10.7490/f1000research.1118900.1.

[16] Council of European Social Science Data Archives (CESSDA), “European Language Social Science Thesaurus ELSST,” 2021. https://thesauri.cessda.eu/en/ (accessed May 08, 2023).

[17] M. Scriven, Evaluation Thesaurus, 3rd Edition. Edgepress, 1981. Accessed: May 08, 2023. [Online]. Available: https://us.sagepub.com/en-us/nam/evaluation-thesaurus/book3562

[18] Papantoniou, Bill et al., The Glossary of Human Computer Interaction. Interaction Design Foundation. Accessed: May 08, 2023. [Online]. Available: https://www.interaction-design.org/literature/book/the-glossary-of-human-computer-interaction

[19] “UK Data Service Vocabularies: HASSET Thesaurus.” https://hasset.ukdataservice.ac.uk/hasset/en/ (accessed May 08, 2023).

[20] S. D. Costa, M. P. Barcellos, R. de A. Falbo, T. Conte, and K. M. de Oliveira, “A core ontology on the Human–Computer Interaction phenomenon,” Data Knowl. Eng., vol. 138, p. 101977, Mar. 2022, doi: 10.1016/j.datak.2021.101977.

[21] V. J. Gawron et al., “Human Factors Taxonomy,” Proc. Hum. Factors Soc. Annu. Meet., vol. 35, no. 18, pp. 1284–1287, Sep. 1991, doi: 10.1177/154193129103501807.

[22] L. Onnasch and E. Roesler, “A Taxonomy to Structure and Analyze Human–Robot Interaction,” Int. J. Soc. Robot., vol. 13, no. 4, pp. 833–849, Jul. 2021, doi: 10.1007/s12369-020-00666-5.

[23] R. A. Schwier, “A Taxonomy of Interaction for Instructional Multimedia.” Sep. 28, 1992. Accessed: May 09, 2023. [Online]. Available: https://eric.ed.gov/?id=ED352044

[24] C. Kelly, J. Miller, A. Redlich, and S. Kleinman, “A Taxonomy of Interrogation Methods,”

AdA Project Public Data Release

This repository holds public data provided by the AdA project (Affektrhetoriken des Audiovisuellen - BMBF eHumanities Research Group Audio-Visual Rhetorics of Affect).

See: http://www.ada.cinepoetics.fu-berlin.de/en/index.html The data is made accessible under the terms of the Creative Commons Attribution-ShareAlike 3.0 License. The data can be accessed also at the project's public GitHub repository: https://github.com/ProjectAdA/public

Further explanations of the data can be found on our AdA project website: https://projectada.github.io/. See also the peer-reviewed data paper for this dataset that is in review to be published in NECSUS_European Journal of Media Studies, and will be available from https://necsus-ejms.org/ and https://mediarep.org

The data currently includes:

AdA Filmontology

The latest public release of the AdA Filmontology: https://github.com/ProjectAdA/public/tree/master/ontology

A vocabulary of film-analytical terms and concepts for fine-grained semantic video annotation.

The vocabulary is also available online in our triplestore: https://ada.cinepoetics.org/resource/2021/05/19/eMAEXannotationMethod.html

Advene Annotation Template

The latest public release of the template for the Advene annotation software: https://github.com/ProjectAdA/public/tree/master/advene_template

The template provides the developed semantic vocabulary in the Advene software with ready-to-use annotation tracks and predefined values.

In order to use the template you have to install and use Advene: https://www.advene.org/

Annotation Data

The latest public releases of our annotation datasets based on the AdA vocabulary: https://github.com/ProjectAdA/public/tree/master/annotations

The dataset of news reports, documentaries and feature films on the topic of "financial crisis" contains more than 92.000 manual & semi-automatic annotations authored in the open-source-software Advene (Aubert/Prié 2005) by expert annotators as well as more than 400.000 automatically generated annotations for wider corpus exploration. The annotations are published as Linked Open Data under the CC BY-SA 3.0 licence and available as rdf triples in turtle exports (ttl files) and in Advene's non-proprietary azp-file format, which allows instant access through the graphical interface of the software.

The annotation data can also be queried at our public SPARQL Endpoint: http://ada.filmontology.org/sparql

• The dataset contains furthermore sample files for two different export capabilities of the web application AdA Annotation explorer: 1) all manual annotations of the type "field size" throughout the corpus as csv files. 2) static html exports of different queries conducted in the AdA Annotation Explorer.

Manuals

The data set includes different manuals and documentations in German and English: https://github.com/ProjectAdA/public/tree/master/manuals

"AdA Filmontology – Levels, Types, Values": an overview over all analytical concepts and their definitions.

"Manual: Annotating with Advene and the AdA Filmontology". A manual on the usage of Advene and the AdA Annotation Explorer that provides the basics for annotating audiovisual aesthetics and visualizing them.

"Notes on collaborative annotation with the AdA Filmontology"

General information

The dataset consists of 4403 labelled subscenes from 155 Sentinel-2 (S2) Level-1C (L1C) products distributed over the Northern European terrestrial area. Each S2 product was oversampled at 10 m resolution for 512 x 512 pixels subscenes. 6 L1C S2 products were labelled fully. Among other 149 S2 products the most challenging ~10 subscenes per product were selected for labelling. In total the dataset represents 4403 labelled Sentinel-2 subscenes, where each sub-tile is 512 x 512 pixels at 10 m resolution. The dataset consists of around 30 S2 products per month from April to August and 3 S2 products per month for September and October. Each selected L1C S2 product represents different clouds, such as cumulus, stratus, or cirrus, which are spread over various geographical locations in Northern Europe.

The classification pixel-wise map consists of the following categories:

• 0 – MISSING: missing or invalid pixels;
• 1 – CLEAR: pixels without clouds or cloud shadows;
• 2 – CLOUD SHADOW: pixels with cloud shadows;
• 3 – SEMI TRANSPARENT CLOUD: pixels with thin clouds through which the land is visible; include cirrus clouds that are on the high cloud level (5-15km).
• 4 – CLOUD: pixels with cloud; include stratus and cumulus clouds that are on the low cloud level (from 0-0.2km to 2km).
• 5 – UNDEFINED: pixels that the labeler is not sure which class they belong to.

The dataset was labelled using Computer Vision Annotation Tool (CVAT) and Segments.ai. With the possibility of integrating active learning process in Segments.ai, the labelling was performed semi-automatically.

The dataset limitations must be considered: the data is covering only terrestrial region and does not include water areas; the dataset is not presented in winter conditions; the dataset represent summer conditions, therefore September and October contain only test products used for validation. Current subscenes do not have georeferencing, however, we are working towards including them in next version.

More details about the dataset structure can be found in README.

Contributions and Acknowledgements

The data were annotated by Fariha Harun and Olga Wold. The data verification and Software Development was performed by Indrek Sünter, Heido Trofimov, Anton Kostiukhin, Marharyta Domnich, Mihkel Järveoja, Olga Wold. Methodology was developed by Kaupo Voormansik, Indrek Sünter, Marharyta Domnich.
We would like to thank Segments.ai annotation tool for instant and an individual customer support. We are grateful to European Space Agency for reviews and suggestions. We would like to extend our thanks to Prof. Gholamreza Anbarjafari for the feedback and directions.
The project was funded by European Space Agency, Contract No. 4000132124/20/I-DT.

This file contains the annotations for the ConfLab dataset, including actions (speaking status), pose, and F-formations.

------------------

./actions/speaking_status:

./processed: the processed speaking status files, aggregated into a single data frame per segment. Skipped rows in the raw data (see https://josedvq.github.io/covfee/docs/output for details) have been imputed using the code at: https://github.com/TUDelft-SPC-Lab/conflab/tree/master/preprocessing/speaking_status

The processed annotations consist of:

./speaking: The first row contains person IDs matching the sensor IDs,

The rest of the row contains binary speaking status annotations at 60fps for the corresponding 2 min video segment (7200 frames).

./confidence: Same as above. These annotations reflect the continuous-valued rating of confidence of the annotators in their speaking annotation.

To load these files with pandas: pd.read_csv(p, index_col=False)

./raw.zip: the raw outputs from speaking status annotation for each of the eight annotated 2-min video segments. These were were output by the covfee annotation tool (https://github.com/josedvq/covfee)

Annotations were done at 60 fps.

--------------------

./pose:

./coco: the processed pose files in coco JSON format, aggregated into a single data frame per video segment. These files have been generated from the raw files using the code at: https://github.com/TUDelft-SPC-Lab/conflab-keypoints

To load in Python: f = json.load(open('/path/to/cam2_vid3_seg1_coco.json'))

The skeleton structure (limbs) is contained within each file in:

f['categories'][0]['skeleton']

and keypoint names at:

f['categories'][0]['keypoints']

./raw.zip: the raw outputs from continuous pose annotation. These were were output by the covfee annotation tool (https://github.com/josedvq/covfee)

Annotations were done at 60 fps.

---------------------

./f_formations:

seg 2: 14:00 onwards, for videos of the form x2xxx.MP4 in /video/raw/ for the relevant cameras (2,4,6,8,10).

seg 3: for videos of the form x3xxx.MP4 in /video/raw/ for the relevant cameras (2,4,6,8,10).

Note that camera 10 doesn't include meaningful subject information/body parts that are not already covered in camera 8.

First column: time stamp

Second column: "()" delineates groups, "<>" delineates subjects, cam X indicates the best camera view for which a particular group exists.

phone.csv: time stamp (pertaining to seg3), corresponding group, ID of person using the phone

Historical treebanks tend to be manually annotated, which is not surprising, since state-of-the-art parsers are not accurate enough to ensure high-quality annotation for historical texts. We show that automatic parsing can be an efficient pre-annotation tool for Old East Slavic texts.

The COCO-WAN benchmark is designed to assess the impact of weakly annotations (combined with auto-annotation tools) noise on instance segmentation models. This benchmark is built upon the COCO dataset and incorporates noise generated through weak annotations, simulating real-world scenarios where annotations might be imperfect due to semi-automated tools. It includes various levels of noise to challenge the robustness and generalization capabilities of segmentation models.

Accurately labeling instance segmentation datasets is a complex and error-prone task, often leading to noisy labels. The COCO-WAN benchmark aims to provide a realistic testing ground for models to handle such noisy annotations. By utilizing foundation models and weak annotations, COCO-WAN simulates semi-automated annotation tools, helping researchers understand how well their models can perform under less-than-ideal labeling conditions. This benchmark includes multiple noise levels (easy, medium, and hard) to reflect varying degrees of annotation imperfections.

Potential Use Cases of the Dataset:

Model Robustness Testing: Researchers can use COCO-WAN to evaluate how different instance segmentation models cope with noisy annotations, allowing for the development of more resilient algorithms. Annotation Tool Improvement: By analyzing model performance on COCO-WAN, developers of annotation tools can identify common pitfalls and work on reducing noise in their outputs.

Semi-Automated Annotation Systems: The benchmark provides insights into how models trained with semi-automated annotations perform, guiding improvements in such systems for better accuracy and efficiency in labeling tasks. The COCO-WAN benchmark offers a crucial resource for advancing the field of instance segmentation by highlighting the challenges posed by noisy labels and fostering the creation of more robust and reliable models.

Model Robustness Testing: Researchers can use COCO-WAN to evaluate how different instance segmentation models cope with spatial, real noisy annotations, allowing for the development of more resilient algorithms.

Semi-Automated Annotation Systems: The benchmark provides insights into how models trained with semi-automated annotations perform, guiding improvements in such systems for better accuracy and efficiency in labeling tasks.

The COCO-WAN benchmark offers a crucial resource for advancing the field of instance segmentation by highlighting the challenges posed by noisy labels and fostering the creation of more robust and reliable models.

We provide annotated fish imagery data for use in deep learning models (e.g., convolutional neural networks) for individual and species recognition. For individual recognition models, the dataset consists of annotated .json files of individual brook trout imagery collected at the Eastern Ecological Science Center's Experimental Stream Laboratory. For species recognition models, the dataset consists of annotated .json files for 7 freshwater fish species: lake trout, largemouth bass, smallmouth bass, brook trout, rainbow trout, walleye, and northern pike. Species imagery was compiled from Anglers Atlas and modified to remove human faces for privacy protection. We used open-source VGG image annotation software developed by Oxford University: https://www.robots.ox.ac.uk/~vgg/software/via/via-1.0.6.html.