When students learn linear regression, they must learn to use diagnostics to check and improve their models. Model-building is an expert skill requiring the interpretation of diagnostic plots, an understanding of model assumptions, the selection of appropriate changes to remedy problems, and an intuition for how potential problems may affect results. Simulation offers opportunities to practice these skills, and is already widely used to teach important concepts in sampling, probability, and statistical inference. Visual inference, which uses simulation, has also recently been applied to regression instruction. This article presents the regressinator, an R package designed to facilitate simulation and visual inference in regression settings. Simulated regression problems can be easily defined with minimal programming, using the same modeling and plotting code students may already learn. The simulated data can then be used for model diagnostics, visual inference, and other activities, with the package providing functions to facilitate common tasks with a minimum of programming. Example activities covering model diagnostics, statistical power, and model selection are shown for both advanced undergraduate and Ph.D.-level regression courses.

This dataset presents a dual-version representation of employment-related data from India, crafted to highlight the importance of data cleaning and transformation in any real-world data science or analytics project.

🔹 Dataset Composition:

It includes two parallel datasets: 1. Messy Dataset (Raw) – Represents a typical unprocessed dataset often encountered in data collection from surveys, databases, or manual entries. 2. Cleaned Dataset – This version demonstrates how proper data preprocessing can significantly enhance the quality and usability of data for analytical and visualization purposes.

Each record captures multiple attributes related to individuals in the Indian job market, including: - Age Group
- Employment Status (Employed/Unemployed)
- Monthly Salary (INR)
- Education Level
- Industry Sector
- Years of Experience
- Location
- Perceived AI Risk
- Date of Data Recording

Transformations & Cleaning Applied:

The raw dataset underwent comprehensive transformations to convert it into its clean, analysis-ready form: - Missing Values: Identified and handled using either row elimination (where critical data was missing) or imputation techniques. - Duplicate Records: Identified using row comparison and removed to prevent analytical skew. - Inconsistent Formatting: Unified inconsistent naming in columns (like 'monthly_salary_(inr)' → 'Monthly Salary (INR)'), capitalization, and string spacing. - Incorrect Data Types: Converted columns like salary from string/object to float for numerical analysis. - Outliers: Detected and handled based on domain logic and distribution analysis. - Categorization: Converted numeric ages into grouped age categories for comparative analysis. - Standardization: Uniform labels for employment status, industry names, education, and AI risk levels were applied for visualization clarity.

Purpose & Utility:

This dataset is ideal for learners and professionals who want to understand: - The impact of messy data on visualization and insights - How transformation steps can dramatically improve data interpretation - Practical examples of preprocessing techniques before feeding into ML models or BI tools

It's also useful for: - Training ML models with clean inputs
- Data storytelling with visual clarity
- Demonstrating reproducibility in data cleaning pipelines

By examining both the messy and clean datasets, users gain a deeper appreciation for why “garbage in, garbage out” rings true in the world of data science.

Context

There's a story behind every dataset and here's your opportunity to share yours.

Content

This dataset aims to state the simple visual interaction between objects. In image format 100x100 px, 25 frames.

An object moving in the frame.

Music21 is an untrimmed video dataset crawled by keyword query from Youtube. It contains music performances belonging to 21 categories. This dataset is relatively clean and collected for the purpose of training and evaluating visual sound source separation models.

DESCRIPTION
For this task, we use a subset of the MIRFLICKR (http://mirflickr.liacs.nl) collection. The entire collection contains 1 million images from the social photo sharing website Flickr and was formed by downloading up to a thousand photos per day that were deemed to be the most interesting according to Flickr. All photos in this collection were released by their users under a Creative Commons license, allowing them to be freely used for research purposes. Of the entire collection, 25 thousand images were manually annotated with a limited number of concepts and many of these annotations have been further refined and expanded over the lifetime of the ImageCLEF photo annotation task. This year we used crowd sourcing to annotate all of these 25 thousand images with the concepts.

On this page we provide you with more information about the textual features, visual features and concept features we supply with each image in the collection we use for this year's task.

TEXTUAL FEATURES
All images are accompanied by the following textual features:

- Flickr user tags
These are the tags that the users assigned to the photos their uploaded to Flickr. The 'raw' tags are the original tags, while the 'clean' tags are those collapsed to lowercase and condensed to removed spaces.

- EXIF metadata
If available, the EXIF metadata contains information about the camera that took the photo and the parameters used. The 'raw' exif is the original camera data, while the 'clean' exif reduces the verbosity.

- User information and Creative Commons license information
This contains information about the user that took the photo and the license associated with it.

VISUAL FEATURES
Over the previous years of the photo annotation task we noticed that often the same types of visual features are used by the participants, in particular features based on interest points and bag-of-words are popular. To assist you we have extracted several features for you that you may want to use, so you can focus on the concept detection instead. We additionally give you some pointers to easy to use toolkits that will help you extract other features or the same features but with different default settings.

- SIFT, C-SIFT, RGB-SIFT, OPPONENT-SIFT
We used the ISIS Color Descriptors (http://www.colordescriptors.com) toolkit to extract these descriptors. This package provides you with many different types of features based on interest points, mostly using SIFT. It furthermore assists you with building codebooks for bag-of-words. The toolkit is available for Windows, Linux and Mac OS X.

- SURF
We used the OpenSURF (http://www.chrisevansdev.com/computer-vision-opensurf.html) toolkit to extract this descriptor. The open source code is available in C++, C#, Java and many more languages.

- TOP-SURF
We used the TOP-SURF (http://press.liacs.nl/researchdownloads/topsurf) toolkit to extract this descriptor, which represents images with SURF-based bag-of-words. The website provides codebooks of several different sizes that were created using a combination of images from the MIR-FLICKR collection and from the internet. The toolkit also offers the ability to create custom codebooks from your own image collection. The code is open source, written in C++ and available for Windows, Linux and Mac OS X.

- GIST
We used the LabelMe (http://labelme.csail.mit.edu) toolkit to extract this descriptor. The MATLAB-based library offers a comprehensive set of tools for annotating images.

For the interest point-based features above we used a Fast Hessian-based technique to detect the interest points in each image. This detector is built into the OpenSURF library. In comparison with the Hessian-Laplace technique built into the ColorDescriptors toolkit it detects fewer points, resulting in a considerably reduced memory footprint. We therefore also provide you with the interest point locations in each image that the Fast Hessian-based technique detected, so when you would like to recalculate some features you can use them as a starting point for the extraction. The ColorDescriptors toolkit for instance accepts these locations as a separate parameter. Please go to http://www.imageclef.org/2012/photo-flickr/descriptors for more information on the file format of the visual features and how you can extract them yourself if you want to change the default settings.

CONCEPT FEATURES
We have solicited the help of workers on the Amazon Mechanical Turk platform to perform the concept annotation for us. To ensure a high standard of annotation we used the CrowdFlower platform that acts as a quality control layer by removing the judgments of workers that fail to annotate properly. We reused several concepts of last year's task and for most of these we annotated the remaining photos of the MIRFLICKR-25K collection that had not yet been used before in the previous task; for some concepts we reannotated all 25,000 images to boost their quality. For the new concepts we naturally had to annotate all of the images.

- Concepts
For each concept we indicate in which images it is present. The 'raw' concepts contain the judgments of all annotators for each image, where a '1' means an annotator indicated the concept was present whereas a '0' means the concept was not present, while the 'clean' concepts only contain the images for which the majority of annotators indicated the concept was present. Some images in the raw data for which we reused last year's annotations only have one judgment for a concept, whereas the other images have between three and five judgments; the single judgment does not mean only one annotator looked at it, as it is the result of a majority vote amongst last year's annotators.

- Annotations
For each image we indicate which concepts are present, so this is the reverse version of the data above. The 'raw' annotations contain the average agreement of the annotators on the presence of each concept, while the 'clean' annotations only include those for which there was a majority agreement amongst the annotators.

You will notice that the annotations are not perfect. Especially when the concepts are more subjective or abstract, the annotators tend to disagree more with each other. The raw versions of the concept annotations should help you get an understanding of the exact judgments given by the annotators.

ShapeNetCore is a subset of the full ShapeNet dataset with single clean 3D models and manually verified category and alignment annotations. It covers 55 common object categories with about 51,300 unique 3D models. The 12 object categories of PASCAL 3D+, a popular computer vision 3D benchmark dataset, are all covered by ShapeNetCore.

Globally, the growth of plastic production has increased exponentially from 1.5 million metric tons (Mt) in 1950 to 400.3 Mt in 2022, resulting in a substantial increase of marine litter along the coastal region. Presently, there is a growing interest in using an artificial intelligence (AI) based automatic and cost-effective approach to identify marine litter for clean-up processes. This study aims to understand the spatial distribution of marine litter along the central east coast of India using the conventional method and AI based object detection approach. From the field survey, a total of 4588 marine litter items could be identified, with an average of 1.147 ± 0.375 items/m2. Based on clean coast index, 37.5% of beaches were categorized as ‘dirty’ and 62.5% of beaches as ‘extremely dirty’. For the machine learning approach ‘You Only Look Once (YOLOv5)’ model was used to detect and classify various types of marine litter items. A total of 9714 images representing seven categories of marine litter (plastic, metal, glass, fabric, paper, processed wood, and rubber) were extracted from eight field videos recorded across diverse beach settings. The efficiency of the trained machine learning model was assessed using different metrices such as Recall, Precision, Mean average precision (mAP) and F1 score (a metric for forecast accuracy). The model achieved a F1 score of 0.797, mAP 0.5 of 0.95, and mAP@0.5-0.95 of 0.76, and these results show that YOLOv5 model could be used in conjunction with conventional marine litter monitoring, classification and detection to provide quick and accurate results.

Due to the significant resemblance in visual appearance, pill misuse is prevalent and has become a critical issue, responsible for one-third of all deaths worldwide. Pill identification, thus, is a crucial concern that needs to be investigated thoroughly. Recently, several attempts have been made to exploit deep learning to tackle the pill identification problem. However, most published works consider only single-pill identification and fail to distinguish hard samples with identical appearances. Also, most existing pill image datasets only feature single pill images captured in carefully controlled environments under ideal lighting conditions and clean backgrounds. In this work, we are the first to tackle the multi-pill detection problem in real-world settings, aiming at localizing and identifying pills captured by users during pill intake. Moreover, we also introduce a multi-pill image dataset taken in unconstrained conditions. To handle hard samples, we propose a novel method for constructing heterogeneous a priori graphs incorporating three forms of inter-pill relationships, including co-occurrence likelihood, relative size, and visual semantic correlation. We then offer a framework for integrating a priori with pills’ visual features to enhance detection accuracy. Our experimental results have proved the robustness, reliability, and explainability of the proposed framework. Experimentally, it outperforms all detection benchmarks in terms of all evaluation metrics. Specifically, our proposed framework improves COCO mAP metrics by 9.4% over Faster R-CNN and 12.0% compared to vanilla YOLOv5. Our study opens up new opportunities for protecting patients from medication errors using an AI-based pill identification solution.

Subscribers can find out export and import data of 23 countries by HS code or product’s name. This demo is helpful for market analysis.

Subscribers can find out export and import data of 23 countries by HS code or product’s name. This demo is helpful for market analysis.

An overview of existing public datasets for the task of image-based pill detection.

Description

vidore/vdsid_french dataset that we processed for a visual question answering task where answer is a caption.

  Citation

@misc{faysse2024colpaliefficientdocumentretrieval, title={ColPali: Efficient Document Retrieval with Vision Language Models}, author={Manuel Faysse and Hugues Sibille and Tony Wu and Bilel Omrani and Gautier Viaud and Céline Hudelot and Pierre Colombo}, year={2024}, eprint={2407.01449}, archivePrefix={arXiv}… See the full description on the dataset page: https://huggingface.co/datasets/CATIE-AQ/caption-vidore-vdsid_french-clean.

Context

This can be a great Computer Vision or Multi-Class Deep Learning Project.

Content

The Folder names are self-explanator, they contain the names of the Pieces and the Images are singular images so that the dataset tidy. Multiple object images have been deleted making it an easy dataset to work with.

Inspiration

I am an amateur chess player and a Chess fan, plus I did not come across any great datasets like this on the Internet and so decided to make this one.

Description

manu/tabfquad_retrieving dataset that we processed for a visual question answering task where answer is a caption.

  Citation

https://huggingface.co/datasets/manu/tabfquad_retrieving

Preprocessed Stage 2 CSV's so they work easier with Image Data Generators.

Original data came from this Kaggle competition:

https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/data

Description

vidore/tabfquad_test_subsampled dataset that we processed for a visual question answering task where answer is a caption.

  Citation

@misc{faysse2024colpaliefficientdocumentretrieval, title={ColPali: Efficient Document Retrieval with Vision Language Models}, author={Manuel Faysse and Hugues Sibille and Tony Wu and Bilel Omrani and Gautier Viaud and Céline Hudelot and Pierre Colombo}, year={2024}, eprint={2407.01449}… See the full description on the dataset page: https://huggingface.co/datasets/CATIE-AQ/caption-vidore-tabfquad_test_subsampled-clean.

Description

This dataset is a processed version of liboaccn/OPUS-MIT-5M to make it easier to use, particularly for a visual question answering task where answer is an OCR transcription.Specifically, the original dataset has been processed to provide the image directly as a PIL rather than a path in an image column.We've also created a question column containing around 40 prompts based on via tutoiement, vouvoiement and imperative forms. Note that this dataset contains only the… See the full description on the dataset page: https://huggingface.co/datasets/lbourdois/OCR-liboaccn-OPUS-MIT-5M-clean.

Description

French part of the neulab/PangeaInstruct dataset (OCR data only) that we processed for a visual question answering task where answer is a caption.

  Citation

@article{yue2024pangeafullyopenmultilingual, title={Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages}, author={Xiang Yue and Yueqi Song and Akari Asai and Seungone Kim and Jean de Dieu Nyandwi and Simran Khanuja and Anjali Kantharuban and Lintang Sutawika and Sathyanarayanan Ramamoorthy… See the full description on the dataset page: https://huggingface.co/datasets/lbourdois/OCR-neulab-PangeaInstruct-OCR-clean.

Description

French part of the floschne/xm3600 dataset that we processed for a visual question answering task where answer is a caption.

  Citation

@inproceedings{ThapliyalCrossmodal2022, author = {Ashish Thapliyal and Jordi Pont-Tuset and Xi Chen and Radu Soricut}, title = {{Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset}}, booktitle = {EMNLP}, year = {2022} }