Dataset Card for GLUE

  Dataset Summary

GLUE, the General Language Understanding Evaluation benchmark (https://gluebenchmark.com/) is a collection of resources for training, evaluating, and analyzing natural language understanding systems.

  Supported Tasks and Leaderboards

The leaderboard for the GLUE benchmark can be found at this address. It comprises the following tasks:

  ax

A manually-curated evaluation dataset for fine-grained analysis of system… See the full description on the dataset page: https://huggingface.co/datasets/nyu-mll/glue.

Dataset

This dataset was created by Learn and Progress

Contents

It contains the following files:

SuperGLUE (https://super.gluebenchmark.com/) is a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a new public leaderboard.

To use this dataset:

import tensorflow_datasets as tfds

ds = tfds.load('super_glue', split='train')
for ex in ds.take(4):
 print(ex)

See the guide for more informations on tensorflow_datasets.

SuperGLUE (https://super.gluebenchmark.com/) is a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a new public leaderboard.

GLUE, the General Language Understanding Evaluation benchmark (https://gluebenchmark.com/) is a collection of resources for training, evaluating, and analyzing natural language understanding systems.

The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems. GLUE consists of: A benchmark of nine sentence- or sentence-pair language understanding tasks built on established existing datasets and selected to cover a diverse range of dataset sizes, text genres, and degrees of difficulty, A diagnostic dataset designed to evaluate and analyze model performance with respect to a wide range of linguistic phenomena found in natural language, and A public leaderboard for tracking performance on the benchmark and a dashboard for visualizing the performance of models on the diagnostic set. The format of the GLUE benchmark is model-agnostic, so any system capable of processing sentence and sentence pairs and producing corresponding predictions is eligible to participate. The benchmark tasks are selected so as to favor models that share information across tasks using parameter sharing or other transfer learning techniques. The ultimate goal of GLUE is to drive research in the development of general and robust natural language understanding systems.

The SloWIC dataset is a Slovenian dataset for the Word in Context task. Each example in the dataset contains a target word with multiple meanings and two sentences that both contain the target word. Each example is also annotated with a label that shows if both sentences use the same meaning of the target word. The dataset contains 1808 manually annotated sentence pairs and additional 13150 automatically annotated pairs to help with training larger models. The dataset is stored in the JSON format following the format used in the SuperGLUE version of the Word in Context task (https://super.gluebenchmark.com/).

Dataset Card for "super_glue"

  Dataset Summary

SuperGLUE (https://super.gluebenchmark.com/) is a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a new public leaderboard. BoolQ (Boolean Questions, Clark et al., 2019a) is a QA task where each example consists of a short passage and a yes/no question about the passage. The questions are provided anonymously and unsolicited by users of the Google search… See the full description on the dataset page: https://huggingface.co/datasets/zzzzhhh/test_data.

GLUE, the General Language Understanding Evaluation benchmark (https://gluebenchmark.com/) is a collection of resources for training, evaluating, and analyzing natural language understanding systems.

CoLA Text Classification RCL Dataset

  Overview

This dataset contains textual data structured explicitly for standard text classification tasks using Lumina AI's Random Contrast Learning (RCL) algorithm via the PrismRCL application. Unlike LLM datasets, standard text classification datasets contain individual .txt files organized by class.

  Dataset Structure

The dataset structure for text classification training: CoLA.Classification/ train/ [class_1]/… See the full description on the dataset page: https://huggingface.co/datasets/LuminaAI/CoLA_2_Class-GLUE-Benchmark.

Trained models from the paper:

Lukas Galke, Isabell Cuber, Christoph Meyer, Henrik Ferdinand Noelscher, Angelina Sonderecker, and Ansgar Scherp: General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings, in: International Joint Conference on Neural Networks (IJCNN), 2022.

• File seq2mat_hybrid_bidirectional_sbertlike-100p-bsz512 holds the model from pretraining
• File ws2020_transformer_final_models holds the fine-tuned models for each task of the GLUE benchmark

Model comparison for question pairs detection using 10-fold cross validation.

GLUE, the General Language Understanding Evaluation benchmark (https://gluebenchmark.com/) is a collection of resources for training, evaluating, and analyzing natural language understanding systems.

Dataset Card for Adversarial GLUE

  Dataset Summary

Adversarial GLUE Benchmark (AdvGLUE) is a comprehensive robustness evaluation benchmark that focuses on the adversarial robustness evaluation of language models. It covers five natural language understanding tasks from the famous GLUE tasks and is an adversarial version of GLUE benchmark. AdvGLUE considers textual adversarial attacks from different perspectives and hierarchies, including word-level transformations… See the full description on the dataset page: https://huggingface.co/datasets/AI-Secure/adv_glue.