Chatbot Arena Conversations Dataset

This dataset contains 33K cleaned conversations with pairwise human preferences. It is collected from 13K unique IP addresses on the Chatbot Arena from April to June 2023. Each sample includes a question ID, two model names, their full conversation text in OpenAI API JSON format, the user vote, the anonymized user ID, the detected language tag, the OpenAI moderation API tag, the additional toxic tag, and the timestamp. To ensure the safe release… See the full description on the dataset page: https://huggingface.co/datasets/lmsys/chatbot_arena_conversations.

LLM-jp Chatbot Arena Conversations Dataset

This dataset contains approximately 1,000 conversations with pairwise human preferences, most of which are in Japanese. The data was collected during the trial phase of the LLM-jp Chatbot Arena (January–February 2025), where users compared responses from two different models in a head-to-head format. Each sample includes a question ID, the names of the two models, their conversation transcripts, the user's vote, an anonymized user ID, a… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/llm-jp-chatbot-arena-conversations.

Dataset Card for "chatbot-arena-ja-calm2-7b-chat"

  Chatbot Arena Conversations JA (calm2) Dataset

Chatbot Arena Conversations JA (calm2)はこちらの論文で構築されたRLHFのための日本語Instructionデータセットです。 「英語で公開されているデータセットをオープンソースのツール・モデルのみを使って日本語用に転用し、日本語LLMの学習に役立てることができるか」を検証する目的で作成しております。 指示文（prompt）はlmsys/chatbot_arena_conversationsのユーザ入力（CC-BY 4.0）を和訳したものです。これはChatbot Arenaを通して人間が作成した指示文であり、CC-BY 4.0で公開されているものです。複数ターンの対話の場合は最初のユーザ入力のみを使っています（そのため、このデータセットはすべて１ターンの対話のみになっております）。… See the full description on the dataset page: https://huggingface.co/datasets/cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental.

lmsys-arena-human-preference-winner-43k-unfiltered

This repository contains a dataset derived from the lmsys/lmsys-arena-human-preference-55k dataset, which is licensed under the Apache 2.0 License.

  Dataset Description

The lmsys-arena-human-preference-winner-43k-unfiltered dataset is a collection of 43,000 samples, each containing an instruction (prompt) and an output (winning response) from real-world user and LLM conversations. The dataset is derived from the original… See the full description on the dataset page: https://huggingface.co/datasets/lesserfield/lmsys-arena-human-preference-winner-43k-unfiltered.

LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset

This dataset contains one million real-world conversations with 25 state-of-the-art LLMs. It is collected from 210K unique IP addresses in the wild on the Vicuna demo and Chatbot Arena website from April to August 2023. Each sample includes a conversation ID, model name, conversation text in OpenAI API JSON format, detected language tag, and OpenAI moderation API tag. User consent is obtained through the "Terms of use"… See the full description on the dataset page: https://huggingface.co/datasets/jc-detoxio/lmsys-chat-1m.

Checkout our blog post

Building an affordable and reliable benchmark for LLM chatbots has become a critical challenge. A high-quality benchmark should 1. robustly separate model capability 2. reflect human preference in real-world use cases 3. frequently update to avoid over-fitting or test set leakage

Traditional benchmarks are often static or close-ended (e.g., MMLU multi-choice QA), which do not satisfy the above requirements. On the other hand, models are evolving faster than ever, underscoring the need to build benchmarks with high separability.

We introduce Arena-Hard – a data pipeline to build high-quality benchmarks from live data in Chatbot Arena, which is a crowd-sourced platform for LLM evals.

We compare our new benchmark, Arena Hard v0.1, to a current leading chat LLM benchmark, MT Bench. We show Arena Hard v0.1 offers significantly stronger separability against MT Bench with tighter confidence intervals. It also has a higher agreement (89.1%, see blog post) with the human preference ranking by Chatbot Arena (english-only). We expect to see this benchmark useful for model developers to differentiate their model checkpoints.

A complete list of live websites using the Arena Liveblog And Chat Tool technology, compiled through global website indexing conducted by WebTechSurvey.

VisionArena-Battle: 30K Real-World Image Conversations with Pairwise Preference Votes

200k single and multi-turn chats between users and VLM's collected on Chatbot Arena. WARNING: Images may contain inappropriate content.

  Dataset Details

200K conversations 45 VLM's 138 languages ~43k unique images Question Category Tags (Captioning, OCR, Entity Recognition, Coding, Homework, Diagram, Humor, Creative Writing, Refusal)

  Dataset Description

200,000… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/VisionArena-Chat.

Dataset for Kaggle competition on predicting human preference on Chatbot Arena battles. The training dataset includes over 55,000 real-world user and LLM conversations and user preferences across over 70 state-of-the-art LLMs, such as GPT-4, Claude 2, Llama 2, Gemini, and Mistral models. Each sample represents a battle consisting of 2 LLMs which answer the same question, with a user label of either prefer model A, prefer model B, tie, or tie (both bad).

  Citation

Please cite the… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/arena-human-preference-55k.

Overview

This dataset contains ALL in-the-wild conversation crowdsourced from Search Arena between March 18, 2025 and May 8, 2025. It includes 24,069 multi-turn conversations with search-LLMs across diverse intents, languages, and topics—alongside 12,652 human preference votes. The dataset spans approximately 11,000 users across 136 countries, 13 publicly released models, around 90 languages (including 11% multilingual prompts), and over 5,000 multi-turn sessions. While user… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/search-arena-24k.

⚔️ Arena Preferences

This is a preference dataset based on lmsys/chatbot_arena_conversations. It contains multi-turn conversations (up to 11 turns) and original samples in 39 different languages (no translation).

Chosen answers are answers where GPT-4 was the winner (33k => 2,868 samples) Duplicates were removed (13 samples) GPTisms were removed (166 samples)

  📊 Plots

Here's breakdown of the four most represented languages + an "other" bin in the dataset.

Here's… See the full description on the dataset page: https://huggingface.co/datasets/mlabonne/arena-preferences.

llm-jp/llm-jp-chatbot-arena-conversationsを整形したデータセットです

Overview

This dataset contains 7k leaderboard conversation votes collected from Search Arena between March 18, 2025 and April 13, 2025. All entries have been redacted for PII and sensitive user information to ensure privacy. Each data point includes:

Two model responses (messages_a and messages_b) The human vote result A timestamp Full system metadata, LLM + web search trace, and post-processed metadata for controlled experiments (conv_meta)

To reproduce the leaderboard results… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/search-arena-v1-7k.

a-namizuka/chatbot-arena-ja-calm2-7b-chat-experimental-autotrain-dpo-formatted-10 dataset hosted on Hugging Face and contributed by the HF Datasets community

Introduction

We release the annotated data used in Dissecting Human and LLM Preferences. Original Dataset - The dataset is based on lmsys/chatbot_arena_conversations, which contains 33K cleaned conversations with pairwise human preferences collected from 13K unique IP addresses on the Chatbot Arena from April to June 2023. Filtering and Scenario-wise Sampling - We filter out the conversations that are not in English, with "Tie" or "Both Bad" labels, and the multi-turn… See the full description on the dataset page: https://huggingface.co/datasets/Preference-Dissection/preference-dissection.

Chatbot Arena user conversations between Llama-3-70b VS GPT-4-1025 or Llama-3-70b VS Claude-3-Opus with user preference votes. Single turn. Excludes ties. Used in Llama Data Analysis blog post and "VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models" (Paper, Code).

  Citation

@article{dunlap_vibecheck, title={VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models}, author={Lisa Dunlap and Krishna Mandal and Trevor… See the full description on the dataset page: https://huggingface.co/datasets/lmarena-ai/Llama-3-70b-battles.

chatbot-arena-ja-calm2-7b-chatをフィルタリングし、karakuri-lm-8x7b-chat-v0.1-awqでchosenを生成しました

WildVision-Chat

WildVisvion-Chat is the publicly released chat data collected from WildVision-Arena. We hope our released data can promote the development of a multimodal language model.

  Models

the WildVision datasets contain user conversations with PaliGemma, GPT-4T, GPT-4o, Phi 3 vision, Gemini 1.5, Neva 22b, Claude 3 Haiku, Idefics2-8b, Qwen-VL plus, Claude 3.5 Sonnet, Qwen-VL max, Yi-VL plus, MiniCPM LLama3, Claude 3 Sonnet, Claude 3 Opus, and GPT-4 Vision preview.… See the full description on the dataset page: https://huggingface.co/datasets/WildVision/wildvision-chat.

AI_Ecosystem_Exploitation

  Dataset Details





  Dataset Description

The dataset used in the "The Attribution Crisis in LLM Search Results" paper.

  Links

Repository: Ecosystem_Exploitation_In_Search_Results Paper: Link

  Uses

This dataset contains 7000 conversations sourced from the lmsys search arena, it has been post processed to distinguish between search results and in-text citations. This dataset is usefull to observe the patterns of LLM… See the full description on the dataset page: https://huggingface.co/datasets/Disclosures-SSRC/The_Attribution_Crisis_in_LLM_Search_Results.

WildVision-Battle

WildVisvion-Battle is the publicly released battle data collected from WildVision-Arena, where user's preferences are collected across multiple models and wild tasks from real-world users.

  Models

the WildVision datasets contain user conversations with PaliGemma, GPT-4T, GPT-4o, Phi 3 vision, Gemini 1.5, Neva 22b, Claude 3 Haiku, Idefics2-8b, Qwen-VL plus, Claude 3.5 Sonnet, Qwen-VL max, Yi-VL plus, MiniCPM LLama3, Claude 3 Sonnet, Claude 3 Opus, and… See the full description on the dataset page: https://huggingface.co/datasets/WildVision/wildvision-battle.