jondurbin/truthy-dpo-v0.1 in ChatML format, ready to use in HuggingFace TRL's DPO Trainer. Python code used for conversion: from datasets import load_dataset

dataset = load_dataset("jondurbin/truthy-dpo-v0.1", split="train")

def format(columns): prompt = f"<|im_start|>user {columns['prompt']}<|im_end|> <|im_start|>assistant "

if (columns['system']):
  prompt = f"<|im_start|>system

{columns['system']}<|im_end|> {prompt}"

return {
  "prompt": prompt… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-truthy-dpo-v0.1.

How to load the dataset (Usage code): from datasets import load_from_disk

saved_dataset_path = "path/to/dataset/folder" loaded_preference_dataset = load_from_disk(saved_dataset_path) print(f" Successfully loaded dataset from {saved_dataset_path}:") print(loaded_preference_dataset) # Now 'loaded_preference_dataset' can be used directly with DPOTrainer # Example: # dpo_trainer = DPOTrainer(..., train_dataset=loaded_preference_dataset, ...) except Exception as e:… See the full description on the dataset page: https://huggingface.co/datasets/Genesis-AI-Labs/GAIL-gsm8k-preference-small.

NeuralNovel/Neural-DPO in ChatML format, ready to use in HuggingFace TRL's DPO Trainer. Python code used for conversion: from datasets import load_dataset

dataset = load_dataset("NeuralNovel/Neural-DPO", split="train")

def format(columns): prompt = f"<|im_start|>user {columns['question']}<|im_end|> <|im_start|>assistant "

if (columns['system']):
  prompt = f"<|im_start|>system

{columns['system']}<|im_end|> {prompt}"

return {
  "prompt": prompt… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-Neural-DPO.

Justin Law's Q&A DPO Dataset

This is a very small dataset meant to provide guidance for further DPO data generation or for direct fine-tuning using Hugging Face's DPO trainer. This initial dataset was generated using OpenAI's GPT4o, along with Justin Law's publicly available resume and cover letter.