huggingface data_error ai_generated true

键错误：数据集未找到分割 'validation'。可用分割：['train', 'test']

KeyError: "Split 'validation' not found in dataset. Available splits: ['train', 'test']"

ID: huggingface/dataset-split-key-error

其他格式: JSON · Markdown 中文 · English
95%修复率
90%置信度
1证据数
2023-04-05首次发现

版本兼容性

版本状态引入弃用备注
datasets>=2.14.0 active — — —
transformers>=4.30.0 active — — —
Python>=3.8 active — — —

根因分析

请求的数据集分割名称（如 'validation'）在数据集配置中不存在；数据集可能使用不同的命名约定，如 'val' 或 'test'。

English

The requested dataset split name (e.g., 'validation') does not exist in the dataset configuration; the dataset may use a different naming convention like 'val' or 'test'.

generic

官方文档

https://huggingface.co/docs/datasets/en/loading#splits

解决方案

  1. Check available splits before loading and use the correct name:
from datasets import get_dataset_split_names
splits = get_dataset_split_names('org/dataset')
print(splits)  # e.g., ['train', 'test']
# Then use the correct split:
dataset = load_dataset('org/dataset', split='test')
  2. If no validation split exists, create one from the training data:
from datasets import load_dataset
dataset = load_dataset('org/dataset', split='train')
train_val = dataset.train_test_split(test_size=0.1, seed=42)
train_dataset = train_val['train']
val_dataset = train_val['test']

无效尝试

常见但无效的做法:

  1. 60% 失败

    Using split='all' to load all data and then manually splitting may cause data leakage if the dataset has predefined splits.

  2. 90% 失败

    Renaming the split after loading with rename_column does not create a new split; it only renames a column.