llm data_error ai_generated true

openai.BadRequestError: 训练文件无效：第42行的'messages'必须是对象列表，但得到的是<class 'str'>

openai.BadRequestError: Invalid training file: 'messages' must be a list of objects, but got <class 'str'> for line 42

ID: llm/fine-tuning-data-format-invalid

其他格式: JSON · Markdown 中文 · English
88%修复率
90%置信度
1证据数
2024-06-01首次发现

版本兼容性

版本状态引入弃用备注
openai==1.30.0 active — — —
python==3.11 active — — —

根因分析

用于微调的JSONL训练文件中存在格式错误的行，其中'messages'字段是字符串而不是消息对象列表，通常是由于缺少逗号或JSON序列化不正确所致。

English

A JSONL training file for fine-tuning contains a malformed line where the 'messages' field is a string instead of a list of message objects, often due to a missing comma or incorrect JSON serialization.

generic

官方文档

https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset

解决方案

  1. Validate the training file locally before uploading: `import json; with open('training.jsonl') as f: for i, line in enumerate(f, 1): try: obj = json.loads(line); assert isinstance(obj['messages'], list), f'Line {i}: messages is not a list'; for msg in obj['messages']: assert 'role' in msg and 'content' in msg, f'Line {i}: message missing role or content' except (json.JSONDecodeError, AssertionError) as e: print(f'Error on line {i}: {e}')`
  2. Use the OpenAI CLI tool to validate the file: `openai api fine_tunes.create -t training.jsonl -v` which provides detailed error messages for each malformed line.

无效尝试

常见但无效的做法:

  1. 75% 失败

    The error may be caused by a systemic issue (e.g., incorrect serialization logic) that affects multiple lines; fixing only line 42 won't prevent the same error on other lines.

  2. 90% 失败

    Standard JSON validators will pass a string value for 'messages' as valid JSON; the error is a semantic validation that the OpenAI API performs.

  3. 95% 失败

    If the code that writes JSONL is buggy, regenerating will produce the same error on the same or different lines.