data encoding_error ai_generated true

JSON解析器静默将BOM字符读入第一个键名

JSON parser silently reads BOM character as part of first key name

ID: data/json-utf8-bom-silent-read

其他格式: JSON · Markdown 中文 · English
95%修复率
88%置信度
1证据数
2023-09-20首次发现

版本兼容性

版本状态引入弃用备注
Python 3.11 active — — —
Python 3.12 active — — —
Node.js 20 active — — —
jq 1.7 active — — —

根因分析

Windows编辑器保存的UTF-8 BOM文件（字节顺序标记\ufeff）导致未去除BOM的JSON解析器将其包含在第一个键名中，造成键查找失败和静默数据损坏。

English

Files saved with UTF-8 BOM (Byte Order Mark, \ufeff) by Windows editors cause JSON parsers that do not strip BOM to include it in the first key name, leading to key lookup failures and silent data corruption.

generic

官方文档

https://docs.python.org/3/library/json.html

解决方案

  1. Open file with utf-8-sig encoding before parsing: `with open('data.json', encoding='utf-8-sig') as f: data = json.load(f)`
  2. Strip BOM from file content before parsing: `content = open('data.json', 'rb').read().lstrip(b'\xef\xbb\xbf'); data = json.loads(content)`

无效尝试

常见但无效的做法:

  1. 50% 失败

    Python's json.load does not accept encoding parameter; you must use open() with encoding='utf-8-sig' before json.load.

  2. 40% 失败

    This modifies the source file and may be forgotten in CI/CD pipelines; also requires root access in some environments.

  3. 90% 失败

    This breaks all key lookups in downstream code and makes the code non-portable across different editors.