python system_error ai_generated partial

worker 'gw3' 在运行 'tests/test_heavy.py::test_parallel' 时崩溃 [gw3] 节点宕机: 未正常终止 正在替换崩溃的 worker gw3

worker 'gw3' crashed while running 'tests/test_heavy.py::test_parallel' [gw3] node down: Not properly terminated Replacing crashed worker gw3

ID: python/pytest-xdist-worker-crash-fork

其他格式: JSON · Markdown 中文 · English
80%修复率
82%置信度
0证据数
2024-11-19首次发现

版本兼容性

版本状态引入弃用备注
3.6.x active — — —
3.7.x active — — —

根因分析

测试在 xdist worker 内使用了 fork() 或 fork 启动方式的 multiprocessing；fork 一个已经包含线程的进程（xdist worker 就是）会导致死锁或 SIGSEGV。

English

A test uses fork() or multiprocessing with the fork start method inside an xdist worker; forking a process that already has threads (xdist workers do) leads to deadlocks or SIGSEGV.

generic

解决方案

  1. 93% 成功率 
    import multiprocessing as mp

if __name__ == '__main__':
    mp.set_start_method('spawn', force=True)

# In test fixture:
ctx = mp.get_context('spawn')
with ctx.Pool(2) as p:
    ...
  2. 88% 成功率 
    # In conftest.py
def pytest_collection_modifyitems(items):
    for it in items:
        if 'heavy' in it.keywords:
            it.add_marker(pytest.mark.xdist_group('serial'))

# Run:
pytest -n auto --dist loadgroup

无效尝试

常见但无效的做法:

  1. 80% 失败

    Masks the crash and slows the suite; the underlying fork-in-threaded-process bug still occurs on every run.

  2. 70% 失败

    Defeats parallelism entirely; on large suites CI time can triple and the test that triggered the crash still runs unsafely.