python system_error ai_generated partial

worker 'gw3' crashed while running 'tests/test_heavy.py::test_parallel' [gw3] node down: Not properly terminated Replacing crashed worker gw3

ID: python/pytest-xdist-worker-crash-fork

Also available as: JSON · Markdown · 中文
80%Fix Rate
82%Confidence
0Evidence
2024-11-19First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.6.x active — — —
3.7.x active — — —

Root Cause

A test uses fork() or multiprocessing with the fork start method inside an xdist worker; forking a process that already has threads (xdist workers do) leads to deadlocks or SIGSEGV.

generic

中文

测试在 xdist worker 内使用了 fork() 或 fork 启动方式的 multiprocessing；fork 一个已经包含线程的进程（xdist worker 就是）会导致死锁或 SIGSEGV。

Workarounds

  1. 93% success 
    import multiprocessing as mp

if __name__ == '__main__':
    mp.set_start_method('spawn', force=True)

# In test fixture:
ctx = mp.get_context('spawn')
with ctx.Pool(2) as p:
    ...
  2. 88% success 
    # In conftest.py
def pytest_collection_modifyitems(items):
    for it in items:
        if 'heavy' in it.keywords:
            it.add_marker(pytest.mark.xdist_group('serial'))

# Run:
pytest -n auto --dist loadgroup

Dead Ends

Common approaches that don't work:

  1. 80% fail

    Masks the crash and slows the suite; the underlying fork-in-threaded-process bug still occurs on every run.

  2. 70% fail

    Defeats parallelism entirely; on large suites CI time can triple and the test that triggered the crash still runs unsafely.