python runtime_error ai_generated true

AssertionError: assert 0 == 1 # fails with pytest-randomly, passes with -p no:randomly

ID: python/pytest-test-ordering-random-seed

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
0Evidence
2025-12-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.15 active — — —

Root Cause

pytest-randomly shuffles test order each run; order-dependent tests or shared state cause intermittent failures that were hidden by alphabetical ordering.

generic

中文

pytest-randomly 每次运行打乱测试顺序；顺序相关的测试或共享状态导致间歇性失败，这些失败在字母序下被隐藏。

Workarounds

  1. 90% success 
    pytest -p randomly --randomly-seed=12345 tests/ -x
# identify the leak, then fix with autouse reset fixture
  2. 92% success 
    @pytest.fixture(autouse=True)
def reset_globals():
    myapp.state.clear()
    yield
    myapp.state.clear()

Dead Ends

Common approaches that don't work:

  1. 90% fail

    Hides order dependencies and lets bugs ship; the same issue will reappear under xdist or CI parallelism.

  2. 85% fail

    Masks the bug for that seed; other seeds still fail and the underlying leak remains.

  3. 80% fail

    Requires pytest-ordering and creates brittle coupling; doesn't fix leaked state.