python runtime_error ai_generated true

FAILED tests/test_state.py::test_b - AssertionError: assert [] == [1, 2] # passes when run alone, fails when run after test_a

ID: python/pytest-randomly-fails-order-dependent

Also available as: JSON · Markdown · 中文
80%Fix Rate
84%Confidence
0Evidence
2024-09-25First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.x active — — —
4.x active — — —

Root Cause

pytest-randomly shuffles test order and exposes shared mutable state (module-level list, class attribute, or global cache) that leaks between tests.

generic

中文

pytest-randomly 打乱测试顺序，暴露了模块级列表、类属性或全局缓存等共享可变状态在测试之间泄漏的问题。

Workarounds

  1. 95% success 
    import pytest

@pytest.fixture(autouse=True)
def reset_state():
    import myapp.cache
    myapp.cache.clear()
    yield
    myapp.cache.clear()
  2. 90% success 
    @pytest.fixture
def items():
    return []

def test_a(items):
    items.append(1)

def test_b(items):
    assert items == []  # always fresh

Dead Ends

Common approaches that don't work:

  1. 90% fail

    Hides the state leak; the bug returns in production or when order changes in CI.

  2. 85% fail

    Reproduces the same order but does not fix the leaking state.