python runtime_error ai_generated partial

FAILED (failures=0) # but subTest failures printed: FAIL: test_batch (tests.test_batch.TestBatch.test_batch) (i=3) AssertionError: 3 != 4

ID: python/unittest-subtest-failure-swallowed

Also available as: JSON · Markdown · 中文
80%Fix Rate
80%Confidence
0Evidence
2025-05-19First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.10 active — — —
3.12 active — — —

Root Cause

subTest failures are recorded against the parent test but some runners or custom result classes do not increment the failures counter, so the summary reports failures=0 even though individual subtests failed.

generic

中文

subTest 失败记录在父测试上，但某些运行器或自定义结果类不会增加 failures 计数器，导致摘要显示 failures=0 而个别子测试实际失败。

Workarounds

  1. 90% success 
    Run with a result class that counts subtests, or use pytest which reports them:

python -m pytest tests/test_batch.py -v
  2. 85% success 
    Inspect result.failures + result.errors + subtest count in a custom runner:

result = unittest.TextTestRunner().run(suite)
assert not result.failures and not result.errors

Dead Ends

Common approaches that don't work:

  1. 85% fail

    Defeats the purpose of subTest (continue-on-failure) and stops at the first failing case.

  2. 90% fail

    The process may still exit 0 when failures counter is zero, so CI passes a broken suite.