mongodb resource_error ai_generated true

MongoServerError：计划执行器错误：聚合阶段'lookup'导致内存压力：估计大小200MB超过100MB限制

MongoServerError: PlanExecutor error: aggregation stage 'lookup' caused memory pressure: estimated size 200MB exceeds 100MB limit

ID: mongodb/aggregation-lookup-unwind-memory-pressure

其他格式: JSON · Markdown 中文 · English
82%修复率
86%置信度
1证据数
2024-09-30首次发现

版本兼容性

版本状态引入弃用备注
mongodb 6.0 active — — —
mongodb 7.0 active — — —
mongodb 8.0 active — — —

根因分析

在大型集合上使用$lookup和$unwind阶段导致中间结果超过100MB内存限制，通常是由于缺少索引或笛卡尔积连接。

English

A $lookup stage with $unwind on large collections exceeded the 100MB memory limit for intermediate results, often due to missing indexes or cartesian product joins.

generic

官方文档

https://www.mongodb.com/docs/manual/reference/operator/aggregation/lookup/#memory-considerations

解决方案

  1. Create an index on the foreign field used in $lookup: `db.orders.createIndex({ customerId: 1 })` to reduce the size of matched documents and avoid full collection scans.
  2. Restructure the pipeline: use $lookup with a pipeline to filter documents before joining, e.g., `{ $lookup: { from: 'orders', let: { custId: '$_id' }, pipeline: [ { $match: { $expr: { $eq: ['$customerId', '$$custId'] } } }, { $limit: 100 } ], as: 'orders' } }` to limit matched documents.
  3. Split the aggregation: perform $lookup in a separate aggregation, write results to an intermediate collection, then run $unwind on that smaller dataset.

无效尝试

常见但无效的做法:

  1. 85% 失败

    allowDiskUse does not apply to intermediate results of $lookup before $unwind; it only helps for sorting and grouping stages after the join.

  2. 95% 失败

    The 100MB limit is hard-coded for $lookup intermediate results and cannot be changed; attempting to set it has no effect.

  3. 70% 失败

    While this may reduce memory, it changes the output structure; the root cause is the large join size, not the unwind itself.