database sharding_error ai_generated partial

pymongo.errors.OperationFailure: Would violate shard key index uniqueness. Shard key update is not allowed.

ID: database/mongo-shard-key-violation

Also available as: JSON · Markdown
70%Fix Rate
80%Confidence
38Evidence
2023-06-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
7 active

Root Cause

An operation violated sharding constraints. Common scenarios: attempting to update a document's shard key field (allowed since MongoDB 4.2 but only within a transaction), running a query without the shard key (scatter-gather), or inserting a document missing the shard key field. The shard key is immutable by default and determines document placement across shards.

generic

Workarounds

  1. 80% success Update shard key values within a multi-document transaction
    Since MongoDB 4.2, shard key updates are allowed within a transaction: session.start_transaction(); db.collection.updateOne({_id: id}, {$set: {shardKeyField: newValue}}, {session}); session.commit_transaction(); The document may be migrated to a different shard. This must run on a mongos, not directly on a shard.
  2. 85% success Include the shard key in all queries and design the schema around the shard key
    Always include the shard key in query filters to enable targeted queries. If queries frequently need to find documents by a non-shard-key field, create a mapping collection or use the shard key as a prefix in compound indexes. Choose shard keys with high cardinality and even distribution.

Dead Ends

Common approaches that don't work:

  1. Resharding a large production collection to fix a bad shard key choice 60% fail

    Resharding (available since MongoDB 5.0) is extremely resource-intensive on large collections. It creates a new collection, copies all data, and rebuilds indexes. On collections with billions of documents, this can take days and significantly impact cluster performance.

  2. Using scatter-gather queries as a workaround for not including the shard key 70% fail

    Queries without the shard key must be broadcast to all shards (scatter-gather). This does not scale: as you add more shards, query latency increases because every shard must respond. It negates the performance benefits of sharding for those queries.

Error Chain

Leads to:
Preceded by:
Frequently confused with: