Is your MongoDB stuttering? Discover the villain behind write conflicts

If you work with MongoDB, you know it promises scalability and flexibility. But what happens when that promise is broken? When operations that should be simple start to freeze, logs fill with errors, and latency skyrockets, leaving your users frustrated and your IT team in a panic?

The culprit is often a silent but deadly villain: write conflicts. They not only compromise performance but also jeopardize data integrity and your database’s availability. This article is for you—the DBA, DevOps engineer, Tech Lead, or IT manager—who has felt the pain of seeing a MongoDB cluster choke.

Here, we will unravel the causes behind write conflicts and, most importantly, show how HTI Technology helps medium and large businesses maintain the health of their MongoDB databases, ensuring performance, security, and sustainability.

What Are MongoDB Write Conflicts and Why Do They Happen?

Simply put, a write conflict occurs when two or more write operations (updates, inserts, or deletes) try to modify the same document or the same data block simultaneously. By design, MongoDB prioritizes consistency, which can lead to a “blocking” problem that delays or even causes transactions to fail.

Imagine two users trying to update the balance of the same bank account at the same time. If the system doesn’t manage this concurrency well, the last update might overwrite the first, leading to incorrect data. In MongoDB, this scenario manifests in more subtle but equally dangerous ways, especially in environments with a high concurrency rate.

Example of a write conflict scenario:

Consider the following bank account document:

// Original account document
{
  "_id": "conta123",
  "saldo": 1000.00,
  "historicoTransacoes": [
    { "tipo": "deposito", "valor": 1000, "data": ISODate("2023-01-01T10:00:00Z") }
  ]
}

Now, imagine two users (or processes) trying to withdraw money at the same time:

Operation 1 (User A): Tries to withdraw $200

db.contas.updateOne(
  { "_id": "conta123", "saldo": { "$gte": 200 } }, // Checks balance before withdrawing
  { "$inc": { "saldo": -200 } }
);

Operation 2 (User B): Tries to withdraw $300

db.contas.updateOne(
  { "_id": "conta123", "saldo": { "$gte": 300 } }, // Checks balance before withdrawing
  { "$inc": { "saldo": -300 } }
);

If both operations read the saldo as 1000.00 simultaneously before either one writes, and there is no proper control (like transactions or findAndModify with retry), one might overwrite the other, resulting in an incorrect balance or a “write conflict” failure if MongoDB detects the concurrent modification.

Common Causes of Write Conflicts in Your MongoDB Cluster

Write conflicts in MongoDB don’t just appear out of nowhere. They are symptoms of flaws in architecture, schema design, or inadequate operational practices. The main causes include:

High Volume of Concurrency: Applications with many concurrent users or processes frequently trying to modify the same documents in your MongoDB database.
Inadequate Schema Design: Denormalization is a significant advantage of MongoDB, but if not well-planned, it can create “hot spots” where a single document is the target of many operations. Excessive use of arrays for data that can grow indefinitely (unbounded arrays) is a classic cause of write conflicts in MongoDB.

Example of a Hot Spot due to Inadequate Schema (Unbounded Array):

Imagine a “Product” document that stores all customer reviews in an array within the document itself.

// Product document with reviews in an unbounded array
{
  "_id": "prod_X",
  "nome": "Smartphone XYZ",
  "descricao": "...",
  "avaliacoes": [
    { "usuario": "user1", "nota": 5, "comentario": "Adorei!" },
    { "usuario": "user2", "nota": 4, "comentario": "Bom produto." },
    // ... hundreds or thousands of reviews ...
  ]
}

Every time a new review is added, the entire document needs to be rewritten. If many reviews arrive at the same time for the same product, this becomes a “hot spot,” generating many write conflicts.

Suggested solution for the example above (Reference instead of Embed):

Instead of embedding, we could create a separate collection for reviews and reference the product, avoiding the hot spot in the product document.

// Products collection (without the reviews array)
db.produtos.insertOne({
  "_id": "prod_X",
  "nome": "Smartphone XYZ",
  "descricao": "..."
});

// Reviews collection
db.avaliacoes.insertOne({
  "produtoId": "prod_X",
  "usuario": "user1",
  "nota": 5,
  "comentario": "Adorei!"
});

Now, adding a new review doesn’t touch the product document, reducing concurrency.

Multi-document Transactions: Although MongoDB 4.0 and later versions support ACID transactions within a single cluster, the complexity of managing locking and concurrency can increase the likelihood of conflicts, especially if transactions are long or access many documents. Managing transactions in MongoDB requires care.

Example of a Multi-document Transaction (potentially conflict-generating):

Imagine a transaction that moves funds between two accounts.

// Starts a session for the transaction
const session = db.getMongo().startSession();
session.startTransaction();

try {
  // Withdraws from the source account
  db.contas.updateOne(
    { _id: "contaOrigem", saldo: { $gte: 100 } },
    { $inc: { saldo: -100 } },
    { session }
  );

  // Deposits to the destination account
  db.contas.updateOne(
    { _id: "contaDestino" },
    { $inc: { saldo: 100 } },
    { session }
  );

  session.commitTransaction();
  print("Transaction successfully completed.");
} catch (error) {
  session.abortTransaction();
  print("Transaction aborted: " + error);
} finally {
  session.endSession();
}

If several of these transactions are operating on the same accounts simultaneously, transaction locks can lead to write conflicts or deadlocks, depending on the order of operations and the load.

Batch Write Operations (Bulk Writes): While efficient, if a large number of batch writes are directed at nearby or related documents, conflicts can arise. It is crucial to plan these operations to avoid bottlenecks in your MongoDB database.

Example of a Bulk Write with conflict potential:

If a bulkWrite tries to update many documents that are physically close on disk (or in the same “chunk” in a sharded cluster) and other operations are also trying to access them.

db.produtos.bulkWrite([
  { updateOne: { filter: { _id: "prod_A" }, update: { $set: { "preco": 10.99 } } } },
  { updateOne: { filter: { _id: "prod_B" }, update: { $set: { "preco": 25.50 } } } },
  { updateOne: { filter: { _id: "prod_C" }, update: { $set: { "preco": 5.00 } } } }
  // ... hundreds of updates for related products or in the same _id range
]);

If the _id of products A, B, C… Z are very close, this operation can generate a broader lock or write collisions with other operations trying to change those same documents or adjacent documents at the same time.

7 Signs That Your MongoDB Health is at Risk

Recognizing the symptoms is the first step toward diagnosis. If your IT team or internal DBA has noticed some of these signs, it’s a red alert that your MongoDB performance needs urgent intervention.

Unexplained Increase in Write Latency: The time it takes for a write operation to complete in MongoDB starts to grow, even without a proportional increase in the volume of requests. This is a classic sign that MongoDB is struggling to handle concurrency.
Conflict Errors in Logs (Write Conflict Error): Your system log fills with messages like WriteConflictException, Operation Failed, or similar messages, indicating operation failures. This is a fatal error for any application using MongoDB.

Example of a WriteConflictException error in logs (simplified format):

2023-10-27T10:30:15.123+0000 I COMMAND  [conn123] write command: update { update: "myCollection", updates: [ { q: { _id: ObjectId(...) }, u: { $set: { "field": "newValue" } } } ], ordered: true, writeConcern: { w: 1 } }
2023-10-27T10:30:15.125+0000 W WRITE    [conn123] Failed to apply write: WriteConflictException: WriteConflict: Transaction 123456 encountered write conflict with transaction 789012.

This is a clear and direct indicator that write conflicts are occurring.

Increased CPU and RAM Usage: The MongoDB database system starts consuming more resources than usual, a sign that it is struggling to resolve collisions.
Decreased Throughput: The number of write operations per second begins to decrease in your MongoDB, even with stable user demand.
Application Timeouts: The application starts to display “timeout” errors because MongoDB cannot process requests in time.
The System “Freezes” during Peak Hours: MongoDB performance degrades significantly during periods of high traffic when concurrency is highest.
Failing Update and Delete Operations: Functions that were previously stable in your MongoDB start to show intermittent failures, requiring the development team to create retry logic.

The Power of DBA Outsourcing: Why Your Company Doesn’t Need to Suffer

Many medium and large companies believe the only solution is to hire a senior internal DBA to solve these problems in their MongoDB database. But this approach can be expensive, time-consuming, and, in many cases, inefficient. This is where outsourcing MongoDB DBA services proves to be a superior strategy.

A specialized company like HTI Technology offers 24/7 consultation, support, and maintenance, with a team of specialists who have already solved complex problems in hundreds of environments. This means you don’t just hire one person, but an entire team of experts in different technologies (MongoDB, but also PostgreSQL, MySQL, Oracle, and others) ready to act.

Advantages of DBA Outsourcing with HTI Technology

Deep Technical Focus and Specialization: Our team of senior DBAs has vast and focused experience, allowing them to identify and solve performance and security problems that a generalist DBA might not be able to. They know the secrets of MongoDB and other technologies, as we detail in our article on the differences between SQL and NoSQL. HTI’s expertise in MongoDB is unquestionable.
Risk Reduction and Operational Continuity: With 24/7 support, we ensure that any availability or performance issue in your MongoDB database—be it a write conflict or a failover—is resolved immediately, minimizing the impact on your business.
Cost Reduction: Hiring, training, and maintaining an internal senior MongoDB DBA is expensive. Outsourcing converts a high fixed cost into a flexible and predictable service, allowing your company to direct resources to its core business.
External and Proactive View: An external consultant brings a new perspective. The HTI team acts proactively, monitoring your MongoDB environment, anticipating problems, and suggesting improvements in database architecture before write conflicts become a catastrophe.

The Depth of HTI Technology’s Solution for MongoDB Problems

Resolving write conflicts goes far beyond a simple configuration adjustment. It is a task that requires a deep understanding of MongoDB’s internal workings and the specificities of your application.

1. Diagnosis and Detailed Environment Analysis

Our specialists perform a deep analysis of your MongoDB cluster, evaluating schema design, workload, index configuration, and system behavior. We use advanced monitoring tools to capture real-time metrics and identify the exact points where write conflicts are occurring. The detailed analysis of the oplog and MongoDB logs allows us to track the root cause of the problems.

Example of a Command for Analysis (WiredTiger State):

// In the MongoDB shell, to check WiredTiger storage engine statistics
db.serverStatus().wiredTiger.concurrentTransactions

This can show the number of active and queued read and write transactions, helping to diagnose bottlenecks. Other important metrics to observe in db.serverStatus() include opcounters.insert, opcounters.update, opcounters.delete, and metrics.queryExecutor.scanned to identify expensive queries.

2. Schema and Index Optimization

Many MongoDB write conflicts can be mitigated with smarter schema design. Our team works with your developers to refactor documents and collections, reducing “hot spots” and optimizing how data is accessed. Additionally, we ensure that indexes are configured to accelerate read operations without compromising write operations. Correct indexing is crucial for MongoDB performance.

Example of Composite Index Creation for Optimization:

For a pedidos (orders) collection where you frequently query by clienteId (customerId) and status, a composite index is essential.

db.pedidos.createIndex({ "clienteId": 1, "status": 1 });

This accelerates queries and can reduce lock retention time, decreasing the chance of conflicts.

3. MongoDB Configuration Tuning

MongoDB’s default settings are not always ideal for high-concurrency environments. Our specialists adjust locking, sharding, and other configurations to ensure the database behaves robustly and efficiently, even under great stress. Tuning WiredTiger and other internal MongoDB mechanisms is one of our specialties.

Example of Write Concern Configuration:

For some less critical operations, you can adjust the write concern to a more relaxed level to avoid waiting for many confirmations, although this should be used with caution.

// Example of write concern for a specific operation
db.myCollection.insertOne(
  { "data": "value" },
  { writeConcern: { w: 0 } } // 'w:0' means the driver does not wait for server confirmation. Highly risky for integrity.
);

// For operations that require more durability, use 'majority'
db.myCollection.updateOne(
  { "_id": "doc1" },
  { "$set": { "status": "completed" } },
  { writeConcern: { w: "majority", j: true, wtimeout: 5000 } } // Waits for majority, journal, and timeout
);

Adjusting w (the number of replicas that must confirm the write) and j (ensuring the write has gone to the journal) directly impacts durability and performance.

4. Implementation of Concurrency Strategies

Depending on your application, we can suggest and implement strategies such as using atomic operations, designing nested documents for more efficient upsert and update operations, or even revising workflows that generate excessive concurrency. Using findAndModify for atomic operations or restructuring operations to use multi for multi-document updates are examples of how HTI helps optimize MongoDB.

Example of findAndModify for Atomic Operations (Balance Withdrawal):

Instead of a simple updateOne that can lead to race conditions, findAndModify ensures that the read, modification, and write occur atomically for the document.

// Function to withdraw money atomically
function withdrawMoney(accountId, amount) {
  const result = db.contas.findAndModify({
    query: { "_id": accountId, "saldo": { "$gte": amount } },
    update: { "$inc": { "saldo": -amount } },
    new: true // Returns the modified document
  });

  if (result) {
    print("Withdrawal of " + amount + " successfully performed. New balance: " + result.saldo);
    return true;
  } else {
    print("Failed to withdraw " + amount + ". Insufficient balance or account not found.");
    return false;
  }
}

// Example usage
withdrawMoney("conta123", 200);

This drastically reduces the chance of write conflicts for critical single-document operations.

5. Proactive Monitoring and Continuous Support

The work doesn’t end after the problem is resolved. HTI offers continuous monitoring for your MongoDB environment, using tools that alert us to potential performance issues before they become critical. This allows us to intervene proactively, ensuring the availability and security of your database 24/7. Our specialized MongoDB support guarantees peace of mind.

Don’t Let Write Conflicts Undermine Your Application’s Success

Write conflicts in MongoDB are more than a technical problem; they are a symptom of a data system that is not optimized for your workload. Ignoring them is opening the door to slowness, instability, and ultimately, a loss of customer trust. The complexity of MongoDB can be a challenge, but you don’t have to face it alone.

HTI Technology is the partner your company needs to ensure your MongoDB not only works but thrives. Our mission is to take care of your database so you can focus on what really matters: innovating and growing your business.

Don’t wait for the next collapse.

Talk to one of our MongoDB specialists and discover how HTI Technology can guarantee the performance, availability, and security your company deserves. Schedule a no-obligation meeting.

Schedule a meeting here