Batch Operations

📦 NoSQL Batch Operations: The Pizza Delivery Story

Imagine you’re running a pizza delivery service. Instead of sending one driver for each pizza, you load up a van with 50 pizzas and deliver them all at once. That’s what batch operations do for your database!

🍕 The Big Picture

When you need to work with lots of data in NoSQL databases, doing things one-by-one is slow—like delivering pizzas one at a time on a bicycle.

Batch operations let you:

• Send many requests in one trip (Bulk Operations)
• Read large results piece by piece (Cursor Operations)
• Show data in manageable chunks (Pagination)

🚚 Part 1: Bulk Operations

What Are Bulk Operations?

Think of bulk operations like a moving truck vs a backpack.

Without bulk operations:

Save pizza order 1 → Wait → Done
Save pizza order 2 → Wait → Done
Save pizza order 3 → Wait → Done
... (100 more times)

With bulk operations:

Load all 100 orders into truck
Send truck once → All done!

Real MongoDB Example

// 🐌 Slow way (one by one)
await db.orders.insertOne({pizza: "Pepperoni"})
await db.orders.insertOne({pizza: "Margherita"})
await db.orders.insertOne({pizza: "Hawaiian"})

// 🚀 Fast way (bulk insert)
await db.orders.insertMany([
  {pizza: "Pepperoni"},
  {pizza: "Margherita"},
  {pizza: "Hawaiian"}
])

Types of Bulk Operations

graph TD
    A["📦 Bulk Operations"] --> B["Insert Many"]
    A --> C["Update Many"]
    A --> D["Delete Many"]
    A --> E["Mixed Batch"]
    B --> F["Add 1000 users at once"]
    C --> G["Mark all orders as shipped"]
    D --> H["Remove old records"]
    E --> I["Insert + Update + Delete together"]

Mixed Bulk Operations

Sometimes you need to do different things in one batch:

// MongoDB bulkWrite example
await db.orders.bulkWrite([
  // Insert a new order
  { insertOne: {
      document: {pizza: "BBQ Chicken"}
  }},
  // Update an existing order
  { updateOne: {
      filter: {_id: "order123"},
      update: {$set: {status: "delivered"}}
  }},
  // Delete cancelled orders
  { deleteMany: {
      filter: {status: "cancelled"}
  }}
])

💡 Why it matters: One network trip instead of three. Faster. Cheaper. Better!

🎯 Part 2: Cursor Operations

What is a Cursor?

Imagine you have a library with 10,000 books. You can’t carry them all at once!

A cursor is like a bookmark that helps you:

• Read books a few at a time
• Remember where you stopped
• Continue from that spot later

graph TD
    A["📚 10,000 Documents"] --> B["Cursor Created"]
    B --> C["Fetch First 100"]
    C --> D["Process Them"]
    D --> E["Fetch Next 100"]
    E --> F["Process Them"]
    F --> G["... Continue Until Done"]

Simple Cursor Example

// Create a cursor (doesn't fetch data yet!)
const cursor = db.users.find({active: true})

// Process documents one at a time
while (await cursor.hasNext()) {
  const user = await cursor.next()
  console.log(user.name)
}

Cursor Methods You’ll Use

Batch Size Control

// Fetch 50 documents at a time
const cursor = db.orders
  .find({})
  .batchSize(50)

// Now each network call gets 50 docs

🧒 Think of it like: Taking 50 steps at a time instead of 1. You get there faster!

📄 Part 3: Pagination

What is Pagination?

Open your phone. Go to any shopping app. See how products appear on pages?

• Page 1: Products 1-20
• Page 2: Products 21-40
• Page 3: Products 41-60

That’s pagination! Breaking big lists into small, easy pages.

graph TD
    A["1000 Products"] --> B["Page 1: 1-20"]
    A --> C["Page 2: 21-40"]
    A --> D["Page 3: 41-60"]
    A --> E["... Page 50: 981-1000"]

Two Ways to Paginate

Method 1: Skip and Limit (Easy but Slow)

// Page 1
db.products.find().skip(0).limit(20)

// Page 2
db.products.find().skip(20).limit(20)

// Page 3
db.products.find().skip(40).limit(20)

Problem: Skip gets slower as pages increase. Skipping 10,000 items takes time!

Method 2: Cursor-Based (Fast and Smart)

// First page
db.products
  .find()
  .sort({_id: 1})
  .limit(20)

// Next page (use last ID from previous page)
db.products
  .find({_id: {$gt: lastSeenId}})
  .sort({_id: 1})
  .limit(20)

Why it’s better: Database jumps directly to the right spot!

Comparison Table

Complete Pagination Example

async function getPage(lastId = null) {
  const pageSize = 20

  // Build the query
  const query = lastId
    ? {_id: {$gt: lastId}}
    : {}

  // Fetch the page
  const items = await db.products
    .find(query)
    .sort({_id: 1})
    .limit(pageSize + 1)  // +1 to check if more exist
    .toArray()

  // Check if there's a next page
  const hasMore = items.length > pageSize
  if (hasMore) items.pop()  // Remove extra item

  return {
    items,
    hasMore,
    nextCursor: items[items.length - 1]?._id
  }
}

🎓 Quick Recap

🌟 Why This Matters

1. Speed: Less waiting, faster apps
2. Memory: Your app doesn’t explode with big data
3. User Experience: People see results quickly
4. Cost: Fewer database calls = lower bills

🚀 Pro Tips

🔥 Bulk Tip: Always use bulkWrite() when doing 10+ operations

🔥 Cursor Tip: Set batchSize() based on your document size

🔥 Pagination Tip: Use cursor-based pagination for infinite scroll

Remember: Batch operations are like smart delivery routes. Do more with less trips! 🍕