Corrective Action: Sanity Batch Transaction Silent Failure

← Back to Corrective Actions 2026-04-03

Corrective Action: Sanity Batch Transaction Silent Failure

Date: 2026-04-03 Category: API Behavior Assumption Impact: 49 episodes remained stuck in "processing" after batch-patch reported success. Pipeline clearance delayed ~30 minutes. Resolution Time: ~30 minutes (from discovering discrepancy to re-running with batched transactions)

Incident

What Happened

A batch-patch script was created to reset 49 Sanity episodes from transcriptionStatus: "processing" to "pending". The script used a single Sanity transaction containing all 49 patch operations. The transaction committed without error, reported "Patched 49 documents" with document IDs, and exited cleanly. However, querying the documents immediately after showed all 49 still had transcriptionStatus: "processing" — none were actually updated.

Timeline

Time Event
~09:45 CT batch-patch.cjs created with single-transaction design
~09:46 CT Script reports "Patched 49 documents" — appears successful
~09:47 CT transcribe-pending.ts finds 0 episodes with Mux IDs ready (expected 44)
~09:50 CT Direct Sanity query confirms all 49 still in "processing"
~09:52 CT Single-document patch via existing patch.js tool succeeds and persists
~09:55 CT batch-patch.cjs updated to use batches of 10
~09:57 CT Re-run succeeds: 5 batches, 48 documents patched and verified

Root Cause

Sanity's transaction API has undocumented size limits on the number of mutations per transaction. When a transaction exceeds this limit, the API returns a success response (including document IDs) but does not persist the changes. There is no error, no warning, and no partial application — the entire transaction is silently dropped.

The client.transaction().patch().commit() pattern returns a result object with documentIds even when nothing was written. This violates the principle of least surprise: a successful response with document IDs implies the operation completed.

Category: API Behavior Assumption

This is the same pattern as previous incidents (HubSpot Broadcast API no draft mode, Sanity GROQ shell escaping): assuming API behavior matches documentation or reasonable expectations, without verifying the actual effect.

Fix Applied

Immediate Resolution

Updated batch-patch.cjs to process documents in batches of 10 instead of a single transaction.

Code/Configuration Changes

File Change
scripts/sanity/batch-patch.cjs Changed from single transaction to batches of BATCH_SIZE = 10. Each batch commits separately and reports count.

Verification

Found 48 documents matching query.
  Batch 1: patched 10 docs
  Batch 2: patched 10 docs
  Batch 3: patched 10 docs
  Batch 4: patched 10 docs
  Batch 5: patched 8 docs
Patched 48 documents total.

Subsequent Sanity query confirmed processing: 0, pending: 74.

Prevention Measures

Rules Added

Layer File Rule
Critical Lessons MEMORY.md Sanity transactions silently fail above ~20 mutations. Always batch in groups of 10. Always verify writes with a follow-up query.
Batch Tools scripts/sanity/batch-patch.cjs BATCH_SIZE = 10 hardcoded
Batch Tools scripts/sanity/batch-delete.cjs Should also be updated to batch (currently unbatched)

Detection Triggers

If writing a Sanity script that patches or deletes more than 10 documents in a single transaction, stop and batch it.

Lessons

Never trust API success responses for batch operations without verifying the actual state change. This is the third incident in this system where an API returned "success" while doing nothing (HubSpot Broadcast, Sanity GROQ, now Sanity transactions). The pattern is clear: after any batch write to any external system, query the result and confirm the change persisted before proceeding.

Related Incidents

  • HubSpot Broadcast API (2026-02-18): Omitting triggerAt published immediately instead of saving as draft. API returned success.
  • Sanity GROQ shell escaping (2026-03-04): curl GROQ queries silently returned wrong results. API returned success.
  • Substring date matching (2026-02-21): "Feb 2" matched "Feb 20-29" in batch operations. Operations appeared to succeed.