Implementing Bulk Operations

1. Designing Bulk API Endpoints

AspectDetail
EndpointPOST /resources/bulk
PayloadArray of operations
Max itemsCap (e.g., 1000 per call)
IdempotencyPer-request key

2. Implementing Bulk Insert Logic

Example: JDBC batch

jdbc.batchUpdate(
    "INSERT INTO orders(id, customer_id, total) VALUES (?,?,?)",
    new BatchPreparedStatementSetter() {
        public void setValues(PreparedStatement ps, int i) throws SQLException {
            var o = orders.get(i);
            ps.setObject(1, o.id());
            ps.setObject(2, o.customerId());
            ps.setLong(3, o.totalCents());
        }
        public int getBatchSize() { return orders.size(); }
    });

3. Implementing Bulk Update Logic

Example: Set-based UPDATE

UPDATE orders SET status = 'archived'
WHERE created_at < now() - interval '180 days'
  AND status = 'completed';

4. Implementing Bulk Delete Logic

PracticeDetail
ChunkedDELETE in batches (e.g., 1k rows)
Soft delete firstReversible
ConfirmationTwo-step UI for destructive ops
ThrottleAvoid replication lag

5. Implementing Bulk Import from CSV

Example: Streaming CSV reader

try (var reader = Files.newBufferedReader(path);
     CSVParser p = CSVFormat.DEFAULT.builder().setHeader().build().parse(reader)) {
    var batch = new ArrayList<Order>(1000);
    for (var rec : p) {
        batch.add(toOrder(rec));
        if (batch.size() == 1000) { repo.saveAll(batch); batch.clear(); }
    }
    if (!batch.isEmpty()) repo.saveAll(batch);
}

6. Implementing Bulk Export

PracticeDetail
AsyncBackground job for large sets
StreamDon't materialize entire result in memory
Compressgzip / zip output
ResumeCheckpoint cursor

7. Handling Partial Failures

StrategyDetail
All-or-nothingSingle TX; one failure rolls back all
Best-effortPer-item TX; continue on errors
Stop on first errorUseful for validation
Per-item resultReturn success/failure per row

8. Implementing Bulk Operation Progress

MechanismDetail
Progress endpointGET /jobs/{id} returns percent
SSE streamPush updates
Countersprocessed / succeeded / failed
ETABased on throughput

9. Implementing Bulk Operation Validation

PhaseDetail
Pre-checkValidate all items before any side effect
SchemaRequired fields, types
Business rulesReferential integrity
Per-item errorsReturn list with line numbers

10. Implementing Bulk Operation Rollback

ApproachDetail
DB transactionSingle TX rollback
CompensationReverse each successful step
SagaDistributed compensating steps
SnapshotRestore from pre-op snapshot

11. Implementing Bulk Operation Throttling

MechanismDetail
Rate per tenantAvoid noisy neighbor
Concurrency capLimit parallel bulk jobs
DB pause windowsDon't run during peak
Adaptive throttleSlow down on replication lag

12. Implementing Bulk Operation Reporting

OutputDetail
SummaryCounts: ok, failed, skipped
Error CSVFailed rows + reasons (downloadable)
Audit entryWho, when, what
NotificationsEmail on completion