Implementing DataLoader Pattern
1. Understanding N+1 Problem
Query: posts { author { name } }
Without DataLoader:
1 query for posts
+ N queries (one per post.author)
= N+1 problem
With DataLoader:
1 query for posts
+ 1 batched query for all unique authorIds
= 2 queries total
2. Creating DataLoader Instance
Example: Basic loader
import DataLoader from "dataloader";
const userLoader = new DataLoader(async (ids) => {
const users = await db.user.findMany({ where: { id: { in: ids } } });
const map = new Map(users.map(u => [u.id, u]));
return ids.map(id => map.get(id) ?? null);
});| Option | Default | Detail |
|---|---|---|
| batch | true | Group calls in same tick |
| cache | true | Cache by key per loader |
| maxBatchSize | Infinity | Cap batch size |
| cacheKeyFn | identity | Hash function |
3. Implementing Batch Function
| Rule | Detail |
|---|---|
| Input | Array of unique keys |
| Output | Array of values, SAME length, SAME order |
| Misses | Return null/undefined or Error in slot |
| Async | Returns Promise<Array<V|Error>> |
4. Using DataLoader in Resolver
Example: Loader in resolver
Post: {
author: (post, _, { loaders }) => loaders.userById.load(post.authorId)
}5. Batching Database Queries
Example: SQL IN batch
const batchUsers = async (ids) =>
db("users").whereIn("id", ids).then(rows => {
const map = new Map(rows.map(r => [r.id, r]));
return ids.map(id => map.get(id) ?? null);
});6. Caching Loaded Data
| Scope | Lifetime |
|---|---|
| Per-request | Default; safe |
| Cross-request | Risky — stale auth/permissions |
| Redis-backed | Custom cacheMap |
7. Clearing DataLoader Cache
| Method | Effect |
|---|---|
| .clear(key) | Invalidate one entry |
| .clearAll() | Reset entire cache |
| After mutations | Clear affected entries |
8. Priming DataLoader Cache
Example: Prime after fetch
const posts = await db.post.findMany({ include: { author: true } });
posts.forEach(p => loaders.userById.prime(p.author.id, p.author));
return posts;9. Handling Load Errors
| Strategy | Detail |
|---|---|
| Return Error in slot | load() rejects only for that key |
| Throw in batch fn | ALL pending loads reject |
| Silent miss | Return null for not-found (recommended) |
10. Creating Multiple Loaders
| Loader | Key |
|---|---|
| userById | User id → User |
| postsByAuthor | authorId → [Post] |
| commentsByPost | postId → [Comment] |
| tagsByPost | postId → [Tag] |
11. Using Loader with Foreign Keys
Example: One-to-many loader
const postsByAuthor = new DataLoader(async (authorIds) => {
const rows = await db.post.findMany({ where: { authorId: { in: authorIds } } });
const grouped = new Map(authorIds.map(id => [id, []]));
rows.forEach(p => grouped.get(p.authorId).push(p));
return authorIds.map(id => grouped.get(id));
});