Implementing Time Series Data
1. Using Sorted Sets for Time Series
| Schema | Detail |
|---|---|
| Score | Unix timestamp (ms) |
| Member | Unique value (e.g. ts:payload-id) |
2. Storing Timestamped Data
ZADD metric:cpu 1716200000000 "65.4|host1"
ZADD metric:cpu 1716200060000 "67.1|host1"
| Pattern | Note |
|---|---|
| Member uniqueness | Include ts in member to avoid collisions |
3. Querying by Time Range
| Command | Description |
|---|---|
ZRANGE key from to BYSCORE | Inclusive range |
ZRANGE key from to BYSCORE LIMIT off cnt | Paginate |
4. Aggregating Time Series Data
| Aggregation | Approach |
|---|---|
| Sum/avg | Pull range, compute client-side or Lua |
| Bucketed | HINCRBY into metric:hour:<ts> |
5. Implementing Downsampling
| Strategy | Detail |
|---|---|
| Periodic job | Aggregate raw → minute → hour → day |
| Different keys | Separate retention per resolution |
6. Setting Data Retention Policies
| Method | Detail |
|---|---|
ZREMRANGEBYSCORE | Drop entries older than threshold |
| Per-bucket EXPIRE | Auto-drop with TTL |
7. Using EXPIRE for Auto-Cleanup
| Pattern | Detail |
|---|---|
| Per-hour key | metric:cpu:2026051912 with TTL 7 days |
8. Handling High-Frequency Updates
| Technique | Detail |
|---|---|
| Pipeline ZADD | Batch metric writes |
| Local buffer + flush | Reduce server load |
9. Using RedisTimeSeries Module
| Command | Description |
|---|---|
TS.CREATE key RETENTION ms LABELS k v | Create series |
TS.ADD key ts value | Insert sample (use * for now) |
TS.RANGE key from to AGGREGATION avg 60000 | Bucketed query |
TS.MRANGE from to FILTER label=value | Multi-series query |
TS.CREATERULE src dst AGGREGATION sum bucketMs | Downsampling rule |
10. Implementing Compaction Strategies
| Tier | Resolution / Retention |
|---|---|
| Hot | 1s / 1 hour |
| Warm | 1min / 7 days |
| Cold | 1h / 1 year |