Modeling Temporal and Historical Data

1. Understanding Temporal Data Types

TermMeaning
Valid TimeWhen the fact is true in reality
Transaction TimeWhen stored in DB
BitemporalBoth valid and transaction time tracked
SCDSlowly changing dimension (type 1/2/3/4/6)
Application timeSQL:2011 PERIOD FOR app_time
System versioningSQL:2011 SYSTEM_TIME (auto history)

2. Implementing Bi-Temporal Tables

Example: Bi-Temporal Row

CREATE TABLE policy_price (
  id           BIGSERIAL,
  policy_id    BIGINT,
  price        NUMERIC(10,2),
  valid_from   DATE NOT NULL,
  valid_to     DATE NOT NULL,
  recorded_at  TIMESTAMPTZ DEFAULT NOW(),
  superseded_at TIMESTAMPTZ,
  PRIMARY KEY (id)
);
-- Query: what did we know on 2026-01-15 about price for valid 2026-03?
SELECT price FROM policy_price
WHERE policy_id = ? AND '2026-03-01' BETWEEN valid_from AND valid_to
  AND recorded_at <= '2026-01-15'
  AND (superseded_at IS NULL OR superseded_at > '2026-01-15');

3. Modeling Effective Dating

ColumnUse
effective_from / effective_toValidity window
Open-endedNULL or '9999-12-31' for current
ConstraintEXCLUSION on overlapping ranges per entity
UpdateClose current row, insert new with new dates

4. Designing Historical Tracking Tables

PatternDetail
Shadow tableMirror schema + op + changed_at
SQL:2011 SYSTEM_TIMEAuto-history (SQL Server, MariaDB, DB2)
CDC streamDebezium → log/store
JSON diffStore only changed fields

5. Implementing Event Sourcing Schemas

ConceptDetail
Event logAppend-only sequence of events
Schemaevent_id, aggregate_id, version, type, payload, occurred_at
ProjectionDerive current state by replaying events
SnapshotPeriodic checkpoint for faster replay
CQRS pairingWrite events; read projections

6. Modeling Point-in-Time Queries

Example: AS OF Query (SQL:2011)

SELECT * FROM employees
FOR SYSTEM_TIME AS OF TIMESTAMP '2025-12-31 23:59:59';
ApproachDetail
SQL:2011 AS OFSupported in MariaDB, SQL Server, DB2, Oracle Flashback
ManualWHERE effective_from <= t AND effective_to > t
Event replayApply events up to timestamp

7. Designing Temporal Foreign Keys

ApproachDetail
PERIOD FK (SQL:2011)FOREIGN KEY ... PERIOD app_time REFERENCES ...
Validity overlapChild period must be within parent period
ManualTrigger or check constraint enforcing temporal containment

8. Handling Temporal Data Updates

OperationPattern
Update fieldClose current row (set effective_to), insert new row
Correct pastUpdate existing row in place (mark recorded_at)
DeleteSet effective_to = NOW(); never physical delete
Bulk reloadInsert new generation, switch pointer

9. Implementing Slowly Changing Dimensions

SCD TypeBehavior
Type 0No change tracking
Type 1Overwrite (no history)
Type 2New row per change with effective dates
Type 3Previous value column (last N changes)
Type 4Current table + history table
Type 6 (hybrid)Type 1 + 2 + 3 combined

10. Optimizing Temporal Query Performance

TipDetail
Index temporal columns(entity_id, effective_to DESC)
Range types + GiST (PG)Fast overlap queries
Partial index for currentWHERE effective_to IS NULL
Partition history by periodDrop old data fast
Separate current vs historyHot table stays small