Working with Metadata

1. Defining Metadata Schema

FieldTypeExample
titlestring"Vector DBs 101"
tagslist[string]["ml", "db"]
scorenumber0.95
publishedbooleantrue
timestampnumber (unix)1716067200
Note: Pinecone doesn't enforce a schema. Enforce one in your application layer for filter consistency.

2. Adding Metadata to Vectors

Example: With Metadata

index.upsert(vectors=[{
    "id": "doc1",
    "values": [...],
    "metadata": {"source": "wiki", "page": 12, "tags": ["ml"]},
}])

3. Updating Existing Metadata

Example: Partial Update

index.update(id="doc1", set_metadata={"views": 42})
OpEffect
set_metadataMerge (overwrite per-key)
UpsertFull replace

4. Validating Metadata Types

Example: Validation

ALLOWED = (str, int, float, bool)
def valid(md):
    for k, v in md.items():
        if isinstance(v, list):
            if not all(isinstance(x, str) for x in v): return False
        elif not isinstance(v, ALLOWED): return False
    return True

5. Understanding Metadata Size Limits

LimitValue
Per vector40 KB total
List itemsCounted in total size
Nested objectsNot supported

6. Organizing Data with Metadata

PatternUse
Categorical filterscategory, language, status
Temporal filterscreated_at, expires_at
Access controlowner_id, visibility
Source trackingdoc_id, chunk_index

7. Implementing Version Tracking

Example: Version Field

index.upsert(vectors=[{
    "id": "doc1",
    "values": [...],
    "metadata": {"version": 3, "embedding_model": "text-embedding-3-small"},
}])

8. Adding Timestamps

FormatRecommendation
Unix epoch (int)Best for range filters
ISO 8601 stringHuman-readable; no range queries

9. Storing Document References

Example: Doc References

{
    "id": "chunk_abc_3",
    "metadata": {
        "doc_id": "abc",
        "chunk_index": 3,
        "source_url": "https://example.com/abc",
        "text_snippet": "...",
    },
}

10. Handling Complex Metadata Structures

Warning: Nested objects are not supported. Flatten with dot notation or store JSON-as-string (no filtering).
StrategyExample
Flatten keysauthor.nameauthor_name
Stringify JSONraw_json: "{...}" (not filterable)
External lookupStore doc ID; fetch full record from DB