Using Pinecone Inference API

1. Understanding Inference API

FeatureDescription
Hosted modelsEmbeddings + reranking, no external provider
IntegratedSingle API key + billing
Auto-upsertEmbed + upsert in one call (integrated inference) NEW

2. Enabling Inference for Index

Example: Integrated Inference Index

pc.create_index_for_model(
    name="docs",
    cloud="aws",
    region="us-east-1",
    embed={
        "model": "multilingual-e5-large",
        "field_map": {"text": "chunk_text"},
    },
)

3. Generating Embeddings

Example: Embed

r = pc.inference.embed(
    model="multilingual-e5-large",
    inputs=["document text"],
    parameters={"input_type": "passage", "truncate": "END"},
)
ParamValues
input_typepassage (docs) / query
truncateEND / NONE

4. Selecting Embedding Models

ModelDimUse
multilingual-e5-large1024100+ languages
llama-text-embed-v21024English-focused
pinecone-sparse-english-v0Sparse for hybrid
pinecone-rerank-v0Reranking

5. Batch Embedding Generation

Example: Batch Embed

texts = ["doc 1", "doc 2", "doc 3"]
r = pc.inference.embed(model="multilingual-e5-large",
    inputs=texts, parameters={"input_type": "passage"})
vectors = [e["values"] for e in r.data]
LimitValue
Max inputs per call96
Max tokens per input512

6. Understanding Rate Limits

PlanLimit (approx)
Free/StarterRPM-limited, low
StandardHigher RPM, request increase
EnterpriseCustom

7. Handling Inference Errors

ErrorCause
429Rate limited; backoff
400 truncationInput exceeds token limit
404Model name typo

8. Optimizing Inference Performance

TipEffect
Batch up to 96 inputsOne round-trip
Same region as indexLower latency
Integrated inferenceAvoids client-side embed step

9. Comparing Costs

ProviderApprox Cost (per 1M tokens)
OpenAI 3-small$0.02
OpenAI 3-large$0.13
Pinecone InferencePlan-dependent (bundled)

10. Integrating Inference with Upsert

Example: Upsert Records (Auto-Embed)

index.upsert_records(
    namespace="docs",
    records=[
        {"id": "d1", "chunk_text": "Vector DB intro", "source": "wiki"},
        {"id": "d2", "chunk_text": "Pinecone basics", "source": "blog"},
    ],
)
# Pinecone embeds chunk_text automatically using the index's model