Using Pinecone Inference API
1. Understanding Inference API
| Feature | Description |
|---|---|
| Hosted models | Embeddings + reranking, no external provider |
| Integrated | Single API key + billing |
| Auto-upsert | Embed + upsert in one call (integrated inference) NEW |
2. Enabling Inference for Index
Example: Integrated Inference Index
pc.create_index_for_model(
name="docs",
cloud="aws",
region="us-east-1",
embed={
"model": "multilingual-e5-large",
"field_map": {"text": "chunk_text"},
},
)
3. Generating Embeddings
Example: Embed
r = pc.inference.embed(
model="multilingual-e5-large",
inputs=["document text"],
parameters={"input_type": "passage", "truncate": "END"},
)
| Param | Values |
|---|---|
input_type | passage (docs) / query |
truncate | END / NONE |
4. Selecting Embedding Models
| Model | Dim | Use |
|---|---|---|
| multilingual-e5-large | 1024 | 100+ languages |
| llama-text-embed-v2 | 1024 | English-focused |
| pinecone-sparse-english-v0 | — | Sparse for hybrid |
| pinecone-rerank-v0 | — | Reranking |
5. Batch Embedding Generation
Example: Batch Embed
texts = ["doc 1", "doc 2", "doc 3"]
r = pc.inference.embed(model="multilingual-e5-large",
inputs=texts, parameters={"input_type": "passage"})
vectors = [e["values"] for e in r.data]
| Limit | Value |
|---|---|
| Max inputs per call | 96 |
| Max tokens per input | 512 |
6. Understanding Rate Limits
| Plan | Limit (approx) |
|---|---|
| Free/Starter | RPM-limited, low |
| Standard | Higher RPM, request increase |
| Enterprise | Custom |
7. Handling Inference Errors
| Error | Cause |
|---|---|
| 429 | Rate limited; backoff |
| 400 truncation | Input exceeds token limit |
| 404 | Model name typo |
8. Optimizing Inference Performance
| Tip | Effect |
|---|---|
| Batch up to 96 inputs | One round-trip |
| Same region as index | Lower latency |
| Integrated inference | Avoids client-side embed step |
9. Comparing Costs
| Provider | Approx Cost (per 1M tokens) |
|---|---|
| OpenAI 3-small | $0.02 |
| OpenAI 3-large | $0.13 |
| Pinecone Inference | Plan-dependent (bundled) |