Designing Search System Architecture
1. Designing Full-Text Search Architecture
| Component | Detail |
|---|---|
| Engine | Elasticsearch / OpenSearch / Solr / Vespa / Typesense / Meilisearch |
| Inverted index | Term → docs |
| Analyzer | Tokenize + filter (lower, stem, synonym) |
| Query DSL | match, term, bool, function_score |
| Replicas / shards | Scale + HA |
2. Designing Search Indexing Strategy
| Mode | Detail |
|---|---|
| Full reindex | Schema change; build alias-swap |
| Incremental | CDC/event-driven |
| Bulk API | Batches of 1–10MB |
| Refresh interval | Trade freshness vs throughput |
| Pipeline | App → Kafka → consumer → ES |
3. Designing Search Relevance and Ranking
| Signal | Detail |
|---|---|
| BM25 | Default text relevance |
| Field boost | title^3 over body |
| Function scoring | Recency, popularity decay |
| Learning-to-rank | LightGBM/RankLib over features |
| A/B test rankers | Online evaluation |
4. Designing Autocomplete and Typeahead
| Approach | Detail |
|---|---|
| Edge n-grams | Index "tes", "test", "testi"… |
| Completion suggester | FST-based; very fast |
| Trie | In-memory structure |
| Popularity weighting | Top suggestions first |
| p99 latency | <50ms for good UX |
5. Designing Faceted Search and Filters
| Feature | Detail |
|---|---|
| Aggregations | terms, range, date_histogram |
| Multi-select facets | Filter context (no scoring) |
| Facet counts | Per-bucket doc counts |
| Hierarchical | Categories with parent paths |
6. Designing Search Pagination
| Method | Detail |
|---|---|
| from/size | Simple; deep paging expensive |
| search_after | Cursor; preferred for deep paging |
| scroll | Snapshot iteration (export use) |
| PIT (point-in-time) | Stable cursor across refreshes |
7. Designing Search Query Performance
| Lever | Detail |
|---|---|
| Filter cache | Use filter context for cacheable |
| Index sorting | Pre-sort for early termination |
| Doc values | Columnar for sort/agg |
| Profile API | Find slow query parts |
| Shard sizing | 10–50GB per shard typical |
8. Designing Fuzzy Search and Typo Tolerance
| Mechanism | Detail |
|---|---|
| Levenshtein | fuzziness=AUTO (1–2 edits) |
| Phonetic | Soundex, Metaphone |
| Did-you-mean | Suggester / spellcheck |
| Synonym sets | Index- or query-time |
9. Designing Search Index Update Strategies
| Strategy | Detail |
|---|---|
| Alias swap | Build new index → atomic alias point |
| Partial update | Update by doc id; full doc reindex internally |
| Version control | Optimistic concurrency via version |
| Backfill | For new fields |
10. Designing Search Analytics
| Metric | Detail |
|---|---|
| Top queries | Trending |
| Zero-result rate | Improve coverage / synonyms |
| CTR | Position-aware |
| NDCG / MRR | Ranking quality |
| Query latency | p50/p95/p99 |
11. Designing Vector / Semantic Search
| Element | Detail |
|---|---|
| Embeddings | OpenAI, Cohere, sentence-transformers |
| Vector DB | pgvector, Pinecone, Weaviate, Milvus, Qdrant; ES kNN |
| Index | HNSW, IVF, ScaNN |
| Hybrid search | BM25 + vector (RRF / weighted) |
| Use cases | RAG, similar items, dedup |
12. Designing Search Personalization
| Signal | Detail |
|---|---|
| User profile | Past clicks, purchases |
| Context | Location, time, device |
| Re-ranking layer | Personalize top-K |
| Privacy | Opt-in; anonymize signals |