Enterprise search rarely fails because it returns nothing at all. More often it returns too many wrong results, documents are stale, product language differs from customer language or users do not know the exact term. Semantic search can bridge those wording gaps by representing content and queries as embeddings and retrieving by similarity.
A vector alone does not create dependable search. Product numbers, names, prices, dates, tenants, roles and approval status remain exact signals. Production search therefore combines semantic candidates with keywords, filters, reranking and a measured quality process.
What embeddings provide
An embedding model converts supported content into numerical vectors. Related meanings tend to be closer in the vector space. A query such as “light jacket for heavy rain” can retrieve a waterproof outdoor jacket even when the product text uses different wording. In documents, “policy for lost devices” may find mobile-device management or device-loss guidance.
Embeddings are not a database with guaranteed factual understanding. They compress model patterns and can create surprising neighbours. Model, language, content type and segmentation influence results, so every important domain needs its own evaluation queries.
Keyword, vector and hybrid search
| Method | Strengths | Typical limitations |
|---|---|---|
| Keyword search | exact terms, IDs, names, specialist vocabulary and transparent weighting | misses synonyms and free-form problem descriptions |
| Vector search | meaning, natural language, similar content and multilingual relationships where supported | can dilute exact signals and return plausible but wrong neighbours |
| Hybrid search | combines exact and semantic candidates with filters and ranking | requires evaluation and calibrated weighting |
Hybrid search is often the strongest baseline, but it is not one fixed slider for every query. A product number should favour exact matches, while a natural-language problem can use more semantic weight. Query rules and reranking can represent these differences.
Data preparation determines result quality
Poor source data does not become good through an embedding. Documents need stable IDs, source, timestamp, language, type and permissions. Products need title, description, category, attributes, seller, price and availability. Images need usage rights, visibility and either a description or a multimodal embedding.
Long documents are split into sections. Segments that are too large mix several topics; segments that are too small lose context. Headings, chapters, tables and lists should guide segmentation. Every segment still points back to its document, version and authorized users.
- detect empty, duplicate and clearly stale content before indexing;
- choose chunk boundaries from document structure, not only character count;
- use stable source and segment IDs for updates and deletions;
- version the embedding model and preparation logic;
- document every filter field in an index contract.
Authorization is applied before output
A business search result must not be hidden only in the interface. Candidate retrieval already needs to enforce tenant, role, user, owner and approval state. Otherwise titles, snippets, facets or result counts can reveal protected documents.
Negative tests matter as much as relevant results: a user in tenant A must never receive a candidate from tenant B. An expired document must leave search and recommendations. A role without HR access must receive neither results nor generated summaries derived from HR records.
Semantic image search
Multimodal models can place images and text in a compatible vector space. A user can search for “person in a red jacket in front of a mountain” without manually maintained tags. Shops can retrieve visually or conceptually similar products; media archives can search subjects, situations or visual similarity.
Metadata remains essential. Copyright, licence, client, campaign, date, approval and visibility cannot be inferred safely from similarity. The vector produces candidates; filters determine which candidates a user may see.
Weaviate as a possible search component
Weaviate can combine vector retrieval, hybrid search and metadata filters. TEX8 can prepare and version content through a controlled in-house embedding pipeline before indexing it in Weaviate. The concrete architecture still depends on volume, tenant separation, backup, latency, hardware, model operations and recovery.
The search index remains a derived projection. The catalogue, CMS, document system or object store keeps business truth. This allows a controlled rebuild without reconstructing source data from search results.
Freshness and re-indexing
A current source does not prove a current search index. Updates, deletions and permission changes must be processed as repeatable events. HTTP 200 from the search service proves reachability, not that every change was delivered.
- each event has a stable ID and can be processed idempotently;
- queue length, oldest event and failed attempts are monitored;
- the last confirmed index state is visible per source;
- a full rebuild can be planned without a public data gap;
- deletions receive the same verification as new content.
Evaluation with real search jobs
A demo with three selected questions does not prove quality. A useful evaluation set includes frequent queries, specialist vocabulary, natural language, misspellings, synonyms, ambiguity, IDs, forbidden content and queries with no meaningful result. Domain owners mark expected results or assess the top candidates.
| Measure | Question | Purpose |
|---|---|---|
| Top-result precision | How many early results are genuinely relevant? | limits plausible but distracting neighbours |
| Recall | Are important known documents retrieved? | finds gaps in data, chunking or model |
| No-result quality | Does the system admit when no good result exists? | prevents random answers and frustration |
| Authorization failures | Does any forbidden result or snippet appear? | tests the central security boundary |
| Freshness and latency | How quickly are updates and queries available? | makes operations measurable |
Separate semantic search from RAG
Retrieval-augmented generation first finds relevant sources and then gives them to a language model. A convincing answer can still be based on poor retrieval. Search and generation are therefore evaluated separately. Retrieval must first provide permitted, current and relevant sources; only then is the answer checked for correct use of those sources.
An AI agent also receives only actions permitted for the user's role. Finding knowledge and authorizing a change are separate contracts.
A controlled rollout
- Define the goal: users, sources, permissions and three to five measurable search jobs.
- Inspect a data sample: quality, metadata, duplicates and deletion path on real content.
- Measure a baseline: compare keyword search against vector and hybrid variants.
- Integrate a pilot: connect one user group and one surface.
- Close operations: monitor queue, freshness, feedback, cost, latency and rebuild.
- Expand: add sources or multimodal retrieval only after measured value.
Next step
Our semantic search development page covers architecture, permissions, evaluation and integration. Typical consumers include an AI marketplace, mobile application, desktop software or an AI agent using approved knowledge.