Semantic Search Service

A Spring Boot service that indexes documents and answers search queries by retrieving on vector similarity, then re-ranking on lexical BM25, metadata boosts, and recency decay. A React dashboard for query testing and relevance inspection is compiled into the same JAR and served at the root path.

The query it is demoed on is keeping p95 response time low, which shares exactly one token with the document it retrieves. That is the case the whole design exists for.

An earlier version scored both signals over the whole corpus and blended them. That is simple to describe and expensive to run: a script-score query touches every vector in the index. The current path is two stages.

1. The query is embedded and the index returns nearest neighbours by cosine similarity, over-fetching 5x the requested number of results, capped at 200. Against Elasticsearch this is an approximate kNN search over the HNSW graph built for the vector field, with metadata filters pushed inside the traversal rather than applied afterwards. 2. Candidates are re-scored: the vector score is blended with a BM25 score computed from corpus-wide term statistics, metadata boosts are added, and recency decay is applied. 3. Anything below minScore is dropped, the list is sorted on the final score, and truncated to limit.

Because retrieval is vector-first, lexical scoring refines the ordering of candidates rather than widening recall. The over-fetch is what gives it room to change the outcome — re-ranking 10 candidates to return 10 cannot do anything.

Cosine scores are normalised across both the in-memory and Elasticsearch backends, so a profile tuned against the local index behaves the same way in a real cluster.

minScore applies to the blended score after boosts and decay, not to the raw vector score, which is always lower. Its default of 0.2 is calibrated against the local embedder: over the gold set, the best match for a natural-language query scores between 0.32 and 0.53. Every query still returns something at a floor of 0.3, and none do at 0.4.

That calibration is provider-specific. A hosted model spreads scores differently and wants a different floor — which is the kind of thing that silently degrades a search product when nobody writes it down.

Recency decay is on by default with a seven-day half-life, so a score drifts slowly downward as a document ages.

Two providers, selected by EMBEDDING_LOCAL_ENABLED:

• Local (default). A feature-hashing vectoriser over word tokens and character n-grams with sublinear term-frequency weighting. No API key, no model download, fully deterministic. It is honestly a lexical model: it scores shared words and fragments, so "ranking" and "ranked" land close together, but it does not know that "car" and "automobile" are related.
• OpenAI. text-embedding-3-small at 1536 dimensions.

Vectors from different models are not comparable, so switching providers or changing the dimension requires an index rebuild. The cache keys encode provider, model, and dimensions precisely so that a switch cannot serve stale vectors from the previous configuration.

Writes go the other way through DocumentService: content hash, dedupe, persist, embed, upsert into the index under the document id, then invalidate the search cache.

Redis holds two caches — embeddings and results — keyed by provider, model, dimensions, and request identity. Results are invalidated on any corpus mutation.

The interesting failure is a partial one: the document persisted to PostgreSQL but the index upsert did not land. Upserts are idempotent so a retry is always safe, and a reconcileUnindexed repair path sweeps documents that exist in the database but not in the index. Without it, a document is silently unfindable and nothing in the system reports that.

GET /api/v1/eval/run walks a configurable gold set of query and document judgments and reports MRR, NDCG@k, and Recall@k, so a ranking change can be compared rather than guessed at. CI gates on MRR, NDCG@5, and Recall@5 — a scoring change that improves one query and quietly breaks four fails the build.

Testcontainers runs the integration suite against a real Elasticsearch index rather than a stub, which is what catches HNSW and filter-pushdown behaviour that an in-memory fake will happily fake. Prometheus metrics and k6 service checks cover the runtime side.

Java 21, Spring Boot, Elasticsearch (dense_vector with HNSW), PostgreSQL for document storage, Redis for embedding and result caching, React for the dashboard, Docker Compose for local development.

The demo profile needs none of it — H2, an in-process vector index, local embeddings, and no authentication, seeded with a small corpus at startup so a clone returns search results immediately:

./mvnw clean package
java -jar target/semantic-search-java-1.0.0.jar --spring.profiles.active=demo