Semantic Search Service

A Spring Boot service that indexes documents and answers search queries by blending lexical (BM25) and dense vector retrieval into a single relevance score. The React dashboard for query testing and relevance inspection is bundled into the same JAR and served at the root path.

Every document is indexed twice. The first pass writes the tokenised text into Elasticsearch's inverted index for BM25. The second pass embeds the content with a pluggable embedding provider and writes the vector into an Elasticsearch dense_vector field with HNSW indexing for kNN.

At query time both signals run in parallel. Their scores are normalised, then combined using a scoring profile. The default profile (hybrid_v1) is a weighted sum of normalised BM25 and cosine similarity, with optional recency decay and metadata boosts.

POST /api/v1/search
Content-Type: application/json

{
  "query": "kubernetes pod restart loop",
  "filters": { "category": "ops" },
  "limit": 10,
  "profile": "hybrid_v1"
}

Switching profiles is a configuration change, not a redeploy, so different surfaces (autocomplete, full page search, related docs) can use different blends side by side.

GET /api/v1/eval/run walks a configurable gold set of query and document judgments, then reports MRR, NDCG@k, and Recall@5. CI exports a JSON report to target/eval/report.json on every build, which makes it cheap to A/B two scoring profiles before flipping the default.

External dependencies are wrapped with circuit breakers and fallbacks. If Elasticsearch is unreachable the service can swap to an in memory stub via ELASTICSEARCH_STUB_ENABLED=true. The embedding provider has the same toggle for deterministic vectors during tests. Both stubs run in CI so the test suite never needs a real cluster.

A k6 smoke test in perf/k6-smoke.js targets p95 under 400 ms. Health endpoints and Prometheus metrics are exposed on the actuator port for scrape based monitoring.

Java 21, Spring Boot 3.4, Elasticsearch (kNN), PostgreSQL for document metadata, Redis for query result caching, React for the dashboard, Docker Compose for local development. Tests use hand rolled fakes rather than Mockito to stay clean on Java 21+ records and pattern matching.